Expanding AI-Driven Innovation with Side-by-Side SAP and Red Hat Architectures

Most enterprise generative AI pilots still fail to reach production, and SAP customers face an additional challenge: adding powerful AI without breaking clean-core mandates or increasing operational risk. Azizur Rahman, managing partner, ZMAN Consulting LLC, explained the implications at SAPinsider Las Vegas in the presentation: “Expanding SAP with AI: Side-by-Side Architecture for Clean Core, Hybrid AI and Regulated Industries.”

How Side-by-Side AI Changes Day-to-Day Responsibilities

Rahman said about 95% of enterprise generative AI pilots deliver no measurable business impact. “All these projects never goes to production and people are wasting billions.”

For SAP technology leaders, the root causes map to three structural traps: proprietary cloud lock-in with high recurring costs and limited on-premises options, build-it-yourself AI programs that spend years and millions without reaching production, and attempts to rely only on native SAP tools that struggle with multimodal, edge and highly regulated use cases.

The side-by-side pattern reframes daily work for SAP architects and platform owners. Instead of embedding models directly in SAP S/4HANA or customizing core tables, teams keep the core clean and treat AI as a separate, governed platform that connects via released APIs, events, and SAP Business Technology Platform Edge Integration Cell runtimes. This preserves quarterly upgrade agility, reduces regression testing, and shifts effort toward designing modular services, event contracts, and MLOps pipelines that can scale from one to more than 100 use cases.

Red Hat Enterprise Linux, OpenShift and the Red Hat AI stack provide the execution fabric for those extensions, including RHEL AI for curated models, OpenShift AI for lifecycle management, and a high-throughput inference server based on vLLM for low-latency responses. SAP Business AI, including Joule and SAP RPT-1 tabular models, focuses on ERP-centric predictions and copilots, while Red Hat platforms handle open models, multimodal workloads, and edge deployments in air-gapped or heavily regulated environments. Day to day, platform teams manage policy-driven model routing, zero-trust security, and audit logging across hybrid landscapes rather than tuning custom ABAP in the core.

Model context protocol and agent coordination add another layer of change. Running MCP on OpenShift AI lets multiple agents for sales, credit, logistics, and finance share context across an order-to-cash process without manual reentry or brittle, point-to-point integrations. For enterprise architects, this shifts focus toward designing agent workflows, context schemas, and role-based access policies that orchestrate complex, cross-domain processes while keeping database credentials and sensitive data shielded behind standardized interfaces.

Evaluation Criteria, Use Cases and Best Practices

Rahman outlined criteria SAP customers should apply when evaluating AI platforms and partners. Any solution must respect clean core principles by avoiding direct table modifications and relying on released APIs, events and extension points.

“The ultimate goal is to keep the clean core clean,” Rahman said. “If that’s the case, then how do you bring those innovations like GenAI without breaking clean core principles?”

Leaders also should demand hybrid and air-gapped deployment options that satisfy sector-specific regulations in energy, utilities, and regulated manufacturing. Platforms further need integrated governance and observability so every prediction, recommendation, and automated action is traceable with risk scores, explanations and audit-ready evidence.

Operational teams in service and operations also see tangible changes. In SAP Service Cloud environments, AI-based ticket routing and SLA risk prediction shorten resolution times and lower SLA penalties by steering cases to the right teams and generating draft responses with full traceability.

Substation and operational technology (OT) security scenarios fuse edge telemetry with SAP enterprise context on OpenShift to detect insider threats, cyberattacks and maintenance issues in real time.

Clean core AI success depends less on individual models and more on disciplined architecture, governance and an incremental rollout path that moves organizations from assessment to proof of value and then to production-scale MLOps within months instead of years.

What This Means for SAPinsiders

Clean-core side-by-side AI becomes a strategic mandate. Enterprise leaders should prioritize architectures that move AI out of the ERP core while tightening API governance, upgrade safety, and regulatory compliance across hybrid SAP landscapes.

Hybrid, open AI platforms reshape partner strategy. SAP vendors and integrators must align offerings with co-engineered, cloud- and edge-ready stacks that avoid lock-in, support air-gapped deployments, and deliver measurable ROI across finance, operations, and security.

Agentic workflows redefine process automation. Transformation leaders should plan for multi-agent, MCP-enabled orchestration that coordinates SAP and non-SAP systems, elevating automation from isolated use cases to governed, end-to-end business execution.