ERP transformations are still among the most complex and expensive initiatives CIOs lead. The stakes are also rising: data suggests that by the end of 2024, only 39% of SAP ECC customers had migrated to S/4HANA, and at the current pace, nearly half may still be on ECC in 2027.
At the same time, the accelerator market is exploding. AI-enabled solutions now show up across various phases of ERP transformation, from discovery through migration and operations. Many of these tools deliver tangible productivity—yet many programs still struggle to translate pilots into scaled, repeatable impact.
However, AI value doesn’t scale just by adding tools; it scales when you build a system that makes tools reusable, governable, and replaceable.
The provider landscape: Why “tool sprawl” happens
Most AI acceleration options in the ERP ecosystem fall into four categories:
- ERP-native capabilities such as SAP Joule or Oracle AI agents. These align with product direction and reduce integration friction but may not cover every niche or transformation phase in a complex, customized landscape.
- Established value-added software companies – Process intelligence platforms like Celonis, ALM/testing platforms, data quality tools, or automation suites that use AI to enhance existing products or add additional features. These are often useful within specific slices of the ERP transformation lifecycle.
- New entrants and startups bringing rapid innovation for specific tasks like code remediation, test generation, and documentation automation—with varying enterprise maturity and longevity.
- Consulting and system integration (SI) players bring accelerators for business value and delivery topics—useful in practice but sometimes creating hidden dependencies and inconsistent toolchains across programs.
This diversity of options should bring more benefits and leverage to enterprises. However, organizations still struggle to convert tool leverage to scalable impact.
The common failure pattern is as follows: each individual workstream buys the best point solution for their tasks, but knowledge remains trapped inside tools or vendors, governance fragments, and switching among the tools to unleash AI benefits becomes disruptive.
The missing capability: intelligent orchestration layer
For an immediate and compounding value, organizations should treat point solutions as what they are: plugins. It’s more important to invest in something more broadly useful than any point solution: an intelligent orchestration layer.
At McKinsey, we see that the intelligent orchestration layers provide measurable value to enterprises in two timelines:
- Immediate value comes from coordinating point solutions through consistent workflows: routing work to the right capability, standardizing handoffs, enforcing approvals, and improving traceability.
- Compounding value comes from pairing orchestration with an intelligence layer that accumulates valuable context over time: gathering, organizing, and showing process variants, fit-gap decisions, exceptions, control points, defect patterns, test assets, migration rules, and outcomes.
This “intelligence” created for the orchestration layer is not a traditional keyword-based knowledge repository or relationship mapping among objects. Instead, it’s a cause-and-effect loop.
Diagram provided courtesy of McKinsey & Company, all rights reserved.
Over time, an intelligent orchestration layer can also self-propose better prompts, recommend next-best actions, and even suggest or assemble “skills” (specialized agent workflows) based on what worked previously—while still keeping humans in control for high-risk decisions.
Example architecture-layered stack
Below is a pragmatic layered stack you can implement incrementally. The key design principle: users interact with stable enterprise workflows, while tools evolve behind the scenes.
Diagram provided courtesy of McKinsey & Company, all rights reserved.
Evaluate AI accelerators across these 5 criteria
With intelligent orchestration as your anchor, evaluation becomes simpler and more durable. Consider these five questions when making your selection:
- Does the tool plug in cleanly (APIs, exportable artifacts, integration patterns)?
- Can you retain what it produces as reusable knowledge — not just outputs, but rationale and lineage?
- Can it operate under your governance model, with the needed security, auditability, and approvals?
- Can you measure end-to-end outcomes (like cycle time, rework, predictability, and quality), not just task speed?
- Can you replace it without retraining the organization?
Make your ERP transformation more intelligent
AI acceleration for ERP transformation is real, but the market is moving too fast to standardize on a single stack for the duration of a multi-year program.
The CIO’s most strategic move is to get immediate value from point solutions and build a compounding advantage through enterprise-owned intelligent orchestration, plus an evolving intelligence layer — so outputs become higher quality, faster, and less dependent on scarce experts over time, powering a cycle of continuous improvement.
Durable value at scale comes from connecting AI ambition to the ERP backbone rather than working around it.
About the author
Rajiv Jha is an Expert Associate Partner with McKinsey with over 18 years of experience in enterprise architecture and tech transformation. His current focus is value creation through AI-enabled business processes and ERP transformations. You can contact him on LinkedIn.