Why AI Agents Hallucinate on ERP Data (And What Actually Fixes It)

Your AI agent pulled an AR aging number that didn't match what your Controller already knew. It sounded certain. It was wrong. This is why AI agents hallucinate on ERP data: not because the model guessed badly, but because two systems in your stack each hold a different record for the same customer, and the agent had no way to know which one was authoritative. If you have run a pilot in the last two years, you have likely seen some version of this. del.ai builds AI-ready ERP migrations for a living, so read the rest of this with that in mind: we have a commercial interest in the diagnosis below, and we still think it is correct. This article covers what an AI agent actually needs to stop hallucinating on financial data, why closed ERPs block that need structurally, and a short test you can run on your own last pilot before you accept another vendor's explanation.

The Agent Gave a Confident Wrong Number

Picture the scene. A Controller asks an AI agent for the AR aging summary before the Monday close review. The agent returns a number in under a second, formatted cleanly, no hedge in the language. It is $340,000 off. Not because the agent misread anything: it pulled real invoice records from the ERP and real records from a downstream billing tool, then reconciled them as if they described the same eleven customers. They did not. The billing tool created a new "Acme Corp" record eight months ago because the original NetSuite record had a typo in the tax ID field. The agent had no way to know these were the same buyer. It answered the question it was asked, using the data it was given, and produced a confidently wrong number. This is why AI agents make things up on financial data in the first place: not a broken model, but two contradictory records presented as if they agree. By the time anyone notices, the number has often already been dropped into a board deck or a cash forecast, and undoing that costs far more than catching it up front would have.

This is not a rare glitch, though it is worth being careful about how strong a claim that supports. The figure people quote is MIT NANDA's: 95% of enterprise generative AI pilots delivered no measurable P&L impact, from The GenAI Divide: State of AI in Business 2025. State the sample when you use it — 52 executive interviews, 153 survey responses and analysis of roughly 300 public enterprise deployments — because it is an enterprise sample, it is contested, and a mid-market finance team is not the population it measured. It is a strong signal about the shape of the failure, not a probability you should apply to your own next pilot. Most postmortems on a dead pilot blame the model, the vendor, or the team that scoped it. Before accepting any of those explanations, it is worth asking a narrower question: is this a model problem, or is it a problem with what the model was reading in the first place?

It's Not the Model. It's What the Model Is Reading.

The instinct after a failed pilot is to wait for a better model, or switch to a bigger one. That instinct treats the wrong variable as the bottleneck.

Frontier models are strong at open-ended reasoning in the abstract, but independent benchmarking of LLMs on real-world financial research tasks found the best-performing model tested, OpenAI's o3, reached 46.8% accuracy (Finance Agent Benchmark, 2025). A month-end close needs closer to 99.9 percent, because a single wrong journal entry either gets caught by a human anyway, which defeats the point of automating it, or it doesn't get caught and becomes a restatement risk. That gap is not the kind a smarter model closes on its own. It is the kind that closes when the thing the model is reading stops contradicting itself.

A foundation model, on its own, has no hands into your systems and no default context on your business. It is raw reasoning ability, not a wired-in operator. It does not know that "Acme Corp" and "ACME CORPORATION" are the same customer unless something upstream told it so, reliably, every time. Waiting a year for the next model release does not fix a database where the same buyer exists under three different names; it just delays finding that out. There is an older, macro-level version of this shape worth borrowing carefully. Brynjolfsson, Rock and Syverson's productivity J-curve work found that gains from a general-purpose technology lag the technology itself, because complementary intangible capital has to be built first (American Economic Journal: Macroeconomics, January 2021). Read it as an analogy and nothing more: it is an aggregate exercise measuring hardware and software intangibles through 2017, not a measurement of AI, and it says nothing about any individual firm's pilot. The same shape shows up here. A better frontier model is a tailwind, not a fix, until the schema underneath it stops feeding it contradictions. That is the mechanical core of most AI hallucination on financial data: the model is asked to reconcile inputs that were never reconciled upstream.

This is the reframe the rest of this article works from. The hallucination problem sits below the model, in the substrate the model has no choice but to trust.

The 5 Things Every AI Pilot Needs: Closed ERPs Block All 5

Strip away the vendor language, and every AI pilot that touches financial data depends on the same five primitives. Miss any one and the agent either cannot act, cannot be trusted, or both.

What Does an AI Agent Need to Not Hallucinate?

An AI agent needs five things to avoid hallucinating on financial data: clean data, a unified ontology, an action space it can write to, evaluation loops that catch drift, and memory that persists across sessions. Clean data means one authoritative record per entity, not several contradictory ones spread across systems. A unified ontology means "customer," "SKU," and "GL account" resolve to the same definition everywhere the agent looks. Action space means the agent can post a corrected entry, not just describe one. Evals mean scoring output against ground truth on a schedule. Memory means the agent does not relearn your chart of accounts every session. A vendor's own AI layer can supply the action space inside its own application: Oracle's MCP Standard Tools SuiteApp creates and updates NetSuite records, so the claim that agents on NetSuite cannot act is wrong. What one application's AI layer cannot supply is an ontology spanning systems whose data it does not own, or a schema you can correct yourself.

Source: Oracle, "Available Tools in the MCP Standard Tools SuiteApp," NetSuite Applications Suite docs, 2026. ↗

None of this is a setting you can flip. On a vendor-hosted ERP, the tool surface is defined and versioned by the vendor, on the vendor's release cycle. Oracle documents the shape of it plainly: MCP tools run under the authorising user's role and never with Administrator or full permissions, cannot invoke Suitelets or SuiteScript that runs with elevated privileges, cannot make HTTP requests to external destinations, and have "Run as role" disabled. The custom SuiteQL tool accepts read-only queries. None of that is a scandal — it is a coherent design for a platform where every tenant runs on the same code. It is also a ceiling somebody else sets, and one you cannot raise.

That distinction is the honest one, and it is narrower than the version of this argument we used to make. The problem is not that agents cannot act on NetSuite. It is that the ontology sits inside one application, and the schema underneath it is not yours to correct. Bolting a chat interface onto existing screens does not solve that either: the chat interface inherits the same contradictory records the humans were already working around by hand. See why AI pilots die before they touch the ERP for the infrastructure argument in full.

How a Unified Ontology Stops Hallucination

Of the five primitives, two do the most work against hallucination specifically: clean data and unified ontology. This is the mechanism finance teams actually depend on when they say an AI agent runs on a unified ontology, more than any prompt-engineering trick. The other three, action space, evals, and memory, matter for reliability more broadly, but a wrong number in an AR aging report traces back to these two almost every time.

How Does a Unified Ontology Reduce AI Hallucination?

A unified ontology reduces AI hallucination by giving every system one canonical definition of "customer," "SKU," and "GL account," so the agent stops guessing which conflicting record is authoritative. Without it, an agent asked for AR aging silently chooses between two "Acme Corp" records, one in the ERP, one in a billing tool, presenting its pick with full confidence and no signal a choice was made. With a unified ontology, that choice was already resolved once, by a person who traced both records to the same tax ID and merged them. Gartner's analysis of why generative AI projects fail puts poor data quality among the recurring causes, producing unreliable outputs and retrieval implementations that never work properly — which is the same finding read from the other end. Migration is one of the few points in a company's lifecycle where this gets rebuilt as a deliverable rather than patched with a retrieval layer on top of the same schema.

Source: Gartner, "Why Half of GenAI Projects Fail," 2025. ↗

Clean data reduces AI hallucination on its own, but a unified ontology is what keeps it that way as new records get added on top. This is also why a copilot bolted onto an existing ERP cannot replicate the fix. A copilot reads whatever the existing schema already contains; it has no mechanism to go back and merge two contradictory customer records, because that requires editing the underlying tables, not just querying them. Migration does touch the underlying tables. For qualifying migrations within the signed scope document, the customer, SKU, and GL mappings get resolved once during the 90-day build, and every correction a Controller makes in that window sharpens the data model permanently. The gap only widens as more agents get added on top of the same unresolved records, because each new agent inherits the same ambiguity the first one did. A vendor selling a copilot on top of your current NetSuite instance cannot sell you this, because the fix requires owning the migration, not renting an assistant on top of it.

If the number you actually care about is what staying on NetSuite costs versus migrating, skip ahead. See the five-year math in The Real Cost of NetSuite Nobody Publishes: license, Alliance Partner retainer, SuiteApps, BI/ETL tooling, and internal headcount, side by side against the cost of migrating.

The Ambiguity Tax: Model Failure or Data Failure?

This is the question we put to Controllers and CFOs early in a conversation, because it separates two very different problems: when the agent gives you a wrong number, how do you know whether it is the model's fault or your data's fault? Most teams have never had to answer it, and that gap has a cost worth naming.

How Do You Tell If an AI Pilot Failed Because of the Model or the Data?

On a closed, undocumented ERP schema, you usually cannot tell whether an AI pilot failed because of the model or because of the data underneath it, and that ambiguity carries a real cost: each debug cycle means tracing the failure back through an API you do not control, usually with outside help, and by the time the trace finishes the pilot has often been quietly shelved. On a versioned, documented ontology built during migration, the same failure is inspectable instead of ambiguous. A wrong AR number traces to a specific record, a specific merge decision, or a specific stale field, not a shrug. That difference is what "grounding" buys operationally: not a claim about better AI, but the ability to point at one row in one table and say, this is what was wrong, and here is exactly when it got fixed.

Source: del.ai diagnostic methodology, 2026

Call this the ambiguity tax: the cost of not knowing why a pilot failed, paid out in consultant hours and in a shelved project nobody reopens. It is also close to the real question behind "your AI agent on my GL, what stops it from screwing up," one of the first things Controllers ask on a call. Part of that answer is the clean, migrated ontology described above, which lowers the hallucination rate at the source. The other half, permissioned write access with a human sign-off on anything that moves money, is a separate governance layer still being built out and is not the subject of this article. The data-layer half is real today, and it is the practical shape AI agent data grounding delivers on an ERP system: not a philosophy, a specific record you can point to. The governance half is a fair thing to ask about directly.

What This Looks Like in Production

Here is what we intend to ship, stated as intent. del.ai was founded in May 2026 and is pre-revenue: no completed migrations, no customers, and the agents below exist as demos rather than as deployments with references behind them. For qualifying migrations within a signed scope document, two agents are scoped to run on the cleaned ontology at go-live: a month-end close agent that reconciles AR, AP and bank transactions against the general ledger, and an FP&A draft agent that pulls actuals against budget and drafts a variance narrative. Both read the same unified customer, SKU and GL definitions described above, because both are built on the same migration. Neither needs a separate integration project to see across systems, because by go-live there is one system of record left to read from.

The bar for a production agent is not "impressive in a demo." It is "a Controller trusts the output enough to act on it without re-checking every line." That is a much higher bar, and it is the one that actually matters. An agent whose output always needs manual verification has not saved anyone time. It has just moved the work downstream.

Here is the honest caveat this article will not skip: clean data and a unified ontology remove the largest source of hallucination, but they do not finish the reliability job by themselves. An agent still needs ongoing evaluation against ground truth, drift monitoring as the business changes, and operational hygiene around who can approve what. That is a separate layer of ongoing work, not a one-time fix baked permanently into the migration. This article fixes the substrate the model reads. It does not claim to be the entire reliability stack, and any migration partner who tells you otherwise is describing a demo, not a production system.

Run the Ambiguity-Tax Test on Your Own Pilot

Before booking a call with anyone, including us, run one test on your own team. Think back to the last time an AI pilot gave your team a number that turned out to be wrong, and ask the question directly: could we point at the exact record that caused it? Most teams have never actually run this test, because nobody thought to ask it in the room where the pilot quietly got killed.

If the answer is yes, immediately, with a specific row in a specific table, your data layer is probably in reasonable shape, and the failure was likely a scope problem or the wrong model for the job.

If the answer is no, if the honest response is some version of "we're not sure, it was probably a data issue but nobody traced it," the problem is not the model you tried. It is the substrate underneath it, and switching models again will reproduce the same failure with a different vendor's logo on it.

That is the diagnostic this whole article has been building toward. The next step, if you want it traced instead of guessed at, is a conversation, not another pilot.

Built for mid-market companies on NetSuite spending $120k+/yr whose AI pilot stalled or gave you a number you couldn't trust.

30 minutes. Bring your last AI-pilot failure and we will trace together whether it was the model or the data. No pitch. You will leave the call knowing which one it was, and what it would take to fix the actual cause.