Data Provenance: Essential for AI-Ready Data in Regulated Financial Environments

Every Tier 1 financial institution I talk to is asking a version of the same question about their AI initiatives: How do we know this data is what we say it is?

It sounds simple. It is not. Behind that question sits the capability that determines whether an AI system can be trusted, explained, or defended to a regulator – and most institutions don’t have a name for it yet. The name is data provenance, and it is quietly becoming the most important foundation for AI-ready data in regulated environments.

Provenance is routinely mistaken for documentation – a record you produce after the fact, a compliance chore, a diagram that lives in a wiki nobody reads. That framing badly undersells it. Data provenance is not a description of your data. It is the evidence behind it. In a world where AI outputs increasingly drive decisions that carry regulatory, financial, and reputational weight, that evidence layer is no longer a nice-to-have. It is the difference between AI you can stand behind and AI you have to hope nobody asks about.

What is data provenance?

Data provenance is the verifiable record of where a piece of data came from, how it was created, and every transformation it underwent on the way to its current state. It answers three questions with evidence rather than assertion:

• What is the origin of this data?
• What has been done to it?
• And can we prove both?

The word itself is borrowed from the art world, where provenance is the documented chain of ownership that establishes a work is authentic and not a forgery. The concept translates almost perfectly to data. A number in a report, a feature in a model, a field in a migrated system – each has an origin and a history. Provenance is the ability to trace that history back to its source and demonstrate that nothing along the way is unaccounted for.

That word – demonstrate – is what separates provenance from ordinary metadata. Plenty of systems can tell you what a field is called and what type it holds. Provenance tells you where the value came from, what logic shaped it, and gives you the receipts to prove it. In regulated financial services, the receipts are the entire point.

Data provenance vs. data lineage

Because the two terms are often used interchangeably, it’s worth being precise: data lineage and data provenance are related but not the same, and the distinction matters more than it appears.

Data lineage maps the path data travels – the flow from source systems through pipelines, transformations, and joins to the tables and reports where it lands. Lineage is directional and structural. It shows you the plumbing.

Data provenance is broader and deeper. It encompasses lineage but adds the evidentiary dimension: not just the path, but proof of origin, the logic applied at each step, the context that explains why a transformation happened, and the ability to reconstruct and defend the full history. Lineage tells you the water flows from A to B. Provenance proves the water is clean, tells you what was added along the way, and lets you certify it to an auditor.

For most organizations, lineage is the necessary substrate and provenance is the capability built on top of it. You cannot have credible provenance without lineage – but lineage alone, especially when it stops at the boundary of legacy systems, is not enough to make data AI-ready.

Your provenance program is only as complete as its weakest link, and for most financial institutions that link is the core. Contextual Data Lineage is how you establish provenance where it is hardest to establish: inside legacy code. It reads the calculations, conditions, and business rules embedded in COBOL, RPG, and PL/1 – so the transformations happening in the systems you can see least are not just traced, but explained.

Provenance is the evidence layer for AI-ready data

Here’s why this is landing now rather than five years ago.

AI systems are only as trustworthy as the data underneath them, and trustworthiness in a regulated context is not a feeling – it’s a burden of proof. When a model produces an output that informs a credit decision, a risk calculation, or a regulatory report, “the model said so” is not a defensible answer. You have to be able to explain the output, and explaining the output means being able to account for the data that produced it, all the way back to origin.

This is one anchor of the enterprise AI-readiness conversation: explainability is what makes data AI-ready and trusted in a regulated environment. And explainability is impossible without provenance. You cannot explain what you cannot trace. You cannot defend what you cannot prove. Provenance is the evidence layer that makes explainability real rather than aspirational.

The research analyst community has converged on the same conclusion. In discussions of the emerging “context layer” that AI agents depend on, provenance is increasingly named as one of its core components – the mechanism that lets an agent (or the humans accountable for it) know that the data it’s reasoning over is authentic and appropriate for the task. Others have framed lineage and provenance explicitly as the substrate for trust scoring, data quality, and observability across AI-ready data pipelines. Different vocabulary, same underlying point: provenance is moving from a back-office concern to a front-line requirement for AI.

This connects directly to the broader case for AI-ready data in financial services, where usability and trustworthiness are the two pillars that determine whether AI initiatives can scale at all. Provenance is what makes the trustworthiness pillar load-bearing.

In regulated environments, provenance is a defense – not documentation

Outside of regulated industries, weak provenance is a quality problem. Inside them, it’s an exposure.

Financial institutions already live under standards that are, in effect, data provenance mandates. BCBS 239 – the Basel Committee’s principles for risk data aggregation – requires that banks be able to prove where their risk numbers come from and trust their accuracy under pressure. The expectations behind model risk management guidance, audit trails, and regulatory reporting all rest on the same foundation: the ability to trace a figure to its source and defend every transformation along the way. (For more on why that standard is the right one, see our open letter to bank regulators on BCBS 239.)

Now layer AI on top of that regime; AI expands both the volume of data-driven decisions and the distance between a decision and the human who can explain it. Every model that touches regulated data inherits the institution’s burden of proof. If the data feeding that model has gaps in its provenance, those gaps don’t disappear – they compound. An unexplained transformation in a source system becomes an unexplained feature in a model becomes an indefensible output in front of a regulator.

This is the trap of data derivatives – abstractions built on abstractions, where each layer of transformation moves the working data further from its origin and quietly erodes traceability. We’ve written before about why business leaders need the data lineage they aren’t asking for: the danger isn’t that the data is wrong, it’s that no one can prove it’s right. Provenance is the antidote. It’s the difference between an AI program that accelerates under regulatory scrutiny and one that stalls the moment someone asks a hard question.

Provenance, data quality, and AI governance

Provenance is also where data quality for AI and AI data governance stop being separate initiatives and start being the same one.

You cannot meaningfully assess the quality of data you cannot trace. Completeness, accuracy, and fit-for-purpose are all judgments that depend on knowing where data came from and what happened to it. A dataset can look pristine and still be unfit for a given AI use case because of a transformation three systems upstream that nobody documented. Provenance surfaces exactly that kind of hidden defect.

The same is true for data governance for AI. Governance frameworks set policies about how data can be used, by whom, and for what – but a policy you can’t enforce or audit quickly loses value. Provenance is the enforcement mechanism. It’s what lets a governance program verify that training data was appropriately sourced, that sensitive fields carry their handling requirements through every transformation, and that a model’s inputs are consistent with the rules the institution has committed to. Strong provenance turns AI governance from a set of aspirations into a set of verifiable facts.

How to establish data provenance

Provenance is earned, not declared. Building it into an AI-ready data foundation comes down to a few disciplines:

Start at the source, including the legacy source.

The hardest provenance gaps almost always live in the oldest systems – the “legacy core”, or the mainframe and AS/400 applications written in COBOL and RPG, with decades-old business rules that no current employee fully understands. If provenance stops at the boundary of those systems, it isn’t provenance. Extracting and making that embedded logic explicit is what turns an opaque legacy estate into a traceable one.

This is the specific job Contextual Data Lineage does. A provenance strategy that treats the legacy core as out of scope isn’t a provenance strategy – it’s a provenance strategy with a hole in exactly the place regulators look first.

Capture transformation logic, not just data movement.

Knowing that data moved from A to B is lineage. Knowing what rule was applied in the move – and being able to show it – is provenance. Every transformation should carry its logic and its rationale as first-class information, not as a comment buried in a script.

Preserve context through change.

Data migrations, system consolidations, and platform modernizations are exactly the moments provenance is most likely to break – and most important to keep intact. Treating migration as an ongoing capability rather than a one-time project is what keeps the evidence chain unbroken as systems change underneath it.

Make it queryable and defensible.

Provenance that lives in static documentation is provenance you can’t use under pressure. The goal is a living, traceable record you can interrogate on demand – when a regulator asks, when a model behaves unexpectedly, when an AI output needs to be explained.

Done well, these disciplines don’t just satisfy auditors. They compound. Every system change that preserves provenance makes the next AI use case faster to stand up, because the trust work is already done.

Provenance is a strategic capability, not a technical afterthought

The institutions that will win with AI in regulated markets are not the ones with the flashiest models. They’re the ones that can move fast because they can trust and access their data with explainability. Provenance is what lets them say yes to an AI opportunity that a less-prepared competitor has to decline – not because the competitor lacks the model, but because they can’t defend the data underneath it.

That’s the reframe worth internalizing: data provenance is not a documentation task you complete once and file away. It is a strategic capability that determines how quickly, and how safely, your institution can turn data into AI-driven value. In regulated environments, it is the foundation everything else is built on.

The question every institution is asking – how do we know this data is what we say it is? – has an answer. The answer is provenance. The only question left is whether you’ll build it before your regulators, your auditors, or your own AI ambitions force the issue.

Zengines helps financial institutions build AI-ready data foundations through a Turnkey Data Migration Platform and Contextual Data Lineage – making modern and legacy data not just movable, but explainable.

Get a demo →

Frequently asked questions

What is data provenance?
Data provenance is the verifiable record of where data originated, how it was created, and every transformation it underwent to reach its current state. Unlike basic metadata, provenance provides evidence – proof of origin and history – not just description.

What is the difference between data provenance and data lineage?
Data lineage maps the path data travels from source to destination. Data provenance encompasses that path but adds the evidentiary layer: proof of origin, the logic applied at each transformation, and the ability to reconstruct and defend the full history. Lineage shows the flow; provenance proves it.

Is contextual data lineage the same as data provenance?
No. Data provenance is an outcome that spans the entire data estate. Contextual Data Lineage is a capability that delivers provenance in the one place it is hardest to reach – inside legacy code – by extracting the calculations and business rules embedded in COBOL, RPG, and PL/1 so legacy transformations can be evidenced rather than assumed. Provenance is only as complete as its weakest link, and for most financial institutions that link is the core.

Why is data provenance important for AI?
AI outputs are only as trustworthy as the data behind them. In regulated environments, “the model said so” is not defensible – you must be able to explain an output by tracing its data back to origin. Provenance is the evidence layer that makes that explainability possible, which is what makes data genuinely AI-ready.

How does data provenance support regulatory compliance?
Standards like BCBS 239 effectively require institutions to prove where their data comes from and defend every transformation. Provenance provides that proof, giving auditors and regulators a traceable, defensible record of how each figure and model input was produced.

Is data provenance the same as data quality?
No, but they’re inseparable. You can’t credibly assess data quality without knowing where data came from and what happened to it. Provenance surfaces hidden defects introduced upstream, making it the foundation on which data quality for AI is judged.

On June 18, 2026 Gartner published a prediction that should make every CIO sponsoring a mainframe exit pause:

“More than 70% of mainframe exit projects initiated in 2026 will fail to produce the intended benefits due to an overestimation of generative AI (GenAI) tooling capabilities.”

I agree with Gartner. We see it every week.

Gartner’s recommendation underneath the headline

It’s worth clarifying what Gartner means. Mainframe modernization encompasses both migrating off the platform and modernizing in place. The 70% failure figure applies specifically to full exits. For most workloads, Gartner is recommending in-place modernization instead.

The 70% figure will generate most of the talk, but the body of the release is making a sharper point.

“For many mainframe customers, GenAI can be more effectively used to enable modernization in place rather than accelerate migration off the platform.” — Alessandro Galimberti, VP Analyst at Gartner

Gartner is recommending a platform-smart approach – evaluating workloads individually and placing them in the right environments, rather than chasing a wholesale exit. Organizations should balance strategies focused on optimizing existing mainframe investments while limiting full platform exits to select, case-by-case scenarios – efforts that, in Gartner’s words, require high-risk transformation and often result in suboptimal outcomes.

That isn’t a story about better migration tooling. It’s a story about asking better questions and a fact-based assessment before the migration question is even on the table.

What GenAI can and can’t do in a mainframe exit

Gartner contends that the high failure rate is due to the expectation that generative AI will fix complex legacy code easily. Based on my experience with multiple transformation initiatives involving mainframes, I’ve observed and dealt with what generative AI can and cannot do well.

What GenAI can do well: read code at scale, surface technical debt, summarize what a module appears to be doing, generate first-pass documentation. It’s useful and time-saving work.

What GenAI can’t do, at least not reliably, in a mission-critical mainframe environment:

• Tell you why a calculation produces the result it does today.
• Provide reliably consistent and complete explanation, especially when functional threads traverse nested modules.
• Tell you what a branching condition in a COBOL module is actually checking against, and what business rule that condition encodes.
• Tell you whether a hard-coded value in a 1998 program was a temporary patch or a deliberate business decision that downstream systems now depend on.

These gaps – between code description and business meaning – are where mainframe exit projects fail. And it’s these gaps that a generative model, however capable, can’t close on its own.

The cost of not understanding what’s there – today

The Gartner finding focuses on exits – but the cost of not understanding your mainframe shows up long before any exit project starts.

Every time a business requirement changes – a new regulation requires a different calculation methodology, a product team needs to update how interest is accrued, an auditor asks where a number came from – someone has to go into the mainframe and answer the question. Before they can change a single line of code, they need to trace what the change will affect: which modules read the variable, which tables get updated, which downstream processes depend on the output, which conditional branches treat it differently.

In a typical environment, that investigation can take weeks or months – and it depends on a shrinking pool of mainframe specialists who are simultaneously running the system. The risk of getting it wrong is real: unintended consequences that show up weeks later in a reconciliation break or a misstated customer statement.

This is the recurring cost the modernization conversation usually skips over. It’s the cost of operating the mainframe without visibility into it – every quarter, every change request, every audit cycle. The 70% of exits that will fail isn’t the only story. The other story is the daily tax that organizations are paying for systems they can’t fully explain.

Explainability is the precondition

Explainability is what makes data AI-ready in a regulated environment. The same principle applies to legacy systems: a system is decision-ready – for modernization, for regulators, for the next code change – when you can explain, with traceable evidence, how it produces what it produces.

Gartner’s framing is correct: AI is being asked to do work it cannot reliably do. But the deeper lesson in the 70% failure number is that the work AI is being asked to skip is the strategic work that determines whether the right path was chosen in the first place. And that work pays for itself long before modernization day – it pays for itself every time the business asks a question the mainframe is supposed to answer.

What the teams who get this right are doing

The mainframe programs that succeed – whether the answer is a full exit, modernization in place, a hybrid, or just running the system safely for the next several years – share a pattern. They treat understanding the legacy environment as a first-order capability, not a one-time pre-migration task.

They invest in surfacing the actual business logic embedded in mainframe code – the calculation logic, the conditional branches, the field-level relationships, the cross-module dependencies – before they decide what to replicate, retire, redesign, or leave alone. That investment doesn’t just serve the eventual migration. It makes today’s mainframe safer to manage, today’s regulatory questions faster to answer, and today’s code changes less risky to make.

At one Fortune 100 financial institution where Zengines Contextual Data Lineage is used every day, the team’s question started with “For each module that touches a regulated calculation, what does it actually do, and what depends on it?” That question scopes an honest mainframe modernization program – what to exit, what to modernize in place, what to leave alone, and how to operate the mainframe safely in the meantime.

The difference isn’t only faster code conversion. The difference is that they know what they have – every day, not just on modernization day.

Mainframes aren’t going anywhere

Despite the industry’s push toward cloud migration and modernization, many financial institutions still rely on mainframe systems to process millions of daily transactions, calculate interest accruals, manage account records, and run core business operations. And they will for years to come.

Modernization is the eventual reality for most organizations still running on mainframes. For many financial institutions, a full modernization effort is on the roadmap but years away – dependent on budget cycles, vendor timelines, regulatory considerations, and a hundred other competing priorities. For others – and this is increasingly what Gartner is pointing toward – modernization will mean working with the mainframe, not off of it.

Either way, the system runs every day in between. And every day, it has to be safely managed, changed, audited, reconciled, and explained.

How Zengines bridges today and tomorrow

This is the bridge Zengines was built to be.

Zengines Contextual Data Lineage parses COBOL, RPG, and PL/1 at scale and surfaces what is actually inside legacy code: the data paths, calculation logic, conditional branches, hard-coded values, and module-to-module dependencies that determine how a legacy system produces what it produces. Analysts get the answer to a reverse-engineering question in minutes – in plain English, with business context – instead of waiting weeks for a mainframe SME to dig through code by hand.

That visibility pays off on two timelines. Today, it makes the mainframe safer to manage: business requirement changes get scoped accurately, regulators get answers in hours instead of months, and engineers can make code changes with confidence about the blast radius. Tomorrow, whenever modernization day arrives – whether that means a full exit, modernization in place, or a workload-by-workload approach – the team isn’t starting from scratch. The understanding is already there.

The mainframe isn’t the problem. The lack of visibility into it is.

If you are managing a mainframe today, planning to modernize tomorrow, or – as Gartner is increasingly suggesting – deciding whether modernization should mean staying on the platform and changing how you work with it, we’d like to show you what Contextual Data Lineage surfaces in your environment.

Get a demo →

In this episode of the Finovate Podcast, host Greg Palmer sits down with Caitlyn Truong, CEO and Co-founder of Zengines, fresh off the company's Best of Show win at FinovateSpring 2026.

Caitlyn traces her path from hardware and software engineering in telecom to financial services consulting, where she and her co-founders kept running into the same gap: critical business logic locked inside legacy core applications written in COBOL, RPG, and PL/1. With 92 of the top 100 banks running COBOL mainframe cores and over half of credit unions and regional banks operating on RPG cores, that black box isn't an edge case — it's the industry norm.

Key points from their discussion

• Beyond pathway tracking: Traditional lineage tools show where data travels. Zengines Contextual Data Lineage ingests entire legacy codebases to reveal not just what happens to data, but why and how — the calculations, conditions, and business rules embedded in the code itself.
• Answers in seconds, not months: Business analysts, data analysts, compliance teams, and technical staff get self-service answers to questions that previously required waiting on scarce subject matter experts.
• Three use cases driving urgency: Meeting regulatory compliance requirements, de-risking modernization and transformation programs, and making legacy data AI-ready with the trust and traceability regulated institutions need.
• The Finovate experience: Caitlyn shares how the Sherlock Holmes-themed demo brought "shining a light into the black box" to life on stage — and her advice for first-time demoers on using seven minutes to plant hooks that turn into real booth conversations.

Listen to the full episode

Watch the demo replay