Customer Data Onboarding: How Software Companies and Service Providers Get New Clients Live Faster

I’ve spent enough time on data migration programs to know what the factory floor actually looks like.

You’ve got a team profiling the source data in one corner. Someone else is building mapping and transformation instructions in another. A third group is then writing scripts for those transformation rules, and they may or may not have the full business context. And somewhere down the line, a QA team is running reconciliation tests against a target system they’ve only seen in documentation.

Every station is staffed with skilled people doing their part well. But there’s no shared conveyor belt connecting them. Context passes by hand in a “data lossy” manner. Gaps appear between stations. And the whole operation moves at the speed of its slowest handoff.

This is what most data migrations look like before something changes. Here’s why the factory floor breaks down, what it looks like when every station is connected, why you can’t build a reliable factory until you understand the machine you’re replacing — and why a connected factory doesn’t just finish faster. It produces data your business can actually trust and use.

The Island Problem in Every Data Migration

The typical migration setup isn’t broken because the people are wrong. It’s broken because nobody owns the whole picture. The work got split into stations for efficiency, and each handoff between them bleeds context. What starts as a decision becomes a guess by the time it reaches the next station. Analysis happens in one system. Mapping happens in a spreadsheet. Transformation rules get written in SQL or passed to engineers over email. Testing happens in yet another environment. And critical knowledge — the kind that determines whether a field should be split, coerced, or dropped entirely — lives in someone’s inbox or, worse, someone’s head.

The result is predictable: rework, misinterpretation, delays, and the constant feeling that your migration is one miscommunication away from a serious problem. Teams spend more time coordinating than converting. And the business analyst who knows the answer is waiting on the engineer who knows the syntax — a bottleneck that didn’t need to exist.

What the Conveyor Belt Should Look Like

Imagine the same factory floor, but now there’s a single conveyor running through every station, and what’s on it doesn’t get consumed and passed on, it stays visible and active the whole way through. For example, data profiling doesn’t just inform upfront analysis. It feeds mapping. It feeds transformation predictions. It’s still there when load files get generated. Every station pulls from the same live metadata instead of inheriting someone else’s interpretation of it. And every party — your migration data teams, the target platform vendor, any third-party consultants, your internal transformation team — works from the same data picture.

No critical knowledge living in someone’s inbox. No “lost in translation” between the person who knows the business rule and the person who builds the data rule. The person who knows the answer can act on it directly.

This isn’t aspirational. This is what it looks like when analysis, mapping, transformation, and reconciliation live in one platform — where AI assists at every step but the business analyst stays in control.

Explore the Zengines Turnkey Data Migration Platform →

Why Iterations Beat Waterfalls

There’s a reason high-volume, sensitive data reveals itself through iterations rather than one massive end-of-project event. Migration is inherently iterative. You profile, you map, you load, you reconcile, you find something unexpected, and you go back. That’s not failure — that’s how good migrations work.

The problem is when the tooling doesn’t support iteration. When generating a single load file takes days because it requires coordination across three teams and two environments, you iterate slowly. And slow iteration means surprises pile up until they’re program-threatening.

When you can generate a load file in minutes, test it immediately, and adjust — that changes everything. Confidence builds progressively. Issues get flushed out early, fast, and often. The go-live conversation shifts from “are we ready?” to “we’ve already validated this twelve times.”

Built for the Business Analyst, Not Just the Engineer

One of the most expensive patterns in data migration is the handoff between business analysts and engineers. The BA knows what the data should look like in the target system. The engineer knows how to write the transformation syntax. Between them is a queue, a potential misunderstanding, and wasted time.

When transformation rules can be generated from plain English prompts — when a BA can describe “split this field and give me only the last name” and get working syntax back in seconds — you’ve collapsed that handoff. The BA doesn’t need to know SQL. The engineer doesn’t need to be pulled off another project. The work just gets done.

That’s not about replacing engineers. It’s about freeing them for the work that actually requires engineering — and letting business users drive the process where business knowledge can become or remove the bottleneck.

When Data Migrations Involve Mainframe or AS/400

If your data migration involves mainframe or AS/400, another problem appears: nobody fully knows how that legacy application works anymore. The rule that decides how an interest accrual is calculated, the condition that makes a record branch one way instead of another, the place a given value actually originates — that logic was written into COBOL, RPG, or PL/1 decades ago, often by people who have long since retired. It’s a black box.

You can’t build a reliable migration factory around a machine you can’t see inside. If you don’t know what the current system does, every mapping decision is a guess, every transformation rule is a hypothesis, and every reconciliation difference turns into a multi-month investigation. The old system says the accrual is $5.00; the new one says $5.62; and no one can explain the gap without someone manually tracing thousands of lines of code to reverse-engineer a requirement nobody documented.

This is where Contextual Data Lineage becomes part of building the factory — not a separate project you bolt on afterward. By parsing the actual legacy code — COBOL, RPG, PL/1, AS/400 — Zengines extracts the calculation logic, conditional branching, field-level relationships, and business rules buried in the system and renders them as something a business analyst can read in plain English. Raw lineage becomes actionable intelligence: the blueprint of the machine you’re replacing, available in minutes instead of months.

That visibility does three things at once. It lets teams manage the legacy systems they still depend on today, modernize them with confidence when the time is right — reverse-engineering the why, where, and how of the old code before they touch the new system — and meet the regulatory compliance requirements that come with moving sensitive financial data, generating audit-ready evidence for frameworks like BCBS-239 and ORSA.

Learn more about Contextual Data Lineage →

This Is What Zengines Was Built For

The inefficient factory floor — disconnected stations, context passing by hand, skilled people slowed down by their own tooling — can be functional. But it’s manual, slow, and risky, usually resting on one keyholder: the SME who knows which scripts run in what order, which stored procedures touch which fields, what to adjust and where. He’s not just holding knowledge. He’s the only one who can conduct it into a working sequence. Leaders budget for time and cost. They rarely budget for what happens when the conductor leaves.

These are the exact problems we designed Zengines to solve. Our platform is the conveyor belt.

Zengines covers the full data migration lifecycle — analysis, mapping, transformation, rule execution and reconciliation — in a single, end-to-end platform.

Because the platform is purpose-built for fast iteration, you’re not waiting days for a load file that requires coordination across three teams. You generate it in minutes, test it immediately, and adjust. Confidence builds progressively. Issues surface early. And AI assists at every step — predicting field mappings, auto-generating transformation rules from plain English prompts, and profiling data quality before it becomes a crisis — while the business analyst stays in control.

When legacy mainframes are in the picture, the same platform extends into Contextual Data Lineage, so the team that builds the migration is working from the actual logic of the system they’re leaving — not their best guess at it.

The results speak for themselves: migrations move 50-80% faster, business analysts are 6x more productive, and the handoff between business and engineering — the bottleneck that slows down every migration I’ve ever seen — largely disappears.

From “Done” to AI-Ready

All of this matters more now than it did five years ago, for one reason: the data you migrate isn’t going to sit quietly in a new system. It’s going to feed dashboards, models, and — increasingly — AI. And AI is only ever as good as the data underneath it.

Two things make data AI-ready:

It has to be usable, and usable starts with being accessible: available in a form something else can pull and act on without a person translating it first. Data locked in a mainframe file format nobody outside the original team can read isn’t accessible, no matter how clean it is. Neither is data still sitting in an acquired company’s legacy system, unreachable until it’s brought into the parent company’s environment. Accessible data has made that move: available in a system others can actually use, not stranded in the one it came from. That’s what a connected migration factory produces.

It has to be trustworthy: you have to be able to explain where a number came from and the logic that produced it. That’s what lineage gives you. Explainability is what makes data AI-ready in a regulated environment.

Two capabilities — data migration and Contextual Data Lineage — one outcome: data that your business, your auditors, and your AI can all rely on. Most teams treat that as three separate problems solved by three separate tools. It’s one problem, and it’s the whole point of building the belt instead of buying more gears.

Stop Adding Gears. Build the Belt.

The organizations that get through data migrations successfully aren’t the ones with the most people on the floor. They’re the ones who stopped re-tooling one gear at a time and connected every station into a single system — one that not only finishes the migration faster, but hands the business data it can explain and trust on the other side. That’s what Zengines does. And it’s why the teams using it don’t just finish faster. They finish with confidence.

Ready to see what the conveyor belt looks like for your migration?

Get a demo →

Every Tier 1 financial institution I talk to is asking a version of the same question about their AI initiatives: How do we know this data is what we say it is?

It sounds simple. It is not. Behind that question sits the capability that determines whether an AI system can be trusted, explained, or defended to a regulator – and most institutions don’t have a name for it yet. The name is data provenance, and it is quietly becoming the most important foundation for AI-ready data in regulated environments.

Provenance is routinely mistaken for documentation – a record you produce after the fact, a compliance chore, a diagram that lives in a wiki nobody reads. That framing badly undersells it. Data provenance is not a description of your data. It is the evidence behind it. In a world where AI outputs increasingly drive decisions that carry regulatory, financial, and reputational weight, that evidence layer is no longer a nice-to-have. It is the difference between AI you can stand behind and AI you have to hope nobody asks about.

What is data provenance?

Data provenance is the verifiable record of where a piece of data came from, how it was created, and every transformation it underwent on the way to its current state. It answers three questions with evidence rather than assertion:

• What is the origin of this data?
• What has been done to it?
• And can we prove both?

The word itself is borrowed from the art world, where provenance is the documented chain of ownership that establishes a work is authentic and not a forgery. The concept translates almost perfectly to data. A number in a report, a feature in a model, a field in a migrated system – each has an origin and a history. Provenance is the ability to trace that history back to its source and demonstrate that nothing along the way is unaccounted for.

That word – demonstrate – is what separates provenance from ordinary metadata. Plenty of systems can tell you what a field is called and what type it holds. Provenance tells you where the value came from, what logic shaped it, and gives you the receipts to prove it. In regulated financial services, the receipts are the entire point.

Data provenance vs. data lineage

Because the two terms are often used interchangeably, it’s worth being precise: data lineage and data provenance are related but not the same, and the distinction matters more than it appears.

Data lineage maps the path data travels – the flow from source systems through pipelines, transformations, and joins to the tables and reports where it lands. Lineage is directional and structural. It shows you the plumbing.

Data provenance is broader and deeper. It encompasses lineage but adds the evidentiary dimension: not just the path, but proof of origin, the logic applied at each step, the context that explains why a transformation happened, and the ability to reconstruct and defend the full history. Lineage tells you the water flows from A to B. Provenance proves the water is clean, tells you what was added along the way, and lets you certify it to an auditor.

For most organizations, lineage is the necessary substrate and provenance is the capability built on top of it. You cannot have credible provenance without lineage – but lineage alone, especially when it stops at the boundary of legacy systems, is not enough to make data AI-ready.

Your provenance program is only as complete as its weakest link, and for most financial institutions that link is the core. Contextual Data Lineage is how you establish provenance where it is hardest to establish: inside legacy code. It reads the calculations, conditions, and business rules embedded in COBOL, RPG, and PL/1 – so the transformations happening in the systems you can see least are not just traced, but explained.

Provenance is the evidence layer for AI-ready data

Here’s why this is landing now rather than five years ago.

AI systems are only as trustworthy as the data underneath them, and trustworthiness in a regulated context is not a feeling – it’s a burden of proof. When a model produces an output that informs a credit decision, a risk calculation, or a regulatory report, “the model said so” is not a defensible answer. You have to be able to explain the output, and explaining the output means being able to account for the data that produced it, all the way back to origin.

This is one anchor of the enterprise AI-readiness conversation: explainability is what makes data AI-ready and trusted in a regulated environment. And explainability is impossible without provenance. You cannot explain what you cannot trace. You cannot defend what you cannot prove. Provenance is the evidence layer that makes explainability real rather than aspirational.

The research analyst community has converged on the same conclusion. In discussions of the emerging “context layer” that AI agents depend on, provenance is increasingly named as one of its core components – the mechanism that lets an agent (or the humans accountable for it) know that the data it’s reasoning over is authentic and appropriate for the task. Others have framed lineage and provenance explicitly as the substrate for trust scoring, data quality, and observability across AI-ready data pipelines. Different vocabulary, same underlying point: provenance is moving from a back-office concern to a front-line requirement for AI.

This connects directly to the broader case for AI-ready data in financial services, where usability and trustworthiness are the two pillars that determine whether AI initiatives can scale at all. Provenance is what makes the trustworthiness pillar load-bearing.

In regulated environments, provenance is a defense – not documentation

Outside of regulated industries, weak provenance is a quality problem. Inside them, it’s an exposure.

Financial institutions already live under standards that are, in effect, data provenance mandates. BCBS 239 – the Basel Committee’s principles for risk data aggregation – requires that banks be able to prove where their risk numbers come from and trust their accuracy under pressure. The expectations behind model risk management guidance, audit trails, and regulatory reporting all rest on the same foundation: the ability to trace a figure to its source and defend every transformation along the way. (For more on why that standard is the right one, see our open letter to bank regulators on BCBS 239.)

Now layer AI on top of that regime; AI expands both the volume of data-driven decisions and the distance between a decision and the human who can explain it. Every model that touches regulated data inherits the institution’s burden of proof. If the data feeding that model has gaps in its provenance, those gaps don’t disappear – they compound. An unexplained transformation in a source system becomes an unexplained feature in a model becomes an indefensible output in front of a regulator.

This is the trap of data derivatives – abstractions built on abstractions, where each layer of transformation moves the working data further from its origin and quietly erodes traceability. We’ve written before about why business leaders need the data lineage they aren’t asking for: the danger isn’t that the data is wrong, it’s that no one can prove it’s right. Provenance is the antidote. It’s the difference between an AI program that accelerates under regulatory scrutiny and one that stalls the moment someone asks a hard question.

Provenance, data quality, and AI governance

Provenance is also where data quality for AI and AI data governance stop being separate initiatives and start being the same one.

You cannot meaningfully assess the quality of data you cannot trace. Completeness, accuracy, and fit-for-purpose are all judgments that depend on knowing where data came from and what happened to it. A dataset can look pristine and still be unfit for a given AI use case because of a transformation three systems upstream that nobody documented. Provenance surfaces exactly that kind of hidden defect.

The same is true for data governance for AI. Governance frameworks set policies about how data can be used, by whom, and for what – but a policy you can’t enforce or audit quickly loses value. Provenance is the enforcement mechanism. It’s what lets a governance program verify that training data was appropriately sourced, that sensitive fields carry their handling requirements through every transformation, and that a model’s inputs are consistent with the rules the institution has committed to. Strong provenance turns AI governance from a set of aspirations into a set of verifiable facts.

How to establish data provenance

Provenance is earned, not declared. Building it into an AI-ready data foundation comes down to a few disciplines:

Start at the source, including the legacy source.

The hardest provenance gaps almost always live in the oldest systems – the “legacy core”, or the mainframe and AS/400 applications written in COBOL and RPG, with decades-old business rules that no current employee fully understands. If provenance stops at the boundary of those systems, it isn’t provenance. Extracting and making that embedded logic explicit is what turns an opaque legacy estate into a traceable one.

This is the specific job Contextual Data Lineage does. A provenance strategy that treats the legacy core as out of scope isn’t a provenance strategy – it’s a provenance strategy with a hole in exactly the place regulators look first.

Capture transformation logic, not just data movement.

Knowing that data moved from A to B is lineage. Knowing what rule was applied in the move – and being able to show it – is provenance. Every transformation should carry its logic and its rationale as first-class information, not as a comment buried in a script.

Preserve context through change.

Data migrations, system consolidations, and platform modernizations are exactly the moments provenance is most likely to break – and most important to keep intact. Treating migration as an ongoing capability rather than a one-time project is what keeps the evidence chain unbroken as systems change underneath it.

Make it queryable and defensible.

Provenance that lives in static documentation is provenance you can’t use under pressure. The goal is a living, traceable record you can interrogate on demand – when a regulator asks, when a model behaves unexpectedly, when an AI output needs to be explained.

Done well, these disciplines don’t just satisfy auditors. They compound. Every system change that preserves provenance makes the next AI use case faster to stand up, because the trust work is already done.

Provenance is a strategic capability, not a technical afterthought

The institutions that will win with AI in regulated markets are not the ones with the flashiest models. They’re the ones that can move fast because they can trust and access their data with explainability. Provenance is what lets them say yes to an AI opportunity that a less-prepared competitor has to decline – not because the competitor lacks the model, but because they can’t defend the data underneath it.

That’s the reframe worth internalizing: data provenance is not a documentation task you complete once and file away. It is a strategic capability that determines how quickly, and how safely, your institution can turn data into AI-driven value. In regulated environments, it is the foundation everything else is built on.

The question every institution is asking – how do we know this data is what we say it is? – has an answer. The answer is provenance. The only question left is whether you’ll build it before your regulators, your auditors, or your own AI ambitions force the issue.

Zengines helps financial institutions build AI-ready data foundations through a Turnkey Data Migration Platform and Contextual Data Lineage – making modern and legacy data not just movable, but explainable.

Get a demo →

Frequently asked questions

What is data provenance?
Data provenance is the verifiable record of where data originated, how it was created, and every transformation it underwent to reach its current state. Unlike basic metadata, provenance provides evidence – proof of origin and history – not just description.

What is the difference between data provenance and data lineage?
Data lineage maps the path data travels from source to destination. Data provenance encompasses that path but adds the evidentiary layer: proof of origin, the logic applied at each transformation, and the ability to reconstruct and defend the full history. Lineage shows the flow; provenance proves it.

Is contextual data lineage the same as data provenance?
No. Data provenance is an outcome that spans the entire data estate. Contextual Data Lineage is a capability that delivers provenance in the one place it is hardest to reach – inside legacy code – by extracting the calculations and business rules embedded in COBOL, RPG, and PL/1 so legacy transformations can be evidenced rather than assumed. Provenance is only as complete as its weakest link, and for most financial institutions that link is the core.

Why is data provenance important for AI?
AI outputs are only as trustworthy as the data behind them. In regulated environments, “the model said so” is not defensible – you must be able to explain an output by tracing its data back to origin. Provenance is the evidence layer that makes that explainability possible, which is what makes data genuinely AI-ready.

How does data provenance support regulatory compliance?
Standards like BCBS 239 effectively require institutions to prove where their data comes from and defend every transformation. Provenance provides that proof, giving auditors and regulators a traceable, defensible record of how each figure and model input was produced.

Is data provenance the same as data quality?
No, but they’re inseparable. You can’t credibly assess data quality without knowing where data came from and what happened to it. Provenance surfaces hidden defects introduced upstream, making it the foundation on which data quality for AI is judged.

On June 18, 2026 Gartner published a prediction that should make every CIO sponsoring a mainframe exit pause:

“More than 70% of mainframe exit projects initiated in 2026 will fail to produce the intended benefits due to an overestimation of generative AI (GenAI) tooling capabilities.”

I agree with Gartner. We see it every week.

Gartner’s recommendation underneath the headline

It’s worth clarifying what Gartner means. Mainframe modernization encompasses both migrating off the platform and modernizing in place. The 70% failure figure applies specifically to full exits. For most workloads, Gartner is recommending in-place modernization instead.

The 70% figure will generate most of the talk, but the body of the release is making a sharper point.

“For many mainframe customers, GenAI can be more effectively used to enable modernization in place rather than accelerate migration off the platform.” — Alessandro Galimberti, VP Analyst at Gartner

Gartner is recommending a platform-smart approach – evaluating workloads individually and placing them in the right environments, rather than chasing a wholesale exit. Organizations should balance strategies focused on optimizing existing mainframe investments while limiting full platform exits to select, case-by-case scenarios – efforts that, in Gartner’s words, require high-risk transformation and often result in suboptimal outcomes.

That isn’t a story about better migration tooling. It’s a story about asking better questions and a fact-based assessment before the migration question is even on the table.

What GenAI can and can’t do in a mainframe exit

Gartner contends that the high failure rate is due to the expectation that generative AI will fix complex legacy code easily. Based on my experience with multiple transformation initiatives involving mainframes, I’ve observed and dealt with what generative AI can and cannot do well.

What GenAI can do well: read code at scale, surface technical debt, summarize what a module appears to be doing, generate first-pass documentation. It’s useful and time-saving work.

What GenAI can’t do, at least not reliably, in a mission-critical mainframe environment:

• Tell you why a calculation produces the result it does today.
• Provide reliably consistent and complete explanation, especially when functional threads traverse nested modules.
• Tell you what a branching condition in a COBOL module is actually checking against, and what business rule that condition encodes.
• Tell you whether a hard-coded value in a 1998 program was a temporary patch or a deliberate business decision that downstream systems now depend on.

These gaps – between code description and business meaning – are where mainframe exit projects fail. And it’s these gaps that a generative model, however capable, can’t close on its own.

The cost of not understanding what’s there – today

The Gartner finding focuses on exits – but the cost of not understanding your mainframe shows up long before any exit project starts.

Every time a business requirement changes – a new regulation requires a different calculation methodology, a product team needs to update how interest is accrued, an auditor asks where a number came from – someone has to go into the mainframe and answer the question. Before they can change a single line of code, they need to trace what the change will affect: which modules read the variable, which tables get updated, which downstream processes depend on the output, which conditional branches treat it differently.

In a typical environment, that investigation can take weeks or months – and it depends on a shrinking pool of mainframe specialists who are simultaneously running the system. The risk of getting it wrong is real: unintended consequences that show up weeks later in a reconciliation break or a misstated customer statement.

This is the recurring cost the modernization conversation usually skips over. It’s the cost of operating the mainframe without visibility into it – every quarter, every change request, every audit cycle. The 70% of exits that will fail isn’t the only story. The other story is the daily tax that organizations are paying for systems they can’t fully explain.

Explainability is the precondition

Explainability is what makes data AI-ready in a regulated environment. The same principle applies to legacy systems: a system is decision-ready – for modernization, for regulators, for the next code change – when you can explain, with traceable evidence, how it produces what it produces.

Gartner’s framing is correct: AI is being asked to do work it cannot reliably do. But the deeper lesson in the 70% failure number is that the work AI is being asked to skip is the strategic work that determines whether the right path was chosen in the first place. And that work pays for itself long before modernization day – it pays for itself every time the business asks a question the mainframe is supposed to answer.

What the teams who get this right are doing

The mainframe programs that succeed – whether the answer is a full exit, modernization in place, a hybrid, or just running the system safely for the next several years – share a pattern. They treat understanding the legacy environment as a first-order capability, not a one-time pre-migration task.

They invest in surfacing the actual business logic embedded in mainframe code – the calculation logic, the conditional branches, the field-level relationships, the cross-module dependencies – before they decide what to replicate, retire, redesign, or leave alone. That investment doesn’t just serve the eventual migration. It makes today’s mainframe safer to manage, today’s regulatory questions faster to answer, and today’s code changes less risky to make.

At one Fortune 100 financial institution where Zengines Contextual Data Lineage is used every day, the team’s question started with “For each module that touches a regulated calculation, what does it actually do, and what depends on it?” That question scopes an honest mainframe modernization program – what to exit, what to modernize in place, what to leave alone, and how to operate the mainframe safely in the meantime.

The difference isn’t only faster code conversion. The difference is that they know what they have – every day, not just on modernization day.

Mainframes aren’t going anywhere

Despite the industry’s push toward cloud migration and modernization, many financial institutions still rely on mainframe systems to process millions of daily transactions, calculate interest accruals, manage account records, and run core business operations. And they will for years to come.

Modernization is the eventual reality for most organizations still running on mainframes. For many financial institutions, a full modernization effort is on the roadmap but years away – dependent on budget cycles, vendor timelines, regulatory considerations, and a hundred other competing priorities. For others – and this is increasingly what Gartner is pointing toward – modernization will mean working with the mainframe, not off of it.

Either way, the system runs every day in between. And every day, it has to be safely managed, changed, audited, reconciled, and explained.

How Zengines bridges today and tomorrow

This is the bridge Zengines was built to be.

Zengines Contextual Data Lineage parses COBOL, RPG, and PL/1 at scale and surfaces what is actually inside legacy code: the data paths, calculation logic, conditional branches, hard-coded values, and module-to-module dependencies that determine how a legacy system produces what it produces. Analysts get the answer to a reverse-engineering question in minutes – in plain English, with business context – instead of waiting weeks for a mainframe SME to dig through code by hand.

That visibility pays off on two timelines. Today, it makes the mainframe safer to manage: business requirement changes get scoped accurately, regulators get answers in hours instead of months, and engineers can make code changes with confidence about the blast radius. Tomorrow, whenever modernization day arrives – whether that means a full exit, modernization in place, or a workload-by-workload approach – the team isn’t starting from scratch. The understanding is already there.

The mainframe isn’t the problem. The lack of visibility into it is.

If you are managing a mainframe today, planning to modernize tomorrow, or – as Gartner is increasingly suggesting – deciding whether modernization should mean staying on the platform and changing how you work with it, we’d like to show you what Contextual Data Lineage surfaces in your environment.

Get a demo →