Migrating legacy core systems without stopping the business

Core migrations fail at the cut-over. Not in the design, not in the build, not in the testing: at the weekend when the old system is switched off, the new one is switched on, and the business opens on Monday to find out what nobody knew. The pattern is common enough to have a name, the big bang, and common enough that most executives who commission a core replacement have lived through one.

The position here is that the big bang is a choice, and a poor one. The alternative is well understood in engineering and rarely demanded by the people who sign the contract. A compatibility layer stands in front of the old system. Consumers move to it one at a time. The new core takes over behind it, domain by domain. Data moves as a stream, reconciled daily, not as a single night's load. Every step has a rollback that has been run, timed and owned. The programme is slower to show a dramatic result and far more likely to finish.

Why the big bang fails

Three things go wrong, and they go wrong together.

The freeze. To cut over on a date, the business must stop changing the old system for weeks or months beforehand so that the new one can catch up to a fixed target. Product launches wait. Regulatory changes are patched in by hand. The backlog grows, the date slips, the freeze extends, and the organisation learns to work around its own core. By the time the cut-over happens, the new system replaces a version of the old one that no longer exists.

The unrehearsed rollback. The rollback plan is a document. It describes how, if the new system fails on Monday, the old one will be restored. It has never been run, because running it would mean taking down the old system in production. When Monday comes and the new system is failing in a way nobody predicted, the team discovers that the rollback takes longer than the business can wait, that the data written since the cut-over cannot be moved back, and that the decision to roll back has no owner. So they push forward, into a system that is not working, because going back is worse.

The unknown consumers. Every core system that has run for a decade has consumers nobody listed. A spreadsheet in finance that reads a table directly. A batch job a departed contractor wrote. A partner integration that authenticates with a shared credential. A report the regulator receives monthly. The dependency map was built from documentation and interviews, and the documentation was wrong and the interviewees did not know. The big bang discovers these consumers at the moment they break.

The rollback plan has never been run, because running it would mean taking down the old system. When Monday comes, the team discovers what that means.

The alternative pattern

The pattern has a handful of components, and each removes one of the failure modes above.

A compatibility layer, sometimes called a façade, is placed in front of the legacy core. It speaks the legacy system's existing contracts, the same interfaces, the same file formats, the same message shapes, so that no consumer has to change in order to talk to it. On the first day it does nothing but pass calls through. It is the seam along which everything else is done.

Strangler-style replacement. Behind the layer, the new core is built one domain at a time. When a domain is ready, the layer routes that domain's traffic to the new implementation and continues to route everything else to the old one. The legacy system is not switched off; it is starved, domain by domain, until nothing depends on it. Then it is retired, on a day that matters to nobody.

One consumer at a time. Consumers are moved onto the compatibility layer individually, in an order chosen for risk: the least critical first, the ones with the best-known behaviour, the ones whose owners are in the building. Each move is small enough to be understood and reversed. The unknown consumers surface as the known ones move, because traffic that still reaches the legacy system directly is, by definition, from a consumer not yet on the map. The map completes itself in production, safely.

Dual-running and reconciliation. For each domain, the new core runs alongside the old for an agreed period. Both receive the same inputs. Their outputs are compared, record by record, every day. The reconciliation report is the evidence that the new system does what the old one did, including the behaviour nobody documented. Where the two differ, the difference is investigated, and it is often the old system that turns out to have been wrong.

Feature flags. The routing decisions in the compatibility layer are controlled by configuration, not by deployment. A flag moves a consumer or a domain to the new core; the same flag moves it back. A release and a cut-over are separated: code ships continuously, and behaviour changes when someone decides it should.

Data migration as a stream. The data does not move in one night. A change-data-capture feed or an equivalent mechanism copies records from old to new continuously, so that the new core is always close to current. The final synchronisation for any domain is minutes of delta, not hours of load. If the domain is rolled back, the stream reverses, or the old system was never stopped writing in the first place.

Rehearsed rollback with defined triggers

A rollback is a procedure, not a plan. The difference is that a procedure has been run.

Every cut-over step, whether moving a consumer, routing a domain or retiring a component, has a rollback that has been executed in an environment shaped like production, timed, and documented with the actual duration. The person who will run it in production has run it in rehearsal. The step is not taken until the rehearsal has passed.

The triggers are defined before the step. A rollback trigger is a measurable condition agreed in advance: an error rate above a threshold for a stated duration, a reconciliation difference above an agreed count, a named business process failing, a regulatory report that cannot be produced. When a trigger fires, the rollback is run. There is no meeting. The meeting happened when the trigger was agreed, and it included the people who would otherwise have to be woken to decide.

The decision has a named owner. One person holds the authority to roll back each step, is present when the step is taken, and is not the person who built the thing being cut over.

Observability and reconciliation as acceptance

The acceptance criterion for a migrated consumer or domain is not a sign-off meeting. It is a reconciliation report showing zero difference between old and new for the agreed period, and an observability view showing that the new system is behaving within its expected bounds under real load.

Reconciliation compares outputs at the level that matters to the business: balances, appointments, policies, invoices, whatever the core is the system of record for. It runs daily, is reviewed by someone who understands the domain, and every difference is either explained and accepted or fixed. The report is kept, because the auditor will ask for it.

Observability is designed with the operators who will read it. Structured logs, metrics and traces answer the questions they actually ask when something is wrong: what changed, who is affected, since when, and whether it is getting worse. Alerts fire on conditions that require action and stay quiet otherwise. An alert nobody acts on is noise, and noise hides the alert that matters.

Together, these replace the confidence a big bang asks the board to take on trust. A director can be shown the reconciliation and the dashboards and decide whether the next step is safe, on evidence rather than assurances.

The organisation around the migration

The engineering pattern fails if the organisation around it still behaves as though a big bang were coming.

A change board. A standing group with representatives from technology, operations, the affected business units and compliance meets on a fixed cadence and approves each cut-over step against the evidence: the rehearsal log, the reconciliation report, the trigger definitions, the named rollback owner. It is small and it decides. It is not a status meeting.

Freeze windows negotiated, not imposed. Some steps still need a quiet period: a short one, for a specific domain, at a time the business chooses. The business unit names the window and the migration fits into it, rather than the migration declaring a freeze and the business working around it. A negotiated window is honoured. An imposed one is circumvented.

Training as a step, not an afterthought. The people who operate the core, at the counter, on the ward, at the trading desk, are moved with the consumer they use. They are trained on the new behaviour before their consumer moves, with the old system still available, and the runbook they use has been tested by someone who was not on the project.

Regulated contexts

In a hospital network or a bank, the migration is a regulated change, and the pattern above is also how the obligations are met.

Under HIPAA, a hospital network in the United States handles protected health information, and the security rule requires administrative, physical and technical safeguards for it, including access controls, audit controls and integrity controls. A migration touches all three. Dual-running means protected health information exists in two systems for a period, and both must meet the safeguards. The reconciliation report is an integrity control. Access logs on the compatibility layer are an audit control. The business associate agreements with the vendors involved must cover the migration period explicitly.

In Switzerland, FINMA's expectations for supervised institutions include requirements on outsourcing, on operational resilience and on the management of information and communications technology risk. Stated generically, a supervised bank is expected to retain control over material outsourced functions, to identify its critical functions and the tolerances for their disruption, and to be able to demonstrate to the supervisor that a significant change to a core system was governed, tested and reversible. A migration that moves one consumer at a time with rehearsed rollback and daily reconciliation produces that evidence as a by-product. A big bang produces a promise.

In every regulated context, audit trails are not optional. Every routing change in the compatibility layer, every flag flipped, every reconciliation result and every rollback is logged with who, when and why, and the log is retained for the period the regulator requires. The pipeline that deploys the code produces its own trail: what was changed, who reviewed it, which tests passed. Compliance becomes a property of the system rather than a spreadsheet beside it.

How to contract for it

The contract shapes the behaviour. A contract with a single go-live date and a payment attached to it produces a big bang, whatever the method statement says, because the vendor is paid for the date.

Contract by consumer migrated. Define the consumers and domains in the dependency map, agree an order, and attach a milestone to each: the consumer is on the compatibility layer, reconciliation has shown zero difference for the agreed period, the rollback has been rehearsed, and the operators have run the runbook. Payment follows the milestone. The date is a forecast, not a term.

Write the definition of done into the contract: tests, observability, documentation, and the rehearsal log for each step. Write exit into the contract from the beginning: source in your repositories from the first week, and a gate that requires your team to have released, rolled back and resolved an incident without the vendor before the final payment.

The signs it is becoming a big bang anyway

A well-drafted contract does not stop the drift on its own. Programmes that start incremental move towards the big bang under schedule pressure, and the signs are visible months ahead.

The compatibility layer is being skipped for "simple" consumers, and the count of consumers moved directly to the new core is growing. The dual-running period is being shortened to fit a date. Reconciliation differences are being accepted without explanation because investigating them takes time. Rollback rehearsals have stopped, and the last logged one is for a step three steps ago. A freeze has been announced rather than negotiated. The change board has become a status meeting that approves everything. A single date has appeared in a board paper with the word go-live beside it. Someone has proposed moving the remaining consumers together, to save time.

Any one of these is a conversation. Three together is a big bang with a different name, and the board should ask for the rehearsal logs before the next step.

What to do on Monday

1. Ask for the dependency map, and ask how it was built. If it came from documentation and interviews alone, commission the traffic analysis that finds the consumers nobody listed.
2. Ask for the rollback rehearsal log for the most recent cut-over step. If there is none, stop the next step until there is.
3. Ask for the reconciliation report for the most recently migrated domain, and the name of the person who reviews it daily.
4. Read the contract. If payment is attached to a date rather than to consumers migrated, renegotiate before the date becomes the plan.
5. Convene the change board with the authority to approve or stop each step against evidence, and remove from it anyone who cannot attend.
6. Write the rollback triggers for the next step, with the numbers, and name the person who holds the decision.