Connecting systems that disagree
Getting two systems to share data reliably, starting with the identifier they will have to agree on and ending with a scheduled reconciliation.
- First deliverable
- An identifier map across the systems in scope
- Typical connection
- Two to four weeks once the mapping is agreed
- Failure handling
- Retries, a dead letter queue and an alert to a person
- Documentation
- An interface specification you can hand to another supplier
Integration work is mostly reconciliation. Two systems hold the same site, customer or asset under different names and keys, and nothing flows correctly until that is resolved. We write the identifier map first, then the interface specification, then the connection, with failure handling and a scheduled comparison of both sides.
- 01 Enquiry
- 02 Systems audit
- 03 Specification
- 04 Build
- 05 Pilot
- 06 Deploy
- 07 Handover
- 08 Support
Connecting systems that disagree sits at step 3 of the build cycle. You receive a written specification and scope.
Integration fails on identifiers, not on APIs
The connection is the easy half. Deciding what counts as the same thing is the hard one.
Your finance system knows a customer by an account code. Operations knows them by a site name somebody typed. The client portal knows them by a contract number. All three are correct in their own system, and none of them match. Push data between them and you get duplicates, orphans and a monthly reconciliation nobody scheduled.
In Saudi data there is often a better key hiding in plain sight. A supplier's CR number or VAT number is frequently the only identifier two systems already agree on. An employee's iqama or national ID number is another, and it is personal data, so it is used only where you have decided it may be.
So the first piece of work is an identifier map: for each shared entity, which system is the master, what its key is, and how the others map onto it. That document is usually short and is the reason the integration holds. Without it you are building a synchronisation that will drift and then be quietly turned off.
The second thing we insist on is deciding what happens when a message fails. Not if. Systems go down, credentials expire, somebody renames a field. A connection that has no answer for failure looks perfect until the first outage, after which nobody trusts the data again.
The shape of an integration job
| Element | What we do | What you end up holding |
|---|---|---|
| Entity mapping | Decide the master for each shared entity and map every other system's key onto it. | The identifier map |
| Interface design | Direction of flow, trigger, payload, frequency and the rule for conflicts, written before code. | An interface specification |
| Connection | Built against the other system's documented interface, or its export if it has no interface worth using. | Working integration |
| Failure handling | Retries with backoff, a queue for what could not be delivered, and an alert that reaches a person. | A monitoring view |
| Reconciliation | A scheduled comparison of both sides that reports differences rather than assuming success. | A reconciliation report |
| Credential handling | Keys held in your own secret store with an owner and an expiry date on each one. | A credential register |
Documents that survive the integration
If we disappear, these are what let somebody else pick it up. That is the test they are written to.
The identifier map
Every shared entity, its master system, its key, and how each other system maps onto it.
The interface specification
One page per connection: direction, trigger, payload, frequency, conflict rule and failure behaviour.
The monitoring view
Whether each connection ran, what it moved and what it could not deliver, with the queue visible rather than buried in a log.
The reconciliation report
A scheduled comparison of both sides, so drift is discovered by the report and not by a customer.
The credential register
Which key, which system, who owns it, when it expires. Expiry raises its own renewal task.
The runbook for a failed connection
What the alert means, where the undelivered items sit, and the steps your named person takes to release them.
How a connection gets built
- 01
Profile both sides
Pull a real extract from each system and look at the actual data, not the documentation. This is where the surprises live.
- 02
Agree the master
For each shared entity, one system wins. This is a business decision, so it goes to your named data owner, who signs it off.
- 03
Write the interface specification
Before any code. It is short, and it is what a future supplier will read instead of reverse engineering us.
- 04
Build and replay
Built against real historical data replayed through it, so the edge cases appear before go-live rather than after.
- 05
Run in shadow
The connection runs and reports what it would have done, without writing, until the differences are explainable.
- 06
Switch on with monitoring
Live, with alerting to a named person and the reconciliation report scheduled from day one.
One machine, three names
An equipment rental company in Jubail keeps its fleet in three places. Accounting knows each machine by an asset code, the fleet register spreadsheet uses the fleet number painted on the machine, and the maintenance app uses the manufacturer's serial number.
Invoices for rental days and maintenance costs cannot be tied to the same machine, so nobody can say which units lose money. Extracts from all three systems are profiled in the first week.
The identifier map makes the fleet register the master for the machine and records the other two keys against it. The spreadsheet becomes a small register with an owner. A nightly connection carries maintenance costs to accounting against the right asset code, and machines that exist in one system and not another land on a weekly exception list for the fleet manager.
The company holds the identifier map, the interface specification, the reconciliation report and a credential register with renewal dates.
Integration questions we get
Our other system has no API. Is that the end of it?
Usually not. A scheduled export, a database view, or a file drop is often enough for the data to flow in the direction that matters, and one-directional integration solves more problems than people expect. Each of those routes survives the vendor redesigning its screens, which a script clicking through a web page does not.
Who owns the integration after handover?
You do, and the interface specification exists so that ownership is real rather than nominal. The credentials sit in your store, the monitoring view is yours, and another supplier can take it over by reading the specification. If you want us to keep running it, that is a support agreement.
How do you handle data that disagrees?
The conflict rule is decided at specification and written into it. Usually the master system wins and the other side is corrected. Sometimes neither wins and the difference is raised for a human to resolve, which is the right answer for anything involving money or a contractual commitment. What we avoid is last-write-wins, which quietly destroys data and is the default in most quick integrations.
What drives the cost?
The number of systems, the quality of their interfaces, how badly their identifiers disagree, and whether the flow has to be real time or can be scheduled. Two systems with clean keys and a nightly schedule is quick. Four systems that each invented their own site naming is where the weeks go, and that work is reconciliation rather than programming.
Can you integrate with an accounting or ERP system we cannot change?
Often, and the constraint is what that system will let you read and write rather than our end. The honest scoping step is a call with whoever supports it, because their answer about what is possible is worth more than any documentation. We do that before quoting, not after.
Related work
Integrations reference
The methods we connect by, and the Saudi data problems that break a connection.
Read moreDigitalLegacy migration
When one side of the connection is an old system that has to be retired.
Read moreDigitalDashboards and reporting
The usual reason data has to move in the first place.
Read moreDigitalApp development
Where the new system on one side of the connection is one we are building.
Read morePart of one system
Every division keeps records to the same standard, so the second one you need works the way the first one did.
Send us both extracts
A real extract from each system tells us more in an hour than a week of meetings. The identifier map comes out of that, and you keep it.