Audit-Ready CI/CD Evidence Trails Financial Regulators Accept
Answering the Examiner Without a Three-Week Evidence Hunt
A regulator or internal auditor picks a production change from four months ago and asks a small number of very reasonable questions. Who approved this. Who reviewed the code. What testing passed before it went out. Is the thing running in production the thing that was approved. And how do you know nothing else was deployed that day.
In most institutions the answers exist, scattered across a source control system, a ticketing system, a scanning tool, a deployment platform, and somebody's inbox, with retention policies that differ across all five. Building an audit ready CI CD financial evidence trail means emitting those answers as a by-product of shipping, rather than assembling them under time pressure months later.
What does an examiner actually ask for?
A short list of questions, each of which maps to a specific artefact.
| Question | Evidence artefact | Where it should come from |
|---|---|---|
| What business need drove this change? | Change or requirement reference | Work tracking, linked from the commit |
| Who reviewed the code, and were they the author? | Review record with identities | Source control, enforced not conventional |
| Who authorised release to production? | Approval with identity and timestamp | Deployment gate, not an email |
| What testing passed? | Test run results tied to the artefact | Pipeline execution record |
| Was the code scanned for vulnerabilities? | Scan results and disposition of findings | Pipeline execution record |
| Is production running what was approved? | Artefact hash and build provenance | Build platform attestation |
| When and by what identity was it deployed? | Deployment record | Deployment platform log |
| Could it be reversed? | Rollback evidence or forward-fix plan | Change record plus tested procedure |
| Did anything else go out? | Complete change population reconciliation | Pipeline plus production access logs |
The last row is the one that is almost never answerable, and it is often the one that matters most. Everything above it concerns a change you already know about.
Why does evidence collection usually fail?
Because it is treated as a reporting activity rather than an output of the delivery process.
The common pattern is a compliance analyst who receives an audit request, emails four teams, receives screenshots, assembles a pack, and submits it. That process produces something, and what it produces is weaker than it looks: screenshots have no integrity guarantee, the assembly introduces selection, retention gaps get papered over, and the whole exercise takes weeks of skilled time per examination. It also scales badly, since the effort is linear in the number of changes sampled while the underlying control is either operating or not.
Why is reconstructed evidence weaker than evidence emitted at the time?
Because a reviewer cannot distinguish an accurate reconstruction from a convenient one.
If the approval record is a screenshot of a chat message, the reviewer is being asked to trust that the message was not edited, that it referred to this change, and that no contrary message exists. If the approval is an immutable pipeline record created at the moment the gate was passed, carrying the approver's authenticated identity and the artefact hash, none of that trust is required. The distinction is not pedantry. It is the difference between evidence that establishes the control operated and evidence that merely does not contradict the claim.
Could you produce complete evidence for a randomly chosen production change from six months ago?
What is the minimum evidence set per change?
Eight artefacts, all linked by a single identifier, all emitted automatically.
The set is the middle column of the table above, and the essential design property is linkage. A change reference that appears on the commit, the artefact metadata, the test run, and the deployment record turns nine separate records into one traceable chain. Without that identifier the evidence exists but cannot be assembled, which is functionally the same as not existing. Enforce the linkage at commit time by rejecting commits without a valid reference, and carry it through the build into the artefact so it survives into production.
Note that the evidence set describes the control operating rather than replacing the control itself. Whether one person can author and approve their own change is a separate design question, addressed in automating change approval and segregation of duties. Evidence proves what happened; enforcement determines what can happen.
How does build provenance work?
The build platform attests to what it built, from what, and how, and signs the attestation.
SLSA, currently at version 1.2 with Approved status, sets out a build track with three levels. Build L1 requires that provenance exists: the platform automatically generates provenance describing the artifact, the entity that built it, and the top-level inputs to the build. Build L2 adds that the hosted build platform generates and signs the provenance itself, which prevents tampering after the build. Build L3 adds hardened builds, requiring that runs cannot influence one another and that the secret material used to sign provenance is inaccessible to user-defined build steps, which protects against tampering during the build.
| Level | Adds | Audit question it answers |
|---|---|---|
| Build L1 | Provenance exists, generated by the platform | What was this artefact built from, and by whom |
| Build L2 | Platform generates and signs the provenance | Has the artefact or its provenance been altered since the build |
| Build L3 | Isolation between runs, signing secrets out of reach of build steps | Could a build step have forged its own provenance |
For a bank the practical value is that provenance turns "is production running the approved code" from an assertion into a verification. It also gives a defensible answer on dependency origin, which is the exposure discussed in software supply chain attacks.
Why does a signed artefact hash answer several questions at once?
Because it collapses source, build, and deployment identity into one verifiable fact.
If the deployment record names an artefact hash, and the signed provenance for that hash names the reviewed commit and the build platform, then a single verification establishes that the deployed binary came from reviewed source through a controlled build. Three separate evidence questions become one cryptographic check that an auditor can repeat independently rather than take on trust. That independence is what distinguishes strong evidence from thorough documentation.
How do you make evidence immutable and retrievable?
Append-only storage, indexed by change reference, retained for the full period, with synchronised time.
Evidence should be written once to a store that the delivery teams cannot alter, retained for whatever period the institution's record-keeping obligations require, and indexed so a query on a change reference returns the whole chain. Time synchronisation matters more than it sounds, since an approval timestamped after the deployment it authorised is a finding regardless of what actually happened. Event-sourced designs are a natural fit for this shape of problem, as covered in event sourcing and CQRS for audit trails, and the broader lineage discipline is the subject of data lineage and audit trail design.
Why is retrieval time the real test?
Because evidence that takes three weeks to assemble has already failed the examination.
A reviewer who asks for a change's evidence and receives it in minutes draws a conclusion about the control environment before reading the contents. A reviewer who waits three weeks draws a different conclusion, correctly, because the delay is itself information about how integrated the controls are. Set an internal target of minutes for a single change and hours for a sampled population, and treat any failure to meet it as a defect in the evidence design rather than a resourcing issue.
How do you evidence what did not happen?
By proving the change population is complete, which requires closing every path around the pipeline.
This is the hardest requirement and the one most estates cannot meet. It means every production change reconciles to an approved pipeline execution, and that any other route into production is either closed or monitored to the point where its use is detected and investigated. The paths to close are well known: direct server access, manual configuration change, database statements run by a person, vendor-applied patches, and infrastructure changes made through a console rather than code. Each needs either removal or detection, and the detection needs to be reconciled periodically so that an unexplained change is noticed within days rather than at the next audit.
| Bypass path | Control |
|---|---|
| Interactive server access | Removed, or just-in-time with session logging and reconciliation |
| Console-driven infrastructure change | Drift detection against the code definition, alert on divergence |
| Direct database manipulation | Break-glass only, logged, reviewed retrospectively |
| Vendor-applied change | Contractual notification plus independent detection |
| Configuration change outside version control | Configuration drift monitoring |
| Feature flags | Flag changes treated as changes, with their own audit record |
Feature flags deserve the explicit mention. A flag flip that changes customer-facing behaviour is a production change, and an evidence model that captures deployments while ignoring flags has a gap exactly where the behaviour changes fastest.
How do you map evidence to control frameworks?
Once, deliberately, with the mapping maintained as a first-class artefact.
Build a matrix that maps each evidence artefact to the internal controls and external expectations it supports, so a single pipeline output serves multiple examinations rather than being reproduced in different formats for each. NIST's Secure Software Development Framework, version 1.1 published February 2022, gives a recognised vocabulary of high-level secure development practices aimed at reducing vulnerabilities and addressing root causes, which is useful as an organising spine. The EBA's guidelines on ICT and security risk management, applicable from 20 May 2025 and amended to align with DORA's application from 17 January 2025, establish ICT risk requirements for credit institutions, investment firms, and payment service providers. Mapping to a recognised framework also shortens conversations with supervisors during approval processes such as those described in cloud migration regulatory approval.
How do you handle vendor-delivered change?
By requiring the same evidence contractually and verifying independently.
Where a vendor deploys into the institution's environment, or operates a service the institution depends on, the evidence chain crosses an organisational boundary and usually thins out. Specify what is required in the contract, including notification, artefact identification, and change records; then verify independently through drift detection rather than relying on the vendor's assurance. Concentration makes this sharper, since a single provider's change process becomes a shared dependency, which is the exposure examined in third-party concentration risk. Resilience obligations pull in the same direction, as set out in operational resilience and impact tolerances.
How do you keep it honest?
By auditing yourself on the same terms an examiner would, on a schedule.
Run periodic self-tests: pull a random production change, attempt to produce the complete evidence chain within the target time, and record what was missing. Sample the change population reconciliation and investigate anything unexplained. Test the negative assertion specifically, because it decays silently as new deployment paths appear. Publish the results internally, including the failures, since a self-testing programme with no findings is not testing anything. Testing evidence quality is a natural partner to testing the software itself, which is the subject of core banking regression automation.
Do your feature flag changes generate the same audit record as your deployments?
How should this be sequenced?
Linkage first, then automatic emission, then provenance, then completeness, then self-testing.
| Phase | Duration | Deliverable |
|---|---|---|
| Change reference linkage | 1 month | Enforced reference from commit through artefact to deployment |
| Automatic evidence emission | 2 months | Review, approval, test, and scan records written by the pipeline |
| Immutable evidence store | 1 to 2 months | Append-only, indexed by reference, retention aligned to obligations |
| Build provenance | 2 to 3 months | Signed provenance for every artefact, verified at deploy |
| Bypass path closure | 3 to 4 months | Interactive access, console change, and direct database routes controlled |
| Change population reconciliation | 2 months | Every production change tied to an approved execution |
| Feature flag audit records | 1 month | Flag changes captured with the same rigour as deployments |
| Control framework mapping | 1 month | Matrix from artefact to control, maintained |
| Self-testing programme | Ongoing | Scheduled sampling with published results |
Which metrics matter?
Evidence retrieval time, evidence completeness rate, unexplained production changes, provenance coverage, and self-test findings.
Retrieval time for a single change and for a sampled population is the operational measure. Evidence completeness, meaning the proportion of sampled changes with a full chain, is the quality measure. Unexplained production changes should be zero and any occurrence investigated, since a single one undermines the completeness assertion for the whole period. Provenance coverage tracks how much of the estate is verifiable rather than asserted. And self-test findings show whether the programme is examining itself honestly.
The goal is not a better evidence pack. It is an environment where the evidence is a by-product of how software ships, so that an examination becomes a query rather than a project.
Frequently Asked Questions
What does an examiner actually ask for about a deployment?
Who authorised this change, who reviewed the code, what testing passed, what artefact was deployed, when, by what identity, and how you know nothing else went out.
Why is evidence assembled afterwards weaker than evidence emitted at the time?
Because reconstructed evidence depends on memory, screenshots, and inference, and a reviewer cannot distinguish an accurate reconstruction from a convenient one.
What is the minimum evidence set for a single change?
Change reference, code review record, approval identity and timestamp, test results, security scan results, artefact hash with provenance, deployment record, and rollback capability.
What does build provenance attest?
What entity built the package, what build process was used, and what the top-level inputs to the build were, which lets a consumer verify the artefact was built as expected.
Why does a signed artefact hash answer several audit questions at once?
Because it links the deployed binary to a specific build, from specific reviewed source, on a specific platform, which collapses three separate evidence questions into one verifiable fact.
Why is retrieval time the real test of an evidence trail?
Because evidence that exists but takes three weeks to assemble fails the examination in practice, and the delay itself suggests the control is not operating as described.
How do you evidence that something did not happen?
By proving the completeness of the change population: every production change reconciles to an approved record, and any deployment path that bypasses the pipeline is closed and monitored.
Who should test whether the evidence works?
The institution itself, through periodic sampling that pulls a random change and attempts to produce full evidence within a defined time, before an examiner does the same thing.
