CI/CD pipeline architecture: dependencies, artifacts and trust boundaries
Design CI/CD as a dependency graph with verifiable artifacts and bounded credentials. Trace a release from source checks through promotion and production verification.
TL;DR: A delivery pipeline turns a reviewed source revision into a verifiable artifact, then promotes that artifact through controlled environments. Its architecture must define dependency ordering, evidence, credentials and recovery. A successful job only proves the checks that job actually performed.
Follow one release through the system
Take a payment service with a container image, a database migration and an environment-specific configuration file. The source revision is one input. Dependency versions, the base image and the build toolchain are additional inputs. The release record needs enough information to connect these inputs to the exact artifact that reaches production.
Separate the checks that can run concurrently from those that require an earlier result. Unit tests and source linting may run in parallel. Integration tests against the built image depend on image creation. A deployment approval should refer to the selected artifact and its evidence, rather than an unqualified “latest successful run.”
A dependency graph, with two trust boundaries
An AI code reviewer can propose findings, while required checks and accountable review still establish whether a change should proceed.
The build boundary separates untrusted source execution from long-lived production permissions. The deployment boundary authorizes a specific release against a specific environment. A pull request from an untrusted contributor must not obtain the same credentials as the protected deployment job merely because both jobs use the same runner pool.
SLSA's build requirements describe provenance and builder guarantees. Choose a concrete assurance target rather than claiming that generating a JSON file makes a pipeline secure. Provenance is useful only when the deployment policy verifies its relationship to the artifact and the trusted builder.
Make every handoff inspectable
| Handoff | Record to retain | Failure that becomes diagnosable |
|---|---|---|
| Source to build | Resolved commit and dependency inputs | Same branch name produced different code |
| Build to test | Artifact digest and test environment | Tests ran on a different image |
| Test to promotion | Required check results for that digest | An old green run authorized a new artifact |
| Promotion to deployment | Target environment and config revision | Correct image used with incompatible settings |
| Deployment to operation | Rollout revision and outcome signals | Job passed while customers received errors |
Caches accelerate a build but should not become an undocumented source of correctness. Key them around the inputs they contain and ensure a cache miss still produces a valid build. Avoid shared writable caches across differently trusted jobs where one job could poison another's dependencies.
Work through a race between two releases
Release A passes tests and waits for approval. Release B passes later and deploys first. An operator then approves A. Without an environment-level concurrency policy, A can overwrite B and silently move production backwards.
Choose and document the intended behavior. You might cancel superseded deployments, serialize them by environment, or require a fresh approval if the current production revision changed. The lock should cover the deployment operation and its verification, with explicit recovery if the runner dies. A lock on a build job alone does not protect production ordering.
Treat migrations as part of this reasoning. A pipeline cannot safely apply a destructive schema change just because container tests passed. Use an expand-and-contract sequence, verify compatibility across the mixed-version period and identify the point after which an old image is no longer a valid rollback target.
Credentials belong to narrowly scoped jobs
A build job usually needs source access and permission to publish its artifact. A deployment job needs access to the target environment. Prefer short-lived workload credentials where the platform supports them, with policy bound to the repository, protected ref and intended audience. Never assume that masking a variable in logs prevents a malicious script from using it.
GitHub's workflow security reference discusses untrusted input, action pinning and token permissions. Equivalent boundaries must be designed in Jenkins, GitLab and Azure Pipelines even though their configuration syntax differs.
Record an owner and retention policy for failed-run evidence. Logs should explain which operation failed without containing tokens or customer records. The secrets lifecycle includes logs and build artifacts because credentials often escape through those secondary paths.
Decide what “green” means
A deployment job can finish when an API accepts the update, when workloads become ready, or after a period of customer-level verification. These are different guarantees. State which one the stage provides. For a risky service change, reserve a verification stage that examines errors and latency for the new revision, with a minimum observation window and enough requests to make the result meaningful.
In Jenkins, controller and agent architecture separates queue coordination from executor capacity and build trust.
Self-check: an integration test passes, but production pulls a mutable tag that now points at another image. Which stage should change?
Bind tests, approvals and deployments to the same digest, and verify the deployed identity. Adding more integration tests against the first image leaves the handoff defect intact. Practice pipeline stage design, then follow the same architecture in Azure Pipelines.
Test selection and flaky checks develops the validation stage: which dependencies require evidence, what a retry means and when a missing suite must block promotion.