GitOps push and pull architectures: credentials and reconciliation ownership
Compare push deployment and pull reconciliation by credential location, network reachability and drift response. Trace what happens when CI succeeds but the cluster cannot reconcile.
TL;DR: Push deployment gives a delivery actor authority to change the target; pull reconciliation gives a controller authority to fetch desired state and converge the target. Both need credentials and policy. The important difference is where that authority lives and who keeps checking actual state.
Separate artifact production from target mutation
A CI job can build and publish an image without deploying it. Another operation updates a desired-state reference. A controller can then reconcile that reference into a cluster. Calling every step “the pipeline” hides distinct authorization boundaries.
In a push design, a runner or deployment service invokes the target API. In a pull design, a controller observes approved desired state and applies it with its own target permissions. A webhook can accelerate observation in either design; its presence does not determine whether deployment authority is push or pull.
The OpenGitOps principles include declarative, versioned desired state, automatic pulling and continuous reconciliation. A one-time scheduled git pull followed by a script does not automatically provide all those properties.
The diagram compares alternatives; it does not recommend allowing both to mutate the same objects without an ownership agreement.
Credential movement changes the failure model
A pull controller still needs repository access and cluster authority, and private artifacts may require separate runtime credentials. It may also manage remote clusters. Avoid claiming pull architecture eliminates production credentials or guarantees that all authority is inside each workload cluster.
Limit repository scope, destination clusters and resource permissions. A repository write permission can become a path to production change if the controller trusts everything in that repository. Reviewing who may modify desired state is therefore as important as protecting the controller's own credentials.
Argo CD's security documentation explains relevant trust boundaries. The Argo CD architecture page covers its components; this comparison focuses on deployment ownership.
| Design question | Push deployment | Pull reconciliation |
|---|---|---|
| Who contacts the target API? | Runner or deployment service | Reconciliation controller |
| Where is target authority held? | Delivery actor's credentials | Controller or managed-cluster credentials |
| What detects later drift? | Separate drift mechanism unless built in | Continuing reconciliation loop |
| What if source hosting is unavailable? | New delivery can fail | Fetch/convergence to new revisions can fail |
| What proves success? | Target and application verification | Reconciled revision plus health and application verification |
Worked incident: green CI, stale production
Suppose CI builds an image, publishes it and merges a desired-state version change. The job turns green. Production still runs the previous version because the controller's repository credential expired.
Compare the source revision approved for production with the controller's observed and applied revision. Inspect fetch errors and credential expiry. Restarting application Pods does not address the missing desired-state fetch and may simply restart the old image.
Restore scoped repository access and observe reconciliation. Confirm both the deployed artifact and the application behavior; a synchronized manifest can still describe a broken application. Alert on sustained convergence failure with enough context to identify the intended revision.
This incident illustrates why CI/CD pipeline architecture needs a delivery outcome beyond successful artifact publication. CI success means the completed stages passed, not that every downstream system reached the intended state.
Emergency writes require a return path
An operator may need to mitigate an incident directly in the cluster. If the controller remains active, it can reverse the manual change according to policy. Decide whether to pause a specific reconciliation scope, update desired state or use an approved emergency mechanism. Record the temporary divergence and the owner responsible for ending it.
Two independent reconcilers writing the same field can also fight indefinitely. Name field and object ownership before combining deployment tools. A manual fix that is immediately undone may be correct evidence of an active controller rather than an unreliable Kubernetes API.
Does pull mean the CI runner needs no privileges? No. It may still publish trusted artifacts or update production desired state. Those permissions can have production consequences even without direct cluster access.
Does a webhook make GitOps push-based? No. A notification can trigger a controller to fetch state while the controller retains deployment authority. Trace the actor making the target API call and the ongoing reconciliation loop.