Object storage consistency and versioning: visibility, deletion and recovery
Understand S3 object consistency, delete markers and asynchronous replication. Design publication and recovery around the exact guarantees each feature provides.
TL;DR: Read consistency determines which successful write a later request can observe. Versioning retains object history, while replication copies eligible changes elsewhere; neither turns several object writes into a transaction or proves a usable recovery point.
Name the operation and the boundary
For Amazon S3 general purpose buckets, successful object writes and deletes have strong read-after-write consistency, including subsequent listing. An overwrite of one key is atomic: a reader gets an old or new object rather than half of each. This does not create an atomic transaction across unrelated keys. The S3 consistency documentation distinguishes object operations from other configuration changes.
Consider an application publishing a dataset with customers.csv, orders.csv and manifest.json. If it overwrites the two data keys separately, a reader can fetch customers from the new release and orders from the previous release during publication. Every individual read can satisfy the storage guarantee while the combined dataset violates the application's requirement.
Use immutable keys for a release and publish a manifest only after its referenced objects are complete. Readers fetch that manifest once and follow those exact keys or version IDs. Concurrent publishers still need conditional updates or another ownership protocol for the manifest pointer. Strong object consistency cannot decide which publisher was authorized to win.
The diagram separates the visible release from unfinished uploads. This follows the same immutable identity principle as artifact promotion.
A delete marker hides a versioned key
With S3 Versioning enabled, overwriting a key creates another version. An ordinary delete without a version ID creates a delete marker, so a normal current-object read behaves as though the object is absent; retained older versions can still be addressed explicitly. A delete that names a particular version can permanently remove that version, subject to permissions and retention controls. See how S3 Versioning works.
| Mechanism | Question it answers | Additional decision required |
|---|---|---|
| Strong consistency | Can a later read see the acknowledged object change? | Whether several keys form a coherent application release |
| Versioning | Is an earlier version retained? | Which version is correct and who may delete it |
| Replication | Is another configured destination receiving copies? | How lag and failed copies affect recovery |
| Retention policy | How long should eligible data remain? | Whether the policy protects the actual failure scenario |
A hypothetical operator accidentally deletes the current manifest.json. The object listing no longer shows an ordinary current object, but retained versions may still hold the release pointer. Recovery begins by identifying the intended manifest version and verifying every referenced data object. Merely removing a delete marker can reveal a version that was already incorrect before the incident.
Lifecycle expiration also matters: retaining versions increases storage use, and a cleanup policy can remove the history a recovery runbook assumes exists. Define the recovery interval first, then test the corresponding version and lifecycle configuration in a disposable bucket.
Replication is a separate recovery system
S3 replication is asynchronous and requires configuration for the eligible source objects and destination. Do not treat a successful source write as evidence that the destination can serve it. Live replication also does not automatically cover every pre-existing object; batch replication is available for eligible older objects. The replication documentation describes those scopes.
Suppose release 43 has reached the source, but the destination has received only its customers object. Serving the new manifest from the destination would create an incomplete release. A recovery check should validate the entire manifest dependency set there and select the latest complete release. The resulting recovery point may be older than the newest replicated object timestamp.
This is also a useful interview follow-up: how do you know the secondary region is ready? Name the destination versions or immutable keys, verify access through the recovery identity, and read representative data using the recovery application's path. An inventory count alone cannot prove decryption permissions or application compatibility.
Rehearse an overwrite and a deletion
For a local tabletop exercise, write a timeline with releases 41, 42 and 43, their manifest references, source acknowledgments and destination arrivals. Mark an accidental overwrite separately from a source-region outage. The first requires choosing trustworthy history; the second requires trustworthy history that is also available at the recovery location.
Self-check: the latest manifest is strongly consistent in the source bucket and all its data keys exist there. Does that establish a zero-loss regional recovery? No. Destination replication may still lag or fail, and the recovery principal may lack access. Establish the latest complete, readable release at the destination and compare its publication time with the permitted data-loss window. That is the recovery point the application can actually use.