Pipeline caches and reproducible builds: reuse inputs without trusting stale outputs

TL;DR: A cache is optional acceleration whose absence must not change correctness. Identify all inputs needed for the cached result, restrict who can populate it, and periodically compare a clean build with the warm path before trusting the speedup.

Separate a download cache from a finished result

A package download cache stores reusable dependency bytes. A compiler cache stores outputs for particular source and toolchain inputs. A release artifact is the exact output selected for deployment. These objects need different identities and approval rules.

Restoring npm's download cache should still be followed by the lockfile-driven installation and validation steps. Restoring an old application binary and skipping its build because “the cache hit” replaces evidence with an assumption. Build once, promote explains how the final artifact moves between environments after validation.

The GitHub dependency-cache reference distinguishes exact keys, restore prefixes and branch scopes. A partial-key restore can be useful as a pool of downloaded packages. It is weak evidence that a complete dependency tree or compiled output matches the current build inputs.

Make the key express compatibility

Suppose a fictional service builds native Node.js extensions on Linux amd64. Its download-cache key includes package-manager version and lockfile hash. A cache of installed native modules also needs compatible runtime ABI, operating system and architecture; source or compiler settings may add further requirements. A key containing only the branch name will eventually reuse incompatible state.

Do not add the commit hash everywhere by reflex. That prevents useful reuse across source changes that leave dependencies unchanged. Include inputs according to what the cache contains, while treating missing or invalid entries as ordinary cache misses.

The clean path is an independent check on the assumptions in the warm path. It should not secretly restore the same output cache.

Test reproducibility with an intentionally small build

The Reproducible Builds definition requires the same artifacts from the same specified inputs and environment. The following local Python exercise controls archive metadata and entry ordering so two separately created ZIP files match. It is a teaching fixture, not proof that a real application toolchain is reproducible.

import hashlib
import io
import zipfile

def build(files):
    output = io.BytesIO()
    with zipfile.ZipFile(output, "w", compression=zipfile.ZIP_STORED) as archive:
        for name, data in sorted(files.items()):
            info = zipfile.ZipInfo(name, date_time=(2020, 1, 1, 0, 0, 0))
            info.create_system = 3
            info.external_attr = 0o100644 << 16
            archive.writestr(info, data)
    return output.getvalue()

first = build({"b.txt": b"second\n", "a.txt": b"first\n"})
second = build({"a.txt": b"first\n", "b.txt": b"second\n"})
assert first == second
assert first != build({"a.txt": b"changed\n", "b.txt": b"second\n"})
print(hashlib.sha256(first).hexdigest())

The standard-library ZIP interface lets the example choose timestamps and attributes explicitly. Real builds may also embed paths, locale-dependent output or nondeterministic compiler data. Compare artifacts and investigate differences instead of treating a fixed timestamp as a complete solution.

A valid cache still leaves the resolver to choose the graph. Maven and Gradle dependencies compares their conflict rules and the evidence needed to explain a changed runtime classpath.

Include who can write the cache

A perfect key cannot make malicious cached content trustworthy. Review the CI platform's read/write scope, permissions and workflow event context. Untrusted pull-request code must not gain the ability to populate a cache that a privileged release job consumes as executable output. Avoid storing credentials in caches; a cache may be readable by more workflows than its author expects.

GitHub documents that pull requests can access eligible base-branch caches, including fork-originated pull requests. That makes a secret-bearing dependency directory a disclosure risk even if release credentials are withheld from the test job. Exact platform restrictions change, so verify them against the deployed service before designing a cross-workflow cache policy.

For a hypothetical timing comparison, fetching dependencies takes eight minutes cold and two warm, while installation and tests take four more minutes in either case. A warm run saves six of twelve minutes, or 50%, if it retains the checks. Skipping four minutes of validation would improve the headline further while changing what a green run establishes.

Self-check: a broad restore prefix finds a cache created with yesterday's lockfile. Can a job skip dependency installation because restoration succeeded? No. It may reuse verified download bytes while installing the current locked dependency set. For an installed-tree cache, require the full compatibility contract or discard it. A green cache lookup is not a dependency-resolution result.