Memory leaks versus cache growth: prove what remains after work finishes

TL;DR: Compare memory after equivalent workload cycles and identify what still owns the allocations. A rising RSS graph warrants investigation, but retained objects, cache bounds and allocator behavior determine whether the fix is in application lifetime, admission or capacity.

A graph needs a workload beside it

An API process growing from 300 MiB to 900 MiB can be warming a useful cache, retaining completed requests by mistake, or holding allocator pages for later reuse. The next action differs in each case. Record request volume, concurrency, input size and process age alongside the graph before comparing two releases.

Use the same accounting boundary throughout. The virtual memory and page cache concept distinguishes process residency from host and cgroup accounting. A container's file cache growth cannot be diagnosed from a managed-language heap chart alone. Conversely, a stable managed heap does not explain native-extension buffers or every memory mapping.

The Linux cgroup memory interface provides composition and pressure events at the enforced boundary. A workload can reach its container limit while its host still has spare RAM. Preserve that evidence before restarting; a restart clears much of the state needed to explain the growth.

Compare the floor after each cycle

Consider a fictional service with a cache limited to 200 MiB. Replay the same set of 1,000 keys three times, allow in-flight work to finish, and record live application allocations at the same point after each cycle. Floors of 310, 312 and 311 MiB are consistent with settling. Floors of 310, 370 and 430 MiB suggest roughly 60 MiB of additional retention per cycle. Neither sequence alone identifies the owner.

Then change only the key population. If new unique keys make the cache grow beyond its declared bound, inspect eviction and key normalization. A cache can be intentional and still cause an incident: a bound on item count is ineffective when individual values can grow without limit. Document bytes, expiry and maximum entry size as separate constraints.

The branches guide evidence collection. A process may have both a leaking request registry and a healthy file cache.

Run a small retention experiment

This Python standard-library exercise allocates about 2 MiB of payload in a temporary process. It demonstrates a reference retaining memory; it does not simulate an entire production leak. Run it as a standalone script so unrelated notebook objects do not distort the result.

import gc
import tracemalloc

tracemalloc.start()
before = tracemalloc.take_snapshot()
retained = [bytearray(4096) for _ in range(512)]
held = tracemalloc.take_snapshot()
retained.clear()
gc.collect()
after = tracemalloc.take_snapshot()

def growth(snapshot):
    return sum(s.size_diff for s in snapshot.compare_to(before, "lineno"))

print("held bytes:", growth(held))
print("after clear:", growth(after))
for entry in held.compare_to(before, "lineno")[:3]:
    print(entry)

Expect approximately 2 MiB plus object overhead while the list holds the buffers, followed by a much smaller traced difference after clearing it. Exact values depend on Python and the environment. The tracemalloc reference explains snapshots and allocation tracebacks. It traces participating Python allocations; it is not a complete RSS or GPU profiler.

Garbage collection does not free objects that remain reachable from a live owner. In a real service, find the reference chain: a module-level dictionary, unfinished future, callback registry or request history may retain the objects long after their useful work ends. Snapshot differences locate allocation sites; a heap/reference investigation establishes why objects remain reachable.

Mitigate, then reproduce the cause

When memory pressure threatens service, reduce admitted concurrency or roll back a correlated release if the previous version remains compatible. A controlled restart may buy time, but record its role as mitigation. Increasing the limit is justified only with enough node capacity and a bounded expected working set; it postpones a genuine unbounded-growth failure.

For the interview follow-up, describe what would disprove your hypothesis. If the same workload returns to its previous live-object floor after a cache expiry interval, the leak claim weakens. If retained request objects increase after every completed cycle, changing a dashboard's definition of “used memory” will not repair the application.

Self-check: after clearing a cache, traced live allocations fall by 180 MiB but RSS falls by only 20 MiB. Did clearing fail? The evidence shows that the traced objects were released. The remaining residency needs allocator/native accounting and a reuse experiment. Verify that subsequent equivalent work reuses memory without another rising floor before calling the service stable.

Redis memory and latency applies this distinction to eviction accounting, process residency and the downstream cost of cache misses.