Anatomy of a memory leak

The application allocates into Eden — blue for request data, orange for cache entries — and most of it dies young: by the time Eden is full, the older part has already turned grey. That is the weak generational hypothesis, and it is why a minor collection stays cheap: it copies the few live bytes into a survivor space and forgets the rest, in a few milliseconds.

Entries in an unbounded map never die. They survive every minor collection, get promoted and pile up in the old generation; each major collection clears the garbage around them but not them, so the floor it leaves behind keeps rising. Marking cost grows with that live set: the first major pause here takes about 90 ms, the last ones about 300 ms, back to back — until a promotion no longer fits. On a 512 MB heap that happens after about 70 simulated seconds. A 1 GB heap lasts twice as long, and its first major pause is several times longer: more memory postpones a leak, it never fixes one.

Figure 2 runs a leak of the same shape in a JVM of its own, scaled down to a 64 MB heap so that it ends in seconds. Recorded on the production image, it ran out of memory after 13.6 s under Serial, 14.2 s under G1 and 13.4 s under ZGC. No collector can free what is still reachable: the leak sets the deadline, not the collector.

What differs is how the end is spent. Serial and G1 stop the application more and more often as the old generation fills, 568 ms and 813 ms of pauses in all. ZGC hardly pauses, 2,420 times for 10 ms together, but the application waited 139 times for memory the collector had not yet freed, 607 ms in all: a concurrent collector turns the same problem into stalls. With the fix, all three ran their twenty seconds, and every full collection took Serial’s heap back to 6–8 MB.

Performance profiling and JVM tuning are on my CV. A leak is where the two meet: the pauses show up in the profile, and only the heap shows their cause. The numbers in the figure come from the model and from the JVM behind this site, not from a production system.

A WeakReference does not keep its referent alive: once no request holds a value any more, the next collection may clear it — an entry lives only as long as something else still uses it — and the TTL clears the stale map entries left behind. Switch the policy to Weak + TTL forty seconds into the leak and the next major GC takes the floor from about 200 MB down to a few MB. The old generation still fills up with data promoted too early, but with almost nothing live in it a major GC takes about 10 ms: pauses follow the live set, not how full the heap is.

A strong map with only a TTL is bounded too, but it keeps every entry alive for the whole TTL, used or not, so it settles at roughly cache rate × TTL (Little’s law) — about 125 MB of live data here, with major pauses around 90 ms. Until it levels off, its growth looks exactly like a leak.

Two general notes: a SoftReference is cleared only under memory pressure, which makes a cache built on it hard to predict; and the usual modern choice is a bounded cache with size- or time-based eviction. Either way, something has to decide when an entry may go.