store: judge memory pressure by swap RATE, not level #130

Merged
will.anderson merged 1 commits from fix/elc-rebuildable-compiler-builtins into dev 2026-08-16 02:12:24 +00:00
Owner

The guard added minutes earlier checked swap as a level (avail < total/8). Wrong signal — the same host proved it twice within minutes:

47.65 / 48.00 GiB swap used, 2047 swapouts/s  -> genuinely thrashing
26.67 / 28.00 GiB swap used,    0 swapouts/s  -> healthy, 15.6 GiB free

Both are ~97% "used". macOS grows swap on demand and trims lazily, so the level is a high-water mark that says almost nothing about now. A level check calls the second state an emergency and starves the pool for no reason — and a guard that fires on healthy machines gets disabled, and then guards nothing.

What separates them is whether pages are moving. Now samples the swapout counter across calls and judges the delta:

  • > 200 pages/s (~3 MiB/s) sustained → report zero available; refuse to grow, hand frames back.
  • First call primes the baseline and reports no pressure — one sample cannot have a rate, and inferring one from a single reading is precisely the mistake being removed.

Thresholds measured, not guessed: idle 0/s; recovery burst 24,845/s while the compressor drained (transient, and correctly not a growth decision since growth is only evaluated on eviction passes); real thrash ~2000/s. 200/s sits clearly above noise and far below either.

Compressor-footprint subtraction stays — that RAM is spoken for regardless of paging rate.

The guard added minutes earlier checked swap as a **level** (`avail < total/8`). Wrong signal — the same host proved it twice within minutes: ``` 47.65 / 48.00 GiB swap used, 2047 swapouts/s -> genuinely thrashing 26.67 / 28.00 GiB swap used, 0 swapouts/s -> healthy, 15.6 GiB free ``` Both are ~97% "used". macOS grows swap on demand and trims lazily, so the level is a high-water mark that says almost nothing about now. A level check calls the second state an emergency and starves the pool for no reason — **and a guard that fires on healthy machines gets disabled, and then guards nothing.** What separates them is whether pages are *moving*. Now samples the swapout counter across calls and judges the delta: - **> 200 pages/s (~3 MiB/s) sustained** → report zero available; refuse to grow, hand frames back. - First call primes the baseline and reports no pressure — one sample cannot have a rate, and inferring one from a single reading is precisely the mistake being removed. **Thresholds measured, not guessed:** idle 0/s; recovery burst 24,845/s while the compressor drained (transient, and correctly not a growth decision since growth is only evaluated on eviction passes); real thrash ~2000/s. 200/s sits clearly above noise and far below either. Compressor-footprint subtraction stays — that RAM is spoken for regardless of paging rate.
will.anderson added 1 commit 2026-08-16 02:12:12 +00:00
store: judge memory pressure by swap RATE, not swap level
El SDK CI - dev / build-and-test (pull_request) Failing after 11m59s
19cc99e57d
The guard I added minutes ago checked swap availability as a level
(avail < total/8 -> report zero available). That is the wrong signal, and the
same host proved it twice within minutes:

    47.65 / 48.00 GiB swap used, 2047 swapouts/s  -> genuinely thrashing
    26.67 / 28.00 GiB swap used,    0 swapouts/s  -> healthy, 15.6 GiB free

Both are ~97% "used". macOS grows swap files on demand and trims them lazily,
so the level says almost nothing about now — it is a high-water mark. The level
check calls the second state an emergency and starves the pool for no reason,
which is its own failure mode: a guard that fires on healthy machines gets
disabled, and then guards nothing.

What separates the two is whether pages are moving. So sample the swapout
counter across calls and judge the delta:

  - > 200 pages/s (~3 MiB/s) sustained outward paging => report zero available;
    callers refuse to grow and pc_relieve_pressure hands frames back.
  - The first call primes the baseline and reports no pressure. One sample
    cannot have a rate, and inferring one from a single reading is exactly the
    mistake this commit removes.

Measured thresholds, not guessed: idle sat at 0/s, recovery burst hit 24,845/s
while the compressor drained (transient, correctly not a growth decision since
growth is only evaluated on eviction passes), and real thrash held ~2000/s.
200/s sits clearly above noise and far below either.

The compressor-footprint subtraction stays: that RAM is genuinely spoken for
regardless of paging rate.
will.anderson merged commit 2240d26c32 into dev 2026-08-16 02:12:24 +00:00
Sign in to join this conversation.