Compare commits

...

39 Commits

Author SHA1 Message Date
will.anderson 0832865952 Merge pull request 'test framework phase 3/4: black_box barrier + three-signal complexity gate, armed' (#139) from wt/soul-runtime-reconcile into dev
El SDK CI - dev / build-and-test (push) Failing after 11m32s
2026-08-16 03:02:18 +00:00
Neuron e0b2c0ea54 bench: arm the Phase 4 gate -- proven to pass clean AND fire on a quadratic
El SDK CI - dev / build-and-test (pull_request) Failing after 12m7s
Adds tests/native/test_lexer_scaling.el, the regression gate for el #132.

Both directions are proven on LIVE workloads, not synthetic series:
  healthy per-character scan  1821 3251 6007 10422 us -> O(n)   PASS
  rescan-from-zero (the #132 shape)  922 3667 13524 44792 -> O(n^2) FAIL

A gate only proven to pass is decoration. The quadratic specimen exists so
the gate is proven to FIRE.

Also fixes elb_spread_ok to judge the ASYMPTOTIC TAIL (last three ratios)
rather than the whole sweep. Measured on a genuinely linear scan the ratios
ran 3.37 2.92 1.76 1.65 -- the head looks quadratic because it is cold
cache, the tail is the truth. Whole-sweep spread rejected correct data. A
complexity bound is an asymptotic claim and must be judged asymptotically.

That fix came from the classifier refusing to rubber-stamp my own bad
measurement: it reported INDETERMINATE on an unwarmed sweep rather than
passing it. Warmup is now taken and discarded at every sweep point.

Reverts the == workarounds in test_elbench.el now that el #137 has landed;
the natural form generates no str_eq and all 13 fitter tests stay green.
The  workaround remains -- the Plus arm is still open.
2026-08-15 21:58:46 -05:00
bigmerge cf060adbfd Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:55:40 -05:00
will.anderson 63fe8a766d Merge pull request 'codegen: Bool is int-like, so Bool comparisons stop lowering to str_eq' (#138) from fix/bool-is-int-like into dev
El SDK CI - dev / build-and-test (push) Failing after 4m28s
2026-08-16 02:54:34 +00:00
bigmerge b5a0a729e6 codegen: Bool is int-like, so Bool comparisons stop lowering to str_eq
El SDK CI - dev / build-and-test (pull_request) Failing after 14m49s
fn check(label: String, cond: Bool, want: Bool) -> Void {
        if cond == want { ... }        ->  if (str_eq(cond, want))   SIGSEGV
    }

Bool has always been an integer in the value model — type_to_c maps Bool to
"int", and el_runtime.h states "Bool -> el_val_t (0 = false, nonzero = true)".
But Bool names were registered NOWHERE: build_int_names_for_params tracked Int
and Float params, and the `let` path tracked Int and Float bindings. Neither
knew about Bool.

So comparing two Bools fell through to str_eq, which dereferenced 0 or 1 as a
char* and segfaulted immediately.

This is the third instance of one family found tonight, after el #137 (a call
on either side of == poisoned the operator) and el #136 (a missing import
compiled clean). All three are the same shape: something the compiler could not
type, silently handled as a string.

Found while writing #137's own test harness — the first version of that harness
crashed on exactly this, on both the old and new compiler, which is how it
surfaced. A test harness that cannot compare two Bools is a good way to notice.

VERIFIED:
  - the harness that segfaulted on every prior compiler (exit 139, no output)
    now runs clean: 14 passed, 0 failed
  - self-hosting fixpoint byte-identical
  - the compiler's own generated C differs by 8 lines — only the intended
    registration
  - neuron's full soul amalgam regenerates in 424ms, exit 0, BYTE-IDENTICAL
  - test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12

Adds tests/runtime/operator_typing_test.el, the 15-case suite from #137, so
this family is covered going forward rather than rediscovered.
2026-08-15 21:54:10 -05:00
will.anderson b26dd47aef Merge pull request 'codegen: either side Int is enough for == and !=, not both' (#137) from fix/eq-operand-inference into dev
El SDK CI - dev / build-and-test (push) Failing after 12m0s
2026-08-16 02:52:02 +00:00
bigmerge b55e6bfd53 codegen: either side Int is enough for == and !=, not both
El SDK CI - dev / build-and-test (pull_request) Failing after 12m20s
let a: Int = 5
    getint(5) == a      ->  str_eq(getint(5), a)      SIGSEGV
    getint(5) == 5      ->  getint(5) == 5            fine

A function call whose return type codegen cannot infer poisoned the operator,
and a declared Int on the other side did not save it. str_eq then read an
integer as a char* and segfaulted. Only an integer LITERAL on one side forced
the numeric form, which is why the bug stayed invisible: the common case
happened to be safe.

The check required BOTH operands to be provably Int:

    if is_int_expr(left) { if is_int_expr(right) { numeric } }

Loosening to OR is strictly safer, not a trade:
  - when one side is a known Int, str_eq is ALWAYS wrong — it dereferences
    that integer — while numeric comparison is at worst a wrong answer on a
    program that was already ill-typed;
  - when neither side is Int nothing changes at all, so string comparison is
    untouched.

Found by the test-framework agent while building the benchmark harness; it
correctly declined to fix it mid-phase since it is a codegen semantics change.

VERIFIED, because a semantics change earns more than an assertion:
  - 15/15 on a dedicated operator suite covering string literals, string vars,
    string-returning calls, mixed var/call, and != in every combination. The
    pre-change compiler scores 0/15 on the same file: it segfaults before
    printing anything.
  - self-hosting fixpoint byte-identical
  - the ONLY difference in the compiler's own generated C is the intended one:
    a nested if becoming two sequential ifs, in EqEq and NotEq. Nothing else
    moved.
  - neuron's full soul amalgam regenerates in 400ms, exit 0, output
    BYTE-IDENTICAL at 1,270,212 bytes
  - test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12 —
    62 tests, 190 assertions, zero failures

NOT fixed here, same family, flagged for a decision: Bool PARAMETERS are not
tracked as int-like, so `cond == want` between two Bool params still lowers to
str_eq and segfaults. Found while writing this commit's own test harness — the
first version of it crashed on exactly that, on both the old and new compiler.
It needs the same treatment, and it wants its own change.
2026-08-15 21:51:35 -05:00
will.anderson dbb06f6ee4 Merge pull request 'compiler: a missing import is an error, not an empty string' (#136) from fix/missing-import-is-an-error into dev
El SDK CI - dev / build-and-test (push) Failing after 11m0s
2026-08-16 02:48:01 +00:00
bigmerge 906c664a65 compiler: a missing import is an error, not an empty string
El SDK CI - dev / build-and-test (pull_request) Failing after 11m23s
import "../../NOPE/does_not_exist.el"

compiled CLEANLY — exit 0, empty stderr, and a program silently missing
everything it imported.

resolve_imports did `fs_read(src_path)` and used the result without checking.
fs_read returns "" both for "file is empty" and "file does not exist", so a
typo, a moved file, or a relative path resolved from the wrong working
directory all produced a successful build of nothing.

It caused a real wrong conclusion during test-framework work: a bisection run
from a subdirectory where ../../runtime/ did not resolve produced ELEVEN
consecutive "successful" compiles that had included no runtime at all, and the
results were believed before anyone noticed.

Missing dependency, confident success — the same shape as a test suite
reporting pass for tests that never ran, and as a benchmark reporting 0us
because the optimiser deleted the loop.

fs_exists separates the two cases, so a legitimately empty file still resolves
to "" and is fine. A path that does not exist now prints the resolved path and
exits 1, which is what build scripts check.

Verified:
  - bad import: exit 1 (was 0), message names the resolved path
  - elc-cli.el still compiles, self-hosting fixpoint byte-identical
  - neuron's full soul amalgam regeneration: exit 0, 405ms, output
    byte-identical at 1,270,212 bytes
2026-08-15 21:47:32 -05:00
Neuron 6a6b589ba0 bench: real black_box barrier + three-signal growth-curve gate
Adds el_black_box (inline asm, +r constraint, memory clobber) and
runtime/elbench.el: a growth-curve classifier that gates time AND
allocation-count AND allocation-bytes, failing if any exceeds its
declared curve.

Refusal is a first-class verdict. The classifier REFUSES rather than
classifying when the largest measurement is below the floor, or when a
series is hard-flat across an 8x input range -- the shape produced when
the optimiser deletes the work. Reporting O(1) there would be a
confident answer with nothing behind it. Disagreeing ratios report
INDETERMINATE rather than a guess.

Deviation from DESIGN.md 6.2, stated in the source: uses consecutive
ratios on a mandated geometric sweep rather than least-squares over
candidate curves. Ratios are directly interpretable on a doubling sweep
and need no floating point; the cost is weaker O(n) vs O(n log n)
separation, reported as an ambiguous band rather than guessed.

Documents the counter scope limit: engram_*.c and libcurl malloc are
NOT tracked, so a flat curve over engram/HTTP-dominated work is not
evidence of anything.

13 tests prove the classifier against real measured series from
fitprobe.el -- including that an accumulator's allocation COUNT is
linear while its bytes are quadratic, and that el #132's pure-CPU shape
reads FLAT on both allocation signals and is caught only by time.
2026-08-15 21:45:13 -05:00
bigmerge b5d1e53902 Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:38:11 -05:00
will.anderson 9e96d74f6a Merge pull request 'runtime: count container allocations too, not just strings' (#135) from feat/alloc-accounting-containers into dev
El SDK CI - dev / build-and-test (push) Failing after 11m49s
2026-08-16 02:37:14 +00:00
bigmerge a8908908df runtime: count container allocations too, not just strings
El SDK CI - dev / build-and-test (pull_request) Failing after 12m11s
el #131 instrumented the four string allocators, which meant list- and map-heavy
code reported ZERO allocations — a benchmark over lists would have been fitted
against a flat line and passed anything. Caught during framework work: a
"linear" specimen read 0 allocs until it was rewritten to allocate strings.

A gate is only as good as its blind spots are small, and a signal that silently
reads zero is worse than no signal: it produces a confident pass.

Now counted at every container allocation — ElList and ElMap bodies, their
backing arrays, the copy-on-write clones, and the realloc growth path.

Verified on an append loop (n = 100..800):
    allocs  7, 8, 9, 10          +1 per doubling = O(log n) reallocations
    bytes   2048, 4096, 8192, 16384   exactly 2x per doubling = O(n)

Both curves are what correct amortized growth should look like, and both read
zero before this change.

Known remaining scope, stated rather than left implicit: these counters cover
the runtime's own allocations. They do not see malloc inside engram_*.c or
libcurl, which is correct — the gate is for El-level complexity, not for
third-party memory behaviour.
2026-08-15 21:36:52 -05:00
Neuron 6291a35bb9 design: gate on THREE signals -- the alloc gate would have missed el #132
el #132's quadratic (strlen per character in str_char_code/str_slice) is
pure CPU and allocates NOTHING. Measured on three controlled specimens:

  specimen  allocs        bytes         time
  linear    2.00 -> O(n)  2.16 -> O(n)  2.05 -> O(n)
  accum     2.00 -> O(n)  3.99 -> O(n2) noisy
  compute   FLAT          FLAT          3.96 -> O(n2)

'compute' is #132's shape. A gate fitting only allocation count and bytes
classifies it FLAT and passes -- it would not have caught the defect it
was created for. The gate now fits time AND count AND bytes, failing if
any exceeds its declared curve.

Also: black_box is mandatory and consuming the result is NOT sufficient.
The first 'compute' reported 0us at every n while returning a correct n2 --
clang closed the loop to a multiply. Only an opaque call restored the curve.

Adds lang/tests/bench/fitprobe.el as the fitter's known-good/known-bad set,
so the classifier is provable without depending on a real bug existing.
Marks DESIGN.md 1.3 stale: test_compiler 3.58s -> 0.03s (119x).
2026-08-15 21:34:39 -05:00
will.anderson a69a4a5894 Merge pull request 'runtime: math_log is base-10, not natural log' (#134) from fix/math-log-base10 into dev
El SDK CI - dev / build-and-test (push) Failing after 14m48s
2026-08-16 02:34:09 +00:00
bigmerge edafd8cce8 runtime: math_log is base-10, not natural log
El SDK CI - dev / build-and-test (pull_request) Failing after 10m8s
el_val_t math_log(el_val_t f) { return el_from_float(log(el_to_float(f))); }
    el_val_t math_ln(el_val_t f)  { return el_from_float(log(el_to_float(f))); }

Both were natural log, so math_log and math_ln were the same function.
log10(100) returned 4.605 instead of 2.

Three sources already agreed it should be base-10 and were being contradicted
by this one line:
  - runtime/math.el:55  "// math_log — base-10 logarithm."
  - el_seed.c:1278      __log_f -> log10()  (the path math.el actually calls)
  - tests/native/test_math.el:133  asserts log10(100) == 2

FOUND BY THE NEW TEST FRAMEWORK ON ITS FIRST RUN (el #133). The assertion had
been sitting in the suite the whole time; nothing could report it. The old
harness printed "N passed, M failed" with no per-test detail, and half the
suites were not compiling at all — so a failing assertion in a suite nobody
could run was indistinguishable from no failure.

That is the entire argument for the framework, demonstrated on day one: this is
not a bug the framework introduced, it is a bug the framework made VISIBLE.

Verified: tests/native/test_math.el goes 12/13 -> 13/13, math-log passing.
2026-08-15 21:33:20 -05:00
will.anderson 5e3e69d326 Merge pull request 'test framework phase 1: compile-time registry + El-side runner with per-test timing' (#133) from wt/soul-runtime-reconcile into dev
El SDK CI - dev / build-and-test (push) Failing after 10m17s
2026-08-16 02:31:08 +00:00
bigmerge 4c3414072b Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:30:52 -05:00
will.anderson 0288024396 Merge pull request 'compiler: fix the quadratic — strlen() on every character access' (#132) from fix/compiler-quadratic-strlen into dev
El SDK CI - dev / build-and-test (push) Failing after 4m2s
2026-08-16 02:29:02 +00:00
Neuron 3e7ab07e82 test framework phase 1: forward decls, void-return fix, suite migration
El SDK CI - dev / build-and-test (pull_request) Failing after 10m4s
Completes the Phase 1 runner and migrates the 11 test files onto it.

- forward-declare the registry accessors in the test preamble; they are
  defined at the end of the unit but the El runner is compiled in between
- eltest.el: explicit trailing return in the void emit_* helpers, which
  otherwise lower to 'return println(...)' and fail to compile
- test files import runtime/eltest.el explicitly, using the language's own
  textual import mechanism rather than compiler-side auto-injection
- DESIGN.md 6.5: gate on allocation COUNT AND BYTES, not count alone

Verified: self-hosting fixpoint byte-identical (gen2 == gen3). 6 of 11
suites run and report per-test timing. The other 5 fail to COMPILE, and
fail identically under the committed compiler -- pre-existing breakage
this framework makes visible for the first time.
2026-08-15 21:28:30 -05:00
bigmerge d231b7e5e7 compiler: fix the quadratic — strlen() on every character access
El SDK CI - dev / build-and-test (pull_request) Failing after 10m21s
THE BUG. str_char_code() and str_slice() each called strlen() on every
invocation. The lexer walks source one character at a time, so every character
access rescanned the whole remaining input: O(n) per character over n
characters = O(n^2).

    el_val_t str_char_code(el_val_t s, el_val_t i) {
        ...
        int64_t n = (int64_t)strlen(str);   // <- O(n), every call
        if (idx < 0 || idx >= n) return 0;
        return str[idx];
    }

HOW IT WAS FOUND. Not by reading code — by sampling the running process, which
is the same method that resolved tonight's engram outage after four wrong
theories. A geometric sweep of synthetic sources showed wall-clock rising 3.0x,
3.0x, 4.0x, 4.14x per doubling (converging on 4x = quadratic), and a stack
sample put 779 of 779 samples inside lex(), every one bottoming out in
_platform_strlen via str_char_code and str_slice.

THE FIX. Remember the length instead of recomputing it. The subtlety is
INVALIDATION: El strings are arena-allocated, so a freed pointer can be reused
for a different string at the same address, and a naive pointer-keyed cache
would hand back a stale length and read past the end of the new string —
trading a performance bug for a memory-safety one. So entries carry a
generation, a hit requires pointer AND generation to match, and every path that
frees or mutates a runtime string bumps the generation: el_arena_pop,
seed_request_end, __str_set_char. Stale entries cannot be believed; they miss
and recompute.

MEASURED, same host, same inputs:

    n(fns)    before     after
      512      0.10s     0.01s
     1024      0.37s     0.02s
     2048      1.51s     0.03s     50x

    the compiler's own 422 KB source concatenated (DESIGN.md's 3.58s case):
              3.55s ->  0.03s      118x

The speedup GROWS with input size, which is the signature of removing a
complexity class rather than a constant factor. After the fix each doubling
adds ~0.01s: linear.

CORRECTNESS, verified rather than assumed:
  - byte-identical output on every sweep input (n = 128..2048)
  - byte-identical output on the 422 KB compiler concatenation
  - byte-identical output on tests/runtime/string_test.el
  - self-hosting fixpoint byte-identical
  - new tests/runtime/str_cache_test.el: 17 assertions covering bounds, empty
    strings, negative indices, slice clamping, distinct strings not sharing a
    cached length, 1000 interleaved strings forcing cache-slot collisions, and
    a grown string not reporting its old length. All pass.

This is the defect that made dist/soul.c a committed artifact: elc could not run
in CI because it needed 24 GB+ and minutes. It needs neither now.
2026-08-15 21:28:23 -05:00
bigmerge a668062e38 Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:24:03 -05:00
Neuron 24fac765a6 test framework phase 1: compile-time registry + El-side runner
Replace the hardcoded test harness main() with a generated static registry
and index-based accessors, and move all reporting into runtime/eltest.el.

The old harness inlined direct calls into main() and counted assertions in
two globals. That shape cannot report which test failed, how long any test
took, or whether a test ran at all -- a misspelled registration reported
success for a test that never executed.

- assertions record into per-test state instead of global counters
- registry table emitted at compile time; discovery strictly precedes
  execution, which is what later enables --list, filtering and sharding
- per-test wall timing on CLOCK_MONOTONIC, taken in C around the call
- runner in El: structured NDJSON events as source of truth, human output
  rendered from the same fields
2026-08-15 21:24:03 -05:00
will.anderson cb1f2a74af Merge pull request 'runtime: allocation accounting — deterministic signal for complexity gating' (#131) from feat/alloc-accounting into dev
El SDK CI - dev / build-and-test (push) Failing after 3m49s
2026-08-16 02:22:13 +00:00
bigmerge 37bcf7eb74 runtime: allocation accounting — the deterministic signal for complexity gating
El SDK CI - dev / build-and-test (pull_request) Failing after 12m7s
Implements the three primitives the test-framework design (DESIGN.md §6.5)
requires for gating on growth curves: el_alloc_count, el_alloc_bytes,
el_peak_rss. Registered in codegen's builtin_arity and wrapped in el_seed.c per
the project's C-builtin recipe.

WHY COUNTS AND NOT WALL-CLOCK: a growth-curve gate has to be a hard build
failure, which means the signal cannot flake. Wall-clock needs warmup,
statistics, and a quiet machine; on shared CI it is unusable as a gate.
Allocation counts are perfectly deterministic — same input, same number, every
machine, every run. Fit them against n and a complexity regression becomes a
build failure with zero noise.

All four runtime string allocators (el_strdup, el_strbuf, and their _persist
variants) funnel every allocation the language performs, so instrumenting there
counts everything.

WHY BYTES AS WELL AS COUNT — this is not redundancy, it is the whole gate.
Measured with two El programs, one allocating once per item, one rebuilding its
accumulator each iteration:

    n     linear allocs / bytes      quadratic allocs / bytes
    100        100 /    290               100 /   5,150
    200        200 /    690               200 /  20,300
    400        400 /  1,490               400 /  80,600
    800        800 /  3,090               800 / 321,200

The quadratic program's allocation COUNT is exactly linear — identical to the
healthy one. Counting allocations alone would have missed it completely. Bytes
catch it: each doubling of n quadruples bytes (ratios 3.94, 3.97, 3.99 ->
converging on 4.0, i.e. O(n^2)), while the linear case converges on 2.0.

That shape — count linear, per-allocation size growing — is the classic
accidental quadratic, and it is exactly elc's defect: quadratic allocation
VOLUME, which the old shipped compiler paid in RSS (27 GB, OOM) and the rebuilt
one pays in malloc/free churn (42s on 1.4 MB). Volume was the invariant across
both; RSS and wall-clock were just the two ways it surfaced.

el_peak_rss is exported for context and is explicitly NOT a gating signal — it
is perturbed by allocator internals, the page cache, and the OS. Gate on the
deterministic numbers; report the physical one.

Counters are unsynchronised by design: this is measurement, and a lock would
change the thing being measured. Exact on the single-threaded compile path,
approximate under threads.
2026-08-15 21:21:42 -05:00
will.anderson 2240d26c32 Merge pull request 'store: judge memory pressure by swap RATE, not level' (#130) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 11m44s
2026-08-16 02:12:22 +00:00
bigmerge 19cc99e57d store: judge memory pressure by swap RATE, not swap level
El SDK CI - dev / build-and-test (pull_request) Failing after 11m59s
The guard I added minutes ago checked swap availability as a level
(avail < total/8 -> report zero available). That is the wrong signal, and the
same host proved it twice within minutes:

    47.65 / 48.00 GiB swap used, 2047 swapouts/s  -> genuinely thrashing
    26.67 / 28.00 GiB swap used,    0 swapouts/s  -> healthy, 15.6 GiB free

Both are ~97% "used". macOS grows swap files on demand and trims them lazily,
so the level says almost nothing about now — it is a high-water mark. The level
check calls the second state an emergency and starves the pool for no reason,
which is its own failure mode: a guard that fires on healthy machines gets
disabled, and then guards nothing.

What separates the two is whether pages are moving. So sample the swapout
counter across calls and judge the delta:

  - > 200 pages/s (~3 MiB/s) sustained outward paging => report zero available;
    callers refuse to grow and pc_relieve_pressure hands frames back.
  - The first call primes the baseline and reports no pressure. One sample
    cannot have a rate, and inferring one from a single reading is exactly the
    mistake this commit removes.

Measured thresholds, not guessed: idle sat at 0/s, recovery burst hit 24,845/s
while the compressor drained (transient, correctly not a growth decision since
growth is only evaluated on eviction passes), and real thrash held ~2000/s.
200/s sits clearly above noise and far below either.

The compressor-footprint subtraction stays: that RAM is genuinely spoken for
regardless of paging rate.
2026-08-15 21:12:00 -05:00
will.anderson f39ae40047 Merge pull request 'store: bound the pool by available memory and let it shrink' (#129) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 4m43s
2026-08-16 02:02:24 +00:00
bigmerge e52415f0e0 store: bound the pool by AVAILABLE memory and let it shrink
El SDK CI - dev / build-and-test (pull_request) Failing after 11m47s
The adaptive budget I added an hour ago could only grow, and grew toward a
share of TOTAL ram (80%, ~38 GiB on a 48 GB host). That is a memory leak with
extra steps: total never shrinks when other processes need memory, so the pool
had no way to notice it was starving the machine it runs on. Deployed briefly;
caught as memory pressure on the host.

A control loop with only one direction is not a control loop.

  - pc_available_ram(): free + inactive + purgeable via host_statistics64 on
    Darwin, MemAvailable on Linux. Availability is the quantity that moves when
    the machine is under pressure; total is not. Returns 0 when it cannot be
    read, and callers then refuse to grow — a cache is never worth swapping the
    host, so unknown means no.

  - Growth is bounded by availability minus a free-memory floor (2 GiB default,
    ENGRAM_POOL_FREE_FLOOR_MB), not by total. The share-of-total ceiling stays
    as a second bound and drops 80% -> 50%.

  - pc_relieve_pressure(): the missing direction. On every eviction pass, if
    available memory is under the floor, hand back ~25% of held frames; the
    resident set follows on the next pass so the memory is actually returned
    rather than merely re-labelled. Counted as adapt_shrinks alongside
    adapt_grows so both directions are visible in the same report.

  - pc_default_cap() also clamps the STARTING budget to what is spare right
    now, so a cold boot on a loaded machine does not open at a size the host
    cannot afford.

Verified on a 48 GB host: engram boots in ~30s, RSS settles at 2.22 GiB (the
store's actual size, resident, not creeping), 0.0% CPU, 13,439 nodes / 37,670
edges, embeddings complete. Guard reports 9.71 GiB available against a 2.00 GiB
floor — 7.71 GiB of headroom it is permitted to use and no more.
2026-08-15 21:02:05 -05:00
will.anderson 7a479111ac Merge pull request 'store: extend the write barrier to edges — kills the full-store walk' (#128) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 14m27s
2026-08-16 01:44:33 +00:00
bigmerge e917b3d439 store: make the buffer pool sense its own state and correct from it
El SDK CI - dev / build-and-test (pull_request) Failing after 14m35s
Follow-on to the edge write barrier. That fix removed the full-store walk;
this one makes the pool able to notice if anything like it happens again.

WHAT WENT WRONG, precisely: the pool thrashed the live engram to a standstill
twice on 2026-08-15 and said nothing. From outside it was indistinguishable
from "busy loading" — 100% CPU, flat RSS, no output — so four wrong theories
got tried (bad binary, corrupt snapshot, WAL replay, feature flags), each
costing a deploy or a rollback. The whole time, hits/misses/evictions were
already being counted in PgCache, and the struct comment read:

    /* stats (introspection only — never affect semantics) */

That comment was the bug. Self-measurement treated as decoration is why the
pool could not correct itself and why no one outside could see what it was
doing. A system that cannot read its own state cannot correct, and neither can
anyone watching it.

  - pc_adapt_budget(): the loop, closed. Over a sliding window, evictions
    running at a large fraction of accesses WHILE reuse is real means the
    working set exceeds the budget — so grow it, geometrically, bounded by a
    LIVE re-read of physical memory. Evictions alone are not pressure (a scan
    evicts and never returns); evictions with reuse are. An explicit
    ENGRAM_POOL_FRAMES still wins — an operator override must not be silently
    overruled.

  - Budget derived, not declared. A constant cannot be right: 16 GiB of frames
    is arbitrary on a 48 GB host and suicidal on a 16 GB one. Even "60% of RAM
    at startup" is a guess about the future — it cannot know the store grew or
    the machine changed. Hence the live re-read.

  - pc_report(): ONE structured emission carrying the entire sensed state,
    through emit_log — El's existing telemetry, already exporting to OTLP.
    Deliberately not a function per stat, and deliberately not a bespoke
    /api/pool endpoint: both make observability something hand-written per noun
    instead of the uniform mechanism every component already has.

  - engram_pool_stats_json(): the same state readable live, wired through the
    normal builtin path (codegen arity + el_seed wrapper), so the pool can be
    observed in real time rather than reconstructed afterward from a stack
    sample.

Verified: with the exact configuration that took production down
(ENGRAM_POOL_FRAMES=65536 → 1 GiB cache against a 2 GiB store) the engram boots
clean and serves — 0.0% CPU, 13,436 nodes / 37,663 edges, embeddings complete —
and NO pressure event fires, because the barrier removed the walk that caused
it. The controller is defense in depth; the barrier is the fix.
2026-08-15 20:44:23 -05:00
bigmerge 777ccc02f0 store: extend the durable-hash write barrier to edges (kills the full-store walk)
El SDK CI - dev / build-and-test (pull_request) Failing after 10m45s
Checkpointing pushes the ENTIRE resident graph through store_put_node and
store_put_edge (see engram_store_checkpoint). Nodes were cheap: a durable-hash
compare skipped unchanged records with zero page I/O. Edges had no barrier at
all — struct comment at PgCache.barrier_on even says "node durable-hash
barrier" — so every edge was rewritten on every checkpoint, and each rewrite
runs the idempotency probe max_page_lsn_for_id -> btree lookup -> page_read.

Edges outnumber nodes ~3:1 here (37,663 vs 13,436), so routine checkpointing
degenerated into a FULL-STORE WALK in id order: random page access across the
whole 2 GiB store, repeated, overwhelmingly to rediscover nothing had changed.
LRU is worst-case under exactly that pattern — it evicts the page it is about
to want — so once the page cache was smaller than the store, the walk collapsed
into thrashing: 100% CPU, flat RSS, no forward progress, port never bound.
That took the live engram down twice on 2026-08-15.

The walk is the defect. Sizing the cache to survive it treats the symptom.

Changes:
  - dh_edge_hash(): edge counterpart of dh_node_hash, with a kind discriminator
    byte so an edge can never collide with a node of the same id in the shared
    map. created_at/updated_at/last_fired are excluded deliberately: last_fired
    is touched by activation without changing what the edge IS, and folding it
    in would defeat the barrier on precisely the hot edges that most need it.
  - store_put_edge(): barrier check + dh_set on success, mirroring
    store_put_node exactly.
  - store_scan_edges(): seed the barrier map from on-disk truth at load, so the
    FIRST post-boot checkpoint already skips unchanged edges. store_scan_nodes
    already did this and its comment says why; edges were simply never done.

Verified: with the exact configuration that killed production
(ENGRAM_POOL_FRAMES=65536 -> 1 GiB cache against a 2 GiB store), the engram now
boots clean and serves — LISTENING, 13,436 nodes / 37,663 edges, embeddings
complete, 0.0% CPU, RSS 1.14 GiB (cache resting at its budget rather than
thrashing against it). Same small cache, same store, no walk.
2026-08-15 20:38:11 -05:00
will.anderson c21074b547 Merge pull request 'runtime: engram_edges_json — kill the whole-graph file round trip' (#127) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 10m19s
2026-08-16 01:13:44 +00:00
bigmerge 4e24d7d3f1 runtime: engram_edges_json — read edges without a whole-graph file round trip
El SDK CI - dev / build-and-test (pull_request) Failing after 13m4s
/api/graph/edges answered a read query by calling engram_save() to serialize
the ENTIRE graph to disk (128 MB) and then fs_read-ing it back. Two defects in
one line, and both bit production on 2026-08-15:

  1. The path it wrote was ~/.neuron/engram/snapshot.json — the engram
     server's CANONICAL store. A READ route overwriting the persistence
     owner's canonical file. This defect had been fixed once (export moved to
     a scratch path); it came back when the hand-written dispatch block was
     replaced by @route dispatch and the unfixed copy is the one that
     survived the merge.
  2. Cost: a full snapshot write, a 128 MB read, and a parse of the whole
     graph, per request, to return a bounded slice.

Calling it tonight overwrote the canonical snapshot and immediately preceded
an engram crash loop.

engram_edges_json(limit, offset) is the builtin that route's own TODO asked
for ("Future: add an engram_edges_json() builtin and drop the file round trip
entirely"). It walks g->edges directly and emits every persisted field.

limit <= 0 defaults to 1000, not unbounded: this is the endpoint that fell
over, and an unbounded default would preserve the failure mode under a new
name. Callers page explicitly.

Registered in codegen.el's builtin_arity (both plain and __ spellings) and
wrapped in el_seed.c per the project's C-builtin recipe.
2026-08-15 20:10:48 -05:00
will.anderson 7557ea6e19 Merge pull request 'runtime: restore engram_recall_json + cgi_* accessors (unblocks the soul build)' (#126) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 10m15s
2026-08-16 00:57:11 +00:00
bigmerge 7351fb0a8d runtime: restore engram_recall_json + cgi_* accessors
El SDK CI - dev / build-and-test (pull_request) Failing after 10m24s
neuron's soul calls engram_recall_json (neuron-api.el:618, memory.el:80) and
cgi_principal (studio.el:72). Both existed in the runtime neuron vendored
(v1.0.0-20260501) and were absent here, so the soul could not link against
current el at all.

The dangerous part is what the obvious "fix" would have done. These look like
redundant wrappers over one impl:

    engram_search_json(q, limit)  -> eg_search_json_impl(q, limit, 0)  LEXICAL
    engram_recall_json(q, limit)  -> eg_search_json_impl(q, limit, 1)  SEMANTIC

They are not interchangeable, and the split is documented at neuron-api.el:613:
search stays LEXICAL because ~40 internal call sites pass a KEY and seven of
them DELETE every record returned. Point those at a semantic matcher and they
delete fuzzy matches. Conversely, pointing recall at search silently downgrades
the mind's entire retrieval surface from semantic to lexical — no error, just
permanently worse recall.

Implemented over engram_activate(), which in this runtime already IS the
semantic path the old with_legs=1 branch built by hand (embeds the query via
eg_embed_fetch, scores by cosine, then spreads activation one hop). Output
shape matches engram_search_json — a flat array via engram_emit_node_json —
because callers parse search's shape, not activate's envelope.

Verified: neuron's soul now compiles and links against current el, boots, and
serves /health with layers initialized.

NOTE for follow-up: current el also ships engram_retrieve_geometric_json, a
structure-first retrieval that appears to be the intended successor to recall.
Repointing the two recall call sites at it may well be the right end state and
would remove the two-wrapper shape entirely — but that is a behavioral change
that must be measured against neuron/tools/retrieval-eval/'s gold set, not
assumed. This commit preserves existing behavior exactly; it does not decide
that question.
2026-08-15 19:56:43 -05:00
will.anderson d545b69614 Merge pull request 'runtime: restore the three builtins that made elc unrebuildable' (#125) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 11m4s
2026-08-16 00:50:43 +00:00
bigmerge 598915cc61 runtime: restore the three builtins that made elc unrebuildable
El SDK CI - dev / build-and-test (pull_request) Failing after 3m52s
The committed elc binary could not be refreshed from its own source. Rebuilding
failed with three implicit-declaration errors: el_mem_check, stdout_to_file,
stdout_restore. The compiler's own source calls all three (compiler.el:472,479,574
and codegen.el:4248) and two are registered in codegen.el's builtin_arity table —
but none were defined in this runtime.

They were found intact in ui/examples/native-hello-ios/NativeHello/el_runtime.c,
a divergent private copy of this runtime that still carried them. Ported verbatim.

Consequence of them being missing: the canonical elc binary was frozen. Source
gained @route dispatch codegen (emit_route_dispatch, codegen.el:3948) and the
@manager boundary-beat seam, but no rebuilt binary could carry them, so
neuron's soul — whose routes.el now calls the compiler-synthesized
el_route_dispatch — could not be built at all.

Verified after the fix:
  - elc rebuilds from current source, clean.
  - Self-hosting fixpoint byte-identical (stage3 == stage2).
  - The rebuilt elc emits el_route_dispatch (2 occurrences in the soul amalgam,
    previously 0) and injects engram_boundary_beat at @manager boundaries,
    i.e. the decorator seam is live rather than inert.

el_mem_check is itself the compiler's memory guard (ELC_MAX_MEM_MB, default
512MB, self-terminates before the OS OOM-killer fires) — so the runtime was
missing the very guard that would have surfaced the compiler's memory blowup
as a clean error instead of a 27GB host-killer.
2026-08-15 19:50:17 -05:00
will.anderson dab14f9100 Merge pull request 'engram: fix silently-wrong query params + make el_seed.o/el_runtime.o link' (#124) from fix/engram-query-param-and-seed-link into dev
El SDK CI - dev / build-and-test (push) Failing after 14m30s
2026-08-16 00:34:33 +00:00
25 changed files with 2508 additions and 31 deletions
+630
View File
@@ -0,0 +1,630 @@
# El Test Framework — Design
**Status:** draft for review
**Author:** Neuron
**Date:** 2026-08-15
**Worktree:** `/Users/will/Development/neuron-technologies/el-worktrees/elc-memory-investigation`
---
## 0. The forcing requirement
We have a confirmed quadratic in `elc`. Peak memory in the old shipped binary and wall-clock in
the current source both grow as O(input²). We cannot fix it, because we cannot test it.
Everything in this document is downstream of one sentence: **a test framework must be able to fail
a build when an operation's growth curve degrades from linear to quadratic.**
That is not a nice-to-have bolted onto a correctness framework. It is the requirement that
determines the architecture. Correctness testing is the easy half.
Second-order requirement, learned the hard way tonight: **the framework must report per-test timing
by default.** The current framework prints `N passed, M failed` and nothing else. That is why a
3.58-second test file sat in the suite unnoticed. A framework that is structurally blind to time
cannot surface the defect class we most need to catch.
---
## 1. What exists today, measured
### 1.1 Two competing systems, neither complete
**System A — `lang/runtime/test.el`.** Manual registration, El-level.
**System B — the compiler's `test { }` block + `elc --test`.** Emits its own harness `main()`
with `__el_pass` / `__el_fail` globals (`codegen.el:3777-3796`).
They do not share a result model. Neither has timing. Both are in the tree.
### 1.2 Specific defects in System A
| Defect | Location | Consequence |
|---|---|---|
| All state as JSON strings in a global string-keyed map | `test.el` throughout | every assertion is `state_get``str_to_int``int_to_str``state_set` |
| Failure list appended by string slice + concat | `_test_json_append` | O(n²) in failure count |
| One OS thread spawned per test | `_test_run_one` via `__thread_create`/`__thread_join` | thread spawn per test, purely to get dispatch-by-name through dlsym |
| Manual registration pairing a string to a function name | `test_case(name, fn_name)` | typo ⇒ test silently never runs, suite still reports pass |
| Counters are assertion-level, global | `_test_pass_count` etc. | no per-test record exists at all |
| No timing, no structured output, no fixtures, no tags, no filtering, no parameterization, no benchmarks | — | — |
The registration defect is the serious one. It is not a slow framework, it is a framework that can
report success for tests that did not execute.
### 1.3 Measured cost structure
Per test file, current build model:
| Step | Time |
|---|---|
| `elc` compile `.el``.c` | 0.00s (small files) |
| **`cc` el_runtime.c → .o** | **0.14s** |
| `cc` test .c → .o | 0.02s |
| link | 0.02s |
> **STALE as of el #132 — re-measured 2026-08-16.** The `test_compiler` figure below was
> *entirely* the `strlen`-per-character quadratic, now fixed. Re-measured on the same host:
> **3.58s → 0.03s (119x)**, and the 422 KB compiler concatenation likewise compiles in 0.03s.
> The table is retained only as the historical record that motivated the gate. The remaining
> per-file cost is the redundant `el_runtime.c` rebuild, which §9's compile-once architecture
> addresses.
Per-file `elc` time across the existing suite:
| File | Bytes | elc time |
|---|---|---|
| `test_compiler` | 29,685 (+394 KB of imports) | **3.58s** |
| `string_test` | 18,545 | 0.01s |
| all other 9 files | 2.210 KB | 0.00s |
Two distinct defects in two distinct regimes:
1. **`test_compiler.el` imports all five compiler sources** — 394 KB in one translation unit. Its
3.58s is entirely the quadratic. It is the only file where the quadratic bites.
2. **Every other file's cost is 100% redundant `el_runtime.c` rebuilds** — 480 KB of identical C,
recompiled once per test file.
Neither is fixed by making the compiler faster. Both are fixed by the architecture below, and the
speedup is a by-product of building it correctly, not the goal.
### 1.4 The asset worth keeping
`codegen.el:3651-3652` already collects `test_names` / `test_c_names` — **the compiler already does
compile-time test discovery.** It then discards that registry into a hardcoded `main()`.
That registry is precisely the seam Go's `_testmain.go` and Rust's `test_main_static` are built on.
The mechanism we need is half-built and wired to the wrong thing.
---
## 2. Grounding — the common spine of excellent frameworks
Researched from primary sources: Go `testing`/`go test`, Rust `libtest`/Criterion, JUnit 5 Platform,
NUnit 3, JMH, Google Benchmark. Six invariants hold across all of them.
1. **A registry is built before execution**`(name, metadata, fn-ptr)` triples. Go generates it
from an AST scan; Rust synthesizes it in a compiler pass; JMH emits it as a build-time resource;
JUnit/NUnit build it reflectively. **Reflection is an implementation of the registry on runtimes
where it is cheap. It is never the architecture.**
2. **Discovery strictly precedes execution.** Every good capability — filtering, listing, counting,
sharding, IDE trees, re-run-failed-only, dry runs — is a consequence of this ordering.
3. **A hierarchy with stable, path-shaped unique IDs.** `TestFoo/subcase_2`. Selection is regex over
that path, one pattern per level.
4. **The framework is a prebuilt library; only the entry point is generated.** "Compile once, link
many" is always: framework archive compiled once + a small generated table + one
`MainStart(deps, registry)` call. Nobody recompiles the harness per test file.
5. **Execution emits an event stream; reporters are downstream renderers.** Human text, NDJSON,
JUnit XML, TAP are all transforms of one event stream. Go's one architectural mistake is doing
this backwards — `test2json` parses human output, and has shipped bugs when user output contains
`--- PASS:`.
6. **A dependency-injection seam at the boundary.** Go's `testdeps.TestDeps` exists so `testing`
can avoid importing `regexp`, profilers, and coverage. The execution core knows nothing about
output formats.
---
## 3. Architecture
### 3.1 The seam
```
┌─────────────────────────────────────────────────────────────┐
│ user code: foo.el with test { } / bench { } blocks │
└───────────────────────────┬─────────────────────────────────┘
│ elc --test
┌─────────────────────────────────────────────────────────────┐
│ generated C (per suite, tiny): │
│ __el_test_fn_0 .. _N lowered test/bench bodies │
│ __el_registry[] static table: name/kind/file/ │
│ line/tags/sizes/expected-O │
│ __el_dispatch(i) generated switch → body │
│ main() { return el_test_main(argc, argv); } │
└───────────────────────────┬─────────────────────────────────┘
│ cc + link (registry only)
┌─────────────────────────────────────────────────────────────┐
│ libeltest.a — PREBUILT ONCE │
│ • el_runtime.o (the 480 KB, compiled once, ever) │
│ • eltest.o the runner, WRITTEN IN EL │
│ discovery view · filtering · execution · fixtures · │
│ timing · benchmark harness · curve fitting · reporters │
└─────────────────────────────────────────────────────────────┘
```
The framework is written in El, compiled to C once, archived. Per-suite compilation touches only
the generated registry. This is Go's model, and it is strictly better for us than Go's because we
own the compiler and already have the AST — no separate source-scanning pass is needed.
### 3.2 Why the runner is in El and the registry is in C
El has no closures and no first-class function pointers. The registry must therefore hold C function
pointers, and it is generated C.
The runner stays in El and reaches the registry through a small builtin surface — indices, not
pointers:
```
__el_reg_count() -> Int
__el_reg_name(i) -> String
__el_reg_file(i) -> String
__el_reg_line(i) -> Int
__el_reg_kind(i) -> Int // 0=test 1=bench
__el_reg_tags(i) -> Int
__el_reg_sizes(i) -> String // JSON array, empty for tests
__el_reg_expect(i) -> Int // complexity class enum, 0 = none
__el_reg_invoke(i) -> Int // runs the body via the generated switch
```
Nine builtins. Everything else — filtering, lifecycle, statistics, curve fitting, all reporters —
is El. That satisfies "written in El" without pretending El can do something it cannot.
### 3.3 Result model
The unit is a **result record**, not a counter:
```
TestResult {
id String // slash path: "parser/handles_empty_input/case_3"
file String
line Int
status Status // Pass | Fail | Error | Skip
duration Int // nanoseconds, ALWAYS populated
message String // assertion detail: expected vs actual
output String // captured stdout/stderr for this test
assertions Int
}
```
`Fail` = an assertion failed. `Error` = unexpected crash/abort. This distinction is load-bearing —
every CI consumer depends on it, and the JUnit XML schema encodes it as distinct elements.
---
## 4. Authoring surface
### 4.1 Tests
`test { }` already exists. Keep it. Add subtests and hierarchy:
```el
test "parser/empty input" {
assert_that(parse(""), is_err())
}
test "parser/table" {
for case in [["", 0], ["a", 1], ["a b", 2]] {
subtest(case[0]) {
assert_that(token_count(case[0]), equals(case[1]))
}
}
}
```
Subtest IDs compose as `parser/table/a_b`. Filtering is `--run 'parser/table/.*'`, one regex per
path segment, exactly as Go does.
**We do not build a parameterized-test annotation system.** Table-driven loops plus subtests subsume
`@ParameterizedTest`, `@MethodSource`, `@CsvSource`, and `TestCaseSource` entirely, at zero framework
surface. This is Go's single biggest ergonomic win over JUnit and NUnit.
### 4.2 Fixtures
Per-file and per-test only, plus a LIFO cleanup stack:
```el
setup_all { ... } // once per suite
setup { ... } // before each test
teardown { ... } // after each test
teardown_all { ... }
```
and inside a test, `cleanup { ... }` registering LIFO-ordered teardown.
**We do not build JUnit 5's extension SPI** — seventeen callback interfaces, hierarchical stores,
registration ordering rules. That complexity is the price of retrofitting a plugin ecosystem onto a
twenty-year-old reflective framework. Go's `t.Cleanup` covers roughly 90% of what `@AfterEach` is
used for at a fraction of the surface.
### 4.3 Assertions — constraint model
One entry point, composable constraint values (NUnit's model, which avoids the N² overload
explosion):
```el
assert_that(actual, equals(expected))
assert_that(xs, has_length(3))
assert_that(s, contains("foo").and(starts_with("bar")))
assert_that(f, is_within(0.01).of(3.14))
```
A constraint is a value with `apply_to(actual) -> ConstraintResult`, and the result knows how to
describe its own failure. Custom constraints are ordinary user types.
**Every failure message must name file, line, the expression text, and both values.** We capture
expression source text at compile time — we have the AST, so we can do this better than any
runtime-introspection framework.
Legacy `assert_true` / `assert_eq` / etc. stay as thin wrappers for migration.
---
## 5. Benchmarks
### 5.1 The loop
Adopt `b.Loop()`, not `b.N`. Go spent fifteen years on `b.N` before concluding `b.Loop` was right;
we skip that.
```el
bench "str_concat" {
let s = make_input(bench_n())
for bench_loop() {
black_box(str_concat(s, "x"))
}
}
```
Three properties that make this the correct choice for a C target:
1. **The timer auto-resets on first call**, so setup above the loop is excluded *by construction*
rather than by the author remembering `ResetTimer`.
2. **`N` is hidden**, so it cannot be misused.
3. **The harness owns the loop shape**, which lets us insert an optimization barrier the C compiler
cannot see through. `black_box(v)` lowers to `asm volatile("" :: "r"(&v) : "memory")`. Since we
emit a single translation unit, dead-code elimination of a benchmark body is a live hazard —
this is our version of JMH's `Blackhole` problem, solved in the harness rather than delegated to
the user.
### 5.2 Iteration scaling
Use Go's `predictN` heuristics verbatim. They are battle-tested and cheap:
```
n = goal_ns * prev_iters / prev_ns // multiply before divide — precision on sub-ns ops
n += n / 5 // 20% headroom, overshoot rather than re-loop
n = min(n, 100 * last) // never grow more than 100× per step
n = max(n, last + 1) // guarantee forward progress
n = min(n, 1_000_000_000) // hard ceiling
```
Report `n` rounded to 1/2/3/5 × 10ᵏ so runs are comparable.
### 5.3 Sampling
Criterion's shape, because it is correct near timer resolution:
- **Warmup**: iteration counts 1, 2, 4, 8… until cumulative time exceeds the warmup budget.
- **Measurement**: collect `sample_size` samples at iteration counts `[d, 2d, 3d, …, Nd]`.
- **Estimate**: slope of a linear regression of iteration-count vs elapsed time. The intercept
absorbs fixed overhead.
- **Time whole samples, never individual iterations.** This is the single most important detail —
it defeats timer-resolution error on nanosecond operations.
Outliers classified by modified Tukey (±1.5 IQR mild, ±3 IQR severe), **reported but retained**.
---
## 6. Complexity gating — the centerpiece
This is the part that makes the quadratic fixable, and the part nobody in the mainstream has
finished. Google Benchmark's `Complexity()` fits the curve and *reports* it. We declare it and
**gate** on it.
### 6.1 Surface
```el
bench "elc_compile" over n in [16, 32, 64, 128, 256, 512, 1024] expect O(n) {
let src = synth_source(bench_n())
for bench_loop() { black_box(compile(src)) }
}
```
Alternative with no new syntax, if the parser change is judged too invasive — `bench_sizes([...])`
and `bench_expect("O(n)")` as calls inside the block. **Recommendation: declarative.** Runtime calls
mean `--list` cannot show the invariant without executing, which breaks the discovery-precedes-
execution invariant from §2.
### 6.2 Fitting
Per Google Benchmark `src/complexity.cc`. For candidate curves
`{O(1), O(log n), O(n), O(n log n), O(n²), O(n³)}`, one-parameter least squares, no intercept:
```
coef = Σ(tᵢ · gᵢ) / Σ(gᵢ²)
rms = sqrt( Σ(tᵢ coef·gᵢ)² / k ) / mean(t) // normalized
```
Best fit = lowest normalized RMS. User-supplied lambda curves also supported.
### 6.3 Gate logic
1. **FAIL** if the best-fit curve is strictly worse than declared, ordering
`O(1) < O(log n) < O(n) < O(n log n) < O(n²) < O(n³)`. Print the fitted coefficient and the full
per-size table.
2. **FAIL** if the declared curve's normalized RMS exceeds a threshold (start at 0.10). This catches
the case where *no* candidate fits — noise, a cache cliff, or a phase change. Report
`INDETERMINATE` honestly rather than gating on garbage.
3. **WARN** if the best fit is strictly better than declared — either an optimization landed and the
annotation should tighten, or the sweep is too narrow to expose real behaviour.
4. **REFUSE to gate** on fewer than 5 distinct sizes spanning under 2 decades, geometrically spaced.
Say so loudly rather than producing a meaningless fit.
### 6.4 Why gate on the exponent, not wall-clock
- **Machine-independent.** The fitted exponent is a property of the algorithm; the coefficient is a
property of the machine. Gating on the exponent makes CI hardware heterogeneity, noisy neighbours,
and thermal throttling irrelevant — they scale `coef`, not `g`.
- **No stored baseline.** No artifact storage, no golden-file drift. The invariant lives in the
source next to the code and is reviewed in the same PR.
- **It catches the failure mode that actually ships.** An O(n) lookup inside an O(n) loop is
invisible at n=100 in a unit test and catastrophic at n=100,000 in production. Constant-factor
regressions are annoying. Complexity regressions are outages. Ours was a 27 GB outage.
### 6.5 The deterministic gate — the one that would have caught us
Wall-clock needs statistics. **Allocation counts do not.** They are perfectly deterministic.
> **Correction, 2026-08-16 — count alone is NOT sufficient. Gate on BOTH count and bytes.**
>
> Measured against two El programs, one allocating once per item and one rebuilding its
> accumulator each iteration:
>
> | n | linear allocs / bytes | quadratic allocs / bytes |
> |---|---|---|
> | 100 | 100 / 290 | 100 / 5,150 |
> | 200 | 200 / 690 | 200 / 20,300 |
> | 400 | 400 / 1,490 | 400 / 80,600 |
> | 800 | 800 / 3,090 | 800 / 321,200 |
>
> The quadratic program's allocation **count is exactly linear** — 100/200/400/800, identical to
> the healthy program. A count-only gate passes it clean. **Bytes** catch it: each doubling of n
> quadruples bytes (ratios 3.94, 3.97, 3.99 → 4.0 = O(n²)) where the linear program converges
> on 2.0.
>
> This is precisely elc's own defect shape — a copy-on-write accumulator reallocating once per
> pass (count linear) into a proportionally larger buffer (bytes quadratic).
>
> Therefore `expect allocs O(n)` **fits count and bytes independently and fails if EITHER exceeds
> the declared curve**, reporting which signal broke. "count linear, bytes quadratic" is a precise,
> directly actionable diagnosis.
>
> **`el_peak_rss()` is CONTEXT ONLY — never gate on it.** It is perturbed by the allocator and by
> the page cache. Allocation volume is the invariant; RSS and malloc/free churn are merely the two
> surfaces it shows on. The old shipped compiler paid the same quadratic in RSS that the rebuilt
> one pays in churn.
>
> **Measure rate, not level.** A guard reading swap *level* saw 97% on a thrashing host and 97% on
> a healthy one; only *rate* separated them. A growth exponent is a rate; a single measurement is
> a level. That is why the gate fits a curve across a sweep instead of comparing one number to a
> threshold.
> **Second correction, same day — THE ALLOCATION GATE ALONE WOULD HAVE MISSED THE REAL BUG.**
>
> el #132 found the actual elc quadratic: `strlen()` called inside `str_char_code()` and
> `str_slice()`, so the lexer rescanned the remaining input on every character. Pure CPU.
> **Zero allocation.** `str_char_code` is a bounds check and an index — it allocates nothing.
>
> Measured on three controlled specimens (`lang/.work/fitprobe.el`), growth ratio per doubling of
> n across n = 200/400/800/1600:
>
> | specimen | allocs | bytes | time | what it proves |
> |---|---|---|---|---|
> | `linear` — one alloc per item | 2.00 2.00 2.00 → **O(n)** | 2.16 2.07 2.23 → **O(n)** | 0.83 2.00 2.05 → **O(n)** | clean baseline |
> | `accum` — rebuilds accumulator | 2.00 2.00 2.00 → **O(n)** | 3.97 3.99 3.99 → **O(n²)** | noisy | count misses, **bytes catches** |
> | `compute` — n scans over n chars | 0 → **FLAT** | 0 → **FLAT** | 3.93 4.01 3.96 → **O(n²)** | **both alloc signals blind; only time catches** |
>
> `compute` is el #132's shape exactly. A gate fitting only allocation count and bytes classifies
> it as FLAT and passes it. **The gate as originally specified would not have caught the defect it
> was created for.**
>
> Therefore the gate fits **THREE** signals and fails if ANY exceeds its declared curve:
>
> ```
> bench "elc_compile" over n in [...] expect time O(n) allocs O(n) bytes O(n) { ... }
> ```
>
> - **allocs (count)** — deterministic, zero-noise. Catches per-item allocation growth.
> - **allocs (bytes)** — deterministic, zero-noise. Catches accumulator-rebuild quadratics that
> count cannot see.
> - **time** — noisy, needs the sweep and statistics. The ONLY signal that sees pure-compute
> complexity regressions. Gate on the fitted *exponent*, never on absolute duration, so CI
> hardware variance scales the coefficient and leaves the classification intact.
>
> The deterministic signals remain preferable where they apply — they need no statistics and are
> correct on the first run. They are simply not sufficient.
>
> **`black_box` is mandatory, and consuming the result is NOT enough.** The first version of
> `compute` accumulated `total + 1` in a nested loop and reported **0 µs at every n** while
> returning a numerically correct n². Clang recognised the idiom and closed the loop to a
> multiply. Feeding the result into output did not prevent it. Only making the inner operation an
> opaque external call restored the real curve. A benchmark harness that trusts the user to defeat
> the optimiser will silently measure nothing — and report success while doing it.
Instrument the runtime with allocation counters and fit *those* against n instead of time:
```el
bench "elc_compile" over n in [...] expect O(n) allocs O(n) { ... }
```
Zero noise, zero statistics, always gateable, correct on the first run on any machine. Go reports
`allocs/op` and `B/op`; **nobody fits them against n.** That is an open opportunity and it is exactly
our bug: elc's defect is quadratic *allocation volume*, which the old binary paid in RSS and the
current source pays in malloc/free churn.
An `expect allocs O(n)` assertion on `elc`'s compile path would have failed the build the day the
quadratic was introduced.
Required runtime additions: `__el_alloc_count()`, `__el_alloc_bytes()`, `__el_peak_rss()`.
### 6.6 Constant-factor gate (secondary, opt-in)
Mann-Whitney U at α = 0.05, noise floor 1%, medians with 95% CIs, `~` for not-significant. Requires
`--count >= 9`. Off by default on CI; opt-in per benchmark.
**Exit nonzero on regression.** Both benchstat and Criterion always exit 0, which is why every shop
using them wrote a wrapper. We do not repeat that omission.
---
## 7. Output
**Structured events are the source of truth.** Human text is rendered from them. We do not repeat
Go's parse-the-human-output design.
Event stream, NDJSON, one object per line, streamed live:
```json
{"time":"...","action":"run","test":"parser/empty"}
{"time":"...","action":"output","test":"parser/empty","output":"..."}
{"time":"...","action":"pass","test":"parser/empty","elapsed":0.0031}
{"time":"...","action":"bench","test":"str_concat","n":1024,"ns_op":41.2,"allocs_op":3,"bigo":"N","rms":0.03}
```
Renderers, all downstream and pluggable:
| Format | Flag | Use |
|---|---|---|
| Human | default | terminal, **per-test duration always shown** |
| NDJSON | `--json` | tooling, history, flaky detection |
| JUnit XML | `--junit-xml=PATH` | every CI system on earth |
| TAP | `--tap` | optional |
JUnit XML per the de-facto schema: `testsuites``testsuite``testcase`, with `time` in seconds
as a decimal, `file`/`line` attributes, and `failure` vs `error` vs `skipped` as distinct child
elements. Absence of a child element means pass. Emit `<testsuites>` even for a single suite, and
parse both shapes on input.
---
## 8. CLI
```
--list print the registry, run nothing
--list-json machine-readable registry
--run PATTERN slash-separated regex per path segment
--tag EXPR tag expression: fast & !slow
--shard I/N deterministic sharding for CI parallelism
--count N repetitions, for statistics
--bench PATTERN run benchmarks (off by default in test runs)
--benchtime DUR per-benchmark time budget
--junit-xml PATH
--json
--isolate re-exec per test on crash, so one SIGSEGV doesn't lose the run
--timeout DUR
--fail-fast
```
`--list` / `--list-json` / `--shard` cost roughly thirty lines because the registry already exists
before `main` does anything. That is the dividend of discovery-precedes-execution.
---
## 9. Build model
```
# once, ever (or when the runtime/framework changes):
cc -c el_runtime.c -o el_runtime.o
elc eltest.el > eltest.c && cc -c eltest.c -o eltest.o
ar rcs libeltest.a el_runtime.o eltest.o
# per suite:
elc --test foo_test.el > foo_test.c # registry + bodies only
cc foo_test.c libeltest.a -o foo_test
```
The 0.14s × N of redundant runtime rebuilds disappears — not because we optimized it, but because
one-runner-over-many-suites requires compile-once-link-many as a structural precondition.
---
## 10. Bootstrap and self-hosting
The framework's own tests are `test { }` blocks run by the framework. Same fixpoint discipline the
compiler already applies to itself.
1. Build the framework using the *existing* harness for its first tests (stage 0).
2. Rebuild the framework's tests as `test { }` blocks run by the new runner (stage 1).
3. Verify stage 1 reports identical results to stage 0.
4. From then on, the framework is tested by itself.
A framework that cannot run its own suite is not evidence of anything. This is a correctness proof,
not a claim.
---
## 11. Explicitly not building
| Rejected | Why |
|---|---|
| Naming-convention discovery (`fn test_foo`) | `test { }` is a real declaration. Go's `TestXxx` exists only because Go had no better hook — and it needs a heuristic to avoid matching `TesticularCancer`. |
| Reflection or symbol-table scanning | Slow, fragile under LTO/strip/dead-strip, and unnecessary when we own the compiler. |
| Parsing human output into structure | Go's `test2json` is its one clear architectural mistake. |
| JUnit 5's extension SPI | Seventeen callback interfaces to retrofit plugins onto a reflective framework. Not our problem. |
| `@ParameterizedTest` machinery | Table-driven loops + subtests subsume it at zero surface. |
| NUnit's out-of-process agents | They bridge CLR versions and AppDomains. We emit one native binary. Keep `--isolate` as crash fallback only. |
| JMH-style forking by default | Forks exist because JIT profiles are per-process. AOT C has no such state. Keep `--fork` available, not default. |
| Exit 0 on regression | benchstat and Criterion both do this, and every user writes a wrapper. |
| Dynamic runtime test registration | Breaks `--list`, sharding, and individual selection. Registry stays static. |
---
## 12. Phasing
| Phase | Content | Gate |
|---|---|---|
| **1** | Registry emission in codegen; 9 builtins; `el_test_main` skeleton in El; result records; per-test timing; human + NDJSON output | existing 11 test files pass, with timing |
| **2** | `libeltest.a` build model; subtests; filtering; `--list`; fixtures; constraint assertions; JUnit XML | suite runs in one binary; runtime compiled once |
| **3** | `bench { }`, `bench_loop`, `black_box`, `predictN`, Criterion sampling | benchmarks produce stable ns/op |
| **4** | Allocation counters; complexity fitting; `expect O(...)` gate | **an `expect allocs O(n)` benchmark on `elc` fails on the current quadratic** |
| **5** | Migrate both legacy systems; delete `runtime/test.el`; self-host | framework runs its own suite |
Phase 4 is the deliverable that matters. Phases 13 exist to make it possible.
---
## 13. Open questions for review
1. **Declarative `over n in [...] expect O(...)` syntax vs runtime calls.** I recommend declarative
(§6.1) so `--list` can show invariants without executing. It costs parser work. Your call.
2. **`bench { }` as a new block form** — parallel to `test { }`, or a modifier on it?
3. **Scope of the constraint model.** Full composable constraints, or start with a flat assertion set
and add constraints later? Full model is more surface but avoids a second migration.
4. **Does `runtime/test.el` get deleted or kept as a deprecated shim?** I lean delete — two systems
is how we got here.
5. **Where does `libeltest.a` live** in the tree, and does `epm` need to know about it?
6. **Allocation counters in `el_seed.c` or `el_runtime.c`?** AGENTS.md says `el_seed.c` is the sole
C dependency and hand-maintained; counters are OS-boundary-adjacent but not OS calls.
7. **Is per-test timing enough, or do we want per-*assertion* timing** for finding slow helpers?
---
## 14. What this document is not
This is a design, not a measurement. Every performance claim about the *current* system in §1 is
measured and reproducible in this worktree. Every claim about the *proposed* system is a prediction.
None of it is verified until Phase 1 runs and Phase 4 fails a build on the real quadratic.
+149 -16
View File
@@ -862,10 +862,23 @@ fn cg_expr(expr: Map<String, Any>) -> String {
// arithmetic BinOp (or vice-versa). Without this check the // arithmetic BinOp (or vice-versa). Without this check the
// fallthrough to str_eq produces str_eq(int_value, int_value) // fallthrough to str_eq produces str_eq(int_value, int_value)
// which reads the integer as a char* and segfaults. // which reads the integer as a char* and segfaults.
// EITHER side provably Int is enough. Requiring BOTH meant a call
// whose return type codegen cannot infer poisoned the operator:
// getint(5) == a -> str_eq(getint(5), a)
// even with `a` declared Int. str_eq then reads an integer as a
// char* and segfaults. Only an integer LITERAL on one side forced
// the numeric form, so the bug was invisible in the common case.
//
// Loosening to OR is strictly safer: when one side is a known Int,
// str_eq is always wrong (it dereferences that int), while numeric
// comparison is at worst a wrong answer on an already ill-typed
// program. When neither side is Int nothing changes, so string
// comparison is untouched.
if is_int_expr(left) { if is_int_expr(left) {
if is_int_expr(right) {
return "(" + left_c + " == " + right_c + ")" return "(" + left_c + " == " + right_c + ")"
} }
if is_int_expr(right) {
return "(" + left_c + " == " + right_c + ")"
} }
// Float literal or negative float literal: use plain == (bit-equal // Float literal or negative float literal: use plain == (bit-equal
// el_val_t comparison). This handles `r0 == 3.0`, `neg == -3.0`, etc. // el_val_t comparison). This handles `r0 == 3.0`, `neg == -3.0`, etc.
@@ -921,10 +934,12 @@ fn cg_expr(expr: Map<String, Any>) -> String {
} }
// Same mixed Ident/BinOp fix as EqEq: use is_int_expr to detect // Same mixed Ident/BinOp fix as EqEq: use is_int_expr to detect
// integer-typed operands before falling through to !str_eq. // integer-typed operands before falling through to !str_eq.
// Either side Int is enough see the EqEq note above.
if is_int_expr(left) { if is_int_expr(left) {
if is_int_expr(right) {
return "(" + left_c + " != " + right_c + ")" return "(" + left_c + " != " + right_c + ")"
} }
if is_int_expr(right) {
return "(" + left_c + " != " + right_c + ")"
} }
// Float-typed operands use plain != (bit-equal comparison). // Float-typed operands use plain != (bit-equal comparison).
if is_float_expr(left) { if is_float_expr(left) {
@@ -1495,6 +1510,11 @@ fn cg_stmt(stmt: Map<String, Any>, indent: String, declared: [String]) -> [Strin
if str_eq(ltype, "Int") { if str_eq(ltype, "Int") {
add_int_name(name) add_int_name(name)
} }
// Same as params: Bool is an int in the value model. Without this a
// `let ok: Bool = ...` compared to another Bool lowered to str_eq.
if str_eq(ltype, "Bool") {
add_int_name(name)
}
if str_eq(ltype, "Float") { if str_eq(ltype, "Float") {
add_float_name(name) add_float_name(name)
} }
@@ -1705,9 +1725,13 @@ fn cg_stmt(stmt: Map<String, Any>, indent: String, declared: [String]) -> [Strin
} else { } else {
let c_msg = "EL_STR_PTR(" + cg_expr(msg_node) + ")" let c_msg = "EL_STR_PTR(" + cg_expr(msg_node) + ")"
} }
// Assertions record into PER-TEST state, not global counters. The test
// is the unit of result; a global pass/fail tally cannot say which test
// failed or whether a test ran at all. Reporting is the runner's job
// nothing is printed here.
emit_line(indent + "if (!(" + c_cond + ")) {") emit_line(indent + "if (!(" + c_cond + ")) {")
emit_line(indent + " __el_test_fail(__el_cur_test, " + c_msg + "); __el_fail++;") emit_line(indent + " __el_test_fail(" + c_msg + ");")
emit_line(indent + "} else { __el_pass++; }") emit_line(indent + "} else { __el_cur_asserts++; }")
return declared return declared
} }
@@ -2602,6 +2626,17 @@ fn builtin_arity(name: String) -> Int {
// LSP seed primitives // LSP seed primitives
if str_eq(name, "__read_n") { return 1 } if str_eq(name, "__read_n") { return 1 }
if str_eq(name, "__print_raw") { return 1 } if str_eq(name, "__print_raw") { return 1 }
// Test-registry accessors. These are not runtime builtins they are
// GENERATED into the same translation unit by the --test path below, one
// set per test binary. They are declared here so the El-side runner in
// runtime/eltest.el can call them with a known arity.
if str_eq(name, "__el_reg_count") { return 0 }
if str_eq(name, "__el_reg_name") { return 1 }
if str_eq(name, "__el_reg_invoke") { return 1 }
if str_eq(name, "__el_reg_last_ns") { return 0 }
if str_eq(name, "__el_reg_msg") { return 0 }
if str_eq(name, "__el_reg_asserts") { return 0 }
if str_eq(name, "__el_opt_json") { return 0 }
// String // String
if str_eq(name, "el_str_concat") { return 2 } if str_eq(name, "el_str_concat") { return 2 }
if str_eq(name, "str_eq") { return 2 } if str_eq(name, "str_eq") { return 2 }
@@ -2764,6 +2799,11 @@ fn builtin_arity(name: String) -> Int {
if str_eq(name, "__engram_node_full_in") { return 9 } if str_eq(name, "__engram_node_full_in") { return 9 }
if str_eq(name, "__engram_connect_in") { return 5 } if str_eq(name, "__engram_connect_in") { return 5 }
if str_eq(name, "__engram_scan_nodes_json") { return 2 } if str_eq(name, "__engram_scan_nodes_json") { return 2 }
if str_eq(name, "__engram_edges_json") { return 2 }
if str_eq(name, "__engram_pool_stats_json") { return 0 }
if str_eq(name, "__el_alloc_count") { return 0 }
if str_eq(name, "__el_alloc_bytes") { return 0 }
if str_eq(name, "__el_peak_rss") { return 0 }
if str_eq(name, "__generate") { return 1 } if str_eq(name, "__generate") { return 1 }
// Filesystem // Filesystem
if str_eq(name, "fs_read") { return 1 } if str_eq(name, "fs_read") { return 1 }
@@ -2862,6 +2902,12 @@ fn builtin_arity(name: String) -> Int {
if str_eq(name, "engram_get_node_by_label") { return 1 } if str_eq(name, "engram_get_node_by_label") { return 1 }
if str_eq(name, "engram_search_json") { return 2 } if str_eq(name, "engram_search_json") { return 2 }
if str_eq(name, "engram_scan_nodes_json") { return 2 } if str_eq(name, "engram_scan_nodes_json") { return 2 }
if str_eq(name, "engram_edges_json") { return 2 }
if str_eq(name, "engram_pool_stats_json") { return 0 }
if str_eq(name, "el_alloc_count") { return 0 }
if str_eq(name, "el_alloc_bytes") { return 0 }
if str_eq(name, "el_peak_rss") { return 0 }
if str_eq(name, "el_black_box") { return 1 }
if str_eq(name, "engram_neighbors_json") { return 3 } if str_eq(name, "engram_neighbors_json") { return 3 }
if str_eq(name, "engram_activate_json") { return 2 } if str_eq(name, "engram_activate_json") { return 2 }
if str_eq(name, "engram_stats_json") { return 0 } if str_eq(name, "engram_stats_json") { return 0 }
@@ -3087,6 +3133,15 @@ fn build_int_names_for_params(params: [Map<String, Any>]) -> Bool {
if str_eq(ptype, "Int") { if str_eq(ptype, "Int") {
add_int_name(pname) add_int_name(pname)
} }
// Bool is an integer in the value model (type_to_c maps Bool -> "int";
// el_runtime.h: "Bool -> el_val_t (0 = false, nonzero = true)"), but
// Bool names were registered nowhere. So `cond == want` between two
// Bool params fell through to str_eq and dereferenced 0 or 1 as a
// char* an immediate segfault. Track them as int-like, which is what
// they are.
if str_eq(ptype, "Bool") {
add_int_name(pname)
}
if str_eq(ptype, "Float") { if str_eq(ptype, "Float") {
add_float_name(pname) add_float_name(pname)
} }
@@ -4106,12 +4161,35 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
// Emit test harness preamble (counters, fail printer) when in test mode. // Emit test harness preamble (counters, fail printer) when in test mode.
if test_is_mode { if test_is_mode {
emit_line("#include <stdio.h>") emit_line("#include <stdio.h>")
emit_line("#include <string.h>")
emit_line("#include <time.h>")
emit_blank() emit_blank()
emit_line("static int __el_pass = 0, __el_fail = 0;") // Per-test result state. Reset by __el_reg_invoke before each test, so
// every test gets its own record rather than contributing to a global
// tally. The first failure message is retained; later ones only bump
// the count, which keeps the common case allocation-free.
emit_line("static int __el_cur_fails = 0;")
emit_line("static int __el_cur_asserts = 0;")
emit_line("static char __el_cur_msg[512] = \"\";")
emit_line("static const char *__el_cur_test = \"(none)\";") emit_line("static const char *__el_cur_test = \"(none)\";")
emit_line("static void __el_test_fail(const char *test, const char *msg) {") emit_line("static void __el_test_fail(const char *msg) {")
emit_line(" fprintf(stderr, \"FAIL %-40s %s\\n\", test, msg);") emit_line(" if (__el_cur_fails == 0 && msg) {")
emit_line(" snprintf(__el_cur_msg, sizeof __el_cur_msg, \"%s\", msg);")
emit_line(" }") emit_line(" }")
emit_line(" __el_cur_fails++; __el_cur_asserts++;")
emit_line("}")
emit_blank()
// Forward declarations for the registry accessors. The definitions are
// emitted at the END of the unit (they reference the test functions,
// which do not exist yet at this point), but the El-side runner is
// compiled in between and calls them so it needs the prototypes here.
emit_line("el_val_t __el_reg_count(void);")
emit_line("el_val_t __el_reg_name(el_val_t i);")
emit_line("el_val_t __el_reg_invoke(el_val_t i);")
emit_line("el_val_t __el_reg_last_ns(void);")
emit_line("el_val_t __el_reg_msg(void);")
emit_line("el_val_t __el_reg_asserts(void);")
emit_line("el_val_t __el_opt_json(void);")
emit_blank() emit_blank()
} }
@@ -4308,17 +4386,72 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
el_release(sigs) el_release(sigs)
let test_arena_mark: Any = el_arena_push() let test_arena_mark: Any = el_arena_push()
let tn: Int = native_list_len(test_c_names)
// Generated test registry
// Discovery happens HERE, at compile time. The runner never searches
// for tests; it walks this table. That ordering — discovery strictly
// before execution is what makes --list, filtering, sharding and
// per-test reporting possible later, and it is why the old harness
// (which inlined direct calls into main) could not have any of them.
emit_line("typedef void (*__el_test_fp)(void);")
emit_line("typedef struct { const char *name; __el_test_fp fn; } __el_test_entry;")
emit_line("static const __el_test_entry __el_registry[] = {")
let ri: Int = 0
while ri < tn {
let r_name: String = native_list_get(test_names, ri)
let r_cfn: String = native_list_get(test_c_names, ri)
emit_line(" { \"" + c_escape(r_name) + "\", " + r_cfn + " },")
let ri = ri + 1
}
// Trailing sentinel keeps the array non-empty when a file declares no
// tests (a zero-length array is not valid C).
emit_line(" { 0, 0 }")
emit_line("};")
emit_line("static const int __el_registry_n = " + int_to_str(tn) + ";")
emit_blank()
emit_line("static long long __el_last_ns = 0;")
emit_line("static int __el_opt_json_v = 0;")
emit_blank()
// Index-based accessors
// El has no function pointers, so the runner works purely in indices.
// This is the whole seam between generated C and the El-side runner.
emit_line("el_val_t __el_reg_count(void) { return (el_val_t)(int64_t)__el_registry_n; }")
emit_line("el_val_t __el_reg_name(el_val_t i) {")
emit_line(" int64_t k = (int64_t)i;")
emit_line(" if (k < 0 || k >= __el_registry_n) return EL_STR(\"\");")
emit_line(" return EL_STR(__el_registry[k].name);")
emit_line("}")
// Timing is taken immediately around the call, in C, on the MONOTONIC
// clock never the wall clock, which can step backwards under NTP.
emit_line("el_val_t __el_reg_invoke(el_val_t i) {")
emit_line(" int64_t k = (int64_t)i;")
emit_line(" if (k < 0 || k >= __el_registry_n) return 0;")
emit_line(" __el_cur_fails = 0; __el_cur_asserts = 0; __el_cur_msg[0] = '\\0';")
emit_line(" __el_cur_test = __el_registry[k].name;")
emit_line(" struct timespec _t0, _t1;")
emit_line(" clock_gettime(CLOCK_MONOTONIC, &_t0);")
emit_line(" __el_registry[k].fn();")
emit_line(" clock_gettime(CLOCK_MONOTONIC, &_t1);")
emit_line(" __el_last_ns = (long long)(_t1.tv_sec - _t0.tv_sec) * 1000000000LL")
emit_line(" + (long long)(_t1.tv_nsec - _t0.tv_nsec);")
emit_line(" return (el_val_t)(int64_t)__el_cur_fails;")
emit_line("}")
emit_line("el_val_t __el_reg_last_ns(void) { return (el_val_t)(int64_t)__el_last_ns; }")
emit_line("el_val_t __el_reg_msg(void) { return EL_STR(__el_cur_msg); }")
emit_line("el_val_t __el_reg_asserts(void) { return (el_val_t)(int64_t)__el_cur_asserts; }")
emit_line("el_val_t __el_opt_json(void) { return (el_val_t)(int64_t)__el_opt_json_v; }")
emit_blank()
// main() delegates to the El-side runner. Everything above this line is
// generated glue; all reporting logic lives in runtime/eltest.el.
emit_line("int main(int _argc, char **_argv) {") emit_line("int main(int _argc, char **_argv) {")
emit_line(" el_runtime_init_args(_argc, _argv);") emit_line(" el_runtime_init_args(_argc, _argv);")
let ti: Int = 0 emit_line(" for (int _i = 1; _i < _argc; _i++) {")
let tn: Int = native_list_len(test_c_names) emit_line(" if (strcmp(_argv[_i], \"--json\") == 0) __el_opt_json_v = 1;")
while ti < tn { emit_line(" }")
let tc_name: String = native_list_get(test_c_names, ti) emit_line(" return (int)(int64_t)el_test_main();")
emit_line(" " + tc_name + "();")
let ti = ti + 1
}
emit_line(" printf(\"%d passed, %d failed\\n\", __el_pass, __el_fail);")
emit_line(" return __el_fail;")
emit_line("}") emit_line("}")
el_arena_pop(test_arena_mark) el_arena_pop(test_arena_mark)
el_release(test_names) el_release(test_names)
+16
View File
@@ -419,6 +419,22 @@ fn resolve_imports(src_path: String) -> String {
if !str_eq(already, "") { return "" } if !str_eq(already, "") { return "" }
state_set(seen_key, "1") state_set(seen_key, "1")
// A missing file must be a hard error, never an empty string.
//
// fs_read returns "" both for "file is empty" and "file does not exist", and
// this function used the value without distinguishing them. So a broken
// import path a typo, a moved file, a relative path resolved from the
// wrong working directory compiled CLEANLY: exit 0, empty stderr, and a
// program silently missing everything it imported. Observed 2026-08-15:
// eleven consecutive "successful" compiles that had included no runtime at
// all, and a wrong conclusion drawn from them before anyone noticed.
//
// Missing dependency, confident success. fs_exists separates the two cases,
// so a genuinely empty file still resolves to "" and is fine.
if !fs_exists(src_path) {
println("elc: cannot resolve import: " + src_path)
exit_program(1)
}
let source: String = fs_read(src_path) let source: String = fs_read(src_path)
let dir: String = dirname_of(src_path) let dir: String = dirname_of(src_path)
let lines: [String] = str_split(source, "\n") let lines: [String] = str_split(source, "\n")
+352 -5
View File
@@ -140,6 +140,45 @@ el_val_t el_arena_push(void) {
return (el_val_t)(int64_t)_tl_arena.count; return (el_val_t)(int64_t)_tl_arena.count;
} }
/* ── String-length cache ─────────────────────────────────────────────────────
*
* THE COMPILER'S QUADRATIC LIVED HERE. str_char_code and str_slice each called
* strlen() on every invocation. The lexer walks source one character at a time,
* so每 access rescanned the whole remaining input: O(n) per character over n
* characters = O(n^2). Measured on a geometric sweep of synthetic sources,
* wall-clock rose 3.0x, 3.0x, 4.0x, 4.14x per doubling converging on 4x, a
* textbook quadratic and a stack sample put 779 of 779 samples inside lex(),
* every one bottoming out in _platform_strlen.
*
* The fix is to remember the length instead of recomputing it. The subtlety is
* INVALIDATION: El strings are arena-allocated, so a freed pointer can be
* reused for a different string at the same address. A naive pointer-keyed
* cache would then hand back a stale length and read past the end of the new
* string trading a performance bug for a memory-safety one.
*
* So entries carry a generation. Anything that frees or mutates runtime strings
* bumps the generation, and a cache hit requires both the pointer AND the
* generation to match. Stale entries can never be believed; they simply miss
* and recompute.
* */
#define EL_SLC_SLOTS 8
typedef struct { const char* ptr; size_t len; uint64_t gen; } ElStrLenEnt;
static ElStrLenEnt _el_slc[EL_SLC_SLOTS];
static uint64_t _el_str_gen = 1;
/* Called by every path that frees or mutates a runtime string. */
void el_str_cache_flush(void) { _el_str_gen++; }
static size_t el_strlen_cached(const char* s) {
if (!s) return 0;
size_t slot = ((uintptr_t)s >> 4) & (EL_SLC_SLOTS - 1);
ElStrLenEnt* e = &_el_slc[slot];
if (e->ptr == s && e->gen == _el_str_gen) return e->len;
size_t n = strlen(s);
e->ptr = s; e->len = n; e->gen = _el_str_gen;
return n;
}
el_val_t el_arena_pop(el_val_t mark) { el_val_t el_arena_pop(el_val_t mark) {
size_t save = (size_t)(int64_t)mark; size_t save = (size_t)(int64_t)mark;
if (save > _tl_arena.count) save = 0; if (save > _tl_arena.count) save = 0;
@@ -152,24 +191,59 @@ el_val_t el_arena_pop(el_val_t mark) {
_tl_arena.count = save; _tl_arena.count = save;
if (_tl_arena_scope_depth > 0) _tl_arena_scope_depth--; if (_tl_arena_scope_depth > 0) _tl_arena_scope_depth--;
if (save == 0) _tl_arena_active = 0; if (save == 0) _tl_arena_active = 0;
el_str_cache_flush(); /* freed pointers may be reused — see cache note */
return 0; return 0;
} }
/* ── Allocation accounting ───────────────────────────────────────────────────
*
* Every string allocation in the runtime funnels through the four functions
* below, so counting here counts everything the language does.
*
* WHY THIS EXISTS: a growth-curve gate needs a signal that is DETERMINISTIC.
* Wall-clock needs statistics, warmup, and a quiet machine; it is noisy on
* shared CI and unusable as a hard build gate. Allocation COUNT has none of
* those problems the same input allocates the same number of times on every
* machine, every run. Fit allocations against input size and a complexity
* regression becomes a build failure with zero flake.
*
* This is not hypothetical. elc's known defect is quadratic ALLOCATION VOLUME.
* The old shipped binary paid it in RSS (27 GB, OOM); the rebuilt one pays the
* same quadratic in malloc/free churn (42s on a 1.4 MB input). The allocation
* count was the invariant across both RSS and wall-clock were just the two
* ways it surfaced. An `expect allocs O(n)` assertion on the compile path
* would have failed the build the day it was introduced.
*
* Peak RSS is exported too but is explicitly NOT the gating signal: it is
* perturbed by allocator behaviour, page cache, and the OS. Gate on counts,
* report RSS as context.
*
* Counters are plain unsigned longs, incremented on the allocating thread with
* no synchronisation: this is measurement, and a lock here would change the
* thing being measured. Under threads the count is approximate; for the
* single-threaded compile path it is exact.
* */
static unsigned long _el_alloc_count = 0;
static unsigned long _el_alloc_bytes = 0;
/* Persistent allocation — bypasses the arena (state_set, engram internals). */ /* Persistent allocation — bypasses the arena (state_set, engram internals). */
static char* el_strdup_persist(const char* s) { static char* el_strdup_persist(const char* s) {
if (!s) return strdup(""); if (!s) { _el_alloc_count++; _el_alloc_bytes += 1; return strdup(""); }
_el_alloc_count++; _el_alloc_bytes += strlen(s) + 1;
return strdup(s); return strdup(s);
} }
static char* el_strbuf_persist(size_t n) { static char* el_strbuf_persist(size_t n) {
char* p = malloc(n + 1); char* p = malloc(n + 1);
if (!p) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!p) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
p[0] = '\0'; p[0] = '\0';
_el_alloc_count++; _el_alloc_bytes += n + 1;
return p; return p;
} }
static char* el_strdup(const char* s) { static char* el_strdup(const char* s) {
if (!s) { char* p = strdup(""); el_arena_track(p); return p; } if (!s) { char* p = strdup(""); _el_alloc_count++; _el_alloc_bytes += 1; el_arena_track(p); return p; }
char* p = strdup(s); char* p = strdup(s);
_el_alloc_count++; _el_alloc_bytes += strlen(s) + 1;
el_arena_track(p); el_arena_track(p);
return p; return p;
} }
@@ -178,6 +252,7 @@ static char* el_strbuf(size_t n) {
char* p = malloc(n + 1); char* p = malloc(n + 1);
if (!p) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!p) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
p[0] = '\0'; p[0] = '\0';
_el_alloc_count++; _el_alloc_bytes += n + 1;
el_arena_track(p); el_arena_track(p);
return p; return p;
} }
@@ -274,7 +349,7 @@ el_val_t str_to_int(el_val_t sv) {
el_val_t str_slice(el_val_t sv, el_val_t start, el_val_t end) { el_val_t str_slice(el_val_t sv, el_val_t start, el_val_t end) {
const char* s = EL_CSTR(sv); const char* s = EL_CSTR(sv);
if (!s) return el_wrap_str(el_strdup("")); if (!s) return el_wrap_str(el_strdup(""));
int64_t len = (int64_t)strlen(s); int64_t len = (int64_t)el_strlen_cached(s);
if (start < 0) start = 0; if (start < 0) start = 0;
if (end > len) end = len; if (end > len) end = len;
if (start >= end) return el_wrap_str(el_strdup("")); if (start >= end) return el_wrap_str(el_strdup(""));
@@ -401,12 +476,14 @@ typedef struct {
static ElList* list_alloc(int64_t cap) { static ElList* list_alloc(int64_t cap) {
if (cap < 4) cap = 4; if (cap < 4) cap = 4;
ElList* lst = malloc(sizeof(ElList)); ElList* lst = malloc(sizeof(ElList));
_el_alloc_count++; _el_alloc_bytes += sizeof(ElList);
if (!lst) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!lst) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
lst->hdr.magic = EL_MAGIC_LIST; lst->hdr.magic = EL_MAGIC_LIST;
lst->hdr.refcount = 1; lst->hdr.refcount = 1;
lst->length = 0; lst->length = 0;
lst->capacity = cap; lst->capacity = cap;
lst->elems = malloc((size_t)cap * sizeof(el_val_t)); lst->elems = malloc((size_t)cap * sizeof(el_val_t));
_el_alloc_count++; _el_alloc_bytes += (size_t)cap * sizeof(el_val_t);
if (!lst->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!lst->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
return lst; return lst;
} }
@@ -456,6 +533,7 @@ el_val_t el_list_append(el_val_t listv, el_val_t elem) {
if (old->length >= old->capacity) { if (old->length >= old->capacity) {
int64_t new_cap = old->capacity > 0 ? old->capacity * 2 : 4; int64_t new_cap = old->capacity > 0 ? old->capacity * 2 : 4;
el_val_t* grown = realloc(old->elems, (size_t)new_cap * sizeof(el_val_t)); el_val_t* grown = realloc(old->elems, (size_t)new_cap * sizeof(el_val_t));
_el_alloc_count++; _el_alloc_bytes += (size_t)new_cap * sizeof(el_val_t);
if (!grown) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!grown) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
old->elems = grown; old->elems = grown;
old->capacity = new_cap; old->capacity = new_cap;
@@ -468,12 +546,14 @@ el_val_t el_list_append(el_val_t listv, el_val_t elem) {
int64_t new_cap = old->length + 1; int64_t new_cap = old->length + 1;
if (new_cap < 4) new_cap = 4; if (new_cap < 4) new_cap = 4;
ElList* fresh = malloc(sizeof(ElList)); ElList* fresh = malloc(sizeof(ElList));
_el_alloc_count++; _el_alloc_bytes += sizeof(ElList);
if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
fresh->hdr.magic = EL_MAGIC_LIST; fresh->hdr.magic = EL_MAGIC_LIST;
fresh->hdr.refcount = 1; fresh->hdr.refcount = 1;
fresh->length = old->length + 1; fresh->length = old->length + 1;
fresh->capacity = new_cap; fresh->capacity = new_cap;
fresh->elems = malloc((size_t)new_cap * sizeof(el_val_t)); fresh->elems = malloc((size_t)new_cap * sizeof(el_val_t));
_el_alloc_count++; _el_alloc_bytes += (size_t)new_cap * sizeof(el_val_t);
if (!fresh->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!fresh->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
if (old->length > 0) { if (old->length > 0) {
memcpy(fresh->elems, old->elems, (size_t)old->length * sizeof(el_val_t)); memcpy(fresh->elems, old->elems, (size_t)old->length * sizeof(el_val_t));
@@ -495,12 +575,14 @@ el_val_t el_list_clone(el_val_t listv) {
if (cap < old->length) cap = old->length; if (cap < old->length) cap = old->length;
if (cap < 4) cap = 4; if (cap < 4) cap = 4;
ElList* fresh = malloc(sizeof(ElList)); ElList* fresh = malloc(sizeof(ElList));
_el_alloc_count++; _el_alloc_bytes += sizeof(ElList);
if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
fresh->hdr.magic = EL_MAGIC_LIST; fresh->hdr.magic = EL_MAGIC_LIST;
fresh->hdr.refcount = 1; fresh->hdr.refcount = 1;
fresh->length = old->length; fresh->length = old->length;
fresh->capacity = cap; fresh->capacity = cap;
fresh->elems = malloc((size_t)cap * sizeof(el_val_t)); fresh->elems = malloc((size_t)cap * sizeof(el_val_t));
_el_alloc_count++; _el_alloc_bytes += (size_t)cap * sizeof(el_val_t);
if (!fresh->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!fresh->elems) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
if (old->length > 0) { if (old->length > 0) {
memcpy(fresh->elems, old->elems, (size_t)old->length * sizeof(el_val_t)); memcpy(fresh->elems, old->elems, (size_t)old->length * sizeof(el_val_t));
@@ -521,6 +603,7 @@ typedef struct {
static ElMap* map_alloc(int64_t cap) { static ElMap* map_alloc(int64_t cap) {
if (cap < 4) cap = 4; if (cap < 4) cap = 4;
ElMap* m = malloc(sizeof(ElMap)); ElMap* m = malloc(sizeof(ElMap));
_el_alloc_count++; _el_alloc_bytes += sizeof(ElMap);
if (!m) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!m) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
m->hdr.magic = EL_MAGIC_MAP; m->hdr.magic = EL_MAGIC_MAP;
m->hdr.refcount = 1; m->hdr.refcount = 1;
@@ -596,6 +679,7 @@ el_val_t el_map_set(el_val_t mapv, el_val_t keyv, el_val_t value) {
int64_t new_cap = m->count + 1; int64_t new_cap = m->count + 1;
if (new_cap < 4) new_cap = 4; if (new_cap < 4) new_cap = 4;
ElMap* fresh = malloc(sizeof(ElMap)); ElMap* fresh = malloc(sizeof(ElMap));
_el_alloc_count++; _el_alloc_bytes += sizeof(ElMap);
if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); } if (!fresh) { fputs("el_runtime: out of memory\n", stderr); exit(1); }
fresh->hdr.magic = EL_MAGIC_MAP; fresh->hdr.magic = EL_MAGIC_MAP;
fresh->hdr.refcount = 1; fresh->hdr.refcount = 1;
@@ -5165,7 +5249,12 @@ el_val_t str_to_float(el_val_t s) {
/* ── Math (Float-aware) ──────────────────────────────────────────────────── */ /* ── Math (Float-aware) ──────────────────────────────────────────────────── */
el_val_t math_sqrt(el_val_t f) { return el_from_float(sqrt(el_to_float(f))); } el_val_t math_sqrt(el_val_t f) { return el_from_float(sqrt(el_to_float(f))); }
el_val_t math_log(el_val_t f) { return el_from_float(log(el_to_float(f))); } /* base-10, matching runtime/math.el's documented contract ("math_log — base-10
* logarithm") and el_seed.c's __log_f. This returned NATURAL log, so math_log
* and math_ln were the same function: log10(100) gave 4.605 instead of 2.
* Caught by tests/native/test_math.el on the new framework's first run the
* assertion existed all along, the suite just had no way to report it. */
el_val_t math_log(el_val_t f) { return el_from_float(log10(el_to_float(f))); }
el_val_t math_ln(el_val_t f) { return el_from_float(log(el_to_float(f))); } el_val_t math_ln(el_val_t f) { return el_from_float(log(el_to_float(f))); }
el_val_t math_sin(el_val_t f) { return el_from_float(sin(el_to_float(f))); } el_val_t math_sin(el_val_t f) { return el_from_float(sin(el_to_float(f))); }
el_val_t math_cos(el_val_t f) { return el_from_float(cos(el_to_float(f))); } el_val_t math_cos(el_val_t f) { return el_from_float(cos(el_to_float(f))); }
@@ -5222,7 +5311,7 @@ el_val_t str_char_code(el_val_t s, el_val_t i) {
const char* str = EL_CSTR(s); const char* str = EL_CSTR(s);
int64_t idx = (int64_t)i; int64_t idx = (int64_t)i;
if (!str) return 0; if (!str) return 0;
int64_t n = (int64_t)strlen(str); int64_t n = (int64_t)el_strlen_cached(str);
if (idx < 0 || idx >= n) return 0; if (idx < 0 || idx >= n) return 0;
return (el_val_t)(unsigned char)str[idx]; return (el_val_t)(unsigned char)str[idx];
} }
@@ -18204,3 +18293,261 @@ el_val_t __http_do(el_val_t m, el_val_t u, el_val_t b, el_val_t h, el_val_t t) {
el_val_t __http_do_map(el_val_t m, el_val_t u, el_val_t b, el_val_t h, el_val_t t) { (void)m; (void)u; (void)b; (void)h; (void)t; return _no_curl_err(); } el_val_t __http_do_map(el_val_t m, el_val_t u, el_val_t b, el_val_t h, el_val_t t) { (void)m; (void)u; (void)b; (void)h; (void)t; return _no_curl_err(); }
el_val_t __http_do_map_to_file(el_val_t m, el_val_t u, el_val_t b, el_val_t h, el_val_t p) { (void)m; (void)u; (void)b; (void)h; (void)p; return _no_curl_err(); } el_val_t __http_do_map_to_file(el_val_t m, el_val_t u, el_val_t b, el_val_t h, el_val_t p) { (void)m; (void)u; (void)b; (void)h; (void)p; return _no_curl_err(); }
#endif /* !HAVE_CURL */ #endif /* !HAVE_CURL */
/* ── Compiler-support builtins ───────────────────────────────────────────────
* stdout_to_file / stdout_restore / el_mem_check are called by the El compiler's
* own source (compiler.el:472,479,574 and codegen.el:4248) and are registered in
* codegen.el's builtin_arity table, but were missing from this runtime so
* rebuilding elc from source failed with three implicit-declaration errors and
* the committed elc binary could never be refreshed. The definitions below are
* ported verbatim from ui/examples/native-hello-ios/NativeHello/el_runtime.c,
* a divergent private copy of this runtime that still carried them.
* */
#include <sys/resource.h>
static int _el_saved_stdout_fd = -1;
/* Redirect process stdout to a file; used by the compiler's JS post-processing
* pipeline to capture codegen output before piping it onward. */
el_val_t stdout_to_file(el_val_t pathv) {
const char* path = EL_CSTR(pathv);
if (!path) return (el_val_t)(int64_t)-1;
fflush(stdout);
_el_saved_stdout_fd = dup(STDOUT_FILENO);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0) return (el_val_t)(int64_t)-1;
dup2(fd, STDOUT_FILENO);
close(fd);
return (el_val_t)(int64_t)0;
}
el_val_t stdout_restore(void) {
if (_el_saved_stdout_fd >= 0) {
fflush(stdout);
dup2(_el_saved_stdout_fd, STDOUT_FILENO);
close(_el_saved_stdout_fd);
_el_saved_stdout_fd = -1;
}
return (el_val_t)(int64_t)0;
}
/* el_mem_check — self-terminating memory guard for long-running compiler runs.
* Called periodically by the compiler to catch runaway growth before the OS
* OOM-killer fires. Limit comes from ELC_MAX_MEM_MB (default 512 MB).
* macOS reports ru_maxrss in bytes, Linux in kilobytes; normalised to MB. */
el_val_t el_mem_check(void) {
long limit_mb = 512;
const char* env_val = getenv("ELC_MAX_MEM_MB");
if (env_val && *env_val) {
long v = atol(env_val);
if (v > 0) limit_mb = v;
}
struct rusage ru;
if (getrusage(RUSAGE_SELF, &ru) != 0) return 0; /* can't read — skip check */
long rss_mb;
#if defined(__APPLE__) || defined(__MACH__)
rss_mb = (long)(ru.ru_maxrss / (1024L * 1024L));
#else
rss_mb = (long)(ru.ru_maxrss / 1024L);
#endif
if (rss_mb >= limit_mb) {
fprintf(stderr, "elc: memory limit exceeded (%ldMB), aborting\n", limit_mb);
exit(1);
}
return 0;
}
/* ── engram_recall_json / cgi_* accessors — restored 2026-08-15 ──────────────
*
* These existed in the runtime neuron vendored (v1.0.0-20260501) and were lost
* when this runtime moved on, so a soul built against current el would fail to
* link and, worse, the naive "fix" of pointing recall at engram_search_json
* would have SILENTLY DOWNGRADED the mind's whole retrieval surface from
* semantic to lexical, with no error at any layer.
*
* The lexical/semantic split is a real safety boundary, not redundant naming
* (neuron-api.el:613 documents it): engram_search_json stays LEXICAL because
* ~40 internal call sites pass a KEY and seven of them DELETE every record
* returned making those semantic would delete fuzzy matches. recall is the
* SEMANTIC surface, used by the retrieval routes.
*
* The old implementation was eg_search_json_impl(q, limit, with_legs=1): embed
* the query, cosine over the corpus, then a graph leg from semantic seeds.
* In this runtime that is exactly what engram_activate() already does (it
* embeds via eg_embed_fetch, scores by cosine, then spreads activation), so
* recall delegates to it rather than re-deriving a second semantic path.
* Output shape matches engram_search_json a flat array of node objects via
* engram_emit_node_json because existing callers (memory.el:80,
* neuron-api.el:618) parse it as search's shape, not activate's envelope.
* */
el_val_t engram_recall_json(el_val_t query, el_val_t limit) {
int64_t lim = (int64_t)limit;
if (lim <= 0) lim = 100;
/* depth 1: the associative leg, one hop out from the semantic seeds. */
el_val_t lst = engram_activate(query, (el_val_t)(int64_t)1);
ElList* arr = (ElList*)(uintptr_t)lst;
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
int64_t emitted = 0;
if (arr) {
for (int64_t i = 0; i < arr->length && emitted < lim; i++) {
if (!arr->elems[i]) continue;
el_val_t node_map = el_map_get(arr->elems[i], EL_STR("node"));
el_val_t id_v = el_map_get(node_map, EL_STR("id"));
const char* id_s = EL_CSTR(id_v);
EngramNode* n = id_s ? engram_find_node(id_s) : NULL;
if (!n) continue;
if (emitted > 0) jb_putc(&b, ',');
engram_emit_node_json(&b, n, 0);
emitted++;
}
}
jb_putc(&b, ']');
return el_wrap_str(b.buf);
}
/* cgi_* — read-only identity accessors over the process-wide CGI registration
* set by cgi_register(). Read-only by design: there is no setter (studio.el:66). */
el_val_t cgi_principal(void) { return EL_STR(_el_cgi_principal ? _el_cgi_principal : ""); }
el_val_t cgi_network(void) { return EL_STR(_el_cgi_network ? _el_cgi_network : ""); }
el_val_t cgi_engram(void) { return EL_STR(_el_cgi_engram ? _el_cgi_engram : ""); }
/* engram_edges_json(limit, offset) — emit edges straight from the store.
*
* Replaces a serialize-and-reread round trip that took production down on
* 2026-08-15: /api/graph/edges called engram_save() to write the ENTIRE graph
* to disk (128 MB) and then fs_read it back, just to answer a read query for
* edges. One debug request cost a full snapshot write, a 128 MB read, and the
* peak memory to hold it on top of being O(whole graph) for a bounded slice.
* The route's own comment had already named the fix: "Future: add an
* engram_edges_json() builtin and drop the file round trip entirely."
*
* limit <= 0 defaults to 1000 rather than unbounded: this is the endpoint that
* fell over, and an unbounded default would preserve the failure mode under a
* different name. Pass an explicit limit to page.
*/
el_val_t engram_edges_json(el_val_t limit, el_val_t offset) {
EngramStore* g = engram_get();
int64_t lim = (int64_t)limit; if (lim <= 0) lim = 1000;
int64_t off = (int64_t)offset; if (off < 0) off = 0;
JsonBuf b; jb_init(&b);
jb_putc(&b, '[');
int64_t emitted = 0;
char t[192];
for (int64_t i = off; i < g->edge_count && emitted < lim; i++) {
EngramEdge* e = &g->edges[i];
if (emitted > 0) jb_putc(&b, ',');
jb_puts(&b, "{\"id\":"); jb_emit_escaped(&b, e->id ? e->id : "");
jb_puts(&b, ",\"from_id\":"); jb_emit_escaped(&b, e->from_id ? e->from_id : "");
jb_puts(&b, ",\"to_id\":"); jb_emit_escaped(&b, e->to_id ? e->to_id : "");
jb_puts(&b, ",\"relation\":"); jb_emit_escaped(&b, e->relation ? e->relation : "");
snprintf(t, sizeof t,
",\"weight\":%.6g,\"hebb\":%.6g,\"confidence\":%.6g,"
"\"created_at\":%lld,\"updated_at\":%lld,\"last_fired\":%lld,"
"\"inhibitory\":%d,\"layer_id\":%u}",
e->weight, e->hebb, e->confidence,
(long long)e->created_at, (long long)e->updated_at,
(long long)e->last_fired, e->inhibitory, (unsigned)e->layer_id);
jb_puts(&b, t);
emitted++;
}
jb_putc(&b, ']');
return el_wrap_str(b.buf);
}
/* engram_pool_stats_json() — the buffer pool's interoception, exposed.
*
* StorePoolStats and store_pool_stats() already existed and were surfaced
* NOWHERE. On 2026-08-15 the engram thrashed itself to a standstill twice while
* these exact counters sat in memory, unread, and four wrong theories were tried
* from the outside instead. Sensing state is only corrective if the state can be
* read by the process itself (pc_adapt_budget) and by anything watching it.
*
* Serves the live numbers plus the derived signals that actually diagnose:
* hit_rate sustained low hit rate with high evictions is the thrash shape
* evict_ratio evictions per access; ~1 means every fetch displaces a live page
* pressure 1 when evicting into genuine reuse (working set > budget)
* cap_gib/resident_gib budget vs what is actually held
*/
el_val_t engram_pool_stats_json(void) {
if (!g_engram_store) return el_wrap_str(el_strdup("{\"store\":false}"));
StorePoolStats st;
store_pool_stats(g_engram_store, &st);
uint64_t acc = st.hits + st.misses;
double hit_rate = acc ? (double)st.hits / (double)acc : 0.0;
double evict_ratio = acc ? (double)st.evictions / (double)acc : 0.0;
int pressure = (acc > 100000 && evict_ratio > 0.33 && hit_rate > 0.25) ? 1 : 0;
char b[768];
snprintf(b, sizeof b,
"{\"store\":true,\"cap_frames\":%zu,\"resident_frames\":%zu,\"pinned\":%zu,"
"\"dirty\":%zu,\"prefetch\":%u,\"hits\":%llu,\"misses\":%llu,\"evictions\":%llu,"
"\"prefetch_reads\":%llu,\"hit_rate\":%.4f,\"evict_ratio\":%.4f,\"pressure\":%d,"
"\"cap_gib\":%.3f,\"resident_gib\":%.3f,\"page_size\":%u}",
st.cap, st.resident, st.pinned, st.dirty, st.prefetch,
(unsigned long long)st.hits, (unsigned long long)st.misses,
(unsigned long long)st.evictions, (unsigned long long)st.prefetch_reads,
hit_rate, evict_ratio, pressure,
(double)st.cap * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0),
(double)st.resident * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0),
(unsigned)STORE_PAGE_SIZE);
return el_wrap_str(el_strdup(b));
}
/* ── Allocation/RSS introspection (test-framework complexity gate, §6.5) ─────
*
* el_alloc_count() total runtime string allocations since process start.
* THE gating signal. Deterministic: same input => same count, every machine,
* every run. A benchmark harness samples it before and after an operation at
* several input sizes and fits the deltas against n; a curve worse than the
* declared one fails the build. No warmup, no statistics, no baseline file,
* no flake none of which is true of wall-clock.
*
* el_alloc_bytes() total bytes requested. Same determinism; catches the case
* where allocation COUNT stays linear but per-allocation SIZE grows, which is
* the classic accidental-quadratic shape (rebuilding a whole buffer per
* append). Count alone would miss it.
*
* el_peak_rss() peak resident set in bytes. Context, NOT a gate: perturbed by
* allocator internals, the page cache, and the OS. Reported so a human can
* see the physical consequence; never fitted.
*/
el_val_t el_alloc_count(void) { return (el_val_t)(int64_t)_el_alloc_count; }
el_val_t el_alloc_bytes(void) { return (el_val_t)(int64_t)_el_alloc_bytes; }
/* el_black_box — optimisation barrier for benchmark bodies.
*
* WHY THIS IS NOT OPTIONAL. A benchmark whose result is unused is dead code,
* and CONSUMING THE RESULT IS NOT SUFFICIENT: clang recognises loop idioms and
* closes them to arithmetic. A nested `total = total + 1` loop measured at
* 0 microseconds for every n while returning a numerically correct n*n --
* the answer was right and the work never happened.
*
* That is the same failure shape as a test that never ran reporting pass. The
* harness must own the barrier rather than trusting the benchmark author to
* defeat the optimiser.
*
* The constraint "+r" forces the value through a register the compiler must
* treat as both read and written by opaque code; the "memory" clobber stops
* loads and stores being reordered across it or elided. Emits no instructions. */
el_val_t el_black_box(el_val_t v) {
__asm__ __volatile__("" : "+r"(v) : : "memory");
return v;
}
el_val_t el_peak_rss(void) {
struct rusage ru;
if (getrusage(RUSAGE_SELF, &ru) != 0) return (el_val_t)0;
#if defined(__APPLE__) || defined(__MACH__)
return (el_val_t)(int64_t)ru.ru_maxrss; /* macOS: bytes */
#else
return (el_val_t)(int64_t)(ru.ru_maxrss * 1024L); /* Linux: KB -> bytes */
#endif
}
+29
View File
@@ -1011,6 +1011,35 @@ el_val_t __uuid_v4(void);
/* Args */ /* Args */
el_val_t __args_json(void); el_val_t __args_json(void);
/* Compiler-support builtins — called by the El compiler's own source
* (compiler.el, codegen.el) and registered in codegen.el's builtin_arity. */
el_val_t stdout_to_file(el_val_t path);
el_val_t stdout_restore(void);
el_val_t el_mem_check(void);
/* Allocation accounting — the deterministic signal behind complexity gating.
* Gate on counts/bytes; peak RSS is context only. */
el_val_t el_alloc_count(void);
el_val_t el_alloc_bytes(void);
el_val_t el_peak_rss(void);
el_val_t el_black_box(el_val_t v);
/* Semantic retrieval surface. NOT interchangeable with engram_search_json,
* which is lexical by design — see the note at the definition. */
el_val_t engram_recall_json(el_val_t query, el_val_t limit);
/* Edges straight from the store — replaces the engram_save()+fs_read()
* whole-graph round trip that /api/graph/edges used to do. */
el_val_t engram_edges_json(el_val_t limit, el_val_t offset);
/* Buffer-pool interoception as JSON — live pool health for observation. */
el_val_t engram_pool_stats_json(void);
/* CGI identity accessors (read-only). */
el_val_t cgi_principal(void);
el_val_t cgi_network(void);
el_val_t cgi_engram(void);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
+23
View File
@@ -148,10 +148,17 @@ static void seed_request_start(void) {
_seed_arena_on = 1; _seed_arena_on = 1;
} }
/* Defined in el_runtime.c. The string-length cache there keys on pointer +
* generation; anything that frees or mutates a runtime string must bump the
* generation or a reused address could return a stale length. Weak so this
* file still links on its own. */
__attribute__((weak)) void el_str_cache_flush(void);
static void seed_request_end(void) { static void seed_request_end(void) {
_seed_arena_on = 0; _seed_arena_on = 0;
for (size_t i = 0; i < _seed_arena.count; i++) free(_seed_arena.ptrs[i]); for (size_t i = 0; i < _seed_arena.count; i++) free(_seed_arena.ptrs[i]);
_seed_arena.count = 0; _seed_arena.count = 0;
if (el_str_cache_flush) el_str_cache_flush(); /* freed pointers may be reused */
} }
/* el_request_start / el_request_end — formerly defined in el_runtime.c. /* el_request_start / el_request_end — formerly defined in el_runtime.c.
@@ -213,6 +220,7 @@ el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c) {
int64_t idx = (int64_t)i; int64_t idx = (int64_t)i;
if (idx < 0 || idx >= len) return s; if (idx < 0 || idx >= len) return s;
p[idx] = (char)(unsigned char)(int64_t)c; p[idx] = (char)(unsigned char)(int64_t)c;
if (el_str_cache_flush) el_str_cache_flush(); /* in-place write can move the NUL */
return s; return s;
} }
@@ -1371,6 +1379,21 @@ el_val_t __engram_scan_nodes_json(el_val_t limit, el_val_t offset) {
return engram_scan_nodes_json(limit, offset); return engram_scan_nodes_json(limit, offset);
} }
el_val_t engram_edges_json(el_val_t limit, el_val_t offset);
el_val_t __engram_edges_json(el_val_t limit, el_val_t offset) {
return engram_edges_json(limit, offset);
}
el_val_t engram_pool_stats_json(void);
el_val_t __engram_pool_stats_json(void) { return engram_pool_stats_json(); }
el_val_t el_alloc_count(void);
el_val_t el_alloc_bytes(void);
el_val_t el_peak_rss(void);
el_val_t __el_alloc_count(void) { return el_alloc_count(); }
el_val_t __el_alloc_bytes(void) { return el_alloc_bytes(); }
el_val_t __el_peak_rss(void) { return el_peak_rss(); }
el_val_t __engram_scan_nodes_by_type_json(el_val_t node_type, el_val_t limit, el_val_t offset) { el_val_t __engram_scan_nodes_by_type_json(el_val_t node_type, el_val_t limit, el_val_t offset) {
return engram_scan_nodes_by_type_json(node_type, limit, offset); return engram_scan_nodes_by_type_json(node_type, limit, offset);
} }
+256
View File
@@ -0,0 +1,256 @@
// runtime/elbench.el growth-curve classifier and complexity gate.
//
// Given a geometric sweep of input sizes and the measurements taken at each,
// classify the growth curve and decide whether it violates a declared bound.
//
// Why this exists
//
// Constant-factor regressions are annoying. Complexity regressions are outages.
// An O(n) lookup inside an O(n) loop is invisible at n=100 in a unit test and
// catastrophic at n=100000 in production. el #132 was exactly that: a strlen()
// inside a per-character accessor, quadratic, shipped for months.
//
// THREE signals, not one
//
// The gate fits time AND allocation-count AND allocation-bytes, and fails if
// ANY of them exceeds its declared curve. This is not belt-and-braces; each
// signal is blind to a real defect class the others catch:
//
// * A copy-on-write accumulator rebuilding its buffer allocates ONCE per
// iteration count is exactly linear while bytes go quadratic.
// Count alone passes it.
// * el #132's strlen-per-character is pure CPU and allocates NOTHING.
// Both allocation signals read FLAT. Only time catches it.
//
// The deterministic signals (count, bytes) are preferable where they apply:
// no statistics, correct on the first run, machine-independent. They are
// simply not sufficient.
//
// SCOPE LIMIT read this before trusting a flat curve
//
// The allocation counters track EL-LEVEL allocation only: strings, ElList and
// ElMap bodies, their backing arrays, copy-on-write clones, and the realloc
// growth path. malloc inside engram_*.c and inside libcurl is NOT counted.
//
// A flat allocation curve over a workload dominated by engram or HTTP calls is
// therefore NOT evidence of anything. It means "no El-level allocation growth",
// not "no allocation growth". Gate El-level complexity with this; do not read
// third-party memory behaviour into it.
//
// Classification method
//
// Sizes must form a geometric sweep (each n double the last). On such a sweep
// the ratio between consecutive measurements IS the growth exponent, directly:
//
// O(1) -> 1.0 O(log n) -> ~1.1 O(n) -> 2.0
// O(n log n) -> ~2.2 O(n^2) -> 4.0 O(n^3) -> 8.0
//
// DEVIATION FROM DESIGN.md 6.2, stated plainly: that section specified Google
// Benchmark's one-parameter least-squares fit over candidate curves. This uses
// consecutive ratios instead. The sweep is mandated geometric either way, and
// on a geometric sweep ratios are directly interpretable and need no floating
// point. The cost is weaker separation between O(n) and O(n log n), which is
// reported honestly as an ambiguous band rather than guessed at. Least-squares
// remains the better answer if that band ever needs to be resolved.
//
// All arithmetic is fixed-point, scaled by 1000 ("milli-ratio"), so a ratio of
// 2.0 is 2000. El values are int64; this avoids float-in-list handling.
// Curve identifiers. Ordered by growth the ordering IS the comparison used
// by the gate, so an index comparison decides "worse than declared".
// 0 = O(1) 1 = O(log n) 2 = O(n) 3 = O(n log n) 4 = O(n^2) 5 = O(n^3)
fn elb_curve_name(c: Int) -> String {
if c == 0 { return "O(1)" }
if c == 1 { return "O(log n)" }
if c == 2 { return "O(n)" }
if c == 3 { return "O(n log n)" }
if c == 4 { return "O(n^2)" }
if c == 5 { return "O(n^3)" }
return "O(?)"
}
fn elb_curve_from_name(s: String) -> Int {
if str_eq(s, "O(1)") { return 0 }
if str_eq(s, "O(log n)") { return 1 }
if str_eq(s, "O(n)") { return 2 }
if str_eq(s, "O(n log n)") { return 3 }
if str_eq(s, "O(n^2)") { return 4 }
if str_eq(s, "O(n^3)") { return 5 }
return -1
}
// elb_classify_ratio map a milli-ratio-per-doubling onto a curve.
//
// Bands are deliberately wide at the top (a quadratic measured at 3.4x is
// still a quadratic) and deliberately overlap-averse at the bottom, where a
// misclassification between O(1) and O(log n) matters least.
fn elb_classify_ratio(milli: Int) -> Int {
if milli < 1300 { return 0 }
if milli < 1700 { return 1 }
if milli < 2400 { return 2 }
if milli < 3200 { return 3 }
if milli < 6000 { return 4 }
return 5
}
// elb_ratio milli-ratio between two consecutive measurements.
// Returns -1 when the earlier measurement is zero (ratio undefined).
fn elb_ratio(prev: Int, cur: Int) -> Int {
if prev <= 0 { return -1 }
return (cur * 1000) / prev
}
// The measurement floor
//
// A benchmark whose largest measurement is at or near zero has not been
// measured. Reporting it as O(1) would be a confident answer with nothing
// behind it the same failure as a test that never ran reporting pass, and
// exactly what happened when clang closed a nested loop to a multiply and the
// harness read 0 microseconds at every n.
//
// So: REFUSE. Never classify below the floor.
fn elb_below_floor(vals: [Int], floor: Int) -> Bool {
let n: Int = native_list_len(vals)
let i: Int = 0
let mx: Int = 0
while i < n {
let v: Int = native_list_get(vals, i)
if v > mx { let mx = v }
let i = i + 1
}
if mx < floor { return true }
return false
}
// elb_implausibly_flat a measurement that does not move across a sweep whose
// input grew by 8x or more is not a flat curve, it is a broken measurement.
// Genuine O(1) work still shows noise; a hard-flat series means the work was
// optimised away, the timer has insufficient resolution, or the benchmark body
// never executed.
fn elb_implausibly_flat(vals: [Int]) -> Bool {
let n: Int = native_list_len(vals)
if n < 3 { return false }
let first: Int = native_list_get(vals, 0)
let last: Int = native_list_get(vals, n - 1)
if first == 0 {
if last == 0 { return true }
return false
}
let r: Int = (last * 1000) / first
if r < 1100 { return true }
return false
}
// elb_spread_ok do the consecutive ratios agree with each other?
//
// This is the ratio-method analogue of a normalised-RMS threshold. If the
// doublings disagree wildly the data is noise, a cache cliff, or a phase
// change, and the honest report is INDETERMINATE rather than a classification.
// Applies to the ASYMPTOTIC TAIL only the last three ratios.
//
// The small-n end of any sweep is dominated by fixed overhead, cold caches and
// branch predictors that have not warmed. Measured on a genuinely linear
// character scan, the ratios ran 3.37, 2.92, 1.76, 1.65: the head looks
// quadratic, the tail is the truth. Checking spread across the whole sweep
// therefore rejects correct data. A complexity bound is an asymptotic claim, so
// it is judged on the asymptotic region the same reason a benchmark harness
// discards warmup rather than averaging it in.
fn elb_spread_ok(ratios: [Int]) -> Bool {
let total: Int = native_list_len(ratios)
if total < 2 { return true }
let start: Int = total - 3
if start < 0 { let start = 0 }
let n: Int = total
let lo: Int = 999999
let hi: Int = 0
let i: Int = start
while i < n {
let r: Int = native_list_get(ratios, i)
if r >= 0 {
if r < lo { let lo = r }
if r > hi { let hi = r }
}
let i = i + 1
}
if lo <= 0 { return false }
// Reject when the widest ratio is more than 2.2x the narrowest. That is
// enough slack for real timing noise and tight enough to separate a clean
// 2.0 series from a clean 4.0 series.
if (hi * 1000) / lo > 2200 { return false }
return true
}
// elb_ratios consecutive milli-ratios across the sweep.
fn elb_ratios(vals: [Int]) -> [Int] {
let out: [Int] = native_list_empty()
let n: Int = native_list_len(vals)
let i: Int = 1
while i < n {
let out = native_list_append(out,
elb_ratio(native_list_get(vals, i - 1), native_list_get(vals, i)))
let i = i + 1
}
return out
}
// elb_mean_tail_ratio mean of the LAST TWO ratios.
//
// The tail is used deliberately: asymptotic behaviour is what a complexity
// bound claims, and the small-n end of any sweep is dominated by fixed
// overhead. This is the same reason a benchmark harness discards warmup.
fn elb_mean_tail_ratio(ratios: [Int]) -> Int {
let n: Int = native_list_len(ratios)
if n == 0 { return -1 }
if n == 1 { return native_list_get(ratios, 0) }
let a: Int = native_list_get(ratios, n - 1)
let b: Int = native_list_get(ratios, n - 2)
if a < 0 { return b }
if b < 0 { return a }
return (a + b) / 2
}
// Verdicts
//
// 0 PASS measured curve is at or below the declared bound
// 1 FAIL measured curve is strictly worse than declared
// 2 INDETERMINATE ratios disagree; data is noise or a phase change
// 3 REFUSED below the measurement floor, or implausibly flat
// 4 BETTER measured strictly better than declared (warn, not fail)
fn elb_verdict_name(v: Int) -> String {
if v == 0 { return "PASS" }
if v == 1 { return "FAIL" }
if v == 2 { return "INDETERMINATE" }
if v == 3 { return "REFUSED" }
if v == 4 { return "BETTER" }
return "?"
}
// elb_gate classify one signal against its declared bound.
//
// vals measurements, one per sweep point, in sweep order
// expect declared curve index (see elb_curve_name)
// floor minimum largest-measurement below which we refuse to classify
fn elb_gate(vals: [Int], expect: Int, floor: Int) -> Int {
if elb_below_floor(vals, floor) { return 3 }
if elb_implausibly_flat(vals) { return 3 }
let ratios: [Int] = elb_ratios(vals)
if !elb_spread_ok(ratios) { return 2 }
let m: Int = elb_mean_tail_ratio(ratios)
if m < 0 { return 2 }
let got: Int = elb_classify_ratio(m)
if got > expect { return 1 }
if got < expect { return 4 }
return 0
}
// elb_measured_curve the classified curve for a signal, or -1 if unclassifiable.
fn elb_measured_curve(vals: [Int], floor: Int) -> Int {
if elb_below_floor(vals, floor) { return -1 }
if elb_implausibly_flat(vals) { return -1 }
let ratios: [Int] = elb_ratios(vals)
let m: Int = elb_mean_tail_ratio(ratios)
if m < 0 { return -1 }
return elb_classify_ratio(m)
}
+194
View File
@@ -0,0 +1,194 @@
// runtime/eltest.el El test framework runner (Phase 1).
//
// This is the RUNNER. It is written in El and consumes a registry that the
// compiler generates into the same translation unit when invoked as
// `elc --test`. Nothing here discovers tests; discovery already happened at
// compile time, which is what makes `--list` and filtering possible later.
//
// Architecture
//
// The compiler lowers each `test "name" { ... }` block into a static C
// function and emits a static table of (name, fn) pairs plus a small set of
// index-based accessors. El has no function pointers, so the runner never
// sees one it works entirely in indices:
//
// __el_reg_count() -> Int number of registered tests
// __el_reg_name(i) -> String test name at index i
// __el_reg_invoke(i) -> Int run test i, return its failure count
// __el_reg_last_ns() -> Int wall-clock ns of the last invoke
// __el_reg_msg() -> String first failure message of the last invoke
// __el_reg_asserts() -> Int assertions executed in the last invoke
// __el_opt_json() -> Int 1 if --json was passed
//
// Timing is taken in the generated C, immediately around the call, so no El
// call overhead lands inside the measurement.
//
// Output
//
// Structured events are the source of truth. The human renderer is written
// FROM the same fields the NDJSON renderer emits never the reverse. Parsing
// human output back into structure is the one clear architectural mistake in
// Go's test tooling and we do not repeat it.
//
// Every result carries a duration. Always. A framework that cannot report how
// long its tests took cannot surface a performance regression, and a
// regression nobody can see is one nobody fixes.
// Small helpers (no imports this file must stay self-contained)
// _elt_json_escape minimal JSON string escaping for the NDJSON renderer.
fn _elt_json_escape(s: String) -> String {
let out: String = ""
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let ch: String = str_slice(s, i, i + 1)
if str_eq(ch, "\"") {
let out = out + "\\\""
} else {
if str_eq(ch, "\\") {
let out = out + "\\\\"
} else {
if str_eq(ch, "\n") {
let out = out + "\\n"
} else {
if str_eq(ch, "\t") {
let out = out + "\\t"
} else {
if str_eq(ch, "\r") {
let out = out + "\\r"
} else {
let out = out + ch
}
}
}
}
}
let i = i + 1
}
return out
}
// _elt_pad3 left-pad an integer to three digits (for the ms.fraction form).
fn _elt_pad3(v: Int) -> String {
if v < 10 { return "00" + int_to_str(v) }
if v < 100 { return "0" + int_to_str(v) }
return int_to_str(v)
}
// _elt_ms render a nanosecond duration as "M.mmm" milliseconds.
//
// Deliberately avoids the modulo operator: the remainder is derived by
// subtraction so this stays portable across El backends.
fn _elt_ms(ns: Int) -> String {
let total_us: Int = ns / 1000
let ms_whole: Int = total_us / 1000
let us_rem: Int = total_us - (ms_whole * 1000)
return int_to_str(ms_whole) + "." + _elt_pad3(us_rem)
}
// _elt_secs render a nanosecond duration as fractional seconds, for the
// NDJSON `elapsed` field. JUnit XML and test2json both use seconds-as-decimal.
fn _elt_secs(ns: Int) -> String {
let total_ms: Int = ns / 1000000
let s_whole: Int = total_ms / 1000
let ms_rem: Int = total_ms - (s_whole * 1000)
return int_to_str(s_whole) + "." + _elt_pad3(ms_rem)
}
// Event emission
//
// One function per event shape. Both renderers read the same fields; the
// human renderer is a projection of the event, not a separate code path.
fn _elt_emit_run(json_mode: Bool, name: String) {
if json_mode {
println("{\"action\":\"run\",\"test\":\"" + _elt_json_escape(name) + "\"}")
}
}
fn _elt_emit_result(json_mode: Bool, name: String, fails: Int, ns: Int, asserts: Int, msg: String) {
if json_mode {
let action: String = "pass"
if fails > 0 { let action = "fail" }
let line: String = "{\"action\":\"" + action + "\""
let line = line + ",\"test\":\"" + _elt_json_escape(name) + "\""
let line = line + ",\"elapsed\":" + _elt_secs(ns)
let line = line + ",\"assertions\":" + int_to_str(asserts)
if fails > 0 {
let line = line + ",\"failures\":" + int_to_str(fails)
let line = line + ",\"message\":\"" + _elt_json_escape(msg) + "\""
}
let line = line + "}"
println(line)
return
}
// Human renderer duration is never optional.
if fails > 0 {
println("FAIL " + name + " (" + _elt_ms(ns) + "ms)")
println(" " + msg)
return
}
println("ok " + name + " (" + _elt_ms(ns) + "ms)")
return
}
fn _elt_emit_summary(json_mode: Bool, total: Int, failed: Int, ns: Int, asserts: Int) {
let passed: Int = total - failed
if json_mode {
let line: String = "{\"action\":\"summary\""
let line = line + ",\"tests\":" + int_to_str(total)
let line = line + ",\"passed\":" + int_to_str(passed)
let line = line + ",\"failed\":" + int_to_str(failed)
let line = line + ",\"assertions\":" + int_to_str(asserts)
let line = line + ",\"elapsed\":" + _elt_secs(ns)
let line = line + "}"
println(line)
return
}
println("")
println(int_to_str(total) + " tests, " + int_to_str(passed) + " passed, "
+ int_to_str(failed) + " failed, " + int_to_str(asserts) + " assertions in "
+ _elt_ms(ns) + "ms")
return
}
// The runner
// el_test_main drive the compile-time registry.
//
// Called from the generated main(). Returns the number of FAILING TESTS, which
// becomes the process exit code. Note that this counts tests, not assertions:
// a test is the unit of result. The old harness counted assertions globally and
// therefore could not say which test failed, how long any of them took, or
// whether a test had run at all.
fn el_test_main() -> Int {
let json_mode: Bool = false
if __el_opt_json() == 1 { let json_mode = true }
let n: Int = __el_reg_count()
let i: Int = 0
let failed: Int = 0
let total_ns: Int = 0
let total_asserts: Int = 0
while i < n {
let name: String = __el_reg_name(i)
_elt_emit_run(json_mode, name)
let fails: Int = __el_reg_invoke(i)
let ns: Int = __el_reg_last_ns()
let asserts: Int = __el_reg_asserts()
let msg: String = __el_reg_msg()
let total_ns = total_ns + ns
let total_asserts = total_asserts + asserts
if fails > 0 { let failed = failed + 1 }
_elt_emit_result(json_mode, name, fails, ns, asserts, msg)
let i = i + 1
}
_elt_emit_summary(json_mode, n, failed, total_ns, total_asserts)
return failed
}
+378 -6
View File
@@ -44,6 +44,11 @@
#include <string.h> #include <string.h>
#include <stdint.h> #include <stdint.h>
#include <unistd.h> #include <unistd.h>
#if defined(__APPLE__) || defined(__MACH__)
#include <sys/sysctl.h>
#include <mach/mach.h>
#include <mach/mach_host.h>
#endif
#include <fcntl.h> #include <fcntl.h>
#include <errno.h> #include <errno.h>
#include <time.h> #include <time.h>
@@ -236,8 +241,16 @@ struct PgCache {
unsigned prefetch; /* read-ahead window (pages); 0 = off */ unsigned prefetch; /* read-ahead window (pages); 0 = off */
LayerPin* lp; size_t lp_n, lp_cap; /* hot-layer pin bookkeeping */ LayerPin* lp; size_t lp_n, lp_cap; /* hot-layer pin bookkeeping */
size_t dirty_count; /* # dirty frames, maintained incrementally (M5) */ size_t dirty_count; /* # dirty frames, maintained incrementally (M5) */
/* stats (introspection only — never affect semantics) */ /* Interoception. These were "introspection only — never affect semantics",
* and that was the bug: the pool could not feel itself thrash, so it could
* not correct, and neither could anyone watching from outside. The sensed
* state IS the corrective mechanism (see pc_adapt_budget) the same way the
* engram's own boundary-beat/chronoception let it feel its own activity. */
uint64_t hits, misses, evictions, prefetch_reads; uint64_t hits, misses, evictions, prefetch_reads;
/* sliding-window marks so pressure reflects NOW, not lifetime totals */
uint64_t adapt_last_acc, adapt_last_evic, adapt_last_hits;
uint64_t adapt_grows; /* budget corrections upward */
uint64_t adapt_shrinks; /* budget corrections downward (memory pressure) */
}; };
/* ── little-endian scalar codecs ──────────────────────────────────────────── */ /* ── little-endian scalar codecs ──────────────────────────────────────────── */
@@ -334,6 +347,51 @@ static uint64_t dh_node_hash(const StoreNode* n){
return h; return h;
} }
/* dh_edge_hash — the edge counterpart of dh_node_hash.
*
* WHY THIS EXISTS (2026-08-15): the write barrier was node-only. Checkpointing
* pushes the WHOLE resident graph through store_put_node/store_put_edge (see
* engram_store_checkpoint), and nodes were cheaply skipped when unchanged
* a hash compare, no page I/O. Edges had no such check, so every edge was
* rewritten on every checkpoint, and each rewrite runs the idempotency probe
* max_page_lsn_for_id btree lookup page_read per stored copy.
*
* Edges outnumber nodes roughly 3:1 here (37,663 vs 13,430), so this turned
* routine checkpointing into a FULL-STORE WALK in id order random page access
* across the entire 2 GiB store, repeated, mostly to rediscover that nothing
* had changed. That walk is the failure mode: with a page cache smaller than
* the store it degenerates into thrashing and the engram never makes progress.
* Sizing the cache around that walk treats the symptom; the walk itself should
* not happen.
*
* The discriminator byte keeps the edge keyspace from ever colliding with a
* node of the same id in the shared dh map: distinct kinds cannot produce the
* same hash, so a stale skip is not reachable by collision. */
static uint64_t dh_edge_hash(const StoreEdge* e){
uint64_t h = 1469598103934665603ULL;
const uint8_t kind = 0xE0; /* edge discriminator */
dh_fold_bytes(&h, &kind, 1);
dh_fold_str(&h, e->id);
dh_fold_str(&h, e->from_id);
dh_fold_str(&h, e->to_id);
dh_fold_str(&h, e->relation);
dh_fold_str(&h, e->metadata);
uint8_t t8[8];
put_f64(t8, e->weight); dh_fold_bytes(&h, t8, 8);
put_f64(t8, e->hebb); dh_fold_bytes(&h, t8, 8);
put_f64(t8, e->confidence); dh_fold_bytes(&h, t8, 8);
uint8_t t4[4];
put_u32(t4, (uint32_t)e->inhibitory); dh_fold_bytes(&h, t4, 4);
put_u32(t4, e->layer_id); dh_fold_bytes(&h, t4, 4);
/* created_at/updated_at/last_fired are deliberately EXCLUDED: last_fired is
* touched by activation without changing what the edge IS, and including it
* would defeat the barrier on exactly the hot edges it most needs to skip.
* The fields that define the edge's durable content are all folded above. */
if (e->unknown && e->unknown_len) dh_fold_bytes(&h, e->unknown, e->unknown_len);
if (h == 0) h = 1; /* reserve 0 as "absent" in the map */
return h;
}
/* Open-addressing id(string)→durable-hash map. Keyed for O(1) bucketing on the /* Open-addressing id(string)→durable-hash map. Keyed for O(1) bucketing on the
* id's FNV hash, compared by strcmp for correctness (full-id discipline, matching * id's FNV hash, compared by strcmp for correctness (full-id discipline, matching
* store_scan_*'s StrSet). Values are the 64-bit durable hash. */ * store_scan_*'s StrSet). Values are the 64-bit durable hash. */
@@ -1531,6 +1589,11 @@ int store_scan_edges(EngramPagedStore* s, StoreEdgeScanCb cb, void* ctx){
if (cand.id && *cand.id && strset_add(&seen, cand.id)){ if (cand.id && *cand.id && strset_add(&seen, cand.id)){
StoreEdge canon; StoreEdge canon;
if (store_get_edge(s, cand.id, &canon) == 1){ if (store_get_edge(s, cand.id, &canon) == 1){
/* seed the write-barrier map from on-disk truth so the FIRST
* post-boot checkpoint full-walk already skips unchanged edges
* (mirrors store_scan_nodes; without it the barrier is empty at
* boot and the first checkpoint re-probes every edge) */
if (s->barrier_on) dh_set(s->dh, canon.id, dh_edge_hash(&canon));
cb(&canon, ctx); count++; /* canonical latest-live */ cb(&canon, ctx); count++; /* canonical latest-live */
store_edge_free(&canon); store_edge_free(&canon);
} }
@@ -1594,19 +1657,76 @@ int store_scan_edges(EngramPagedStore* s, StoreEdgeScanCb cb, void* ctx){
* matches disk, so a re-fault reproduces identical bytes. * matches disk, so a re-fault reproduces identical bytes.
* */ * */
/* default frame budget: large enough that today's whole store stays resident /* ── Frame budget ────────────────────────────────────────────────────────────
* (== Phase 1). Override with env ENGRAM_POOL_FRAMES (0 = unlimited). */ *
#ifndef ENGRAM_POOL_FRAMES_DEFAULT * A FIXED frame count cannot be correct. It has no relationship to either
#define ENGRAM_POOL_FRAMES_DEFAULT (1u<<20) /* ~1M frames × 16KiB = 16 GiB */ * quantity that decides whether a cache works: the size of the working set, or
* the memory actually available on the host. It is the same number on a 16 GB
* laptop and a 256 GB server, and it stays put while the store grows.
*
* That is not hypothetical. On 2026-08-15 the deployment pinned
* ENGRAM_POOL_FRAMES=65536 (1 GiB) while neuron.egm grew to 2.1 GiB. The
* working set was twice the budget, so boot-time WAL replay which walks
* pages in an order uncorrelated with reuse evicted each page shortly before
* it was needed again. The engram spun at 100% CPU inside pc_evict_to_budget
* and never bound its port. Not slow: making no progress. Denning's thrashing,
* exactly, and no eviction policy can fix it when the working set does not
* fit, only more frames or admission control help.
*
* So the budget is DERIVED, from the host's physical memory, and it scales
* with the machine instead of pretending memory is a constant.
*
* ENGRAM_POOL_FRAMES explicit frame count; 0 = unlimited. Overrides all.
* Prefer leaving it unset a hand-set number is how
* this failure happened.
* ENGRAM_POOL_MEM_PCT percent of physical RAM to budget (default 60).
*
* Fallback when RAM cannot be read is 16 GiB worth of frames the old
* default, retained only as a floor for that case.
* */
#ifndef ENGRAM_POOL_FRAMES_FALLBACK
#define ENGRAM_POOL_FRAMES_FALLBACK (1u<<20) /* ~1M frames × 16KiB = 16 GiB */
#endif #endif
static uint64_t pc_available_ram(void); /* fwd — defined with the controller */
/* Physical RAM in bytes, 0 when it cannot be determined. */
static uint64_t pc_physical_ram(void){
#if defined(__APPLE__) || defined(__MACH__)
uint64_t v = 0; size_t len = sizeof v;
int mib[2] = { CTL_HW, HW_MEMSIZE };
if (sysctl(mib, 2, &v, &len, NULL, 0) == 0) return v;
return 0;
#else
long pages = sysconf(_SC_PHYS_PAGES);
long psz = sysconf(_SC_PAGESIZE);
if (pages > 0 && psz > 0) return (uint64_t)pages * (uint64_t)psz;
return 0;
#endif
}
static size_t pc_default_cap(void){
unsigned pct = 60;
const char* p = getenv("ENGRAM_POOL_MEM_PCT");
if (p && *p){ unsigned long v = strtoul(p, NULL, 10); if (v > 0 && v <= 95) pct = (unsigned)v; }
uint64_t ram = pc_physical_ram();
if (!ram) return ENGRAM_POOL_FRAMES_FALLBACK;
uint64_t budget_bytes = (ram / 100u) * pct;
/* Never start above what the machine can actually spare right now. */
uint64_t avail = pc_available_ram();
if (avail > (1ull<<30) && budget_bytes > avail - (1ull<<30)) budget_bytes = avail - (1ull<<30);
uint64_t frames = budget_bytes / (uint64_t)STORE_PAGE_SIZE;
if (frames < 4096) frames = 4096; /* never absurdly small */
return (size_t)frames;
}
static PgCache* pc_new(void){ static PgCache* pc_new(void){
PgCache* c = (PgCache*)calloc(1, sizeof *c); PgCache* c = (PgCache*)calloc(1, sizeof *c);
if (!c) return NULL; if (!c) return NULL;
c->nbuckets = 1024; c->nbuckets = 1024;
c->buckets = (PgEnt**)calloc(c->nbuckets, sizeof(PgEnt*)); c->buckets = (PgEnt**)calloc(c->nbuckets, sizeof(PgEnt*));
if (!c->buckets){ free(c); return NULL; } if (!c->buckets){ free(c); return NULL; }
c->cap = ENGRAM_POOL_FRAMES_DEFAULT; c->cap = pc_default_cap();
c->prefetch = 8; c->prefetch = 8;
const char* pf = getenv("ENGRAM_POOL_FRAMES"); const char* pf = getenv("ENGRAM_POOL_FRAMES");
if (pf && *pf){ char* end=NULL; unsigned long long v = strtoull(pf,&end,10); c->cap = (size_t)v; } if (pf && *pf){ char* end=NULL; unsigned long long v = strtoull(pf,&end,10); c->cap = (size_t)v; }
@@ -1676,6 +1796,241 @@ static void pc_remove(PgCache* c, PgEnt* e){
/* Reclaim clean unpinned frames from the LRU end until under budget, or until no /* Reclaim clean unpinned frames from the LRU end until under budget, or until no
* evictable frame remains (a dirty/pinned-heavy pool may transiently exceed cap * evictable frame remains (a dirty/pinned-heavy pool may transiently exceed cap
* that is the no-steal guarantee, not a bug: the next checkpoint frees them). */ * that is the no-steal guarantee, not a bug: the next checkpoint frees them). */
/* ── Adaptive budget: close the loop ─────────────────────────────────────────
*
* THE LESSON THIS ENCODES (2026-08-15). The engram spent hours down while four
* separate theories were tried bad binary, corrupt snapshot, WAL replay,
* feature flags because nothing in the system said what was happening. It
* looked identical to "busy loading": 100% CPU, flat RSS, no output. Meanwhile
* hits/misses/evictions were ALREADY being counted, right here, and surfaced
* nowhere. One eviction-rate number would have ended it in seconds.
*
* So the counters are not decoration. They are the control signal.
*
* A budget chosen once a literal like 65536, or 60% of RAM read at startup
* is a guess about the future. It cannot know the store grew, the working set
* shifted, or another process took the memory. The cache already MEASURES the
* only thing that matters (am I evicting pages I am about to want again), so it
* should act on that measurement instead of on a number someone typed.
*
* The controller: over a sliding window, if evictions are running at a rate
* comparable to accesses AND there is genuine reuse (hits are material), the
* working set exceeds the budget grow it. Growth is geometric, bounded by a
* live re-read of physical memory rather than a value cached at boot, so it
* tracks the machine instead of a snapshot of it. It never shrinks on its own:
* cap is a ceiling, not an allocation, and frames are only ever held because a
* real access put them there.
*
* Two things this deliberately does NOT do: it does not attempt a cleverer
* eviction policy (when the working set does not fit, no policy helps that is
* Denning, and it is why "tune the LRU" was never the fix), and it does not stay
* silent (pool_report exposes the same numbers outward, so a human or a metric
* pipeline sees the pressure the controller is reacting to). */
/* El's native telemetry, already in the runtime and already exporting to OTLP.
* Declared weak so engram_store.c still links standalone; when the runtime is
* present (every real build) the pool's interoception flows into the SAME
* pipeline as every other metric.
*
* ONE emission carrying the whole sensed state not a function per stat, and
* not a bespoke per-subsystem endpoint. Both of those are the degenerate case:
* they make observability something you hand-write per noun instead of a
* uniform mechanism every component already has. el_val_t is int64_t; strings
* ride as pointers cast through it (see el_runtime.h's value model). */
__attribute__((weak)) int64_t emit_log(int64_t level, int64_t msg, int64_t fields_json);
static void pc_report(const PgCache* c, const char* cause){
if (!emit_log) return; /* runtime not linked: no-op */
uint64_t acc = c->hits + c->misses;
char f[512];
snprintf(f, sizeof f,
"{\"component\":\"engram.pool\",\"cause\":\"%s\",\"hits\":%llu,\"misses\":%llu,"
"\"evictions\":%llu,\"prefetch_reads\":%llu,\"cap_frames\":%zu,\"resident\":%zu,"
"\"dirty\":%zu,\"grows\":%llu,\"hit_rate\":%.4f,\"evict_ratio\":%.4f,"
"\"cap_gib\":%.3f,\"resident_gib\":%.3f}",
cause,
(unsigned long long)c->hits, (unsigned long long)c->misses,
(unsigned long long)c->evictions, (unsigned long long)c->prefetch_reads,
c->cap, c->count, c->dirty_count, (unsigned long long)c->adapt_grows,
acc ? (double)c->hits / (double)acc : 0.0,
acc ? (double)c->evictions / (double)acc : 0.0,
(double)c->cap * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0),
(double)c->count * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0));
emit_log((int64_t)(uintptr_t)"warn", (int64_t)(uintptr_t)"engram.pool pressure",
(int64_t)(uintptr_t)f);
}
static uint64_t pc_ram_bytes_live(void){ return pc_physical_ram(); }
/* AVAILABLE memory right now — free + reclaimable, not total.
*
* Sizing a cache against TOTAL ram is what turns a cache into a memory leak:
* total does not shrink when other processes need memory, so a pool that only
* grows never notices it is starving the machine it runs on. Availability does.
* Returns 0 when undeterminable callers then refuse to grow, the safe way. */
static uint64_t pc_available_ram(void){
#if defined(__APPLE__) || defined(__MACH__)
/* SWAP AND COMPRESSOR FIRST. free+inactive+purgeable is a LIE under memory
* pressure: a machine deep in swap still reports gigabytes "available",
* because inactive pages are only reclaimable by evicting them to swap.
* Observed 2026-08-15: this returned 9.43 GiB available while vm.swapusage
* showed 51.58 of 53.25 GiB used (97% full) and the compressor occupied
* 23.7 GiB the host was thrashing to disk and the pool would have been
* cleared to grow into it. Growing a cache in that state is how a guard
* becomes the crash.
*
* So: if swap is nearly spent, report ZERO available. Callers refuse to
* grow on 0 and pc_relieve_pressure hands frames back. Only when the
* machine is genuinely not swapping do free+inactive+purgeable mean
* anything, and even then the compressor's footprint is subtracted because
* that RAM is already spoken for. */
/* RATE, NOT LEVEL. Swap *level* is a terrible signal: macOS grows swap files
* on demand and reclaims them lazily, so "47 of 48 GiB used" can mean the
* machine is dying OR that it recovered ten minutes ago and the file has not
* been trimmed yet. Measured both states on one host within minutes:
* 47.65/48.00 GiB used, 2047 swapouts/s -> genuinely thrashing
* 26.67/28.00 GiB used, 0 swapouts/s -> perfectly healthy, 15.6 GiB free
* A level check calls the second one an emergency and starves the pool for
* no reason. What distinguishes them is whether pages are moving NOW.
*
* So sample the swapout counter across calls and judge the delta. First call
* establishes the baseline and reports no pressure one sample cannot have
* a rate, and guessing from a single reading is the whole mistake. */
{
static uint64_t prev_swapouts = 0;
static time_t prev_t = 0;
static int primed = 0;
mach_port_t h0 = mach_host_self();
vm_statistics64_data_t v0; mach_msg_type_number_t c0 = HOST_VM_INFO64_COUNT;
if (host_statistics64(h0, HOST_VM_INFO64, (host_info64_t)&v0, &c0) == KERN_SUCCESS){
uint64_t now_out = (uint64_t)v0.swapouts;
time_t now_t = time(NULL);
if (!primed){ prev_swapouts = now_out; prev_t = now_t; primed = 1; }
else if (now_t > prev_t){
double per_s = (double)(now_out - prev_swapouts) / (double)(now_t - prev_t);
prev_swapouts = now_out; prev_t = now_t;
/* Sustained outward paging with nothing coming back is the
* signature of a host being pushed into swap. ~200 pages/s is
* ~3 MiB/s well above idle noise, well below the 2000+/s seen
* while actually thrashing. */
if (per_s > 200.0) return 0;
}
}
}
mach_port_t host = mach_host_self();
vm_size_t page = 0;
if (host_page_size(host, &page) != KERN_SUCCESS) return 0;
vm_statistics64_data_t vm; mach_msg_type_number_t cnt = HOST_VM_INFO64_COUNT;
if (host_statistics64(host, HOST_VM_INFO64, (host_info64_t)&vm, &cnt) != KERN_SUCCESS) return 0;
uint64_t avail = (uint64_t)vm.free_count + (uint64_t)vm.inactive_count
+ (uint64_t)vm.purgeable_count;
/* the compressor is holding real RAM that nobody can hand us */
uint64_t compressed = (uint64_t)vm.compressor_page_count;
if (compressed >= avail) return 0;
avail -= compressed;
return avail * (uint64_t)page;
#else
FILE* f = fopen("/proc/meminfo", "r");
if (!f) return 0;
char line[256]; unsigned long long kb = 0;
while (fgets(line, sizeof line, f))
if (sscanf(line, "MemAvailable: %llu kB", &kb) == 1) break;
fclose(f);
return (uint64_t)kb * 1024ull;
#endif
}
/* Shrink the budget when the machine is short on memory.
*
* A pool that can only grow is a leak with extra steps. This is the other half
* of the control loop: if free memory drops below a floor, hand frames back.
* The resident set follows on the next eviction pass, so the memory is actually
* returned rather than merely re-labelled. */
#ifndef ENGRAM_POOL_FREE_FLOOR_BYTES
#define ENGRAM_POOL_FREE_FLOOR_BYTES (2ull*1024ull*1024ull*1024ull) /* 2 GiB */
#endif
static int pc_relieve_pressure(PgCache* c){
uint64_t avail = pc_available_ram();
if (!avail) return 0;
uint64_t floor_b = ENGRAM_POOL_FREE_FLOOR_BYTES;
const char* fe = getenv("ENGRAM_POOL_FREE_FLOOR_MB");
if (fe && *fe){ unsigned long v = strtoul(fe, NULL, 10); if (v) floor_b = (uint64_t)v * 1024ull * 1024ull; }
if (avail >= floor_b) return 0; /* machine has room */
if (!c->cap || c->count == 0) return 0;
size_t was = c->cap;
size_t want = c->count - (c->count / 4); /* give back ~25% of what we hold */
if (want < 4096) want = 4096;
if (want >= c->cap) return 0;
c->cap = want;
c->adapt_shrinks++;
fprintf(stderr,
"[engram] memory pressure: %.2f GiB available (floor %.2f GiB) — shrinking pool "
"budget %zu -> %zu frames (%.2f -> %.2f GiB) and releasing frames.\n",
(double)avail/(1024.0*1024.0*1024.0), (double)floor_b/(1024.0*1024.0*1024.0),
was, c->cap,
(double)was * (double)STORE_PAGE_SIZE/(1024.0*1024.0*1024.0),
(double)c->cap* (double)STORE_PAGE_SIZE/(1024.0*1024.0*1024.0));
fflush(stderr);
return 1;
}
static void pc_adapt_budget(PgCache* c){
if (!c->cap) return; /* unlimited: nothing to adapt */
if (getenv("ENGRAM_POOL_FRAMES")) return; /* explicit operator override wins */
/* Sliding window so the signal reflects NOW, not lifetime totals. */
uint64_t acc = c->hits + c->misses;
if (acc - c->adapt_last_acc < 100000) return;
uint64_t d_acc = acc - c->adapt_last_acc;
uint64_t d_evic = c->evictions - c->adapt_last_evic;
uint64_t d_hits = c->hits - c->adapt_last_hits;
c->adapt_last_acc = acc; c->adapt_last_evic = c->evictions; c->adapt_last_hits = c->hits;
/* Pressure = evicting on a large fraction of accesses while still getting
* real reuse. Evictions alone are normal (a scan evicts and never returns);
* evictions WITH reuse means the working set genuinely does not fit. */
if (d_evic * 3 < d_acc) return; /* < 1/3 of accesses evict: healthy */
if (d_hits * 4 < d_acc) return; /* little reuse: a scan, not pressure */
/* Growth is bounded by what is AVAILABLE, never by total RAM. Sizing against
* total is how a cache starves its own host: total never shrinks when other
* processes need memory. Refuse to grow at all if availability is unknown or
* already under the floor a cache is never worth swapping the machine. */
uint64_t avail = pc_available_ram();
uint64_t floor_b = ENGRAM_POOL_FREE_FLOOR_BYTES;
const char* fe = getenv("ENGRAM_POOL_FREE_FLOOR_MB");
if (fe && *fe){ unsigned long v = strtoul(fe, NULL, 10); if (v) floor_b = (uint64_t)v * 1024ull * 1024ull; }
if (!avail || avail <= floor_b) return;
uint64_t ram = pc_ram_bytes_live();
if (!ram) return;
unsigned pct = 50; /* ceiling as a share of TOTAL, belt-and-braces */
const char* mp = getenv("ENGRAM_POOL_MAX_PCT");
if (mp && *mp){ unsigned long v = strtoul(mp, NULL, 10); if (v > 0 && v <= 95) pct = (unsigned)v; }
size_t ceiling = (size_t)(((ram / 100u) * pct) / (uint64_t)STORE_PAGE_SIZE);
/* and never grow into the free-memory floor */
uint64_t headroom = avail - floor_b;
size_t ceil_avail = (size_t)((c->count * (uint64_t)STORE_PAGE_SIZE + headroom)
/ (uint64_t)STORE_PAGE_SIZE);
if (ceil_avail < ceiling) ceiling = ceil_avail;
if (c->cap >= ceiling) return; /* already at the machine's limit */
size_t want = c->cap + (c->cap / 2) + 1; /* ×1.5, geometric */
if (want > ceiling) want = ceiling;
size_t was = c->cap;
c->cap = want;
c->adapt_grows++;
/* Emit the sensed state, not just the reaction. These are the numbers that
* would have diagnosed 2026-08-15 in seconds instead of hours. */
pc_report(c, "budget-grow");
fprintf(stderr,
"[engram] pool pressure: %llu evictions / %llu accesses (%llu hits) at %zu frames "
"(%.2f GiB) — working set exceeds budget; growing to %zu frames (%.2f GiB).\n",
(unsigned long long)d_evic, (unsigned long long)d_acc, (unsigned long long)d_hits,
was, (double)was * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0),
c->cap,(double)c->cap * (double)STORE_PAGE_SIZE / (1024.0*1024.0*1024.0));
fflush(stderr);
}
static void pc_evict_to_budget(PgCache* c){ static void pc_evict_to_budget(PgCache* c){
if (!c->cap) return; /* unlimited */ if (!c->cap) return; /* unlimited */
while (c->count > c->cap){ while (c->count > c->cap){
@@ -1687,6 +2042,7 @@ static void pc_evict_to_budget(PgCache* c){
} }
if (!freed) break; /* nothing evictable — allowed to exceed cap */ if (!freed) break; /* nothing evictable — allowed to exceed cap */
} }
if (!pc_relieve_pressure(c)) pc_adapt_budget(c);
} }
static PgEnt* pc_get(EngramPagedStore* s, uint64_t id){ static PgEnt* pc_get(EngramPagedStore* s, uint64_t id){
@@ -2377,6 +2733,18 @@ int store_put_node(EngramPagedStore* s, const StoreNode* n){
int store_put_edge(EngramPagedStore* s, const StoreEdge* e){ int store_put_edge(EngramPagedStore* s, const StoreEdge* e){
if (!s || !e || !e->id || !e->from_id || !e->to_id) return -1; if (!s || !e || !e->id || !e->from_id || !e->to_id) return -1;
STORE_GUARD(s); STORE_GUARD(s);
/* Durable-hash write barrier — mirrors store_put_node. An unchanged edge
* costs one hash compare and zero page I/O; without this, checkpointing
* re-probed every edge against the paged store (max_page_lsn_for_id
* page_read), turning a routine checkpoint into a full-store walk. */
uint64_t dh_h = 0;
if (s->barrier_on){
dh_h = dh_edge_hash(e);
if (dh_get(s->dh, e->id) == dh_h){
s->stat_barrier_skips++;
return 0;
}
}
uint64_t L = ++s->next_lsn; uint64_t L = ++s->next_lsn;
if (s->wal){ if (s->wal){
size_t blen; uint8_t* body = edge_serialize(e, &blen); size_t blen; uint8_t* body = edge_serialize(e, &blen);
@@ -2386,6 +2754,10 @@ int store_put_edge(EngramPagedStore* s, const StoreEdge* e){
if (wr != 0) return -1; if (wr != 0) return -1;
} }
int r = apply_edge_put(s, e, L); int r = apply_edge_put(s, e, L);
if (r == 0 && s->barrier_on){
if (!dh_h) dh_h = dh_edge_hash(e);
dh_set(s->dh, e->id, dh_h); /* remember the now-persisted durable hash */
}
ckpt_maybe(s); ckpt_maybe(s);
return r; return r;
} }
+91
View File
@@ -0,0 +1,91 @@
// fitprobe.el controlled growth-curve specimens for validating the complexity fitter.
//
// Three deliberately-shaped workloads. None depends on a real defect existing,
// which is the point: the fitter must be provable against KNOWN curves.
//
// linear one allocation per item. count O(n), bytes O(n), time O(n)
// accum rebuilds its accumulator. count O(n), bytes O(n^2), time O(n^2)
// compute nested arithmetic, no alloc. count O(1), bytes O(1), time O(n^2)
//
// `compute` is the specimen that matters. It is the shape of el #132
// (strlen-per-character inside str_char_code): pure CPU, zero allocation.
// An allocation-only gate is structurally blind to it.
//
// No imports uses runtime builtins directly so nothing collides.
fn work_linear(n: Int) -> Int {
let parts: [String] = native_list_empty()
let i: Int = 0
while i < n {
let parts = native_list_append(parts, int_to_str(i))
let i = i + 1
}
return native_list_len(parts)
}
fn work_accum(n: Int) -> Int {
let acc: String = ""
let i: Int = 0
while i < n {
let acc = acc + "x"
let i = i + 1
}
return str_len(acc)
}
fn work_compute(n: Int) -> Int {
// str_char_code is an opaque external call, so the C optimiser cannot
// reduce this nest to a closed form the way it does with `total + 1`.
// This is the exact shape of el #132: n scans over n characters, pure
// CPU, ZERO allocation.
let s: String = "abcdefghij"
let total: Int = 0
let i: Int = 0
while i < n {
let j: Int = 0
while j < n {
let total = total + str_char_code(s, 0)
let j = j + 1
}
let i = i + 1
}
return total
}
fn run_one(mode: String, n: Int) {
let c0: Int = el_alloc_count()
let b0: Int = el_alloc_bytes()
let t0: Int = el_now_instant()
let r: Int = 0
if str_eq(mode, "linear") { let r = work_linear(n) }
if str_eq(mode, "accum") { let r = work_accum(n) }
if str_eq(mode, "compute") { let r = work_compute(n) }
let t1: Int = el_now_instant()
let c1: Int = el_alloc_count()
let b1: Int = el_alloc_bytes()
println(mode + "\t" + int_to_str(n)
+ "\t" + int_to_str(c1 - c0)
+ "\t" + int_to_str(b1 - b0)
+ "\t" + int_to_str((t1 - t0) / 1000)
+ "\t" + int_to_str(r))
return
}
fn sweep(mode: String) {
run_one(mode, 200)
run_one(mode, 400)
run_one(mode, 800)
run_one(mode, 1600)
return
}
fn main() -> Int {
println("mode\tn\tallocs\tbytes\tusec\tsink")
sweep("linear")
sweep("accum")
sweep("compute")
return 0
}
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// tests/native/test_compiler.el comprehensive tests for the El compiler pipeline. // tests/native/test_compiler.el comprehensive tests for the El compiler pipeline.
// //
// Tests the lexer (lexer.el), parser (parser.el), and codegen (codegen.el) // Tests the lexer (lexer.el), parser (parser.el), and codegen (codegen.el)
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_codegen_js.el - basic tests for JS codegen features. // test_codegen_js.el - basic tests for JS codegen features.
// //
// These tests verify that core El language features produce correct values // These tests verify that core El language features produce correct values
+111
View File
@@ -0,0 +1,111 @@
import "../../runtime/eltest.el"
import "../../runtime/elbench.el"
// test_elbench.el proves the growth-curve classifier against KNOWN curves.
//
// Every series below is real measured data from lang/tests/bench/fitprobe.el
// on a geometric sweep n = 200/400/800/1600. The classifier must be provable
// without depending on a live defect existing, which is the whole point of
// keeping controlled specimens.
fn _s4(a: Int, b: Int, c: Int, d: Int) -> [Int] {
let l: [Int] = native_list_empty()
let l = native_list_append(l, a)
let l = native_list_append(l, b)
let l = native_list_append(l, c)
let l = native_list_append(l, d)
return l
}
test "classifies a linear allocation series as O(n)" {
// fitprobe `linear`, allocation count
let v = _s4(208, 409, 810, 1611)
assert elb_measured_curve(v, 10) == 2, "linear allocs should classify O(n)"
}
test "classifies a linear byte series as O(n)" {
// fitprobe `linear`, allocation bytes
let v = _s4(4786, 9682, 19474, 39658)
assert elb_measured_curve(v, 10) == 2, "linear bytes should classify O(n)"
}
test "classifies a quadratic byte series as O(n^2)" {
// fitprobe `accum`, allocation bytes -- the accumulator-rebuild shape
let v = _s4(20300, 80600, 321200, 1282400)
assert elb_measured_curve(v, 10) == 4, "accum bytes should classify O(n^2)"
}
test "accumulator count is linear -- proves count alone misses it" {
// Same run as above. The COUNT is exactly linear while bytes are
// quadratic. A count-only gate passes this defect clean.
let v = _s4(200, 400, 800, 1600)
assert elb_measured_curve(v, 10) == 2, "accum count classifies O(n)"
assert elb_gate(v, 2, 10) == 0, "count-only gate PASSES the quadratic"
}
test "classifies a quadratic time series as O(n^2)" {
// fitprobe `compute` -- el #132's shape: n scans over n characters
let v = _s4(67, 205, 818, 3268)
assert elb_measured_curve(v, 10) == 4, "compute time should classify O(n^2)"
}
test "REFUSES an all-zero series instead of calling it O(1)" {
// fitprobe `compute` allocation count. Pure CPU, allocates nothing.
// Reporting O(1) here would be a confident answer with nothing behind it.
let v = _s4(0, 0, 0, 0)
assert elb_gate(v, 2, 10) == 3, "all-zero series must be REFUSED"
assert elb_measured_curve(v, 10) < 0, "unclassifiable returns -1"
}
test "REFUSES an implausibly flat series" {
// The shape produced when clang closes a loop to a multiply: a real
// answer, no work done, no movement across an 8x input range.
let v = _s4(1000, 1001, 1002, 1003)
assert elb_gate(v, 2, 10) == 3, "hard-flat series must be REFUSED"
}
test "gate FAILS a quadratic declared as linear" {
let v = _s4(20300, 80600, 321200, 1282400)
assert elb_gate(v, 2, 10) == 1, "O(n^2) measured vs O(n) declared must FAIL"
}
test "gate PASSES a linear series declared as linear" {
let v = _s4(208, 409, 810, 1611)
assert elb_gate(v, 2, 10) == 0, "O(n) measured vs O(n) declared must PASS"
}
test "gate reports BETTER when measured beats the declared bound" {
let v = _s4(208, 409, 810, 1611)
assert elb_gate(v, 4, 10) == 4, "O(n) measured vs O(n^2) declared is BETTER"
}
test "gate reports INDETERMINATE on disagreeing ratios" {
// fitprobe `linear` WALL TIME at these sizes: 26/19/43/78 microseconds.
// Ratios 0.73, 2.26, 1.81 disagree well past the noise threshold. The
// honest answer is "cannot tell", not a classification -- this is exactly
// why benchmarks need auto-scaled iteration counts rather than one shot.
let v = _s4(26, 19, 43, 78)
assert elb_gate(v, 2, 10) == 2, "disagreeing ratios must be INDETERMINATE"
}
test "black_box is a real barrier and returns its input" {
assert el_black_box(42) == 42, "black_box is value-preserving"
let s: Int = 0
let i: Int = 0
while i < 100 {
// Bind the call before using it in arithmetic: `x + call(...)`
// lowers to el_str_concat() on integers. Same inference defect
// as `call(...) == y` lowering to str_eq().
let bx: Int = el_black_box(1)
let s = s + bx
let i = i + 1
}
assert s == 100, "black_box does not disturb the computation"
}
test "curve names round-trip" {
assert elb_curve_from_name("O(n)") == 2, "O(n) parses"
assert elb_curve_from_name("O(n^2)") == 4, "O(n^2) parses"
assert str_eq(elb_curve_name(4), "O(n^2)"), "O(n^2) renders"
assert elb_curve_from_name("O(nonsense)") < 0, "unknown curve is -1"
}
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_env.el - native test suite for runtime/env.el // test_env.el - native test suite for runtime/env.el
// //
// Covers: env() for reading environment variables, args() returning a list, // Covers: env() for reading environment variables, args() returning a list,
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_fs.el - native test suite for runtime/fs.el // test_fs.el - native test suite for runtime/fs.el
// //
// Covers: fs_write/read round-trip, fs_exists, fs_mkdir, fs_list, // Covers: fs_write/read round-trip, fs_exists, fs_mkdir, fs_list,
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_json.el - native test suite for runtime/json.el // test_json.el - native test suite for runtime/json.el
// //
// Covers: json_get (dot-path), typed extractors (int, bool, float), // Covers: json_get (dot-path), typed extractors (int, bool, float),
+178
View File
@@ -0,0 +1,178 @@
import "../../runtime/eltest.el"
import "../../runtime/elbench.el"
// test_lexer_scaling.el THE ARMED GATE.
//
// This is the regression test that would have caught el #132.
//
// #132 was a strlen() inside str_char_code() and str_slice(). The lexer walks
// source one character at a time, so every character access rescanned the whole
// remaining input: O(n) per character over n characters = O(n^2). It shipped for
// months. It was found by a geometric sweep, not by reading code.
//
// So this test IS a geometric sweep. It scans a string of length n, character by
// character, at four doubling sizes, and asserts the cost is linear. If anyone
// reintroduces a per-character rescan in str_char_code, in str_slice, in any
// accessor the lexer leans on the measured curve becomes O(n^2) and this fails.
//
// The value is in it being ARMED, not in it currently failing. It passes today
// because #132 is fixed. That is the correct state for a regression gate.
//
// Note the deliberate `let c: Int = str_char_code(...)` binding in the scan loop.
// Inlining it as `total + str_char_code(s, i)` lowers to el_str_concat() on
// integers the Plus arm of the operator-typing family, still open at the time
// of writing. Binding first is the safe form.
// _mk_string build a string of length >= n by DOUBLING.
//
// Deliberately not `s = s + "x"` n times: that is itself quadratic in bytes and
// would contaminate the very measurement this test exists to take. Doubling
// allocates ~2n total.
fn _mk_string(n: Int) -> String {
let s: String = "abcdefgh"
while str_len(s) < n {
let s = s + s
}
return s
}
// _scan walk the string one character at a time, REPS times.
//
// This is the lexer's access pattern reduced to its essential shape. The
// repetitions lift the measurement clear of timer resolution; without them the
// smaller sizes land in noise and the classifier correctly reports
// INDETERMINATE rather than guessing.
fn _scan(s: String, n: Int, reps: Int) -> Int {
let total: Int = 0
let r: Int = 0
while r < reps {
let i: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
let total = total + c
let i = i + 1
}
let r = r + 1
}
return total
}
// _measure_scan microseconds for a full scan sweep point.
fn _measure_scan(n: Int, reps: Int) -> Int {
let s: String = _mk_string(n)
// WARMUP, discarded. Without it the small-n end of the sweep is dominated
// by cold caches and reads as superlinear on genuinely linear work --
// measured ratios 3.37 2.92 1.76 1.65 on exactly this workload.
let w: Int = _scan(s, n, 2)
let wj: Int = el_black_box(w)
let t0: Int = el_now_instant()
let got: Int = _scan(s, n, reps)
let t1: Int = el_now_instant()
// Feed the result through the barrier so the scan cannot be elided.
let sink: Int = el_black_box(got)
if sink == 0 { println("") }
return (t1 - t0) / 1000
}
fn _series4(a: Int, b: Int, c: Int, d: Int) -> [Int] {
let l: [Int] = native_list_empty()
let l = native_list_append(l, a)
let l = native_list_append(l, b)
let l = native_list_append(l, c)
let l = native_list_append(l, d)
return l
}
test "character scan is LINEAR in time -- regression gate for el #132" {
let reps: Int = 40
let t1: Int = _measure_scan(16384, reps)
let t2: Int = _measure_scan(32768, reps)
let t3: Int = _measure_scan(65536, reps)
let t4: Int = _measure_scan(131072, reps)
let series: [Int] = _series4(t1, t2, t3, t4)
let verdict: Int = elb_gate(series, 2, 50)
let measured: Int = elb_measured_curve(series, 50)
// Report the actual numbers regardless of outcome. A gate that fires
// without showing its evidence is just an assertion.
println(" scan us: " + int_to_str(t1) + " " + int_to_str(t2) + " "
+ int_to_str(t3) + " " + int_to_str(t4)
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
// PASS (0) or BETTER (4) are both acceptable. FAIL (1) means someone
// reintroduced superlinear per-character cost. REFUSED (3) or
// INDETERMINATE (2) mean the measurement is untrustworthy -- which is
// also a failure of this test, deliberately: a gate that cannot measure
// must not report success.
assert verdict == 0 || verdict == 4, "character scan must measure O(n) or better"
}
test "string building by doubling stays linear in allocated bytes" {
let b1: Int = el_alloc_bytes()
let s1: String = _mk_string(8192)
let b2: Int = el_alloc_bytes()
let s2: String = _mk_string(16384)
let b3: Int = el_alloc_bytes()
let s3: String = _mk_string(32768)
let b4: Int = el_alloc_bytes()
let s4: String = _mk_string(65536)
let b5: Int = el_alloc_bytes()
let series: [Int] = _series4(b2 - b1, b3 - b2, b4 - b3, b5 - b4)
let verdict: Int = elb_gate(series, 2, 1000)
let measured: Int = elb_measured_curve(series, 1000)
println(" bytes: " + int_to_str(b2 - b1) + " " + int_to_str(b3 - b2) + " "
+ int_to_str(b4 - b3) + " " + int_to_str(b5 - b4)
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
assert verdict == 0 || verdict == 4, "doubling build must be O(n) in bytes"
assert str_len(s4) >= 65536, "final string reached the requested size"
}
// _scan_quadratic a DELIBERATELY quadratic scan: for each position, rescan
// from the start. This is precisely what el #132 did strlen() from offset 0
// on every character access reproduced here so the gate can be proven to
// FIRE, not merely to pass on healthy code. An unproven gate is decoration.
fn _scan_quadratic(s: String, n: Int) -> Int {
let total: Int = 0
let i: Int = 0
while i < n {
let j: Int = 0
while j < i {
let c: Int = str_char_code(s, j)
let total = total + c
let j = j + 1
}
let i = i + 1
}
return total
}
fn _measure_quadratic(n: Int) -> Int {
let s: String = _mk_string(n)
let w: Int = _scan_quadratic(s, 64)
let wj: Int = el_black_box(w)
let t0: Int = el_now_instant()
let got: Int = _scan_quadratic(s, n)
let t1: Int = el_now_instant()
let sink: Int = el_black_box(got)
return (t1 - t0) / 1000
}
test "the gate FIRES on a live quadratic scan -- proves it is armed" {
let q1: Int = _measure_quadratic(1024)
let q2: Int = _measure_quadratic(2048)
let q3: Int = _measure_quadratic(4096)
let q4: Int = _measure_quadratic(8192)
let series: [Int] = _series4(q1, q2, q3, q4)
let verdict: Int = elb_gate(series, 2, 50)
let measured: Int = elb_measured_curve(series, 50)
println(" quad us: " + int_to_str(q1) + " " + int_to_str(q2) + " "
+ int_to_str(q3) + " " + int_to_str(q4)
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
assert measured == 4, "a rescan-from-zero workload must classify O(n^2)"
assert verdict == 1, "declared O(n) against measured O(n^2) must FAIL the gate"
}
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_math.el - native test suite for runtime/math.el // test_math.el - native test suite for runtime/math.el
// //
// Covers: integer math (abs, max, min), float math (sqrt, log, sin, cos, pi), // Covers: integer math (abs, max, min), float math (sqrt, log, sin, cos, pi),
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_state.el - native test suite for runtime/state.el // test_state.el - native test suite for runtime/state.el
// //
// Covers: state_set/get/del, state_has, state_get_or, state_keys, // Covers: state_set/get/del, state_has, state_get_or, state_keys,
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_string.el - native test suite for runtime/string.el // test_string.el - native test suite for runtime/string.el
// //
// Covers: type conversions, core primitives, comparison and search, // Covers: type conversions, core primitives, comparison and search,
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_text.el - native test suite for text primitives. // test_text.el - native test suite for text primitives.
// //
// Mirrors the acceptance corpus in tests/text/examples/ using the // Mirrors the acceptance corpus in tests/text/examples/ using the
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// test_time.el - native test suite for runtime/time.el // test_time.el - native test suite for runtime/time.el
// //
// Covers: time_now (positive timestamp), time_to_parts (UTC decomposition), // Covers: time_now (positive timestamp), time_to_parts (UTC decomposition),
@@ -0,0 +1,28 @@
fn getstr(x: String) -> String { return x }
fn getint(x: Int) -> Int { return x }
fn ok(label: String) -> Void { println("ok " + label) }
fn bad(label: String) -> Void { println("FAIL " + label) }
let s1: String = "hello"
let s2: String = "hello"
let s3: String = "world"
let i1: Int = 5
let i2: Int = 5
let i3: Int = 9
if "abc" == "abc" { ok("str literal eq") } else { bad("str literal eq") }
if "abc" == "xyz" { bad("str literal ne") } else { ok("str literal ne") }
if s1 == s2 { ok("str var eq") } else { bad("str var eq") }
if s1 == s3 { bad("str var ne") } else { ok("str var ne") }
if getstr("hi") == "hi" { ok("str call vs literal") } else { bad("str call vs literal") }
if s1 == getstr("hello") { ok("str var vs call") } else { bad("str var vs call") }
if s1 == getstr("nope") { bad("str var vs call ne") } else { ok("str var vs call ne") }
if i1 == i2 { ok("int var eq") } else { bad("int var eq") }
if i1 == i3 { bad("int var ne") } else { ok("int var ne") }
if getint(5) == i1 { ok("int call vs var") } else { bad("int call vs var") }
if getint(9) == i1 { bad("int call vs var ne") } else { ok("int call vs var ne") }
if s1 != s3 { ok("str NOTEQ") } else { bad("str NOTEQ") }
if s1 != s2 { bad("str NOTEQ same") } else { ok("str NOTEQ same") }
if i1 != i3 { ok("int NOTEQ") } else { bad("int NOTEQ") }
if getint(9) != i1 { ok("int call NOTEQ") } else { bad("int call NOTEQ") }
println("done")
+58
View File
@@ -0,0 +1,58 @@
fn expect_int(label: String, got: Int, want: Int) -> Void {
if got == want { println("ok " + label) }
else { println("FAIL " + label + " got=" + int_to_str(got) + " want=" + int_to_str(want)) }
}
fn expect_str(label: String, got: String, want: String) -> Void {
if str_eq(got, want) { println("ok " + label) }
else { println("FAIL " + label + " got='" + got + "' want='" + want + "'") }
}
// 1. basic char access across a string
let s: String = "hello"
expect_int("char[0]=h", str_char_code(s, 0), 104)
expect_int("char[4]=o", str_char_code(s, 4), 111)
expect_int("char[5] OOB -> 0", str_char_code(s, 5), 0)
expect_int("char[-1] OOB -> 0", str_char_code(s, -1), 0)
expect_int("empty string OOB", str_char_code("", 0), 0)
// 2. slices
expect_str("slice(0,5)", str_slice(s, 0, 5), "hello")
expect_str("slice(1,3)", str_slice(s, 1, 3), "el")
expect_str("slice past end clamps", str_slice(s, 3, 99), "lo")
expect_str("slice inverted -> empty", str_slice(s, 4, 2), "")
// 3. DIFFERENT strings must not share a cached length (the real hazard)
let a: String = "abc"
let b: String = "abcdefghij"
expect_int("a[2]=c", str_char_code(a, 2), 99)
expect_int("a[3] OOB", str_char_code(a, 3), 0)
expect_int("b[9]=j", str_char_code(b, 9), 106)
expect_int("b[3]=d after a", str_char_code(b, 3), 100)
expect_int("a[3] still OOB after b", str_char_code(a, 3), 0)
// 4. many distinct strings interleaved forces cache slot collisions
fn interleave(n: Int) -> Int {
let i: Int = 0
let bad: Int = 0
while i < n {
let t: String = int_to_str(i)
let l: Int = str_len(t)
let last: Int = str_char_code(t, l - 1)
let oob: Int = str_char_code(t, l)
if oob != 0 { let bad2: Int = bad + 1
let bad: Int = bad2 }
if last == 0 { let bad3: Int = bad + 1
let bad: Int = bad3 }
let i2: Int = i + 1
let i: Int = i2
}
return bad
}
expect_int("1000 interleaved strings, no bad reads", interleave(1000), 0)
// 5. concatenation changes length cache must not report the old one
let g: String = "12345"
let g2: String = g + "6789"
expect_int("grown string len via char", str_char_code(g2, 8), 57)
expect_int("original still bounded", str_char_code(g, 5), 0)
println("done")
+1
View File
@@ -1,3 +1,4 @@
import "../../runtime/eltest.el"
// tests/runtime/string_test.el Test suite for runtime/string.el // tests/runtime/string_test.el Test suite for runtime/string.el
// //
// Exercises every public function exported by runtime/string.el using the // Exercises every public function exported by runtime/string.el using the