runtime: land the growth ratchet and engram_text extraction on dev #163
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: neuron-technologies/el#163
Reference in New Issue
Block a user
Delete Branch "fix/runtime-stack-on-dev"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
#161 and #162 never reached
dev. They were stacked — #161 based onfix/runtime-shim-retire, #162 based onfix/runtime-growth-guard— and #160 merged that parent intodevbefore either landed. Both reportedmerged=trueagainst their own base, so the ratchet and the extraction were stranded one level down and invisible indev.This is both commits rebased onto current
dev. The rebase matters: the original branch forked before #159, so merging it as-is would have deletedorgan_cli.el,organ_converse.elandorgan_dsp.eland revertedel speaks. After rebase the diff only adds.Carries:
fe634c4— put el_runtime.c on a ratchet, and actually run the guards.check-single-runtime.shhad never been wired in; its own footer still called the wire-in a TODO, so the anti-fork guard born from the hebb-edge production bug ran nowhere. Now runs in 3 workflows + the hook, alongside a newcheck-runtime-growth.sh.678dac5— extractengram_text.c(+.h), and repair 10 engram harnesses that had silently stopped linking, including the ASan/UBSan parity harness and the hebb-persistence gate.Verified by local build, which is the only bar.
elccompilesengram/src/server.el(1853 lines C) and the full 11-source link set fromlang/runtime/SOURCESplusel_peripheral_null.clinks to a 717,608-byte binary, clean.scripts/check-single-runtime.sh guards against el_runtime.c being COPIED — it was written after a lagging fork shipped to prod and dropped learned hebb edges. Nothing guarded against it GROWING. So it grew: 10,607 -> 20,527 lines, 94% in 3.5 months, the whole time under an explicit commit-message promise that it was a temporary shim about to be deleted. Worse, the copy guard was never wired in. Its own footer described the CI wire-in as a TODO, and the TODO had never been done — the script existed but ran nowhere, in no workflow and in no hook, so it had caught nothing for as long as it has been in the tree. A guard that does not run is a comment. This adds the missing guard and runs both. * lang/runtime/BUDGET — a RATCHET, not a limit. max_lines is set at the current 20,527 with NO headroom: the file cannot grow by one line. A second cap, max_engram_fns (279), counts top-level engram_/eg_/cog_ definitions in it — ~47.5% of the file is engram code and engram already owns six sibling .c files, so this is the scoreboard for moving it out. Both may only go DOWN. * scripts/check-runtime-growth.sh — enforces the ratchet, and three invariants that keep the multi-file runtime honest: every .c in lang/runtime/ is either in SOURCES or explicitly platform-optional (an unaccounted .c is compiled by nothing and is silently dead); install.sh's hardcoded download list matches SOURCES (it cannot call the helper — it runs where there is no checkout — so that copy is checked, not trusted); and an advisory nudge to lower the budget when you have earned it. * Both guards now run as early steps in ci-dev.yaml, ci-stage.yaml and sdk-release.yaml, and in .githooks/pre-commit. The failure message is the point. The guard that existed said what was wrong but not where the code should go, which makes it easy to "fix" by arguing with the guard. This one names the destination: the concern-owning .c, or a new .c plus one line in SOURCES, or c_source in a program's manifest.el — and it prints the `nm` command that proves placement is link-time and that the shipped compiler already links from ten translation units. Every runtime file except el_runtime.c is deliberately uncapped, because that is where code is supposed to go. Proven with negative controls, per lang/AGENTS.md step 5 — each shown FAILING: * +1 line to el_runtime.c -> FAIL (20528/20527) * +1 engram fn, net-zero lines -> FAIL (280/279) * a new unaccounted lang/runtime/*.c -> FAIL * engram_store.c removed from install.sh -> FAIL, names the missing file * el_runtime.c truncated to 20,000 lines -> PASS + "lower max_lines to 20000" * baseline, tree unmodified -> OK, and both guards green el_runtime.c is byte-identical after the controls; this commit changes zero lines of it.First concern moved out of el_runtime.c under the ratchet, and the move is deliberately small: it exists to prove the mechanism end to end before anything large depends on it. engram_text.{c,h} — query tokenization, candidate-token hygiene, word-boundary matching, and the text-damage signature. Four functions, moved verbatim; only `static` was dropped and each doc comment travelled with the code. They touch no EL value type and no engram store type: plain C over <ctype.h>/<string.h> over char buffers. They were never el_runtime.c's business. el_runtime.c 20,527 -> 20,427 lines (BUDGET max_lines ratcheted down) engram fns 279 -> 275 (BUDGET max_engram_fns ratcheted down) The Stage 1 extension point worked as designed: adding the file to lang/runtime/SOURCES was one line, and every build path picked it up. The Stage 2 drift guard then caught that I had NOT added it to install.sh's standalone list — the exact class of drift it was written for, on its first real change, before the commit rather than after a broken SDK shipped. WHY ONLY 100 LINES, AND WHAT ACTUALLY BLOCKS THE REST Measured, not estimated: of 273 engram-domain functions in el_runtime.c (~9,700 lines), only 75 (~1,058 lines) can move today, and they are scattered rather than clustered. The blocker is a single fact: EngramNode, EngramEdge, EngramStore, EngramLayer, EngramWal and EngramIdSlot are typedef'd INSIDE el_runtime.c. No sibling can see them. engram_store.h defines a SEPARATE serializable "node view" struct and maps between the two. So every engram function that takes an EngramNode* — which is most of them, 109 of 273 by direct type reference — cannot compile in engram_store.c until those types move to a shared header. That extraction is the real Stage 3 enabler and it deserves its own change: it touches the most load-bearing struct in the system, and doing it in the same commit as a code move would make a regression impossible to bisect. REPAIRED: 10 engram harnesses that had silently stopped linking Not new breakage from this move — verified against unmodified dev, where el_runtime.c + engram_store.c alone already failed with undefined symbols. They had been dead for as long as el_runtime.c has been calling into the siblings, and nothing noticed because nothing ran them. run_m3_parity, run_m7_traversal, run_m35_hebb_persist, run_interoception_p0..p5 — now build from $(scripts/el-runtime-sources.sh) run_wal_tests — its two TUs #include "el_runtime.c" directly, so it links the SIBLINGS ONLY; adding el_runtime.c to that link line would define every symbol twice (That #include'd .c is worth recording: the runtime does have one, in engram/test/test_wal.c and the generated test_failloud.c.) Verified locally — every one of these was run, not assumed: * m3_parity ............ PASS, incl. ASan+UBSan clean across seed/on/reboot * m7_traversal ......... PASS * m35_hebb_persist ..... PASS (the gate over the original prod hebb bug) * interoception p0..p5 . PASS (all six) * wal_tests ............ 66 passed, 0 failed, + fail-loud exit check * self-host fixpoint ... byte-identical, AND the emitted C is byte-identical to the pre-move compiler output — the move changes nothing the compiler produces * engram/src/server.el . compiles and links * native suites ........ 8 of 13, unchanged from before the move; the same 5 pre-existing failures, no regression * both runtime guards .. green at the new, lower budget Also fixes a block comment left unterminated by the extraction (the deleted range carried its closing */), restoring the compile to its single pre-existing -Wcomment warning.