runtime: extract engram_text.c, and repair 10 harnesses that could not link #162

Merged
will.anderson merged 1 commits from fix/runtime-extract-text into fix/runtime-growth-guard 2026-08-17 00:57:03 +00:00
Owner

Stacked on #161 (which stacks on #160). Base is fix/runtime-growth-guard. This PR's own diff is 16 files.

Root cause

Same shim-never-retired + false-instruction root cause as #160/#161. This is the first concern actually moved out, under the ratchet from #161.

What changed

engram_text.{c,h} — query tokenization, candidate-token hygiene, word-boundary matching, and the text-damage signature. Four functions, moved verbatim: only static was dropped and each doc comment travelled with the code. They touch no EL value type and no engram store type — plain C over <ctype.h>/<string.h> over char buffers. They were never el_runtime.c's business.

Before After
lang/runtime/el_runtime.c 20,527 lines 20,427
engram fns in it 279 275
BUDGET max_lines 20527 / max_engram_fns 279 ratcheted to 20427 / 275

The move is deliberately small. It exists to prove the mechanism end to end before anything large depends on it — and it did:

  • The Stage 1 extension point worked: adding the file to lang/runtime/SOURCES was one line, and every build path picked it up.
  • The Stage 2 drift guard then caught my own omission — I had not added it to install.sh's standalone list. The exact class of drift it was written for, on its first real change, before the commit rather than after a broken SDK shipped.

Why only 100 lines — and what actually blocks the rest

Measured, not estimated. Of 273 engram-domain functions in el_runtime.c (~9,700 lines), only 75 (~1,058 lines) can move today, and they are scattered rather than clustered. One fact blocks the rest:

EngramNode, EngramEdge, EngramStore, EngramLayer, EngramWal and EngramIdSlot are typedef'd inside el_runtime.c. No sibling can see them. engram_store.h defines a separate serializable "node view" struct and maps between the two.

So every engram function taking an EngramNode*109 of 273 by direct type reference — cannot compile in engram_store.c until those types move to a shared header.

That extraction is the real Stage 3 enabler and deserves its own change. It touches the most load-bearing struct in the system, and doing it in the same commit as a code move would make a regression impossible to bisect.

Repaired: 10 engram harnesses that could not link

Not new breakage from this move — verified against unmodified dev, where el_runtime.c + engram_store.c alone already failed with undefined symbols. They had been dead for as long as el_runtime.c has been calling into the siblings, and nothing noticed because nothing ran them.

  • run_m3_parity, run_m7_traversal, run_m35_hebb_persist, run_interoception_p0..p5 — now build from $(scripts/el-runtime-sources.sh)
  • run_wal_tests — its two TUs #include "el_runtime.c" directly, so it links the siblings only; adding el_runtime.c to that link line would define every symbol twice

(Worth recording: the runtime does have an #include'd .c, in engram/test/test_wal.c and the generated test_failloud.c.)

Verified locally — every one of these was run, not assumed

Check Result
run_m3_parity PASS, incl. ASan+UBSan clean across seed/on/reboot
run_m7_traversal PASS
run_m35_hebb_persist PASS — the gate over the original prod hebb bug
run_interoception_p0..p5 PASS (all six)
run_wal_tests 66 passed, 0 failed + fail-loud exit check
Self-host fixpoint byte-identical — and the emitted C is byte-identical to the pre-move compiler output, so the move changes nothing the compiler produces
engram/src/server.el compiles and links
Native suites 8 of 13 — unchanged; same 5 pre-existing failures, no regression
Both runtime guards green at the new, lower budget

Also fixes a block comment left unterminated by the extraction (the deleted range carried its closing */), restoring the compile to its single pre-existing -Wcomment warning.

What remains

  1. The enabler — move EngramNode/EngramEdge/EngramStore/EngramLayer/EngramWal/EngramIdSlot into a shared header. Unblocks the other ~8,700 lines. Recommended next.
  2. Then move engram code by concern into the six sibling files, ratcheting BUDGET after each move.
  3. Still not attempted: make elc emit #include el_seed.h and drop elb's hardcoded runtime path — finishing the 2026-05-03 migration.

Coordination

Does not touch engram/src/server.el, peripheral/, or elp/. Changes under engram/test/ only. No lang/AGENTS.md change here, so no interaction with PR #157.

**Stacked on #161** (which stacks on #160). Base is `fix/runtime-growth-guard`. This PR's own diff is 16 files. ## Root cause Same shim-never-retired + false-instruction root cause as #160/#161. This is the first concern actually moved **out**, under the ratchet from #161. ## What changed **`engram_text.{c,h}`** — query tokenization, candidate-token hygiene, word-boundary matching, and the text-damage signature. Four functions, **moved verbatim**: only `static` was dropped and each doc comment travelled with the code. They touch **no EL value type and no engram store type** — plain C over `<ctype.h>`/`<string.h>` over char buffers. They were never `el_runtime.c`'s business. | | Before | After | |---|---|---| | `lang/runtime/el_runtime.c` | 20,527 lines | **20,427** | | engram fns in it | 279 | **275** | | `BUDGET` | `max_lines 20527` / `max_engram_fns 279` | **ratcheted to 20427 / 275** | **The move is deliberately small.** It exists to prove the mechanism end to end before anything large depends on it — and it did: - The **Stage 1 extension point worked**: adding the file to `lang/runtime/SOURCES` was **one line**, and every build path picked it up. - The **Stage 2 drift guard then caught my own omission** — I had not added it to `install.sh`'s standalone list. The exact class of drift it was written for, **on its first real change, before the commit** rather than after a broken SDK shipped. ## Why only 100 lines — and what actually blocks the rest **Measured, not estimated.** Of 273 engram-domain functions in `el_runtime.c` (~9,700 lines), only **75 (~1,058 lines) can move today**, and they are scattered rather than clustered. One fact blocks the rest: > `EngramNode`, `EngramEdge`, `EngramStore`, `EngramLayer`, `EngramWal` and `EngramIdSlot` are `typedef`'d **inside `el_runtime.c`**. No sibling can see them. `engram_store.h` defines a **separate** serializable "node view" struct and maps between the two. So every engram function taking an `EngramNode*` — **109 of 273** by direct type reference — cannot compile in `engram_store.c` until those types move to a shared header. **That extraction is the real Stage 3 enabler and deserves its own change.** It touches the most load-bearing struct in the system, and doing it in the same commit as a code move would make a regression impossible to bisect. ## Repaired: 10 engram harnesses that could not link **Not new breakage from this move** — verified against unmodified `dev`, where `el_runtime.c` + `engram_store.c` alone already failed with undefined symbols. They had been dead for as long as `el_runtime.c` has been calling into the siblings, **and nothing noticed because nothing ran them.** - `run_m3_parity`, `run_m7_traversal`, `run_m35_hebb_persist`, `run_interoception_p0..p5` — now build from `$(scripts/el-runtime-sources.sh)` - `run_wal_tests` — its two TUs `#include "el_runtime.c"` **directly**, so it links the **siblings only**; adding `el_runtime.c` to that link line would define every symbol twice *(Worth recording: the runtime does have an `#include`'d `.c`, in `engram/test/test_wal.c` and the generated `test_failloud.c`.)* ## Verified locally — every one of these was run, not assumed | Check | Result | |---|---| | `run_m3_parity` | **PASS**, incl. **ASan+UBSan clean** across seed/on/reboot | | `run_m7_traversal` | **PASS** | | `run_m35_hebb_persist` | **PASS** — the gate over the original prod hebb bug | | `run_interoception_p0..p5` | **PASS** (all six) | | `run_wal_tests` | **66 passed, 0 failed** + fail-loud exit check | | Self-host fixpoint | **byte-identical** — and the emitted C is **byte-identical to the pre-move compiler output**, so the move changes nothing the compiler produces | | `engram/src/server.el` | compiles and links | | Native suites | **8 of 13 — unchanged**; same 5 pre-existing failures, no regression | | Both runtime guards | green at the new, lower budget | Also fixes a block comment left unterminated by the extraction (the deleted range carried its closing `*/`), restoring the compile to its single pre-existing `-Wcomment` warning. ## What remains 1. **The enabler** — move `EngramNode`/`EngramEdge`/`EngramStore`/`EngramLayer`/`EngramWal`/`EngramIdSlot` into a shared header. Unblocks the other ~8,700 lines. **Recommended next.** 2. Then move engram code by concern into the six sibling files, ratcheting `BUDGET` after each move. 3. **Still not attempted:** make `elc` emit `#include el_seed.h` and drop `elb`'s hardcoded runtime path — finishing the 2026-05-03 migration. ## Coordination Does not touch `engram/src/server.el`, `peripheral/`, or `elp/`. Changes under `engram/test/` only. No `lang/AGENTS.md` change here, so no interaction with PR #157.
will.anderson added 1 commit 2026-08-16 21:58:47 +00:00
First concern moved out of el_runtime.c under the ratchet, and the move is
deliberately small: it exists to prove the mechanism end to end before anything
large depends on it.

engram_text.{c,h} — query tokenization, candidate-token hygiene, word-boundary
matching, and the text-damage signature. Four functions, moved verbatim; only
`static` was dropped and each doc comment travelled with the code. They touch no
EL value type and no engram store type: plain C over <ctype.h>/<string.h> over
char buffers. They were never el_runtime.c's business.

  el_runtime.c   20,527 -> 20,427 lines   (BUDGET max_lines ratcheted down)
  engram fns        279 -> 275            (BUDGET max_engram_fns ratcheted down)

The Stage 1 extension point worked as designed: adding the file to
lang/runtime/SOURCES was one line, and every build path picked it up. The
Stage 2 drift guard then caught that I had NOT added it to install.sh's
standalone list — the exact class of drift it was written for, on its first
real change, before the commit rather than after a broken SDK shipped.

WHY ONLY 100 LINES, AND WHAT ACTUALLY BLOCKS THE REST

Measured, not estimated: of 273 engram-domain functions in el_runtime.c
(~9,700 lines), only 75 (~1,058 lines) can move today, and they are scattered
rather than clustered. The blocker is a single fact:

  EngramNode, EngramEdge, EngramStore, EngramLayer, EngramWal and EngramIdSlot
  are typedef'd INSIDE el_runtime.c. No sibling can see them. engram_store.h
  defines a SEPARATE serializable "node view" struct and maps between the two.

So every engram function that takes an EngramNode* — which is most of them, 109
of 273 by direct type reference — cannot compile in engram_store.c until those
types move to a shared header. That extraction is the real Stage 3 enabler and
it deserves its own change: it touches the most load-bearing struct in the
system, and doing it in the same commit as a code move would make a regression
impossible to bisect.

REPAIRED: 10 engram harnesses that had silently stopped linking

Not new breakage from this move — verified against unmodified dev, where
el_runtime.c + engram_store.c alone already failed with undefined symbols.
They had been dead for as long as el_runtime.c has been calling into the
siblings, and nothing noticed because nothing ran them.

  run_m3_parity, run_m7_traversal, run_m35_hebb_persist,
  run_interoception_p0..p5   — now build from $(scripts/el-runtime-sources.sh)
  run_wal_tests              — its two TUs #include "el_runtime.c" directly, so
                               it links the SIBLINGS ONLY; adding el_runtime.c
                               to that link line would define every symbol twice

(That #include'd .c is worth recording: the runtime does have one, in
engram/test/test_wal.c and the generated test_failloud.c.)

Verified locally — every one of these was run, not assumed:
  * m3_parity ............ PASS, incl. ASan+UBSan clean across seed/on/reboot
  * m7_traversal ......... PASS
  * m35_hebb_persist ..... PASS   (the gate over the original prod hebb bug)
  * interoception p0..p5 . PASS   (all six)
  * wal_tests ............ 66 passed, 0 failed, + fail-loud exit check
  * self-host fixpoint ... byte-identical, AND the emitted C is byte-identical
                           to the pre-move compiler output — the move changes
                           nothing the compiler produces
  * engram/src/server.el . compiles and links
  * native suites ........ 8 of 13, unchanged from before the move; the same 5
                           pre-existing failures, no regression
  * both runtime guards .. green at the new, lower budget

Also fixes a block comment left unterminated by the extraction (the deleted
range carried its closing */), restoring the compile to its single pre-existing
-Wcomment warning.
will.anderson merged commit 858e9ccd2d into fix/runtime-growth-guard 2026-08-17 00:57:03 +00:00
Sign in to join this conversation.