Merge pull request 'runtime: put el_runtime.c on a ratchet, and actually run the guards' (#161) from fix/runtime-growth-guard into fix/runtime-shim-retire

scripts/check-single-runtime.sh guards against el_runtime.c being COPIED — it
was written after a lagging fork shipped to prod and dropped learned hebb edges.
Nothing guarded against it GROWING. So it grew: 10,607 -> 20,527 lines, 94% in
3.5 months, the whole time under an explicit commit-message promise that it was
a temporary shim about to be deleted.

Worse, the copy guard was never wired in. Its own footer described the CI
wire-in as a TODO, and the TODO had never been done — the script existed but ran
nowhere, in no workflow and in no hook, so it had caught nothing for as long as
it has been in the tree. A guard that does not run is a comment.

This adds the missing guard and runs both.

  * lang/runtime/BUDGET — a RATCHET, not a limit. max_lines is set at the
    current 20,527 with NO headroom: the file cannot grow by one line. A second
    cap, max_engram_fns (279), counts top-level engram_/eg_/cog_ definitions in
    it — ~47.5% of the file is engram code and engram already owns six sibling
    .c files, so this is the scoreboard for moving it out. Both may only go DOWN.

  * scripts/check-runtime-growth.sh — enforces the ratchet, and three
    invariants that keep the multi-file runtime honest: every .c in
    lang/runtime/ is either in SOURCES or explicitly platform-optional (an
    unaccounted .c is compiled by nothing and is silently dead); install.sh's
    hardcoded download list matches SOURCES (it cannot call the helper — it
    runs where there is no checkout — so that copy is checked, not trusted);
    and an advisory nudge to lower the budget when you have earned it.

  * Both guards now run as early steps in ci-dev.yaml, ci-stage.yaml and
    sdk-release.yaml, and in .githooks/pre-commit.

The failure message is the point. The guard that existed said what was wrong but
not where the code should go, which makes it easy to "fix" by arguing with the
guard. This one names the destination: the concern-owning .c, or a new .c plus
one line in SOURCES, or c_source in a program's manifest.el — and it prints the
`nm` command that proves placement is link-time and that the shipped compiler
already links from ten translation units. Every runtime file except el_runtime.c
is deliberately uncapped, because that is where code is supposed to go.

Proven with negative controls, per lang/AGENTS.md step 5 — each shown FAILING:
  * +1 line to el_runtime.c                  -> FAIL (20528/20527)
  * +1 engram fn, net-zero lines             -> FAIL (280/279)
  * a new unaccounted lang/runtime/*.c       -> FAIL
  * engram_store.c removed from install.sh   -> FAIL, names the missing file
  * el_runtime.c truncated to 20,000 lines   -> PASS + "lower max_lines to 20000"
  * baseline, tree unmodified                -> OK, and both guards green

el_runtime.c is byte-identical after the controls; this commit changes zero
lines of it.

.gitea/workflows/ci-dev.yaml

@@ -19,6 +19,16 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Guards must run from the REPO ROOT — override the job's
+      # defaults.run.working-directory: lang
+      - name: Guard - single canonical runtime source
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-single-runtime.sh
+
+      - name: Guard - el_runtime.c growth budget
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-runtime-growth.sh
+
       - name: Install build dependencies
         run: |
           apt-get update -qq

.gitea/workflows/ci-stage.yaml

@@ -29,6 +29,16 @@ jobs:
           fi
           echo "Source branch check passed: ${SOURCE} -> stage"
 
+      # Guards must run from the REPO ROOT — override the job's
+      # defaults.run.working-directory: lang
+      - name: Guard - single canonical runtime source
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-single-runtime.sh
+
+      - name: Guard - el_runtime.c growth budget
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-runtime-growth.sh
+
       - name: Install build dependencies
         run: |
           apt-get update -qq

.gitea/workflows/sdk-release.yaml

@@ -29,6 +29,16 @@ jobs:
           fi
           echo "Source branch check passed: ${SOURCE} -> main"
 
+      # Guards must run from the REPO ROOT — override the job's
+      # defaults.run.working-directory: lang
+      - name: Guard - single canonical runtime source
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-single-runtime.sh
+
+      - name: Guard - el_runtime.c growth budget
+        working-directory: ${{ github.workspace }}
+        run: bash scripts/check-runtime-growth.sh
+
       - name: Install build dependencies
         run: |
           apt-get update -qq

.githooks/pre-commit

@@ -9,6 +9,15 @@ LANG_DIR="$ROOT/lang"
 RUNTIME="$LANG_DIR/runtime"
 ELC="$LANG_DIR/dist/platform/elc"
 
+# Runtime guards — catch drift and growth before they are committed, not in CI.
+#   check-single-runtime.sh : el_runtime.c must not be FORKED (a lagging copy
+#                             shipped to prod and dropped learned hebb edges).
+#   check-runtime-growth.sh : el_runtime.c must not GROW (it is a 2026-05-03
+#                             build shim that was never retired; see BUDGET).
+echo "→ Runtime guards..."
+bash "$ROOT/scripts/check-single-runtime.sh"
+bash "$ROOT/scripts/check-runtime-growth.sh"
+
 # If elc isn't built yet, skip with a warning rather than blocking
 if [ ! -x "$ELC" ]; then
   echo "⚠  elc not found at lang/dist/platform/elc — skipping pre-commit tests"

lang/AGENTS.md

@@ -97,6 +97,8 @@ The runtime is native El (`runtime/*.el`) over a C OS-boundary. **Status (verifi
 
 Choose the file by concern: engram store ops → `engram_store.c`; index → `engram_vindex.c`; geometry/priming → `engram_geometry.c`; reasoning → `engram_reason.c`; grounding/consistency → `engram_verify.c`; think/stance → `engram_cognition.c`. **If no existing file owns it, create one** — add the `.c` to `runtime/SOURCES` (one line) and every build path picks it up. For a builtin that belongs to a downstream program rather than the runtime, declare `c_source "path/to/file.c"` in that program's `manifest.el`; `elb` already links it (`parse_manifest_c_sources`, `lang/elb.el:82`).
 
+> **`el_runtime.c` is on a ratchet and will reject your commit.** `runtime/BUDGET` caps it at its current line count *with no headroom*, and separately caps the number of `engram_*`/`eg_*`/`cog_*` function definitions in it. `scripts/check-runtime-growth.sh` enforces both in CI and in `.githooks/pre-commit`. **The numbers may only ever go down — do not raise them.** Every other runtime file is deliberately uncapped, because that is where the code is supposed to go. When you move code *out*, lower the numbers in the same commit; the guard tells you the new values.
+
 When you add a C builtin (verbatim-emit recipe — the El name is emitted as the exact C symbol; `builtin_arity` is an arity guard only, not a dispatch table):
 1. Implement the C function in the **concern-owning `.c`** (and declare it in that file's `.h`). Add the file to `runtime/SOURCES` if it is new. Only put it in `el_runtime.c` if it is genuinely EL core (val/str/map/list/arena) — that is ~8% of what is in there today.
 2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.

lang/runtime/BUDGET

@@ -0,0 +1,39 @@
+# BUDGET — a RATCHET on lang/runtime/el_runtime.c. Enforced by
+# scripts/check-runtime-growth.sh. These numbers may only ever go DOWN.
+#
+# WHY THIS FILE EXISTS
+# --------------------
+# scripts/check-single-runtime.sh guards against el_runtime.c being COPIED.
+# Nothing guarded against it GROWING. It grew from 10,607 lines to 20,527 —
+# 94% — in 3.5 months, while under an explicit commit-message promise that it
+# was a temporary shim about to be deleted.
+#
+# It grew because lang/AGENTS.md told every agent to grow it: it claimed
+# el_runtime.c was "the authoritative single-file link target" and that a new
+# C builtin "must live there to be linkable". That is false — placement is a
+# link-time concern, `builtin_arity` is an arity guard not a dispatch table,
+# and the shipped elc already links from ten translation units. The claim is
+# corrected, and this file is the mechanism that keeps it corrected.
+#
+# THIS IS A RATCHET, NOT A LIMIT
+# ------------------------------
+# The budget is set at the CURRENT size. There is no headroom, deliberately.
+# The file cannot grow by even one line. Any new code goes in the .c that owns
+# the concern — that is the whole point, and every other runtime file is
+# deliberately UNCAPPED.
+#
+# When you move code OUT, lower the number in the same commit. The guard tells
+# you to when you have earned it.
+#
+# FORMAT: <key> <value>   — `#` comments and blank lines ignored.
+
+# Maximum lines in lang/runtime/el_runtime.c.
+# 2026-08-16: 20,527 — the high-water mark, ratcheted from here.
+max_lines 20527
+
+# Maximum top-level engram/eg_/cog_ function definitions in el_runtime.c.
+# ~47.5% of the file is engram code, and engram already owns six dedicated
+# sibling files (engram_{store,vindex,geometry,reason,verify,cognition}.c).
+# Every one of these belongs in one of them. This is the Stage 3 scoreboard.
+# 2026-08-16: 279.
+max_engram_fns 279

scripts/check-runtime-growth.sh

@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# check-runtime-growth.sh — GROWTH guard for lang/runtime/el_runtime.c.
+#
+# Sibling to scripts/check-single-runtime.sh. That one guards against the file
+# being COPIED (a lagging fork shipped to prod and dropped learned hebb edges).
+# Nothing guarded against it GROWING — so it grew from 10,607 to 20,527 lines in
+# 3.5 months, while under an explicit commit-message promise that it was a
+# temporary shim about to be deleted.
+#
+# This enforces the RATCHET in lang/runtime/BUDGET: the numbers may only go down.
+#
+# It also checks two invariants that keep the multi-file runtime honest:
+#   * every .c in lang/runtime/ is either in SOURCES or explicitly optional
+#   * lang/install.sh's hardcoded download list matches SOURCES
+#
+# Exits non-zero on any violation. Run from anywhere; resolves the repo root.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$ROOT"
+
+RUNTIME_DIR="lang/runtime"
+TARGET="$RUNTIME_DIR/el_runtime.c"
+BUDGET_FILE="$RUNTIME_DIR/BUDGET"
+SOURCES_FILE="$RUNTIME_DIR/SOURCES"
+
+FAIL=0
+
+for f in "$TARGET" "$BUDGET_FILE" "$SOURCES_FILE"; do
+  if [ ! -f "$f" ]; then
+    echo "FATAL: required file missing: $f" >&2
+    exit 1
+  fi
+done
+
+budget() {
+  local key="$1"
+  sed -e 's/#.*//' "$BUDGET_FILE" | awk -v k="$key" '$1==k {print $2; found=1} END{if(!found) exit 1}'
+}
+
+MAX_LINES="$(budget max_lines)"       || { echo "FATAL: no 'max_lines' in $BUDGET_FILE" >&2; exit 1; }
+MAX_ENGRAM="$(budget max_engram_fns)" || { echo "FATAL: no 'max_engram_fns' in $BUDGET_FILE" >&2; exit 1; }
+
+# ---------------------------------------------------------------------------
+# The message every failure prints. The guard that existed before this one told
+# you what was wrong but not where the code should go — so it was easy to
+# "fix" by arguing with the guard. This one names the destination.
+# ---------------------------------------------------------------------------
+where_it_goes() {
+  cat >&2 <<'MSG'
+
+  WHERE THE CODE ACTUALLY GOES
+  ----------------------------
+  Placement is a LINK-TIME concern. The compiler cannot tell which .c a symbol
+  came from: `builtin_arity` in el-compiler/src/codegen.el maps NAME -> ARITY
+  INT only, the El name is emitted as the exact C symbol, and `ld` resolves it.
+  The SHIPPED compiler already links from ten translation units — check it:
+
+      nm lang/dist/platform/elc | grep -E 'T _(engram_think|vindex_insert)'
+
+  So a builtin defined in a sibling .c is EXACTLY as linkable as one defined in
+  el_runtime.c. Pick the file that owns the concern:
+
+      engram store ops ......... lang/runtime/engram_store.c
+      ANN / vector index ....... lang/runtime/engram_vindex.c
+      geometry, priming ........ lang/runtime/engram_geometry.c
+      reasoning operators ...... lang/runtime/engram_reason.c
+      grounding, consistency ... lang/runtime/engram_verify.c
+      think, stance ............ lang/runtime/engram_cognition.c
+
+  No existing file owns it?  Create one, add ONE line to lang/runtime/SOURCES,
+  and every build path picks it up. Every runtime file EXCEPT el_runtime.c is
+  deliberately uncapped.
+
+  Belongs to a downstream program, not the runtime?  Declare
+  `c_source "path/to/file.c"` in that program's manifest.el — elb already links
+  it (parse_manifest_c_sources, lang/elb.el:82).
+
+  See lang/AGENTS.md "Where a new C builtin goes".
+MSG
+}
+
+# --- 1. Line-count ratchet ---------------------------------------------------
+LINES="$(wc -l < "$TARGET" | tr -d ' ')"
+if [ "$LINES" -gt "$MAX_LINES" ]; then
+  echo "FAIL: $TARGET grew past its budget." >&2
+  echo "      now:    $LINES lines" >&2
+  echo "      budget: $MAX_LINES lines (lang/runtime/BUDGET: max_lines)" >&2
+  echo "      over by: $((LINES - MAX_LINES))" >&2
+  echo "" >&2
+  echo "This file is a 2026-05-03 build shim that was scheduled for deletion and" >&2
+  echo "never retired. It does not get to grow. Do NOT raise the budget." >&2
+  where_it_goes
+  FAIL=1
+fi
+
+# --- 2. Engram-concern ratchet ----------------------------------------------
+# ~47.5% of el_runtime.c is engram code, and engram already owns six sibling
+# files. This count is the Stage 3 scoreboard: it may only go down.
+ENGRAM_FNS="$(grep -cE '^(static +)?[A-Za-z_][A-Za-z0-9_ *]*\b(engram|eg|cog)_[a-z0-9_]+\(' "$TARGET" || true)"
+if [ "$ENGRAM_FNS" -gt "$MAX_ENGRAM" ]; then
+  echo "FAIL: new engram/eg_/cog_ function(s) added to $TARGET." >&2
+  echo "      now:    $ENGRAM_FNS definitions" >&2
+  echo "      budget: $MAX_ENGRAM (lang/runtime/BUDGET: max_engram_fns)" >&2
+  echo "" >&2
+  echo "Engram code belongs in the six engram_*.c files that already exist." >&2
+  where_it_goes
+  FAIL=1
+fi
+
+# --- 3. Ratchet-down nudge (advisory, never fails) ---------------------------
+if [ "$LINES" -lt "$MAX_LINES" ]; then
+  echo "NOTE: $TARGET is $((MAX_LINES - LINES)) lines under budget — lower" >&2
+  echo "      'max_lines' to $LINES in $BUDGET_FILE in this same commit, so the" >&2
+  echo "      ground you gained cannot be quietly given back." >&2
+fi
+if [ "$ENGRAM_FNS" -lt "$MAX_ENGRAM" ]; then
+  echo "NOTE: $((MAX_ENGRAM - ENGRAM_FNS)) engram fn(s) moved out — lower" >&2
+  echo "      'max_engram_fns' to $ENGRAM_FNS in $BUDGET_FILE in this same commit." >&2
+fi
+
+# --- 4. Every runtime .c is accounted for ------------------------------------
+# A new .c that is in neither SOURCES nor the optional list will not be
+# compiled by any build path — it would be silently dead. Catch that here.
+OPTIONAL_RE='^(el_android|el_gtk4|el_lvgl|el_sdl2|el_win32|el_runtime_win32|eg_cosine_batch_strategy_ggml|vindex_bench)\.c$'
+mapfile -t IN_SOURCES < <(scripts/el-runtime-sources.sh)
+for path in "$RUNTIME_DIR"/*.c; do
+  base="$(basename "$path")"
+  if printf '%s\n' "${IN_SOURCES[@]}" | grep -qxF "$base"; then continue; fi
+  if [[ "$base" =~ $OPTIONAL_RE ]]; then continue; fi
+  echo "FAIL: $path is in neither lang/runtime/SOURCES nor the platform-optional" >&2
+  echo "      list in this guard. It will not be compiled by any build path." >&2
+  echo "      Add it to SOURCES (one line), or add it to OPTIONAL_RE here if it" >&2
+  echo "      is a platform/strategy variant that is linked in deliberately." >&2
+  FAIL=1
+done
+
+# --- 5. install.sh must not drift from SOURCES -------------------------------
+# install.sh runs on machines with no repo checkout, so it cannot call
+# el-runtime-sources.sh and has to hardcode the list. That copy is exactly the
+# kind of duplicate that silently drifted before — so it is checked, not trusted.
+INSTALL_SH="lang/install.sh"
+if [ -f "$INSTALL_SH" ]; then
+  EXPECTED="$(scripts/el-runtime-sources.sh | sort)"
+  ACTUAL="$(sed -n '/^RUNTIME_SOURCES=(/,/^)/p' "$INSTALL_SH" \
+            | grep -oE '[a-z_0-9]+\.c' | sort)"
+  if [ "$EXPECTED" != "$ACTUAL" ]; then
+    echo "FAIL: $INSTALL_SH RUNTIME_SOURCES has drifted from $SOURCES_FILE." >&2
+    echo "      Only in SOURCES:    $(comm -23 <(echo "$EXPECTED") <(echo "$ACTUAL") | tr '\n' ' ')" >&2
+    echo "      Only in install.sh: $(comm -13 <(echo "$EXPECTED") <(echo "$ACTUAL") | tr '\n' ' ')" >&2
+    echo "      An SDK that ships the wrong set produces a lib/ that cannot link." >&2
+    FAIL=1
+  fi
+fi
+
+if [ "$FAIL" -ne 0 ]; then
+  exit 1
+fi
+
+echo "OK: el_runtime.c within budget ($LINES/$MAX_LINES lines, $ENGRAM_FNS/$MAX_ENGRAM engram fns);"
+echo "    runtime sources accounted for; install.sh in step with SOURCES."

scripts/check-single-runtime.sh

@@ -81,14 +81,14 @@ fi
 echo "OK: single canonical runtime source — $CANONICAL (no un-allowlisted forks)."
 
 # ---------------------------------------------------------------------------
-# CI wire-in:
+# CI wire-in — DONE (2026-08-16). This block used to describe the wire-in as a
-#   foundation/el .gitea/workflows/ci-dev.yaml, ci-stage.yaml, sdk-release.yaml
+# TODO, and it had never been done: the guard existed but ran nowhere, so it
-#   Add an early step (before the build/publish steps). It must run from the
+# caught nothing for as long as it has been in the tree. It is now an early step
-#   REPO ROOT, so override the job's `defaults.run.working-directory: lang`:
+# in ci-dev.yaml, ci-stage.yaml and sdk-release.yaml (each with
+# `working-directory: ${{ github.workspace }}`, since the jobs default to lang/),
+# and it runs in .githooks/pre-commit.
 #
-#     - name: Guard - single canonical runtime source
+# Its sibling scripts/check-runtime-growth.sh is wired in at the same points and
-#       working-directory: ${{ github.workspace }}
+# guards the other half of the problem: this script stops el_runtime.c being
-#       run: bash scripts/check-single-runtime.sh
+# COPIED, that one stops it GROWING.
-#
-#   Also add to .githooks/pre-commit so drift is caught before it is committed.
 # ---------------------------------------------------------------------------