runtime: land the growth ratchet and engram_text extraction on dev #163
@@ -19,6 +19,16 @@ jobs:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# Guards must run from the REPO ROOT — override the job's
|
||||
# defaults.run.working-directory: lang
|
||||
- name: Guard - single canonical runtime source
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-single-runtime.sh
|
||||
|
||||
- name: Guard - el_runtime.c growth budget
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-runtime-growth.sh
|
||||
|
||||
- name: Install build dependencies
|
||||
run: |
|
||||
apt-get update -qq
|
||||
|
||||
@@ -29,6 +29,16 @@ jobs:
|
||||
fi
|
||||
echo "Source branch check passed: ${SOURCE} -> stage"
|
||||
|
||||
# Guards must run from the REPO ROOT — override the job's
|
||||
# defaults.run.working-directory: lang
|
||||
- name: Guard - single canonical runtime source
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-single-runtime.sh
|
||||
|
||||
- name: Guard - el_runtime.c growth budget
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-runtime-growth.sh
|
||||
|
||||
- name: Install build dependencies
|
||||
run: |
|
||||
apt-get update -qq
|
||||
|
||||
@@ -29,6 +29,16 @@ jobs:
|
||||
fi
|
||||
echo "Source branch check passed: ${SOURCE} -> main"
|
||||
|
||||
# Guards must run from the REPO ROOT — override the job's
|
||||
# defaults.run.working-directory: lang
|
||||
- name: Guard - single canonical runtime source
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-single-runtime.sh
|
||||
|
||||
- name: Guard - el_runtime.c growth budget
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: bash scripts/check-runtime-growth.sh
|
||||
|
||||
- name: Install build dependencies
|
||||
run: |
|
||||
apt-get update -qq
|
||||
|
||||
@@ -9,6 +9,15 @@ LANG_DIR="$ROOT/lang"
|
||||
RUNTIME="$LANG_DIR/runtime"
|
||||
ELC="$LANG_DIR/dist/platform/elc"
|
||||
|
||||
# Runtime guards — catch drift and growth before they are committed, not in CI.
|
||||
# check-single-runtime.sh : el_runtime.c must not be FORKED (a lagging copy
|
||||
# shipped to prod and dropped learned hebb edges).
|
||||
# check-runtime-growth.sh : el_runtime.c must not GROW (it is a 2026-05-03
|
||||
# build shim that was never retired; see BUDGET).
|
||||
echo "→ Runtime guards..."
|
||||
bash "$ROOT/scripts/check-single-runtime.sh"
|
||||
bash "$ROOT/scripts/check-runtime-growth.sh"
|
||||
|
||||
# If elc isn't built yet, skip with a warning rather than blocking
|
||||
if [ ! -x "$ELC" ]; then
|
||||
echo "⚠ elc not found at lang/dist/platform/elc — skipping pre-commit tests"
|
||||
|
||||
@@ -97,6 +97,8 @@ The runtime is native El (`runtime/*.el`) over a C OS-boundary. **Status (verifi
|
||||
|
||||
Choose the file by concern: engram store ops → `engram_store.c`; index → `engram_vindex.c`; geometry/priming → `engram_geometry.c`; reasoning → `engram_reason.c`; grounding/consistency → `engram_verify.c`; think/stance → `engram_cognition.c`. **If no existing file owns it, create one** — add the `.c` to `runtime/SOURCES` (one line) and every build path picks it up. For a builtin that belongs to a downstream program rather than the runtime, declare `c_source "path/to/file.c"` in that program's `manifest.el`; `elb` already links it (`parse_manifest_c_sources`, `lang/elb.el:82`).
|
||||
|
||||
> **`el_runtime.c` is on a ratchet and will reject your commit.** `runtime/BUDGET` caps it at its current line count *with no headroom*, and separately caps the number of `engram_*`/`eg_*`/`cog_*` function definitions in it. `scripts/check-runtime-growth.sh` enforces both in CI and in `.githooks/pre-commit`. **The numbers may only ever go down — do not raise them.** Every other runtime file is deliberately uncapped, because that is where the code is supposed to go. When you move code *out*, lower the numbers in the same commit; the guard tells you the new values.
|
||||
|
||||
When you add a C builtin (verbatim-emit recipe — the El name is emitted as the exact C symbol; `builtin_arity` is an arity guard only, not a dispatch table):
|
||||
1. Implement the C function in the **concern-owning `.c`** (and declare it in that file's `.h`). Add the file to `runtime/SOURCES` if it is new. Only put it in `el_runtime.c` if it is genuinely EL core (val/str/map/list/arena) — that is ~8% of what is in there today.
|
||||
2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# BUDGET — a RATCHET on lang/runtime/el_runtime.c. Enforced by
|
||||
# scripts/check-runtime-growth.sh. These numbers may only ever go DOWN.
|
||||
#
|
||||
# WHY THIS FILE EXISTS
|
||||
# --------------------
|
||||
# scripts/check-single-runtime.sh guards against el_runtime.c being COPIED.
|
||||
# Nothing guarded against it GROWING. It grew from 10,607 lines to 20,527 —
|
||||
# 94% — in 3.5 months, while under an explicit commit-message promise that it
|
||||
# was a temporary shim about to be deleted.
|
||||
#
|
||||
# It grew because lang/AGENTS.md told every agent to grow it: it claimed
|
||||
# el_runtime.c was "the authoritative single-file link target" and that a new
|
||||
# C builtin "must live there to be linkable". That is false — placement is a
|
||||
# link-time concern, `builtin_arity` is an arity guard not a dispatch table,
|
||||
# and the shipped elc already links from ten translation units. The claim is
|
||||
# corrected, and this file is the mechanism that keeps it corrected.
|
||||
#
|
||||
# THIS IS A RATCHET, NOT A LIMIT
|
||||
# ------------------------------
|
||||
# The budget is set at the CURRENT size. There is no headroom, deliberately.
|
||||
# The file cannot grow by even one line. Any new code goes in the .c that owns
|
||||
# the concern — that is the whole point, and every other runtime file is
|
||||
# deliberately UNCAPPED.
|
||||
#
|
||||
# When you move code OUT, lower the number in the same commit. The guard tells
|
||||
# you to when you have earned it.
|
||||
#
|
||||
# FORMAT: <key> <value> — `#` comments and blank lines ignored.
|
||||
|
||||
# Maximum lines in lang/runtime/el_runtime.c.
|
||||
# 2026-08-16: 20,527 — the high-water mark, ratcheted from here.
|
||||
max_lines 20527
|
||||
|
||||
# Maximum top-level engram/eg_/cog_ function definitions in el_runtime.c.
|
||||
# ~47.5% of the file is engram code, and engram already owns six dedicated
|
||||
# sibling files (engram_{store,vindex,geometry,reason,verify,cognition}.c).
|
||||
# Every one of these belongs in one of them. This is the Stage 3 scoreboard.
|
||||
# 2026-08-16: 279.
|
||||
max_engram_fns 279
|
||||
Executable
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# check-runtime-growth.sh — GROWTH guard for lang/runtime/el_runtime.c.
|
||||
#
|
||||
# Sibling to scripts/check-single-runtime.sh. That one guards against the file
|
||||
# being COPIED (a lagging fork shipped to prod and dropped learned hebb edges).
|
||||
# Nothing guarded against it GROWING — so it grew from 10,607 to 20,527 lines in
|
||||
# 3.5 months, while under an explicit commit-message promise that it was a
|
||||
# temporary shim about to be deleted.
|
||||
#
|
||||
# This enforces the RATCHET in lang/runtime/BUDGET: the numbers may only go down.
|
||||
#
|
||||
# It also checks two invariants that keep the multi-file runtime honest:
|
||||
# * every .c in lang/runtime/ is either in SOURCES or explicitly optional
|
||||
# * lang/install.sh's hardcoded download list matches SOURCES
|
||||
#
|
||||
# Exits non-zero on any violation. Run from anywhere; resolves the repo root.
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
RUNTIME_DIR="lang/runtime"
|
||||
TARGET="$RUNTIME_DIR/el_runtime.c"
|
||||
BUDGET_FILE="$RUNTIME_DIR/BUDGET"
|
||||
SOURCES_FILE="$RUNTIME_DIR/SOURCES"
|
||||
|
||||
FAIL=0
|
||||
|
||||
for f in "$TARGET" "$BUDGET_FILE" "$SOURCES_FILE"; do
|
||||
if [ ! -f "$f" ]; then
|
||||
echo "FATAL: required file missing: $f" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
budget() {
|
||||
local key="$1"
|
||||
sed -e 's/#.*//' "$BUDGET_FILE" | awk -v k="$key" '$1==k {print $2; found=1} END{if(!found) exit 1}'
|
||||
}
|
||||
|
||||
MAX_LINES="$(budget max_lines)" || { echo "FATAL: no 'max_lines' in $BUDGET_FILE" >&2; exit 1; }
|
||||
MAX_ENGRAM="$(budget max_engram_fns)" || { echo "FATAL: no 'max_engram_fns' in $BUDGET_FILE" >&2; exit 1; }
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The message every failure prints. The guard that existed before this one told
|
||||
# you what was wrong but not where the code should go — so it was easy to
|
||||
# "fix" by arguing with the guard. This one names the destination.
|
||||
# ---------------------------------------------------------------------------
|
||||
where_it_goes() {
|
||||
cat >&2 <<'MSG'
|
||||
|
||||
WHERE THE CODE ACTUALLY GOES
|
||||
----------------------------
|
||||
Placement is a LINK-TIME concern. The compiler cannot tell which .c a symbol
|
||||
came from: `builtin_arity` in el-compiler/src/codegen.el maps NAME -> ARITY
|
||||
INT only, the El name is emitted as the exact C symbol, and `ld` resolves it.
|
||||
The SHIPPED compiler already links from ten translation units — check it:
|
||||
|
||||
nm lang/dist/platform/elc | grep -E 'T _(engram_think|vindex_insert)'
|
||||
|
||||
So a builtin defined in a sibling .c is EXACTLY as linkable as one defined in
|
||||
el_runtime.c. Pick the file that owns the concern:
|
||||
|
||||
engram store ops ......... lang/runtime/engram_store.c
|
||||
ANN / vector index ....... lang/runtime/engram_vindex.c
|
||||
geometry, priming ........ lang/runtime/engram_geometry.c
|
||||
reasoning operators ...... lang/runtime/engram_reason.c
|
||||
grounding, consistency ... lang/runtime/engram_verify.c
|
||||
think, stance ............ lang/runtime/engram_cognition.c
|
||||
|
||||
No existing file owns it? Create one, add ONE line to lang/runtime/SOURCES,
|
||||
and every build path picks it up. Every runtime file EXCEPT el_runtime.c is
|
||||
deliberately uncapped.
|
||||
|
||||
Belongs to a downstream program, not the runtime? Declare
|
||||
`c_source "path/to/file.c"` in that program's manifest.el — elb already links
|
||||
it (parse_manifest_c_sources, lang/elb.el:82).
|
||||
|
||||
See lang/AGENTS.md "Where a new C builtin goes".
|
||||
MSG
|
||||
}
|
||||
|
||||
# --- 1. Line-count ratchet ---------------------------------------------------
|
||||
LINES="$(wc -l < "$TARGET" | tr -d ' ')"
|
||||
if [ "$LINES" -gt "$MAX_LINES" ]; then
|
||||
echo "FAIL: $TARGET grew past its budget." >&2
|
||||
echo " now: $LINES lines" >&2
|
||||
echo " budget: $MAX_LINES lines (lang/runtime/BUDGET: max_lines)" >&2
|
||||
echo " over by: $((LINES - MAX_LINES))" >&2
|
||||
echo "" >&2
|
||||
echo "This file is a 2026-05-03 build shim that was scheduled for deletion and" >&2
|
||||
echo "never retired. It does not get to grow. Do NOT raise the budget." >&2
|
||||
where_it_goes
|
||||
FAIL=1
|
||||
fi
|
||||
|
||||
# --- 2. Engram-concern ratchet ----------------------------------------------
|
||||
# ~47.5% of el_runtime.c is engram code, and engram already owns six sibling
|
||||
# files. This count is the Stage 3 scoreboard: it may only go down.
|
||||
ENGRAM_FNS="$(grep -cE '^(static +)?[A-Za-z_][A-Za-z0-9_ *]*\b(engram|eg|cog)_[a-z0-9_]+\(' "$TARGET" || true)"
|
||||
if [ "$ENGRAM_FNS" -gt "$MAX_ENGRAM" ]; then
|
||||
echo "FAIL: new engram/eg_/cog_ function(s) added to $TARGET." >&2
|
||||
echo " now: $ENGRAM_FNS definitions" >&2
|
||||
echo " budget: $MAX_ENGRAM (lang/runtime/BUDGET: max_engram_fns)" >&2
|
||||
echo "" >&2
|
||||
echo "Engram code belongs in the six engram_*.c files that already exist." >&2
|
||||
where_it_goes
|
||||
FAIL=1
|
||||
fi
|
||||
|
||||
# --- 3. Ratchet-down nudge (advisory, never fails) ---------------------------
|
||||
if [ "$LINES" -lt "$MAX_LINES" ]; then
|
||||
echo "NOTE: $TARGET is $((MAX_LINES - LINES)) lines under budget — lower" >&2
|
||||
echo " 'max_lines' to $LINES in $BUDGET_FILE in this same commit, so the" >&2
|
||||
echo " ground you gained cannot be quietly given back." >&2
|
||||
fi
|
||||
if [ "$ENGRAM_FNS" -lt "$MAX_ENGRAM" ]; then
|
||||
echo "NOTE: $((MAX_ENGRAM - ENGRAM_FNS)) engram fn(s) moved out — lower" >&2
|
||||
echo " 'max_engram_fns' to $ENGRAM_FNS in $BUDGET_FILE in this same commit." >&2
|
||||
fi
|
||||
|
||||
# --- 4. Every runtime .c is accounted for ------------------------------------
|
||||
# A new .c that is in neither SOURCES nor the optional list will not be
|
||||
# compiled by any build path — it would be silently dead. Catch that here.
|
||||
OPTIONAL_RE='^(el_android|el_gtk4|el_lvgl|el_sdl2|el_win32|el_runtime_win32|eg_cosine_batch_strategy_ggml|vindex_bench)\.c$'
|
||||
mapfile -t IN_SOURCES < <(scripts/el-runtime-sources.sh)
|
||||
for path in "$RUNTIME_DIR"/*.c; do
|
||||
base="$(basename "$path")"
|
||||
if printf '%s\n' "${IN_SOURCES[@]}" | grep -qxF "$base"; then continue; fi
|
||||
if [[ "$base" =~ $OPTIONAL_RE ]]; then continue; fi
|
||||
echo "FAIL: $path is in neither lang/runtime/SOURCES nor the platform-optional" >&2
|
||||
echo " list in this guard. It will not be compiled by any build path." >&2
|
||||
echo " Add it to SOURCES (one line), or add it to OPTIONAL_RE here if it" >&2
|
||||
echo " is a platform/strategy variant that is linked in deliberately." >&2
|
||||
FAIL=1
|
||||
done
|
||||
|
||||
# --- 5. install.sh must not drift from SOURCES -------------------------------
|
||||
# install.sh runs on machines with no repo checkout, so it cannot call
|
||||
# el-runtime-sources.sh and has to hardcode the list. That copy is exactly the
|
||||
# kind of duplicate that silently drifted before — so it is checked, not trusted.
|
||||
INSTALL_SH="lang/install.sh"
|
||||
if [ -f "$INSTALL_SH" ]; then
|
||||
EXPECTED="$(scripts/el-runtime-sources.sh | sort)"
|
||||
ACTUAL="$(sed -n '/^RUNTIME_SOURCES=(/,/^)/p' "$INSTALL_SH" \
|
||||
| grep -oE '[a-z_0-9]+\.c' | sort)"
|
||||
if [ "$EXPECTED" != "$ACTUAL" ]; then
|
||||
echo "FAIL: $INSTALL_SH RUNTIME_SOURCES has drifted from $SOURCES_FILE." >&2
|
||||
echo " Only in SOURCES: $(comm -23 <(echo "$EXPECTED") <(echo "$ACTUAL") | tr '\n' ' ')" >&2
|
||||
echo " Only in install.sh: $(comm -13 <(echo "$EXPECTED") <(echo "$ACTUAL") | tr '\n' ' ')" >&2
|
||||
echo " An SDK that ships the wrong set produces a lib/ that cannot link." >&2
|
||||
FAIL=1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$FAIL" -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "OK: el_runtime.c within budget ($LINES/$MAX_LINES lines, $ENGRAM_FNS/$MAX_ENGRAM engram fns);"
|
||||
echo " runtime sources accounted for; install.sh in step with SOURCES."
|
||||
@@ -81,14 +81,14 @@ fi
|
||||
echo "OK: single canonical runtime source — $CANONICAL (no un-allowlisted forks)."
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CI wire-in:
|
||||
# foundation/el .gitea/workflows/ci-dev.yaml, ci-stage.yaml, sdk-release.yaml
|
||||
# Add an early step (before the build/publish steps). It must run from the
|
||||
# REPO ROOT, so override the job's `defaults.run.working-directory: lang`:
|
||||
# CI wire-in — DONE (2026-08-16). This block used to describe the wire-in as a
|
||||
# TODO, and it had never been done: the guard existed but ran nowhere, so it
|
||||
# caught nothing for as long as it has been in the tree. It is now an early step
|
||||
# in ci-dev.yaml, ci-stage.yaml and sdk-release.yaml (each with
|
||||
# `working-directory: ${{ github.workspace }}`, since the jobs default to lang/),
|
||||
# and it runs in .githooks/pre-commit.
|
||||
#
|
||||
# - name: Guard - single canonical runtime source
|
||||
# working-directory: ${{ github.workspace }}
|
||||
# run: bash scripts/check-single-runtime.sh
|
||||
#
|
||||
# Also add to .githooks/pre-commit so drift is caught before it is committed.
|
||||
# Its sibling scripts/check-runtime-growth.sh is wired in at the same points and
|
||||
# guards the other half of the problem: this script stops el_runtime.c being
|
||||
# COPIED, that one stops it GROWING.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user