El's units of encapsulation are the function and the module. Neither can hold
a concern that belongs to the process, so each one had been expressed the only
way it could be -- as a convention: call this at every site. Conventions of
that shape do not hold. Measured here: zero process-identity guards at any
layer, 20 environment variables each with its default written inline at the
read site, 62 persist call sites, 10 per-route auth checks. One absence, four
times.
Step 0 first, because the premise was wrong. El was believed to have no
middleware or effect mechanism. It has one, and it is already load-bearing:
codegen injects engram_boundary_beat at the entry of every @manager/@accessor
fn, decorators take arguments and stack, dharma_emit from a non-@manager fn is
a #error, and the cgi block injects el_cgi_init at the head of main(). So the
correct move was not to invent a mechanism but to generalize the seam that
already existed. The real gap is narrower and is now recorded: the seam is
prologue-only and its callee is a fixed builtin.
Adds a `program` block -- the third program-level declarative block. cgi and
service declare what a program may do; program declares what it is.
program "engram" {
singleton: "engram"
env ENGRAM_BIND: String = ":8742"
env GUIDE_PORT: Int = "8771"
}
singleton takes an exclusive flock before any user statement runs and refuses a
second start, reporting the holder's pid. It is a lock rather than a pidfile so
the kernel releases it on death including SIGKILL -- no stale state, and so no
"delete the lock file to get unstuck" ritual, which would itself be a
convention. It reports the pid because "already running" is not actionable; a
pid is. That is the direct answer to a stale process surviving a pkill and
going on answering probes.
env entries resolve once at startup -- environment wins, declaration supplies
the fallback -- and validate as a whole, reporting every problem at once rather
than costing one restart per variable. config("X") for an undeclared X is
fatal, because an advisory schema is just another convention. Programs without
a program block are unaffected, so migration is per-program.
Only one keyword is added. `config` and `env` could not become keywords -- both
are real identifiers in the tree -- so the block's fields are read as
identifier token values by its own parse loop and stay usable everywhere else.
The init function is emitted at the block site and called from main() rather
than inlined into main(). The live backend is codegen_streaming, which emits in
source order and cannot hold the entry list alive until main(); this way only a
single bool has to survive.
Also fixes: config() was defined in el_runtime.c but never prototyped in
el_runtime.h, so any el program calling it failed to compile under C99.
Spec: section 18 documents what shipped. Section 9 is corrected -- it claimed
decorators had no structural meaning, which has not been true for some time.
Section 19 designs durability-as-an-epilogue-effect and route authorization
and states plainly why neither is implemented here: both land in files under
concurrent modification, and the prerequisite for both is lifting the seam
from prologue-only to prologue/epilogue.
Self-hosting fixpoint verified byte-identical.
fn check(label: String, cond: Bool, want: Bool) -> Void {
if cond == want { ... } -> if (str_eq(cond, want)) SIGSEGV
}
Bool has always been an integer in the value model — type_to_c maps Bool to
"int", and el_runtime.h states "Bool -> el_val_t (0 = false, nonzero = true)".
But Bool names were registered NOWHERE: build_int_names_for_params tracked Int
and Float params, and the `let` path tracked Int and Float bindings. Neither
knew about Bool.
So comparing two Bools fell through to str_eq, which dereferenced 0 or 1 as a
char* and segfaulted immediately.
This is the third instance of one family found tonight, after el #137 (a call
on either side of == poisoned the operator) and el #136 (a missing import
compiled clean). All three are the same shape: something the compiler could not
type, silently handled as a string.
Found while writing #137's own test harness — the first version of that harness
crashed on exactly this, on both the old and new compiler, which is how it
surfaced. A test harness that cannot compare two Bools is a good way to notice.
VERIFIED:
- the harness that segfaulted on every prior compiler (exit 139, no output)
now runs clean: 14 passed, 0 failed
- self-hosting fixpoint byte-identical
- the compiler's own generated C differs by 8 lines — only the intended
registration
- neuron's full soul amalgam regenerates in 424ms, exit 0, BYTE-IDENTICAL
- test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12
Adds tests/runtime/operator_typing_test.el, the 15-case suite from #137, so
this family is covered going forward rather than rediscovered.
let a: Int = 5
getint(5) == a -> str_eq(getint(5), a) SIGSEGV
getint(5) == 5 -> getint(5) == 5 fine
A function call whose return type codegen cannot infer poisoned the operator,
and a declared Int on the other side did not save it. str_eq then read an
integer as a char* and segfaulted. Only an integer LITERAL on one side forced
the numeric form, which is why the bug stayed invisible: the common case
happened to be safe.
The check required BOTH operands to be provably Int:
if is_int_expr(left) { if is_int_expr(right) { numeric } }
Loosening to OR is strictly safer, not a trade:
- when one side is a known Int, str_eq is ALWAYS wrong — it dereferences
that integer — while numeric comparison is at worst a wrong answer on a
program that was already ill-typed;
- when neither side is Int nothing changes at all, so string comparison is
untouched.
Found by the test-framework agent while building the benchmark harness; it
correctly declined to fix it mid-phase since it is a codegen semantics change.
VERIFIED, because a semantics change earns more than an assertion:
- 15/15 on a dedicated operator suite covering string literals, string vars,
string-returning calls, mixed var/call, and != in every combination. The
pre-change compiler scores 0/15 on the same file: it segfaults before
printing anything.
- self-hosting fixpoint byte-identical
- the ONLY difference in the compiler's own generated C is the intended one:
a nested if becoming two sequential ifs, in EqEq and NotEq. Nothing else
moved.
- neuron's full soul amalgam regenerates in 400ms, exit 0, output
BYTE-IDENTICAL at 1,270,212 bytes
- test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12 —
62 tests, 190 assertions, zero failures
NOT fixed here, same family, flagged for a decision: Bool PARAMETERS are not
tracked as int-like, so `cond == want` between two Bool params still lowers to
str_eq and segfaults. Found while writing this commit's own test harness — the
first version of it crashed on exactly that, on both the old and new compiler.
It needs the same treatment, and it wants its own change.
Adds el_black_box (inline asm, +r constraint, memory clobber) and
runtime/elbench.el: a growth-curve classifier that gates time AND
allocation-count AND allocation-bytes, failing if any exceeds its
declared curve.
Refusal is a first-class verdict. The classifier REFUSES rather than
classifying when the largest measurement is below the floor, or when a
series is hard-flat across an 8x input range -- the shape produced when
the optimiser deletes the work. Reporting O(1) there would be a
confident answer with nothing behind it. Disagreeing ratios report
INDETERMINATE rather than a guess.
Deviation from DESIGN.md 6.2, stated in the source: uses consecutive
ratios on a mandated geometric sweep rather than least-squares over
candidate curves. Ratios are directly interpretable on a doubling sweep
and need no floating point; the cost is weaker O(n) vs O(n log n)
separation, reported as an ambiguous band rather than guessed.
Documents the counter scope limit: engram_*.c and libcurl malloc are
NOT tracked, so a flat curve over engram/HTTP-dominated work is not
evidence of anything.
13 tests prove the classifier against real measured series from
fitprobe.el -- including that an accumulator's allocation COUNT is
linear while its bytes are quadratic, and that el #132's pure-CPU shape
reads FLAT on both allocation signals and is caught only by time.
Completes the Phase 1 runner and migrates the 11 test files onto it.
- forward-declare the registry accessors in the test preamble; they are
defined at the end of the unit but the El runner is compiled in between
- eltest.el: explicit trailing return in the void emit_* helpers, which
otherwise lower to 'return println(...)' and fail to compile
- test files import runtime/eltest.el explicitly, using the language's own
textual import mechanism rather than compiler-side auto-injection
- DESIGN.md 6.5: gate on allocation COUNT AND BYTES, not count alone
Verified: self-hosting fixpoint byte-identical (gen2 == gen3). 6 of 11
suites run and report per-test timing. The other 5 fail to COMPILE, and
fail identically under the committed compiler -- pre-existing breakage
this framework makes visible for the first time.
Replace the hardcoded test harness main() with a generated static registry
and index-based accessors, and move all reporting into runtime/eltest.el.
The old harness inlined direct calls into main() and counted assertions in
two globals. That shape cannot report which test failed, how long any test
took, or whether a test ran at all -- a misspelled registration reported
success for a test that never executed.
- assertions record into per-test state instead of global counters
- registry table emitted at compile time; discovery strictly precedes
execution, which is what later enables --list, filtering and sharding
- per-test wall timing on CLOCK_MONOTONIC, taken in C around the call
- runner in El: structured NDJSON events as source of truth, human output
rendered from the same fields
Implements the three primitives the test-framework design (DESIGN.md §6.5)
requires for gating on growth curves: el_alloc_count, el_alloc_bytes,
el_peak_rss. Registered in codegen's builtin_arity and wrapped in el_seed.c per
the project's C-builtin recipe.
WHY COUNTS AND NOT WALL-CLOCK: a growth-curve gate has to be a hard build
failure, which means the signal cannot flake. Wall-clock needs warmup,
statistics, and a quiet machine; on shared CI it is unusable as a gate.
Allocation counts are perfectly deterministic — same input, same number, every
machine, every run. Fit them against n and a complexity regression becomes a
build failure with zero noise.
All four runtime string allocators (el_strdup, el_strbuf, and their _persist
variants) funnel every allocation the language performs, so instrumenting there
counts everything.
WHY BYTES AS WELL AS COUNT — this is not redundancy, it is the whole gate.
Measured with two El programs, one allocating once per item, one rebuilding its
accumulator each iteration:
n linear allocs / bytes quadratic allocs / bytes
100 100 / 290 100 / 5,150
200 200 / 690 200 / 20,300
400 400 / 1,490 400 / 80,600
800 800 / 3,090 800 / 321,200
The quadratic program's allocation COUNT is exactly linear — identical to the
healthy one. Counting allocations alone would have missed it completely. Bytes
catch it: each doubling of n quadruples bytes (ratios 3.94, 3.97, 3.99 ->
converging on 4.0, i.e. O(n^2)), while the linear case converges on 2.0.
That shape — count linear, per-allocation size growing — is the classic
accidental quadratic, and it is exactly elc's defect: quadratic allocation
VOLUME, which the old shipped compiler paid in RSS (27 GB, OOM) and the rebuilt
one pays in malloc/free churn (42s on 1.4 MB). Volume was the invariant across
both; RSS and wall-clock were just the two ways it surfaced.
el_peak_rss is exported for context and is explicitly NOT a gating signal — it
is perturbed by allocator internals, the page cache, and the OS. Gate on the
deterministic numbers; report the physical one.
Counters are unsynchronised by design: this is measurement, and a lock would
change the thing being measured. Exact on the single-threaded compile path,
approximate under threads.
Follow-on to the edge write barrier. That fix removed the full-store walk;
this one makes the pool able to notice if anything like it happens again.
WHAT WENT WRONG, precisely: the pool thrashed the live engram to a standstill
twice on 2026-08-15 and said nothing. From outside it was indistinguishable
from "busy loading" — 100% CPU, flat RSS, no output — so four wrong theories
got tried (bad binary, corrupt snapshot, WAL replay, feature flags), each
costing a deploy or a rollback. The whole time, hits/misses/evictions were
already being counted in PgCache, and the struct comment read:
/* stats (introspection only — never affect semantics) */
That comment was the bug. Self-measurement treated as decoration is why the
pool could not correct itself and why no one outside could see what it was
doing. A system that cannot read its own state cannot correct, and neither can
anyone watching it.
- pc_adapt_budget(): the loop, closed. Over a sliding window, evictions
running at a large fraction of accesses WHILE reuse is real means the
working set exceeds the budget — so grow it, geometrically, bounded by a
LIVE re-read of physical memory. Evictions alone are not pressure (a scan
evicts and never returns); evictions with reuse are. An explicit
ENGRAM_POOL_FRAMES still wins — an operator override must not be silently
overruled.
- Budget derived, not declared. A constant cannot be right: 16 GiB of frames
is arbitrary on a 48 GB host and suicidal on a 16 GB one. Even "60% of RAM
at startup" is a guess about the future — it cannot know the store grew or
the machine changed. Hence the live re-read.
- pc_report(): ONE structured emission carrying the entire sensed state,
through emit_log — El's existing telemetry, already exporting to OTLP.
Deliberately not a function per stat, and deliberately not a bespoke
/api/pool endpoint: both make observability something hand-written per noun
instead of the uniform mechanism every component already has.
- engram_pool_stats_json(): the same state readable live, wired through the
normal builtin path (codegen arity + el_seed wrapper), so the pool can be
observed in real time rather than reconstructed afterward from a stack
sample.
Verified: with the exact configuration that took production down
(ENGRAM_POOL_FRAMES=65536 → 1 GiB cache against a 2 GiB store) the engram boots
clean and serves — 0.0% CPU, 13,436 nodes / 37,663 edges, embeddings complete —
and NO pressure event fires, because the barrier removed the walk that caused
it. The controller is defense in depth; the barrier is the fix.
/api/graph/edges answered a read query by calling engram_save() to serialize
the ENTIRE graph to disk (128 MB) and then fs_read-ing it back. Two defects in
one line, and both bit production on 2026-08-15:
1. The path it wrote was ~/.neuron/engram/snapshot.json — the engram
server's CANONICAL store. A READ route overwriting the persistence
owner's canonical file. This defect had been fixed once (export moved to
a scratch path); it came back when the hand-written dispatch block was
replaced by @route dispatch and the unfixed copy is the one that
survived the merge.
2. Cost: a full snapshot write, a 128 MB read, and a parse of the whole
graph, per request, to return a bounded slice.
Calling it tonight overwrote the canonical snapshot and immediately preceded
an engram crash loop.
engram_edges_json(limit, offset) is the builtin that route's own TODO asked
for ("Future: add an engram_edges_json() builtin and drop the file round trip
entirely"). It walks g->edges directly and emits every persisted field.
limit <= 0 defaults to 1000, not unbounded: this is the endpoint that fell
over, and an unbounded default would preserve the failure mode under a new
name. Callers page explicitly.
Registered in codegen.el's builtin_arity (both plain and __ spellings) and
wrapped in el_seed.c per the project's C-builtin recipe.
transduce() is now THE single mechanism: one function, no content-type
branch inside it. It never asks whether `source` is prose, JSON, or
raw/opaque bytes (audio, etc.) — it runs one algorithm unconditionally:
split on "\n\n" as a universal boundary-marker check, and if that finds
no boundary, fall back to fixed 4096-char windows. Same node/edge wiring
(root -contains-> chunk, chunk -precedes-> next, "#"-prefixed chunk gets
a heading/section_of link) regardless of what's inside a chunk. Dedup is
the existing find_existing_by_content path via merge_manifold, applied
uniformly. The old transduce_structured JSON dataset/records/feature-node
interpretation is deleted outright, not just unused — a JSON file now
gets chunked and deduped like anything else, with no pre-computed
structure. All five ingest_* entry points still exist unchanged in name
and role; ingest_file/ingest_dir/ingest_url/ingest_llm now call the one
transduce() (ingest_stream builds its own turn-nodes directly and never
called either old function, so it's untouched).
This unlocks raw/opaque content (audio, or anything else with no natural
text/JSON shape) without any DSP, LLM call, or external API: transduce()
chunks it exactly like it chunks anything else. There is zero semantic
understanding of audio (or any payload) claimed or built here — any
meaning is expected to emerge later from Neuron's own existing mechanisms
(embedding, spreading activation, dedup) acting on this real geometry
over time.
Two small C builtins added to el_runtime.c/h (fs_size, fs_read_b64_chunk)
because El strings are NUL-unsafe under strlen-based ops and fs_read()'s
result silently truncates at the first embedded NUL, which is routine in
real binary/audio bytes. ingest_file compares fs_read()'s string length
against a real fs_size() stat() count; on mismatch it rebuilds the
payload as base64-encoded fixed 3072-byte windows read directly off disk
(binary-safe in C, verbatim, no invention), joined with the same "\n\n"
marker transduce()'s boundary scan already looks for. This is a
mechanical fidelity fix, not interpretation of content — transduce()
never learns a fallback happened. Registered both builtins' arity in
codegen.el; did not rebuild the elc compiler binary itself (unrelated,
pre-existing gap: self-hosting elc via el_seed.c fails on this worktree
independent of this change, reproduced with codegen.el reverted) — the
existing elc binary compiles calls to unregistered builtins via its
already-existing arity=-1 passthrough, confirmed by an actual clean
`elc ingest.el` + `cc` build against the modified el_runtime.c.
INGEST_KIND keeps existing only as an acquisition-mechanism selector
(dir/file/url/llm/stream — which RPC to use to fetch bytes), not as a
content-type flag; the redundant "structured" value (an alias for "file"
that hinted the now-deleted JSON branch) is removed. ingest_dir drops its
file-extension filter for the same reason: transduce() takes anything now.
Verification: local manifold construction confirmed correct against a
real captured audio file (will_clean.wav, 304288 bytes, and a 12288-byte
real prefix slice) — exact expected node/edge counts both times
(101 nodes/199 edges full file; 5 nodes/7 edges for the slice, matching
ceil(bytes/3072)+1 nodes and 2n-1 edges), with real, verbatim base64
content confirmed decoding back to the actual WAV header bytes. Compiles
clean via the real elc + the modified el_runtime.c/engram_*.c (built and
booted an actual sandbox engram off this exact source with `nsbx create
--branch`).
NOT verified this session, disclosed rather than papered over: end-to-end
server-confirmed persistence (a real before/after /api/stats delta, and a
fetched node by id) for the audio, prose, and JSON-fixture cases. Every
local nsbx sandbox engram tried tonight (two stock pre-#109 binaries
hitting the known O(N*D) brute-force scan bug, then a fresh #109/HNSW
binary built from current dev) took minutes-to indefinitely long on the
final /api/load-merge write's embedding step and hit the client's 60s
HTTP timeout before responding, even for a 5-node write. This is
confirmed as real (if slow) forward progress, not a hang: the sandbox's
WAL file was observed growing steadily across every attempt. The code's
own pre-existing HONESTY GATE correctly refused to report success in
every case, returning "load-merge failed: ..." with a
"nothing below this manifold was confirmed persisted by the server" note
instead — exactly as designed. This is an environment/infrastructure
limitation, not a defect introduced by this change: the engram server
binary itself is untouched by this commit.
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region,
decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency
work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated
seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's
actual current HEAD, plus engram-tiered-storage's still-unique test suite.
RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/
verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/
select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two
branches; #109's copy is a strict superset (adds vindex_harvest_from_store,
used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and
engram_verify.c are also byte-identical. #109's own branch point already
carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing
left to merge into #109 for those files. The one thing engram-tiered-storage
had that #109's tree dropped: its full test suite (test_vindex.c,
test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the
interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh
harnesses) — ported over here unchanged.
WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch
forked from dev on 2026-08-14 15:40 (before restructure-adjacent history
diverged the file's merge-base for `git merge` — it presented as an add/add
conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but
it silently reverted THREE dev fixes landed on 2026-08-14/15, after the
branch point, that the PR's diff had no way to know about:
1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of
the query-aware propagation gate dropped the shift-and-floor rescale
about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate
0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped
around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi]
is computed) is orthogonal to WHAT it gates on and both are kept.
2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes
wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is
diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global).
PR's tree predates this and dropped all three counters + their JSON
stats fields. Restored declarations, all 4 direct increment sites, the
eg_wm_carry_over bll increment, and the act-stats JSON fields --
alongside (not instead of) the PR's own P4 afferent / API-reshape
counters already in that same struct/JSON.
3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev
selects the STRONGEST qualifying candidate for consolidation each call;
PR's tree predates this and reverted to hash-slot order (arbitrary wrt
association strength) for edge formation -- the one path that writes
PERMANENT structure. Restored the strongest-candidate while-loop,
keeping the PR's own genuine improvement at that site
(engram_adj_on_edge_added incremental-index append instead of a bare
adj_dirty=1 full-rebuild flag).
engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env
flags, /api/nodes connected-count in responses) were pure additive: dev's
side was empty, PR's side added the feature. Took PR's side whole.
VERIFIED (nsbx sandbox only, live :8742/:7770 never touched):
- cc -std=c11 -O2, clean link against the real engram/src/server.el via
elc, zero errors.
- vindex_bench (built standalone, read-only harvest) against the real
production store clone (13,671 embedded nodes, 768-dim nomic-embed-text):
recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs
2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s
for the full 13,671-node set -- see the flagged risk below.
- Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned
snapshot of the live store, 13,424 nodes / 37,656 edges) and called
/api/activate for real: first call after boot 41.5s (pays the one-time
HNSW build inline -- matches the standalone bench), second/third calls
356ms/605ms, no crash, correct results, act-stats JSON (including the
restored evict_floor/cap/bll fields) reads correctly.
KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for
this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync
builds the HNSW index synchronously, inline, on the first engram_activate()
call after every process start (or index invalidation). On the real node
count that is a ~46s blocking stall on a single-threaded server -- the first
request after every restart (or its concurrent siblings) waits the full
build. Recommend a background/incremental build (or a bounded per-call build
budget) before this ever reaches the live daemon. See PR description / final
report for the fuller writeup.
Float + previously fell through to string concat (segfault); -, *, /, %
operated on raw IEEE-754 bit patterns as integers (garbage results). Floats
are now tracked via a __float_names typed-binding set (parallel to the
existing int-tracking scheme) and arithmetic is emitted as real C double
ops.
Also fixes math_log, which was wrongly aliased to natural log (duplicating
math_ln) — now uses log10 — and adds the missing <math.h> include. Rebuilt
elc binary included.
Adds engram_assert_json — a grounded "assertion envelope" primitive for a
realizer/op_assert seam (per backlog bl-53/#57) — plus purview-scoped
mutation wrappers engram_node_full_in/engram_connect_in, which refuse
non-default purviews rather than silently mutating the live store. Threads
through el_seed.c/h wrappers and the codegen.el arity table per the
project's existing C-builtin recipe.
Also rewrites lang/AGENTS.md build docs with verified (2026-08-15) findings
that el_seed.c does not compile standalone.
Will waived diff review -> build it for real. Add engram_boundary_beat() to the
runtime (afferent counter++ + engram_chrono_tick + engram_strengthen(self-anchor)
+ dharma_emit) and two act-stats counters (aff_boundary_ops, dharma_emits).
codegen cg_fn injects ONE engram_boundary_beat(op) at the entry of every
@manager/@accessor fn (fn_has_decorator, so it fires under @route @manager too) —
a decorated op self-reports with ZERO hand-written instrumentation. Rebuilt elc
self-host + the cognition engram in the worktree; ran it as the clone daemon on
:8900. Proof (/api/boundary-proof, @manager, empty body, 5x): aff_boundary_ops
0->5, dharma_emits 0->5, self activation_count 1510->1513, chrono stamp advanced.
Brought in feat/cognitive-architecture engram runtime+server for the build.
strengthen = activation bump (not content/edge write) -> identity protection
intact. Live :8742 untouched; no push, no cutover.
Ground-truth the three seams (route/telemetry+interoception/bus) with file:line
evidence. Port the tested @route codegen+parser from feat/el-route-decorators
into the worktree elc (decoration synthesizes el_route_dispatch — no hand-written
90-branch handle_request). Rebuild elc self-host; prove decorate->serve end-to-end
(route_proof.el on :8951). Rewrite surface.el as El-native decorated components:
@route + @accessor/@manager, in-process engram_* builtins (not http_get), @manager
ops emit on the real dharma_* bus (same transport as wt/swarm-ccr). Identity
keystones refused in write/relate/supersede. Gate-1 clone recipe (WAL-aside
cold-boot + ENGRAM_WAL=on) proves the FULL op set live on the clone. Boundary
auto-emit (telemetry/interoception/bus) staged as a reviewable cg_fn diff
(SEAM_STAGED.md) — needs the cognition-engram rebuild to verify link. Live :8742
untouched; no push, no cutover.
Resolve 3 conflicts:
- lang/el-compiler/runtime/el_runtime.c: keep deletion (deprecated runtime fork;
single-source-of-truth is lang/runtime/, enforced by scripts/check-single-runtime.sh).
- lang/releases/v1.0.0-20260501/el_runtime.h: keep deletion (releases/ is a generated
artifact folder, not a source path; a release is a git tag, not a folder).
- lang/runtime/el_platform_win.h: union of dev's Windows port (#80: setsockopt optval
wrapper + curl-less libcurl stubs) and our fsync(->_commit) shim needed by engram_store WAL.
Nothing in dev's build consumes the deprecated fork or releases/ folder.
- Wire paged store into the ENGRAM SERVICE (server.el, the authoritative durable
owner): boot->engram_store_boot, persist_canonical->engram_store_checkpoint,
gated by ENGRAM_STORE.
- elc (lang/elc.c + src/parser.el + codegen.el + elc-combined.el): OOB guard in
tok_kind/tok_value + parse_block progress backstop — fixes the pre-existing
unbounded-memory fold hang on sessions.el.
- engram_prune_telemetry mirrors ISE prune to the store (store_forget) so store
live-count tracks resident and stale telemetry stays bounded.
- Deployed live 2026-08-12: engram :8742 on neuron.egm+WAL, count reconciled 11552.
A cgi block is a top-level declaration, so codegen_streaming classifies it via
is_top_level_decl and releases it. The identity emission then searched
toplevel_exec_stmts for that same block. Declarations are excluded from that list by
construction, so the search could never succeed. A probe printed what it actually
saw for a program whose first statement is a cgi block: [Let, Expr]. It emitted
nothing, silently, with no diagnostic on any channel.
The code documented its own assumption — 'Since cgi blocks are rare and small, they
end up in toplevel_exec_stmts' — and that assumption was false.
Capture the declared values before the release and emit from them. The search is
deleted rather than repaired, so the failure mode is removed rather than relocated.
Proven discriminating (old fails, new passes):
minimal cgi program, old -> 0 el_cgi_init
minimal cgi program, fixed -> el_cgi_init with all four declared values
neuron soul, fixed -> principal present in the compiled binary (0 before),
boots in 2s, interface 110 routes in / 110 out
Consequence: a binary now carries its declared identity as a compiled constant,
which is what the identity protocol requires. Whether the runtime surfaces it to
state_get("soul_principal") is unverified and separate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
chat.el calls the runtime native engram_get_node_by_label to fetch
well-known nodes (conv:history, session:summary) by stable label rather
than by ID — immune to vector-index drift across restarts. The current
runtime never defined it, so the regenerated dist/soul.c fails to link.
Backport the function verbatim (idiom-adapted to jb_finish) from release
runtime v1.0.0-20260501 and register it as an EL builtin exactly like its
siblings: runtime definition + prototype, __-prefixed seed wrapper +
prototype, and codegen arity entry. No search-site code is touched.
Add el_mem_check() to el_runtime.c: reads ELC_MAX_MEM_MB (default 512),
checks RSS via getrusage (macOS bytes / Linux KB normalised to MB), prints
a clear diagnostic to stderr and exits(1) if exceeded.
Wire it into two places:
- compiler.el: upfront check at --emit-header entry point
- codegen.el: per-function check in the streaming loop after each
el_arena_pop, so runaway growth is caught at the earliest function
boundary rather than after the machine is already dying.
The El lexer silently skips '#', so {#each} lexes as LBrace Ident:"each"
and {#if} lexes as LBrace If ... (using the If keyword token, not Hash).
The existing {#each} check used k2=="Hash" which was dead code.
Parser changes (parser.el):
- Add parse_raw_text_content(): collects all tokens as raw text until
</tag_name>, bypassing El expression parsing. Used for <style> and
<script> elements so CSS/JS content isn't parsed as El expressions.
- parse_html_element(): use raw-text mode for <style> and <script> tags.
- parse_html_children(): fix {#each} detection (k2=="Ident", k3=="each"
instead of dead k2=="Hash" check). Add {#if cond}...{#else}...{/if}
support generating HtmlIf AST nodes.
Codegen changes (codegen.el):
- Add cg_html_if(): generates if (cond_c) { then_c } else { else_c }
for HtmlIf nodes.
- cg_html_parts(): dispatch HtmlIf to cg_html_if.
Implement compile_test() entry point that emits a C test harness instead
of a normal program. Test blocks (previously skipped) now compile to
static functions with per-assertion pass/fail tracking. Assert statement
added to parser and codegen. Runtime extended with now_ns, fs_list_json,
json_build_object, json_build_array, json_escape_string, state_has,
state_get_or. Fix float negation codegen, float equality comparisons,
time_to_parts return type (JSON string), time_format empty-fmt, json_set
raw-value semantics, state_keys JSON array return. All 310 native tests
pass across 9 suites (core, text, string, math, env, state, json, time, fs).
test "name" { ... } blocks were not recognized by the self-hosted
compiler. The body { } was parsed as a Map literal, creating a huge
AST with O(n²) string concatenation in the toplevel_exec_stmts loop
(which had no arena scope). A 272-line test file would consume 400MB+
and a 720-line file importing the full compiler source caused 150GB
usage and crashed the machine.
Two fixes:
1. Skip Test tokens in codegen_streaming before parse_one() —
advance past "name" and skip_to_rbrace on the body block.
Test blocks are never compiled; self-hosted compiler has no test runner.
2. Add per-statement arena scope to toplevel_exec_stmts emission loop,
matching the el_main_body loop. Frees intermediate strings after
each statement to prevent O(n²) accumulation from any unrecognized
construct that reaches that path.
Result: test_string.el (272 lines, 27 test blocks): 0MB peak (was 400MB+).
test_compiler.el (720 lines + 8728 imported): 15MB peak (was 150GB).
- Flat token list: lexer emits [kind0, val0, kind1, val1, ...] instead of [{kind,val}, ...]
Eliminates per-token ElMap allocation (~112B × N tokens)
- str_char_code hot loop: char classification via Int codes, no strdup per char
- Batch c_escape: str_slice clean runs instead of char-at per byte
- Parser updated to use tok_at/tok_kind/tok_value stride-2 accessors
Combines two orthogonal optimizations:
1. c_escape batching (from alpha): ASCII runs emitted as str_slice segments instead
of one str_char_at string per byte. O(N) allocs → O(K) where K = special chars.
2. scan_interp_string batching (from beta): char dispatch via str_char_code (Int)
+ clean_start tracking to flush plain runs as str_slice. Eliminates per-char
string allocations in the string-literal scanning hot path.
Result on web/src/main.el: 14.5MB -> 13.4MB peak RSS (-7.6%).
Self-hosting: PASS.
bigmerge
will.anderson