Lands feat/reframe-region-setop (PR #109: native set-based reframe_region, decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's actual current HEAD, plus engram-tiered-storage's still-unique test suite. RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/ verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/ select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two branches; #109's copy is a strict superset (adds vindex_harvest_from_store, used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and engram_verify.c are also byte-identical. #109's own branch point already carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing left to merge into #109 for those files. The one thing engram-tiered-storage had that #109's tree dropped: its full test suite (test_vindex.c, test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh harnesses) — ported over here unchanged. WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch forked from dev on 2026-08-14 15:40 (before restructure-adjacent history diverged the file's merge-base for `git merge` — it presented as an add/add conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but it silently reverted THREE dev fixes landed on 2026-08-14/15, after the branch point, that the PR's diff had no way to know about: 1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of the query-aware propagation gate dropped the shift-and-floor rescale about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate 0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi] is computed) is orthogonal to WHAT it gates on and both are kept. 2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global). PR's tree predates this and dropped all three counters + their JSON stats fields. Restored declarations, all 4 direct increment sites, the eg_wm_carry_over bll increment, and the act-stats JSON fields -- alongside (not instead of) the PR's own P4 afferent / API-reshape counters already in that same struct/JSON. 3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev selects the STRONGEST qualifying candidate for consolidation each call; PR's tree predates this and reverted to hash-slot order (arbitrary wrt association strength) for edge formation -- the one path that writes PERMANENT structure. Restored the strongest-candidate while-loop, keeping the PR's own genuine improvement at that site (engram_adj_on_edge_added incremental-index append instead of a bare adj_dirty=1 full-rebuild flag). engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env flags, /api/nodes connected-count in responses) were pure additive: dev's side was empty, PR's side added the feature. Took PR's side whole. VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): - cc -std=c11 -O2, clean link against the real engram/src/server.el via elc, zero errors. - vindex_bench (built standalone, read-only harvest) against the real production store clone (13,671 embedded nodes, 768-dim nomic-embed-text): recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs 2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s for the full 13,671-node set -- see the flagged risk below. - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned snapshot of the live store, 13,424 nodes / 37,656 edges) and called /api/activate for real: first call after boot 41.5s (pays the one-time HNSW build inline -- matches the standalone bench), second/third calls 356ms/605ms, no crash, correct results, act-stats JSON (including the restored evict_floor/cap/bll fields) reads correctly. KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync builds the HNSW index synchronously, inline, on the first engram_activate() call after every process start (or index invalidation). On the real node count that is a ~46s blocking stall on a single-threaded server -- the first request after every restart (or its concurrent siblings) waits the full build. Recommend a background/incremental build (or a bounded per-call build budget) before this ever reaches the live daemon. See PR description / final report for the fuller writeup.
6.7 KiB
El Language — Agent Guide
El is a self-hosting, statically-typed language that compiles to C. This file orients agents that work on El itself or on programs written in El.
Current work in this worktree — the API reshape / decorated seam (IN PROGRESS, 2026-08-14)
This is the api-reshape worktree. The build here reshapes Neuron's external
surface and how it is declared — proven on isolated dev-port clones only; live
prod engram :8742 is untouched and nothing is promoted. Full framing lives in
neuron/docs/architecture/06-cognitive-architecture.md (Update — 2026-08-14 deep
night) and 02-components.md §5.
- Surface collapse. The ~90 noun-organized CRUD MCP tools collapse to a few
geometry ops —
read(the vantage-read: re-origin + salience/recency + an aperture → a bounded slice, curing the whole-self dump),write,relate,supersede(evolve/tombstone/promote, never a hard delete) — plus the agentic primitivesthink/attend/learn/ground/assert. The old noun is atypeparameter. Implemented intools/api-reshape/surface.elwith a parity harness (parity.sh); aperture proven to bound output. Not yet: compiled into the MCP server, hot-swap, all-alias dispatch. - Decorated seam.
@route(path,method,…)makes codegen synthesizeel_route_dispatch(replacing the hand-writtenhandle_requestif-else) — proven decorate→serve on:8951.@manager/@engine/@accessorare parsed but structurally inert in the shipped compiler today; the@routecodegen lives on the unmerged branchfeat/el-route-decorators. Telemetry-emit and dharma-bus auto-wiring at the boundary are staged, not shipped. In-process, an@accessorreaches the engram viaengram_*builtins, nothttp_get.
Do not edit the protected build sources while this is in flight:
el-compiler/src/codegen.el, el-compiler/runtime/el_seed.c (and the archived
legacy/el_runtime.c), the runtime/engram_*.c boot files, and surface.el
(when present in the reshape tree) — these are owned by the build agents.
What El Is
El compiles .el source → C → native binary. Every El value is el_val_t (int64_t). Strings are heap pointers cast through int64_t. The compiler is written in El (self-hosting).
The compiler pipeline:
elc-cli.el
└─ imports: compiler.el
└─ imports: lexer.el, parser.el, codegen.el, codegen-js.el
The canonical compiler binary is dist/platform/elc. It was produced by running an earlier version of itself on elc-cli.el.
The Two Layers — Know Which One You're In
Layer 1: El programs (.el files)
This is where almost all work belongs. El programs are source files that get compiled by elc. New library functions, application logic, and language-level utilities all go here as .el files.
Do not add C code when El can express it. If functionality can be built from existing El primitives (string ops, exec, fs_read/write, http_post, etc.), write it in El.
Layer 2: The C seed (runtime/el_seed.c)
This is the self-contained C OS-boundary layer. It provides the __-prefixed primitives that compiled El programs call: libcurl HTTP, pthreads, filesystem I/O, arena allocation, etc. It is not generated — it is maintained by hand.
The old el_runtime.c has been archived to runtime/legacy/. The runtime is now native El (runtime/*.el). el_seed.c replaces el_runtime.c as the sole C compilation dependency.
Only edit el_seed.c when you genuinely need OS-level access (raw sockets, GPU calls, new libcurl features). For everything else, write El.
When you do add a C builtin:
- Add the C function to
el_seed.c - Declare it in
el_seed.h - Add it to the
builtin_aritytable inel-compiler/src/codegen.el(so the compiler knows the arg count) - Rebuild the elc binary (see below)
Rebuilding the Compiler
After changing any .el source in el-compiler/src/:
cd /Users/will/Development/neuron-technologies/foundation/el
./dist/platform/elc elc-cli.el > elc-new.c
cc -std=c11 -I runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c runtime/el_seed.c
# Verify self-hosting:
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # should be identical
mv dist/platform/elc-new dist/platform/elc
After changing el_seed.c only (no El source changes), rebuild downstream programs but do NOT need to rebuild the compiler binary itself — the seed is linked at the application level, not the compiler level.
How El Programs Are Built
Each El application has a build.sh that:
- Concatenates all
.elsource files (strippingimportlines) - Runs
elcto produce a.cfile - Runs
cclinking againstel_seed.c
Example (cgi-studio daemon):
cd products/cgi-studio/el-daemon
./build.sh
When you add a new .el file to an application, add it to that application's build.sh concat list.
Parallelism in El
El is single-threaded at the application level. Parallelism is achieved through subprocess fan-out:
// Pattern: write payloads to temp files, exec bash script with & and wait,
// read results back from temp files.
fn http_post_parallel(urls: [String], bodies: [String]) -> [String] {
// ... bash fan-out via exec() ...
}
Use exec() (blocking) or exec_bg() (fire-and-forget) with shell scripts to run concurrent work. There is no goroutine or async/await — parallelism goes through the OS process layer.
Key Files
| Path | What it is |
|---|---|
dist/platform/elc |
Canonical compiler binary (arm64 Mac) |
el-compiler/src/codegen.el |
Code generator — builtin arity table lives here |
el-compiler/src/lexer.el |
Lexer |
el-compiler/src/parser.el |
Parser |
runtime/el_seed.c |
Self-contained C OS-boundary layer (replaces el_runtime.c) |
runtime/el_seed.h |
Seed header (C function declarations) |
spec/language.md |
Language specification |
BOOTSTRAP.md |
How to recover the compiler from scratch |
elc-cli.el |
Compiler entry point |
elc-combined.el |
Pre-merged single-file compiler (used during early bootstrap) |
HTTP Timeout
The El HTTP client (libcurl) defaults to 60 seconds. Override per-process via EL_HTTP_TIMEOUT_MS env var. Set it before spawning any subprocess that makes long API calls:
exec("EL_HTTP_TIMEOUT_MS=300000 " + SOME_BIN + " " + args + " 2>&1")
Rules
- New library functions → write in El
- New OS/hardware primitives → write in C and register in
codegen.elarity table - Never edit
dist/platform/elcdirectly — always rebuild from source - Never modify
el_seed.cto add functionality that El can express