el/lang/AGENTS.md

El Language — Agent Guide

El is a self-hosting, statically-typed language that compiles to C. This file orients agents that work on El itself or on programs written in El.

---

Current work in this worktree — the API reshape / decorated seam (IN PROGRESS, 2026-08-14)

This is the api-reshape worktree. The build here reshapes Neuron's external surface and how it is declared — proven on isolated dev-port clones only; live prod engram :8742 is untouched and nothing is promoted. Full framing lives in neuron/docs/architecture/06-cognitive-architecture.md (Update — 2026-08-14 deep night) and 02-components.md §5.

• Surface collapse. The ~90 noun-organized CRUD MCP tools collapse to a few geometry ops — read (the vantage-read: re-origin + salience/recency + an aperture → a bounded slice, curing the whole-self dump), write, relate, supersede (evolve/tombstone/promote, never a hard delete) — plus the agentic primitives think/attend/learn/ground/assert. The old noun is a type parameter. Implemented in tools/api-reshape/surface.el with a parity harness (parity.sh); aperture proven to bound output. Not yet: compiled into the MCP server, hot-swap, all-alias dispatch.
• Decorated seam. @route(path,method,…) makes codegen synthesize el_route_dispatch (replacing the hand-written handle_request if-else) — proven decorate→serve on :8951. @manager/@engine/@accessor are parsed but structurally inert in the shipped compiler today; the @route codegen lives on the unmerged branch feat/el-route-decorators. Telemetry-emit and dharma-bus auto-wiring at the boundary are staged, not shipped. In-process, an @accessor reaches the engram via engram_* builtins, not http_get.

Do not edit the protected build sources while this is in flight: el-compiler/src/codegen.el, el-compiler/runtime/el_seed.c (and the archived legacy/el_runtime.c), the runtime/engram_*.c boot files, and surface.el (when present in the reshape tree) — these are owned by the build agents.

---

What El Is

El compiles .el source → C → native binary. Every El value is el_val_t (int64_t). Strings are heap pointers cast through int64_t. The compiler is written in El (self-hosting).

The compiler pipeline:

elc-cli.el
  └─ imports: compiler.el
       └─ imports: lexer.el, parser.el, codegen.el, codegen-js.el

The canonical compiler binary is dist/platform/elc. It was produced by running an earlier version of itself on elc-cli.el.

---

The Two Layers — Know Which One You're In

Layer 1: El programs (.el files)

This is where almost all work belongs. El programs are source files that get compiled by elc. New library functions, application logic, and language-level utilities all go here as .el files.

Do not add C code when El can express it. If functionality can be built from existing El primitives (string ops, exec, fs_read/write, http_post, etc.), write it in El.

Layer 2: The C seed (runtime/el_seed.c)

This is the self-contained C OS-boundary layer. It provides the __-prefixed primitives that compiled El programs call: libcurl HTTP, pthreads, filesystem I/O, arena allocation, etc. It is not generated — it is maintained by hand.

The runtime is native El (runtime/*.el) over a C OS-boundary. Status (verified 2026-08-15): the migration to a seed-only boundary is in progress, not done. Two files exist:

• runtime/el_runtime.c (~860 KB) — LIVE. Holds the engram store (EngramStore engram_global) plus the http_*/json_*/state_*/engram_* impls. It is the authoritative single-file link target for the compiler, and tools/install.sh compiles it into libel.a. This is where a new C builtin's implementation must currently live to be linkable.
• runtime/el_seed.c — the intended hand-maintained __-prefixed seed (thin wrappers over the above). It is compiled alongside el_runtime.c by tools/install.sh, but does not compile standalone yet (see the build-path caveat under "Rebuilding the Compiler").

Only edit these when you genuinely need OS-level access (raw sockets, GPU calls, new libcurl features, a new engram store op). For everything else, write El.

When you add a C builtin (verbatim-emit recipe — the El name is emitted as the exact C symbol; builtin_arity is an arity guard only, not a dispatch table):

1. Implement the C function in el_runtime.c (and declare it in el_runtime.h).
2. Add a __-prefixed thin wrapper in el_seed.c and declare it in el_seed.h.
3. Add the name to builtin_arity in el-compiler/src/codegen.el — add both the plain and __-prefixed spellings.
4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical.
5. Prove it with a NEGATIVE CONTROL. Show the test FAILING on a build without your change, then passing with it. A test that has never been seen to fail has proven nothing.

Step 5 is not optional, and step 4 does not cover it. The fixpoint proves the compiler reproduces itself. It says nothing whatsoever about whether your builtin works. A recipe ending at "byte-identical" reads as complete while having verified nothing about the thing just added — which is why this file, until 2026-08-16, produced builtins with no tests at all.

Measured cost of the omission (2026-08-16): engram_node_set_emb, engram_curiosity_json and dream_set_handler were all added in one session with zero tests. Separately, a UTF-8 fix was written, tested, and the test passed on the unpatched build too — the defect was elsewhere entirely, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified.

Two shapes that pass while proving nothing, both hit the same day:

• A test that never exercises your change (the route supplied a default that bypassed the code under test).
• An induction that loses a race. curl --max-time on a large response left both builds alive; only SO_LINGER 0 — a genuine RST, so the peer is provably gone — reproduced the failure. Six of ten attempts is not a control.

Before every probe, confirm your process bound the port (lsof -nP -iTCP:<port>, match the PID). A stale instance answering on the port has silently produced false results here more than once, and pkill -f does not reliably match an argv like ./engram.

Worked example: the engram_assert_json (op_assert seam) and engram_node_full_in/engram_connect_in (purview write-side) primitives added 2026-08-15 follow exactly this recipe.

---

Rebuilding the Compiler

After changing any .el source in el-compiler/src/ (run from the lang/ dir):

# 1. Stage2: current elc compiles the (modified) compiler to C
./dist/platform/elc elc-cli.el > elc-new.c
# 2. Build the new compiler. The C link target is el_runtime.c — it holds the
#    engram store + http/json/state impls the compiler output calls. el_runtime.c
#    self-hosts elc on its own; el_seed.c is the (aspirational) seed layer and does
#    NOT compile standalone under clang (missing prototypes for the el_runtime.c
#    symbols it wraps — see caveat below), so link el_runtime.c here.
cc -std=c11 -I runtime -lcurl -lpthread \
   -o dist/platform/elc-new \
   elc-new.c runtime/el_runtime.c
# 3. Verify self-hosting FIXPOINT (stage3 == stage2 output, byte-identical):
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c   # must be identical
mv dist/platform/elc-new dist/platform/elc

Build-path caveat (verified 2026-08-15). el_seed.c is the intended hand-maintained OS-boundary seed, but it does not compile standalone under modern clang: it wraps ~16 unprefixed el_runtime.c symbols (http_serve, json_*, state_*, http_response) without prototypes, and clang treats implicit declarations as errors (C99+). The productionised install (tools/install.sh) builds libel.a from both el_seed.o + el_runtime.o together, which is why linking succeeds there. To make el_seed.c build on its own, add prototypes for those symbols (or #include "el_runtime.h", reconciling the __http_serve return-type mismatch first). Until then, el_runtime.c is the authoritative single-file link target for the compiler.

After changing el_seed.c only (no El source changes), rebuild downstream programs but do NOT need to rebuild the compiler binary itself — the seed is linked at the application level, not the compiler level.

---

How El Programs Are Built

Each El application has a build.sh that:

1. Concatenates all .el source files (stripping import lines)
2. Runs elc to produce a .c file
3. Runs cc linking against el_seed.c

Example (cgi-studio daemon):

cd products/cgi-studio/el-daemon
./build.sh

When you add a new .el file to an application, add it to that application's build.sh concat list.

---

Parallelism in El

El is single-threaded at the application level. Parallelism is achieved through subprocess fan-out:

// Pattern: write payloads to temp files, exec bash script with & and wait,
// read results back from temp files.
fn http_post_parallel(urls: [String], bodies: [String]) -> [String] {
    // ... bash fan-out via exec() ...
}

Use exec() (blocking) or exec_bg() (fire-and-forget) with shell scripts to run concurrent work. There is no goroutine or async/await — parallelism goes through the OS process layer.

---

Key Files

Path	What it is
dist/platform/elc	Canonical compiler binary (arm64 Mac)
el-compiler/src/codegen.el	Code generator — builtin arity table lives here
el-compiler/src/lexer.el	Lexer
el-compiler/src/parser.el	Parser
runtime/el_seed.c	Self-contained C OS-boundary layer (replaces el_runtime.c)
runtime/el_seed.h	Seed header (C function declarations)
spec/language.md	Language specification
BOOTSTRAP.md	How to recover the compiler from scratch
elc-cli.el	Compiler entry point
elc-combined.el	Pre-merged single-file compiler (used during early bootstrap)

---

HTTP Timeout

The El HTTP client (libcurl) defaults to 60 seconds. Override per-process via EL_HTTP_TIMEOUT_MS env var. Set it before spawning any subprocess that makes long API calls:

exec("EL_HTTP_TIMEOUT_MS=300000 " + SOME_BIN + " " + args + " 2>&1")

---

Rules

• New library functions → write in El
• New OS/hardware primitives → write in C and register in codegen.el arity table
• Never edit dist/platform/elc directly — always rebuild from source
• Never modify el_seed.c to add functionality that El can express