neuron/docs/architecture/05-el-and-build.md

Neuron — El & the Build Pipeline

The soul and the engram are written in El, a self-hosted language that compiles to C11. This document covers the language layer, the amalgamation → soul.c → binary pipeline, how the soul is composed from its layers, and the compile-time capability gates. Sources: manifest.el, soul.el, dist/soul.c, the El toolchain under foundation/el/ (elc.c, elb.el, BOOTSTRAP.md), and .gitea/workflows/.

The El language layer

El is a compiled, Lisp-family language transpiled to C11. Every El program links a shared runtime, el_runtime.c / el_runtime.h, which implements all builtins: the engram graph engine (engram_*), HTTP (http_*), JSON (json_*), crypto, time, LLM calls, and DHARMA primitives (el_runtime.h, BOOTSTRAP.md:599-644). The runtime also provides an arena allocator (server mode) and ARC refcounting. Practically: the runtime is both the standard library and the database — the graph physically lives in el_runtime.c, and El source files are the orchestration/logic on top.

A recurring texture in the source is workaround comments for codegen quirks (e.g. broken %/* operators). These are El-compiler maturity issues, not architecture — but they explain some of the hand-rolled arithmetic in awareness.el/memory.el.

The toolchain: elc, elb, el_runtime

Tool	What it is	Role
elc	the El compiler, self-hosted (written in El)	compiles one El translation unit → C11. Import resolution is textual, depth-first, dedup'd — it inlines all imports into one string and emits forward decls for every fn (BOOTSTRAP.md:927-936).
elb	the build coordinator (elb.el, ~367 lines)	reads manifest.el, walks the import graph, does incremental separate compilation using .elh header files (extern fn decls), links the final binary (".NET-style incremental build", BOOTSTRAP.md:886, 916-925).
el_runtime.c/.h	the C runtime	linked by every compiled El binary; implements all builtins and the graph engine.

The .elh files present in this repo (soul.elh, memory.elh, neuron-api.elh, routes.elh, …) are auto-generated headers (elc --emit-header) — the extern fn interface each module exports. They are the contract surface elb uses for incremental builds, and they double as a concise map of each module's public functions.

Self-hosting fixed point

elc is bootstrapped from a seed binary (dist/platform/elc, Mach-O arm64) and verified by a fixed-point self-recompile: the compiler must compile its own source to a byte-identical binary (BOOTSTRAP.md:7-58, 801-816). Pipeline: elc-cli.el → compiler.el → lexer/parser/codegen.el.

Building the soul: .el → elc → .c → cc → binary

The concrete pipeline (mirrored in the engram build, engram/src/server.el:8-11):

   soul.el (+ imports)
        │  elc  (self-hosted El→C11, inlines imports)
        ▼
   dist/soul.c   (~31,300 lines — single amalgamated translation unit)
        │  cc -std=c11 -O2  soul.c  el_runtime.c
        ▼
   dist/neuron   (native binary)

Why dist/soul.c is committed

dist/soul.c is the authoritative combined translation unit, regenerated on macOS by running elb. It is checked into the repo on purpose: CI compiles it directly and skips elb entirely (ci.yaml). The reason is operational, not aesthetic —

• elb succeeds on arm64/macOS ld, but fails on Linux (duplicate strong symbols), and
• elc uses 24GB+ virtual memory, which OOM-kills the 16GB CI runner.

So the pattern is: compile on the Mac, commit the amalgamation, and let Linux CI do only the cheap cc step. dist/ also holds the per-module .c outputs (memory.c, awareness.c, chat.c, the NLG morphology tables, …) — intermediate artifacts of the same process.

Mechanism note (observed during the self-load regen). The single-TU dist/soul.c is produced by running elc over the flattened import set — every module source in soul.el's transitive import graph, concatenated with import lines stripped, compiled in one pass (elc hoists forward decls for all functions, so concat order doesn't affect correctness). elb on its own emits per-module .c + a linked binary, not the combined soul.c; it is the separate-compilation coordinator, and elc soul.el alone yields only the soul module. Because the amalgamation is regenerated only on demand, it can lag the .el sources: this PR regenerated it after it had fallen behind several source commits, and folded in the inspect_graph relevance-ranked projection (the compact=1 self-load fix, doc 02) so CI ships it.

How the soul is composed (layer stack)

manifest.el declares the build:

package "neuron" { version "0.1.0"  edition "2026" }
build { entry "soul.el" }

The comment in manifest.el:8-16 documents the intended layer composition order: a base layer ../foundation/nlg (the NLG engine — 31-language morphology, grammar, realizer, semantics) with the soul layer (soul.el) injected on top. New layers are added by importing them in soul.el before the soul's own code. (The ../foundation/nlg path is the manifest's stated NLG base; the NLG sources compile into the dist/*.c morphology/grammar tables seen in the tree.)

soul.el itself imports, in order (soul.el:1-10): elp.el, memory.el, safety.el, stewardship.el, imprint.el, awareness.el, chat.el, studio.el, elp-input.el, routes.el — then declares the cgi "neuron-soul" identity block (:12-17): dharma_id, principal, network, and engram: http://localhost:8742. Because elc inlines imports depth-first, this import list is the amalgamation order that produces dist/soul.c.

The cgi block is not just metadata — it sets the program's capability tier (next section).

Compile-time capability gates

El's codegen classifies each program by its top-level declaration and enforces capabilities at compile time (BOOTSTRAP.md:958-965):

Declaration	Tier	Allowed
cgi { … }	full	everything — llm_call_agentic, llm_register_tool, dharma_emit, dharma_field, LLM, DHARMA
service { … }	restricted	no llm_call_agentic / llm_register_tool / dharma_emit / dharma_field
neither	utility	no DHARMA, no LLM

A program that calls a capability its tier forbids fails to compile: codegen emits a C #error naming the forbidding call, so the downstream cc aborts. This is the primary hard gate in the build — capability escalation is caught by the compiler, not at runtime. The soul is a cgi, so it gets the full tier; engram is declared without cgi/service semantics that would grant LLM access (it is a store).

Verification gates

Gate	Where	What it checks
Capability tier	El codegen (BOOTSTRAP.md:958)	no capability escalation; hard #error at compile
Self-hosting fixed point	elc bootstrap (BOOTSTRAP.md:801)	compiler reproduces itself byte-identically
test_soul_guard.el	tests/	the genesis safe_to_seed boot guard — a sparse/oversized snapshot must not clobber the graph
test_layer_contract.el	tests/	JSON interface shapes between composition-stack layers that layered_cycle depends on (e.g. safety_screen always returns an action field)
other tests/*.el	tests/	test_sessions.el, test_safety.el, test_bell_safety.el, test_layered_cycle.el, test_imprint.el, test_stewardship.el, test_api_define_process.el, …
CI smoke test	ci.yaml	dist/neuron --help runs

Flag (unverified/TODO). CI (ci.yaml) runs only the cc compile + the dist/neuron --help smoke test — it does not invoke the tests/*.el soul-guard / layer-contract suites, and .githooks/ is empty. Whether these tests are gated anywhere (a pre-merge hook, a separate workflow, or manual discipline on the Mac before regenerating soul.c) is not evident in the files read. This is the most important build-integrity gap to confirm with a human: the contract tests exist but their enforcement point is unproven.

Practical consequences for a contributor

• You cannot rebuild the whole soul on Linux/CI. Regenerate dist/soul.c on a Mac (elb), commit it, then CI compiles it. Changing an .el file without regenerating soul.c ships nothing.
• The .elh files are your API map. To see what a module exposes, read its .elh — it's the generated extern fn list.
• Memory/activation behavior often can't be changed from this repo. The volatile numeric core is in foundation/el el_runtime.c. Doc 01, Divergence 6 explains why this is the sharpest edge in the architecture.
• The engram is a separate repo. It is cloned and compiled by CI (Dockerfile, .gitea/workflows/), not vendored here. Its source of truth is foundation/el/engram.