No native_js or native_js_call anywhere. Full browser auth flow expressed
with proper El constructs:
- extern fn supabase_create_client(url, key) -> Any
Declares the Supabase CDN global without an El function body.
- client.auth.signInWithOtp(opts)
Direct method call chain on Any-typed value. The client is built by
calling the extern fn; .auth field access and .signInWithOtp(opts)
method call emit clean JS without any escape hatch.
- try { ... } catch (err: Any) { ... }
Wraps the auth call; unexpected runtime errors are caught and shown
to the user rather than crashing silently.
- fn(event: Any) -> Void { ... }
Inline anonymous function literals for DOM event listeners instead
of named forward-declared callbacks.
The rewrite is the proof: every browser JavaScript pattern used in a
real auth flow can now be expressed structurally in El.
import "https://cdn.example.com/lib.js" now emits:
- module mode: import "https://..." at the top of the generated file
- bundle/IIFE mode: // external: https://... comment
El source imports (.el files) are excluded -- they were already inlined
by resolve_imports before codegen. Any import path that doesn't end in
.el or starts with http(s):// is treated as an external JS dependency.
Add to el_runtime.js:
promise_then(p, cb) -- p.then(cb), works with any Promise-returning API
promise_catch(p, cb) -- p.catch(cb)
promise_resolve(val) -- Promise.resolve(val)
promise_reject(msg) -- Promise.reject(new Error(msg))
object_assign(t, s) -- Object.assign({}, t, s) (non-mutating)
object_keys(obj) -- Object.keys(obj)
object_values(obj) -- Object.values(obj)
json_deep_clone(obj) -- JSON.parse(JSON.stringify(obj))
array_from(iterable) -- Array.from(iterable)
type_of(val) -- typeof val
instanceof_check(v, n) -- val instanceof globalThis[name]
All new functions added to __el export object and ES named exports.
codegen-js preamble destructure updated to include all new names.
try { ... } catch (name: Type) { ... } is now a first-class El statement.
Lexer: `try` and `catch` are now keywords (Try, Catch token kinds).
Parser: TryCatch AST node with try_body, catch_name, catch_body.
codegen-js: emits try { ... } catch (name) { ... } directly -- correct
for all browser error handling patterns.
codegen.el (C backend): emits the try body with a comment; exception
handling is a no-op since C has no analogous mechanism. Programs using
try/catch should compile with --target=js.
The catch variable type annotation is parsed and skipped (same treatment
as all other type annotations in El).
fn(params) -> RetType { body } is now valid in expression position.
The parser produces a Lambda AST node. codegen-js emits a hoisted
JS function declaration with a generated name (__lambda_N) and returns
the name as the expression value, so inline callbacks compose cleanly:
dom_listen(btn, "click", fn(event: Any) -> Void { handle(event) })
emits:
function __lambda_1(event) { handle(event); }
dom_listen(btn, "click", __lambda_1);
The hoisted-declaration strategy is debuggable, has no closure-capture
issues, and requires no string-buffer mode in the codegen.
Any-typed receiver method calls now emit obj.method(args) directly
instead of requiring native_js_call. client.auth.signInWithOtp(p)
compiles to client["auth"].signInWithOtp(p) -- no escape hatch needed.
Field access emits obj["field"] (direct bracket notation) instead of
el_get_field, so prototype-inherited JS properties resolve correctly.
el_get_field's hasOwnProperty guard was silently returning null for
real JS objects with inherited fields (Supabase auth, DOM APIs, etc).
El runtime shortform methods (append, len, get, map_get, map_set)
still use the existing method(obj, args) convention for backward compat.
ExternFn statements emit a comment and are excluded from top-level
statement codegen -- the extern declaration tells the compiler the
function exists in the JS environment without emitting a body.
Adds two post-processing flags that produce production-ready browser JS in a
single elc invocation, replacing extract-js.py in the web product pipeline:
elc --target=js --bundle --minify source.el > output.min.js
elc --target=js --bundle --obfuscate source.el > output.obf.js
--minify shells out to terser (passes=2, no drop_console, drop_debugger).
--obfuscate shells out to javascript-obfuscator with the same options as the
old extract-js.py script. --obfuscate implies --minify.
Tool discovery: checks ./node_modules/.bin/, ../node_modules/.bin/ (monorepo),
then falls back to npx. Both flags require --target=js; passing either without
it exits 1 with a clear error.
Both tools receive a reserved-names list of globals referenced from HTML
onclick= attributes (neuronDemoToggle, signInWith, NEURON_CFG, etc.) so they
are not mangled.
Implementation adds stdout_to_file(path)/stdout_restore() builtins to the C
runtime so codegen's println-streamed output can be captured to a temp file
before being piped through the external tools. Temp files use
/tmp/elc-<pid>-<timestamp>.js naming and are cleaned up on success and failure.
Rebuilds dist/platform/elc and dist/platform/elc.c. Self-hosting verified.
Iteration 5:
? nil-propagation: Field and Index handlers in js_cg_expr now detect when
the object expression is a Try node (the AST node for postfix `?`).
When detected, emit JS optional chaining: `(expr)?.["field"] ?? null`.
The `?? null` normalizes JS undefined to El's null. A bare `expr?` not
followed by field/index still passes through unchanged.
browser-auth.el: a realistic 130-line example demonstrating:
- @async function with Supabase via native_js_call
- DOM bridge: get/set value/text/attr, add/remove class, show/hide
- local_storage_get/set for session hints
- window_on_load for initialization
- window_set to expose functions to the browser global scope
- set_timeout for transient state, is_valid_email for input validation
Compiles cleanly with elc --target=js --bundle
Spec updated: status promoted to Phase 4 / ~80% coverage, nil-prop
status updated, new example referenced.
elc --target=js --bundle source.el > output.js produces a single file
with no import statement that can drop directly into a <script> tag.
How it works:
- detect_bundle() reads the --bundle flag from argv
- resolve_runtime_path() looks for el_runtime.js next to the source file
- compile_js_with_bundle() reads the runtime, calls codegen_js_bundle()
- codegen_js_inner(bundle_mode=true):
- emits ;(function() { "use strict"; at the top
- inlines the runtime content (stripping ES export statements which
are invalid inside an IIFE via js_strip_es_exports())
- skips the const {...} = globalThis.__el destructure -- the inlined
function declarations are already in scope within the IIFE
- closes with })(); after main()
Usage: elc --target=js --bundle app.el > app.js
Place el_runtime.js in the same directory as app.el.
Iteration 3: closes the browser API gap needed for real web pages.
New builtins in el_runtime.js:
Extended DOM: dom_set_attr, dom_get_attr, dom_remove_attr, dom_set_html,
dom_get_html, dom_get_parent, dom_contains_class, dom_get_checked,
dom_set_checked
Timers: set_timeout, set_interval, clear_interval
Local storage: local_storage_get, local_storage_set, local_storage_remove
Window: window_location, window_redirect, window_on_load
Debug: console_log
All browser-only functions use _ensureBrowser guard. Timer functions
work in both Node and browser. All new names added to __el export
object, ES named exports, and codegen-js.el destructure preamble.
Spec table updated to document new categories.
type User = { name: String } was silently broken: the parser consumed
the type name then called expect(LBrace) while sitting on the = token.
expect() advances unconditionally on mismatch, so it consumed = and
treated { as the first field name, producing a corrupt TypeDef node.
The FnDef following the broken TypeDef was then parsed incorrectly or
lost entirely -- causing greet() and similar functions to vanish from
JS/C output with no error.
Fix: detect and skip the optional Eq token before expecting LBrace.
Both targets benefit; rebuild elc to pick up the fix.
Parser now handles `SomeEnum::Variant` in match arm patterns, emitting
a Variant pattern node with enum_name and variant fields. Previously
these fell through to Binding, producing broken codegen.
JS codegen: emit str_eq check against the variant name string (El enums
are plain strings at runtime). C codegen: same, via EL_STR + str_eq.
Rebuild elc to pick up the parser change.
- el_runtime.js: add 19 dom_* builtins (browser-only, throw in Node),
window_set/window_get for exposing El functions to the browser global
scope, and native_js/native_js_call escape hatches for third-party libs
- codegen-js.el: destructure all new builtins in generated preamble; add
@async decorator support that emits async function + await at call sites
for known-async HTTP builtins and user-declared @async functions; pre-
registration pass ensures forward calls to @async functions get await
- spec/codegen-js.md: mark Phase 3 (DOM bridge) implemented, document
@async approach and its limitations, update builtin table and status
- examples/browser-counter.el: canonical example showing dom_get_element,
dom_set_text, dom_is_null, window_set, and state_set/get
Replace accumulate-by-concatenation loops with native_list_append + str_join.
Eliminates quadratic memory growth when processing large source files.
This is the v2 compiler state — what produced /tmp/elc-v2.
- .gitea/workflows/sdk-release.yaml: build elc from bootstrap, run tests,
publish latest release, dispatch el-sdk-updated to downstream repos
- install.sh: one-command El SDK install from Gitea release
elc-combined.el had drifted from el-compiler/src/ across three separate
commits that never synced the bundled flat file:
1. 13948f5 - fold fn main() body into C int main() + _argc/_argv rename
(codegen.el updated, elc-combined.el not updated)
2. 742bd0b - bare reassignment Assign AST node
(parser.el + codegen.el updated, elc-combined.el not updated)
3. ed564b6 - Calendar/CalendarTime/Rhythm/LocalDate/LocalTime types
(codegen.el updated, elc-combined.el not updated)
The drift meant that the elc binary (which embeds the correct logic) could
compile test programs correctly, but a fresh self-host pass using gen2 (built
from the stale elc-combined.el) would produce a gen3 that differed in 39
lines: no fn main body fold and broken bare-assignment codegen.
Fix: regenerate elc-combined.el as a flat concatenation of the current
lexer.el + parser.el + codegen.el + codegen-js.el + compiler.el source
files. Self-host fixed point verified: gen2 == gen3 byte-identical at
6450 lines.
Also rebuild dist/platform/elc and dist/platform/elc.c from the fixed
gen2 pass, and carry the pending http dual-stack change in el_runtime.c.
All tests pass: time (6/6), calendar (10/10), text (8/8), html_sanitizer (29/29).
24 new functions covering counting (str_count, str_count_chars,
str_count_bytes, str_count_lines, str_count_words, str_count_letters,
str_count_digits), finding (str_index_of_all, str_last_index_of,
str_find_chars), transforming (str_repeat, str_reverse,
str_strip_prefix/suffix/chars, str_lstrip, str_rstrip), character
classification (is_letter, is_digit, is_alphanumeric, is_whitespace,
is_punctuation, is_uppercase, is_lowercase), and splitting/joining
(str_split_lines, str_split_chars, str_split_n, str_join).
Phase 1 is byte-level + ASCII character classes. Unicode-grapheme
awareness, normalization, and regex are Phase 2 (filed separately).
Lexer-internal helpers is_digit, is_alpha, is_whitespace renamed to
lex_is_digit, lex_is_alpha, lex_is_whitespace to free the public names
for the runtime exports. The El compiler's lexer.el and the bundled
elc-combined.el both updated.
Codegen registrations: builtin_arity entries for all 24 functions,
is_int_call entries for the Int-returning ones (str_count*,
str_last_index_of, str_find_chars) so the + operator dispatches as
arithmetic when applicable.
Tests: tests/text/ corpus with 8 acceptance cases covering the surface
(count-substring, count-overlap-skip, count-lines-words-letters,
index-of-all, transform-suite, char-classes, split-lines, join). All
pass against a fold-fn-main-aware elc bootstrap (see ELC env var
override in run.sh).
Self-host fixed point: elc-combined.el's emit-main pass does not
currently fold the fn main body into C's main, a pre-existing
condition that surfaces as a 39-line gen2/gen3 diff with empty main
in gen3. The committed dist/platform/elc binary has the fold logic
so all tests pass against it. Filing the elc-combined fold-fn-main
fix separately. This commit does not introduce new self-host drift.
Phase 1.5 of time-system. Calendar is pluggable: EarthCalendar
(IANA zones, DST, Gregorian) is the default; MarsCalendar,
CycleCalendar(period), NoCycleCalendar handle non-Earth cases.
Rhythm abstracts recurrence from clock units - rhythm_cycle_phase(0.5)
means "midpoint of cycle" whether the cycle is 24 hours on Earth or
30 hours on a station or 300 years on a long-cycle world.
Phase 1 (Instant + Duration) unchanged. EarthCalendar(zone_local())
is the user-facing default; nobody who doesn't care about non-Earth
calendars sees the abstraction.
Self-host fixed point holds at 6339 lines.
Snapshot tagged at dist/platform/elc.20260502-1321-self-host.
Phase 2 (scheduling primitives every/after/at) lands next, now with
Calendar-aware grounding instead of Earth-time hardcoded.
Backlog: bl-297f66d8 (supersedes bl-b29b3e60)
Replaces the need for product-level denylist sanitizers. Small
state-machine parser; tag-and-attribute allowlist passed as JSON;
URL scheme validation on href/src attrs (http, https, mailto,
fragment, relative); whole-subtree drop for script/style/iframe/
object/embed/form (plus rarer media containers). No comment-
wrapping (was fragile to comment-injection bypass via a literal
--> inside an attacker-supplied attribute value).
Also picks up the codegen and parser changes for first-class
Instant/Duration types (postfix-literal time values, typed binop
dispatch) that were sitting in tree alongside this work.
Test corpus at tests/html_sanitizer/ covers the live attacker
probes (script, iframe, form, javascript:, about:, data:, img
onerror, onclick) plus structural attacks (comment-injection
bypass, tab-in-scheme bypass, encoded payloads, malformed input,
empty input, plain text). 29 cases, all green.
Self-host fixed point holds at 5720 lines via the canonical
el-compiler/src/compiler.el entry. Snapshot tagged at
dist/platform/elc.20260502-1249-self-host.
Backlog: bl-dc55ae07
Previous commit 6d89728 had a misleading message - the rename
itself never landed (Edit-without-Read failure cascaded silently
in the parent shell). 6d89728 incidentally captured 810 lines of
in-flight work from concurrent runtime agents and shipped it under
the wrong message; the in-flight agents will land their final
verified state on top.
This commit is just the actual rename: str_format(template, data)
to str_format(fmt, data). C++ keyword conflict resolved.
template is a reserved keyword in C++; though not in C, it blocks
this header from ever being included from C++ code. Match printf-
family convention with fmt instead.
The deeper question of whether string-template substitution is the
right abstraction for our substrate is filed separately as backlog.
1. Parser+codegen: bare reassignment `x = expr` inside an if-body
was compiling to three orphan expressions with no store. Now
emits a real assignment.
2. Runtime json_get: dot-path segments that are all digits now
correctly traverse array indices. `json_get(s, "0.field")` works.
3. Runtime HTTP writer: response bodies starting with
`{"__status__":<int>,...}` now set the HTTP status header to
that value and strip the marker from the served body. Existing
404/401/503 paths in product code now produce real status codes
instead of HTTP 200 with the status hidden in the body.
Self-host fixed point holds: gen2 == gen3 byte-identical.
Snapshot tagged at dist/platform/elc.20260502-1231-self-host.
Backlog: bl-c121edda
scan_string() is the right gate for this: every El source that embeds JS
or CSS does so as a quoted string literal, and the lexer is the single
chokepoint every backend reads. Strip there and the // line comments
and /* */ block comments never reach the parser, codegen, or the served
HTML.
looks_like_code is intentionally narrow:
- contains "<script" or "<style" (the embedded-asset case), or
- contains "function" AND ";" (a JS body without an opening tag)
Plain prose with stray // sequences passes through verbatim.
strip_code_comments tracks JS string state (single, double, backtick)
and never strips inside one. Backslash escapes inside JS strings consume
the next char verbatim. URL guard: when the char before / is ':', emit
the / literally and advance one — preserves https:// inside string
literals. Block-comment scan walks until the matching '*/' pair.
elc-cli.el is now a one-line `import "el-compiler/src/compiler.el"`
shim. Top-level `let _argv = args()` was clashing with C int main()'s
`char** _argv` parameter once compiler.el's fn main() body got folded
into C main. compiler.el owns the CLI entry point now.
Self-host fixed point reached: gen2 == gen3 byte-identical.
Tagged dist/platform/elc.20260502-1104-self-host alongside dist/platform/elc.
The El compiler self-host has been broken since `fn main()` landed in
compiler.el. Both bootstrap.py and codegen.el skipped emitting an
`el_val_t main()` (correct - it would collide with C's int main),
but neither folded the body anywhere. The C int main() got just
runtime init + return, so any El program that put its work inside
`fn main()` produced a binary that did nothing.
Fix in two places (bootstrap.py and codegen.el, kept symmetric):
1. Capture the body of `fn main()` during the FnDef pass.
2. Emit `int main(int _argc, char** _argv)` so El programs can
declare their own local `argv` / `argc` (compiler.el itself
does this) without colliding.
3. After top-level statements, fold the captured fn main body
into C main alongside them, then return 0.
Self-host fixed point reached: gen 2 and gen 3 of compiler.el's
output are byte-identical (md5 5b4eca2a...). The new elc compiles
products/web/src/main.el natively now - 24 imports resolved, 1,173
lines of C, every imported function (page_open, nav, pricing,
checkout_page, account_page, founding_badge…) emits its forward
decl + body without a concat preprocessor in sight.
Backup of the prior self-hosted binary is at
dist/platform/elc.preselfhost in case we need to fall back.
Added a typed scan function: walks the live nodes once, skips
transparent layers, keeps only entries whose node_type matches the
filter, sorts the survivors by salience, paginates. Header forward
decl in el_runtime.h so callers can find it.
Empty / NULL filter falls through to engram_scan_nodes_json so the
existing GET /api/nodes contract is preserved exactly.
This is what every list-X tool in the MCP wrapper has been wanting:
listProcesses returning only Process nodes, not all of them, without
the wrapper having to fetch + filter client-side.
Per RFC 9110 §9.3.2, HEAD must mirror GET headers + Content-Length
without sending a body. Existing http_worker / http_worker_v2 dropped
HEAD straight to the El handler, which had no idea what to do and
returned the catch-all 404 envelope. Link checkers and SEO bots saw
the 404 and reported the site as broken.
Fix layer is in the runtime, not the El handler:
* http_worker / http_worker_v2 detect HEAD before calling the
handler, dispatch as method="GET" so handler logic is unchanged,
record head_only in a thread-local, then call http_send_response.
* http_send_response reads the thread-local and skips the
final http_send_all of the body. Status line + headers +
Content-Length still go out in full.
Verified locally on engram /health: HEAD returns
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 48
Connection: close
(no body — curl reports size_download=0)
compiler.el: rename `target` → `tgt` in main(); the lexer reserves
`target` as a keyword, and the let-binding position requires Ident.
The naming convention was already followed elsewhere in the file
(compile_dispatch's parameter is tgt for exactly this reason); main
was an outlier that the existing Rust-genesis-built elc happened to
parse but bootstrap.py refused, blocking self-host.
Both bootstrap.py and compiler.el now inline every imported .el file
into a single source string before lex/parse, depth-first with set
deduplication keyed on absolute path. Two forms supported:
import "path/to/file.el" (quoted relative path)
from <module> import { ... } (bare module → <module>.el)
Strict regex matching prevents false positives like CSS keyframes
("from { opacity: 0 }") embedded in El string literals - the prior
naive str.startswith pulled '{' out as a module name and tried to
load src/{.el.
This kills the bash concat preprocessor that web/build-local.sh
needed. A web full build is now just:
python3 bootstrap.py src/main.el > dist/main.c
cc -O2 ... -o dist/neuron-web dist/main.c dist/web_stubs.c \
foundation/el/el-compiler/runtime/el_runtime.c \
-lcurl -lpthread -lssl -lcrypto
Verified end-to-end: bootstrap.py produces 1,151 lines of C from
src/main.el's 24 imports, cc links a 667 KB binary.
Three codegen bugs surfaced repeatedly across the parallel port-to-El
agents and were patched here:
1. Empty array literal '[]' was emitting el_list_new(0, ) — trailing
comma in a varargs call, fails the C parse. Special-cased: n==0
returns 'el_list_empty()' directly.
2. '==' between two identifiers both tracked in __int_names (typed
Int via 'let x: Int = ...') was miscompiling to str_eq. With the
tagged-pointer Int-as-int64 representation, str_eq strcmp's what
are integer values dressed as char* and segfaults on the first
non-printable byte. Added the int-name lookup, mirroring the
dispatch already present for '+' between Int idents. NotEq got
the same treatment.
3. 'm.field' codegen was passing the raw const char* field name to
el_get_field, which expects el_val_t. C compiler warned about int
conversion; runtime read garbage at the address. Wrapped in
EL_STR(...) so the field name lands as a proper el_val_t.
Runtime additions in the same pass:
- el_runtime.c http_read_request: the loop's boundary check was
'line_end >= hdr_end' which broke before processing the LAST
header line — its trailing \r\n IS hdr_end. Real curl clients
put Content-Length last, so POST bodies were silently arriving
as length 0. Changed to '> hdr_end' so the last line is processed.
soma-server agent surfaced this during smoke testing.
- _GNU_SOURCE feature macro: clock_gettime/CLOCK_REALTIME, strcasecmp,
and the dlfcn extensions (RTLD_DEFAULT) all gated behind it on
glibc/Debian. macOS is permissive without; the landing Docker
build needed these for linux/amd64. Adds <strings.h> for
strcasecmp.
- Refactored slot semantics in el_runtime.c (already in tree from
the morning ARC commit): magic-tagged ElHeader at offset 0,
ElList/ElMap with separate elems/keys/values payload allocations,
el_list_append and el_map_set mutate-in-place when refcount<=1
and copy-on-write when shared.
Self-host fixpoint reached at v3: elc → elc.c → cc → elc binary →
elc.c reproduced byte-for-byte. dist/platform/elc and dist/platform/elc.c
updated. The codegen.el and elc-combined.el changes are mirror-edits;
both flow through the bootstrap chain to keep self-hosting clean.
The compiler used to OOM at ~8.7 GB on 4325-line inputs because every
el_list_append allocated a fresh ElList header + elements array. That
was the workaround for an aliasing bug in cg_if_stmt — codegen held a
stale pointer through a realloc. Persistent semantics fixed the bug
but turned every accumulator (decl in cg_stmts, AST construction, the
__int_names CSV) into O(N²) memory.
Real fix in two coordinated parts:
1. Runtime — ElList and ElMap now carry a magic-tagged ElHeader at
offset 0 (uint32 magic, uint32 refcount). The payload arrays live in
separate heap allocations behind a stable header pointer, so realloc-
grow on append never invalidates the caller's reference. el_list_append
and el_map_set mutate in place when refcount <= 1 (the common single-
owner case, amortized O(1)) and copy-on-write when shared. Adds
el_list_clone for explicit shallow copies, plus el_retain/el_release
no-op-on-non-pointers so codegen can emit them on every let-binding
without tracking types. The magic words (0xE1xxxxxx) live above the
printable-ASCII range so they can never collide with a string's first
byte, and looks_like_string in json_stringify already rejects them.
2. Codegen — every place that delegates to a child C scope now clones
`declared` before passing it down: cg_if_stmt for both then/else
branches, cg_for_body for the loop body (which also picks up the
loop variable via append), and cg_stmt's While case. Without the
clones, mutation-in-place would let a sibling scope's let-bindings
leak into the parent's declared list and the parent would emit
`x = ...` against an undeclared name. The clones are cheap shallow
copies of a list of strings.
Result on the landing-combined.el (4325 lines): 8.7 GB → 3.5 GB peak,
0.26s wall clock, compile completes successfully where it previously
OOM'd. Self-hosting fixpoint reached: dist/platform/elc compiled from
elc-combined.el reproduces dist/platform/elc.c byte-for-byte on a
second pass through itself.
Strings still allocate fresh on every concat; that's the next layer of
optimization (probably an arena tied to function scope) but isn't
blocking. The persistent-list aliasing bug remains structurally fixed —
clones are explicit at the codegen sites where the persistence
guarantee matters; everywhere else the compiler runs at mutation speed.
The Rust bootstrap was archived in 4f3543b and removed from the working
tree in e7a49eb. The bytecode tier was retired in 9fca4dc. What remained
on disk was leftover platform binaries (dist/platform/el-macos-universal,
el-windows-x86_64.exe) that nothing should be invoking, the elvm.md spec
for the retired bytecode tier, and the 8.7GB target/ build cache that
was tracked despite being in .gitignore.
Untracks target/, removes the platform binaries and elvm.md, and updates
spec/language.md so its self-hosting section no longer references the
genesis Rust path. The canonical toolchain is dist/platform/elc against
el-compiler/runtime/el_runtime.{h,c} — one compiler, one runtime, one
language.
Capability becomes a compile-time structural property, not a runtime
convention. A program's top-level block determines what runtime
primitives it may call; the codegen rejects forbidden calls with
#error directives so cc fails with a clear message.
Three kinds:
cgi — full self-formation. All primitives.
service — bounded. Cannot call self-formation primitives:
llm_call_agentic, llm_register_tool, dharma_emit,
dharma_field. Single-turn LLM calls allowed.
utility — default (no top-level block). No DHARMA, no LLM.
Pure compute + I/O.
Deep claim: the binary either CAN or CANNOT do a thing. There is no
runtime check, no opt-in, no override. A weather service compiled
with `service { ... }` is structurally incapable of becoming Neuron.
Sponsors of services know exactly what they're vouching for.
Implementation
- Lexer: `service` keyword.
- Parser: parse_service_block parallels parse_cgi_block. Produces
ServiceBlock AST with name/sponsor/domain.
- Codegen entry: scans top-level for cgi/service blocks, sets
__program_kind state ("cgi" / "service" / "utility"). Rejects
programs declaring both kinds.
- cg_expr Call: cap_check_call(fn_name) per emission. Records
violations in __cap_violations CSV. emit_cap_violations() writes
one #error per violation at end of generated C.
- Helpers: is_self_formation_call, is_dharma_call, is_llm_call.
Tests verified:
cgi + llm_call_agentic → compiles ✓
service + llm_call_agentic → cc fails with capability violation
for 'service' on 'llm_call_agentic'
service + llm_call (1-turn) → compiles ✓
utility + dharma_send → cc fails with capability violation
for 'utility' on 'dharma_send'
utility + http/json/state → compiles + runs ✓ ("got: world")
cgi + dharma_emit (manager) → compiles ✓ (VBD also enforced)
cgi + dharma_emit (engine) → cc fails with VBD violation
Three-stage closure: stage1.c == stage2.c (byte-identical).
Engram rebuilt against new compiler — daemon on :8742 healthy,
{"node_count":0,"edge_count":0}.
A bug found and fixed during testing: cap_record_violation had
`csv = ","` (bare assignment, not valid in El) instead of
`let csv = ","`. Without the let, the leading comma never made
it into the accumulator, off-by-one'ing the kind extraction so
"service" appeared as "ervice" in error messages. Pattern
fixed; this confirms once more that El requires `let X = ...`
for all rebindings (codegen converts to assignment when X is
already declared).
Two parallel agent sweeps closing the remaining structural gaps.
== Compiler completions ==
- match codegen: lowers Match into GCC/Clang statement-expression
({ ... }). Patterns: Wildcard, Binding, LitInt (==), LitStr
(str_eq), LitBool. Per-match unique label via state counter.
Verified: classify(0)→"zero", classify(1)→"one", classify(7)→"other".
- cgi block parsing: `cgi "name" { dharma_id, principal, network,
engram }` → CgiBlock AST node → el_cgi_init() emitted as the first
call in main() after el_runtime_init_args. Multiple cgi blocks per
program emit a #error directive. Missing optional fields → EL_NULL.
- VBD compile-time enforcement: parser attaches `decorator: <name>`
to FnDef. Codegen recursively walks fn bodies (Call/BinOp/Not/Neg/
Field/Index/Try/Array/Map/If/For/Match plus Let/Return/Expr/While/
For). If a non-@manager function calls dharma_emit or dharma_field,
emit `#error "VBD violation: ... fn '<name>'"` before the function
body. Verified: @engine fn calling dharma_emit → cc fails with the
message. @manager fn calling dharma_emit → compiles clean.
Three-stage closure: stage1.c == stage2.c == stage3.c (2791 lines
each, byte-identical). dist/platform/elc rebuilt at 165 KB; .prev5
preserved.
== Runtime completions ==
- Real dharma_* primitives, no more stubs. Channel registry,
request/response over HTTP, network-wide spreading activation,
fire-and-forget event emission, blocking dharma_field with
pthread_cond_timedwait (30s default), Hebbian relationship
weights stored as Engram edges between dharma:self and
dharma:peer:<id>, sorted-by-weight peer list. URL/ID arrays
snapshotted before network I/O so mutexes never block on socket.
- New public C contract: el_runtime_dharma_event_arrive(type, payload,
source) — application HTTP handler calls this when /dharma/event
arrives, runtime broadcasts on _dharma_event_cv. Keeps the HTTP
server generic; events flow through the application's router.
- llm_call_agentic real multi-turn loop. Tool registry (mutex-
protected, dlsym-resolved, mirroring http_set_handler). Loop:
build request with tools+messages → POST → dispatch on stop_reason.
end_turn → return text. max_tokens → text + "[truncated]". tool_use
→ walk content[], call registered handler per block, build
tool_result message, append to conversation, loop. Iteration cap
10. Tools not registered return {"error":"tool not registered: X"}
with is_error: true.
- New builtin: llm_register_tool(name, handler_fn_name).
Compile clean: cc -std=c11 -Wall -Wextra -c → zero warnings, zero
errors. Smoke test exercises every new dharma_* primitive +
llm_register_tool round-trip.
Runtime grew 3309→4079 lines (.c, ~155 KB), 312→342 lines (.h).
== Integration ==
Engram rebuilt against the new runtime: 130 KB binary, daemon
swapped on :8742 cleanly, /health and /api/stats both returning
correctly under launchd. No regressions.
== Status of "planned" items in language.md ==
- match codegen → IMPLEMENTED
- cgi block parsing → IMPLEMENTED
- VBD enforcement → IMPLEMENTED
- % operator → IMPLEMENTED (earlier today)
- vessel keyword → lexed (codegen uses package compatible)
- activate construct → still planned (low priority; engram_activate
builtin covers the use case for now)
- sealed block → still planned
- dharma_emit fanout parallelization → potential future work, current
serial behavior matches spec
The .prev/.prev2/.prev3/.prev4 backups were checkpoints during the
self-host bootstrap. Now that closure is verified and we're past the
fragile period, drop them. The current elc and the original elc.legacy
are sufficient — git history preserves the genealogy.
Three changes that turned the runtime into something Engram-the-server
can actually run on top of.
1. engram_*_json accessors. The runtime's engram_get_node/search/scan/
neighbors/activate return ElList/ElMap; passing those through
json_stringify hit the type-erasure wall (an ElList* has no header
that distinguishes it from a string pointer). Added pre-serialized
sibling builtins:
engram_get_node_json(id) -> JSON object
engram_search_json(query, limit) -> JSON array of node objects
engram_scan_nodes_json(limit, offset)
engram_neighbors_json(node_id, max_depth, direction)
engram_activate_json(query, depth)
engram_stats_json()
Each walks the typed C structures and serializes directly, reusing
the existing engram_emit_node_json / engram_emit_edge_json helpers
from the snapshot path.
2. http_set_handler now falls back to dlsym(RTLD_DEFAULT, name) when
the named handler isn't already in the C-level registry. El programs
that define `fn handle_request(method, path, body) -> String` can
register themselves just by calling http_set_handler("handle_request").
No C glue required. Verified live on a real El server.
3. Codegen: extended int-typed dispatch on `+` to handle Calls. New
helper is_int_call recognizes a known-int-returning builtin set:
str_len, str_index_of, str_to_int, str_char_code, native_list_len,
el_list_len, len, json_get_int, json_array_len, engram_node_count,
engram_edge_count, time_now, time_now_utc, time_diff, time_add,
time_from_parts, el_abs/max/min, float_to_int. With this,
`pos + str_len(needle)` compiles to integer arithmetic instead of
string concat. The earlier limitation noted in the previous commit
(Ident + Call returning Int) is now closed.
Also: el_to_float / el_from_float moved to el_runtime.h as static
inlines so generated programs can use them. Eliminates the unused
inline definitions that were duplicating in the .c file.
Closure verified: stage1 vs stage2 byte-identical against the new
runtime. dist/platform/elc rebuilt; .prev4 preserved.
Engram server (engram/src/server.el) end-to-end:
POST /api/nodes ×3 → 3 UUIDs returned
POST /api/edges ×2 → linkage made
GET /api/stats → {"node_count":3,"edge_count":2}
GET /api/search?q=spreading&limit=5 → 1 hit, full node JSON
POST /api/activate {"query":"Hebbian","depth":3}
→ seed node @ hop 0, strength 0.8
→ 1-hop neighbor @ strength 0.392 (= 0.8 × 0.7 weight × 0.7 decay)
GET /api/neighbors/<id>?depth=2 → {node, edge, hops} triple
POST /api/save → {"ok":true,"path":"..."}
Server stays alive across all routes.
Snapshot save/load on restart still TODO — server starts with 0 nodes
even when a snapshot exists; investigation pending.
El programs that define `fn handle_request(method, path, body) -> String`
can now use http_serve directly without C-level glue. http_set_handler
falls back to dlsym(RTLD_DEFAULT, name) when the named handler isn't
already in the registry, picks up the El-compiled symbol, and registers
it transparently.
Closes the gap that made http_serve unusable from pure El. Verified
with a real El server on :17890 — POST /hello with body returned
{"method":"POST","path":"/hello","echo":"test body"} via curl.
dist/platform/elc rebuilt; .prev3 preserved.
Batches 2/3/4 of the runtime extension. The runtime grew from 1620
to 3112 lines (.c) and 247 to 286 lines (.h) — adding 27 new or
real-implementation builtins and replacing every batch-1 stub.
Batch 2 — HTTP / fs (8 builtins)
- http_get, http_post: replaced stubs with real libcurl client.
Network errors return JSON {"error":"..."} so callers can detect.
- http_post_json: sets Content-Type: application/json.
- http_get_with_headers, http_post_with_headers: ElMap → headers.
- http_post_form_auth: form-urlencoded + Authorization header
(Stripe-style API calls).
- http_serve: replaced stub with real POSIX-socket server, threaded,
capped at 64 concurrent connections. Auto-detects content type
(HTML / JSON / plain). Handler dispatch via named registry.
- fs_list: directory listing via opendir/readdir.
Batch 3 — In-process graph store (14 builtins)
- engram_node, engram_node_full: create node, returns UUID.
- engram_get_node, engram_forget, engram_node_count.
- engram_strengthen: Hebbian potentiation (+0.05, clamp 1.0,
bumps last_activated).
- engram_search, engram_scan_nodes: text search, paginated scan.
- engram_connect, engram_edge_between, engram_neighbors,
engram_neighbors_filtered, engram_edge_count.
- engram_activate: real spreading-activation algorithm.
BFS to depth, max-activation merge across paths, decay 0.7/hop,
multiplied by node confidence, filtered by epistemic_confidence
≥ 0.2 (refresh threshold), sorted desc.
- engram_save, engram_load: JSON snapshot persistence.
Batch 4 — LLM (5 builtins)
- llm_call, llm_call_system: Anthropic /v1/messages via libcurl.
ANTHROPIC_API_KEY from env. Default model claude-sonnet-4-5.
- llm_vision: adds image content block. URL / base64 / file path
detected by prefix.
- llm_models: returns the available model list.
- llm_call_agentic: stubbed with TODO (single-turn fallback to
llm_call_system); full tool_use loop is the next iteration.
Codegen fix: emit Float literals as `el_from_float(<v>)`. Without
the wrapper, C implicit conversion truncates 0.8 to 0 when passed to
a builtin expecting el_val_t. Float helpers moved to el_runtime.h
so generated programs can call them.
Compile-time
- cc -std=c11 -Wall -Wextra -c el_runtime.c → no errors, no warnings.
- Link requires -lcurl -lpthread (documented in header comment).
Verified end-to-end
- engram_node × 2, engram_connect, engram_activate("Hebbian", 2)
returns 2 activated nodes with correct epistemic confidence.
- http_get("https://httpbin.org/get") returns 259-byte JSON live.
- Self-host closure: stage1 vs stage2 byte-identical against the
new runtime.
- engram_save → engram_load round-trip preserves graph.
dist/platform/elc rebuilt against the new runtime (147 KB, up from
94 KB due to libcurl link). .prev2 preserves the prior binary.
The bytecode VM was the bootstrap path before C transpilation landed
(commit ede087e). With elc self-hosting, both elvm and the bootstrap
.elc artifacts are no longer on the critical path. Removing:
dist/elvm/elvm-aarch64-apple-darwin (4.3 MB legacy VM binary)
el-compiler/bootstrap/el-compiler.elc (111 KB bytecode bootstrap)
el-compiler/dist/el-compiler.elc (110 KB)
main.elc / main.map.json
llm_test.elc / llm_test.map.json
test_*.elc / test_*.map.json
The compiler is now: source.el → elc → C → cc → native binary.
One tier. No VM. No bytecode in the runtime path.
dist/platform/elc.legacy preserved as backup of the broken pre-fix
binary; will retire once we're confident in the new path.
Will Anderson