Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.
src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
GET /health
GET /api/stats
POST /api/nodes (auth required)
GET /api/nodes
GET /api/nodes/:id
DELETE /api/nodes/:id (auth required)
POST /api/edges (auth required)
GET /api/neighbors/:id
POST /api/activate
GET /api/activate
POST /api/search
GET /api/search
POST /api/strengthen (auth required)
POST /api/save (auth required)
POST /api/load (auth required)
Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.
Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.
Build: dist/platform/elc src/server.el > dist/engram.c
cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
dist/engram.c <runtime>/el_runtime.c
Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.
Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
Three changes that turned the runtime into something Engram-the-server
can actually run on top of.
1. engram_*_json accessors. The runtime's engram_get_node/search/scan/
neighbors/activate return ElList/ElMap; passing those through
json_stringify hit the type-erasure wall (an ElList* has no header
that distinguishes it from a string pointer). Added pre-serialized
sibling builtins:
engram_get_node_json(id) -> JSON object
engram_search_json(query, limit) -> JSON array of node objects
engram_scan_nodes_json(limit, offset)
engram_neighbors_json(node_id, max_depth, direction)
engram_activate_json(query, depth)
engram_stats_json()
Each walks the typed C structures and serializes directly, reusing
the existing engram_emit_node_json / engram_emit_edge_json helpers
from the snapshot path.
2. http_set_handler now falls back to dlsym(RTLD_DEFAULT, name) when
the named handler isn't already in the C-level registry. El programs
that define `fn handle_request(method, path, body) -> String` can
register themselves just by calling http_set_handler("handle_request").
No C glue required. Verified live on a real El server.
3. Codegen: extended int-typed dispatch on `+` to handle Calls. New
helper is_int_call recognizes a known-int-returning builtin set:
str_len, str_index_of, str_to_int, str_char_code, native_list_len,
el_list_len, len, json_get_int, json_array_len, engram_node_count,
engram_edge_count, time_now, time_now_utc, time_diff, time_add,
time_from_parts, el_abs/max/min, float_to_int. With this,
`pos + str_len(needle)` compiles to integer arithmetic instead of
string concat. The earlier limitation noted in the previous commit
(Ident + Call returning Int) is now closed.
Also: el_to_float / el_from_float moved to el_runtime.h as static
inlines so generated programs can use them. Eliminates the unused
inline definitions that were duplicating in the .c file.
Closure verified: stage1 vs stage2 byte-identical against the new
runtime. dist/platform/elc rebuilt; .prev4 preserved.
Engram server (engram/src/server.el) end-to-end:
POST /api/nodes ×3 → 3 UUIDs returned
POST /api/edges ×2 → linkage made
GET /api/stats → {"node_count":3,"edge_count":2}
GET /api/search?q=spreading&limit=5 → 1 hit, full node JSON
POST /api/activate {"query":"Hebbian","depth":3}
→ seed node @ hop 0, strength 0.8
→ 1-hop neighbor @ strength 0.392 (= 0.8 × 0.7 weight × 0.7 decay)
GET /api/neighbors/<id>?depth=2 → {node, edge, hops} triple
POST /api/save → {"ok":true,"path":"..."}
Server stays alive across all routes.
Snapshot save/load on restart still TODO — server starts with 0 nodes
even when a snapshot exists; investigation pending.
El programs that define `fn handle_request(method, path, body) -> String`
can now use http_serve directly without C-level glue. http_set_handler
falls back to dlsym(RTLD_DEFAULT, name) when the named handler isn't
already in the registry, picks up the El-compiled symbol, and registers
it transparently.
Closes the gap that made http_serve unusable from pure El. Verified
with a real El server on :17890 — POST /hello with body returned
{"method":"POST","path":"/hello","echo":"test body"} via curl.
dist/platform/elc rebuilt; .prev3 preserved.
Batches 2/3/4 of the runtime extension. The runtime grew from 1620
to 3112 lines (.c) and 247 to 286 lines (.h) — adding 27 new or
real-implementation builtins and replacing every batch-1 stub.
Batch 2 — HTTP / fs (8 builtins)
- http_get, http_post: replaced stubs with real libcurl client.
Network errors return JSON {"error":"..."} so callers can detect.
- http_post_json: sets Content-Type: application/json.
- http_get_with_headers, http_post_with_headers: ElMap → headers.
- http_post_form_auth: form-urlencoded + Authorization header
(Stripe-style API calls).
- http_serve: replaced stub with real POSIX-socket server, threaded,
capped at 64 concurrent connections. Auto-detects content type
(HTML / JSON / plain). Handler dispatch via named registry.
- fs_list: directory listing via opendir/readdir.
Batch 3 — In-process graph store (14 builtins)
- engram_node, engram_node_full: create node, returns UUID.
- engram_get_node, engram_forget, engram_node_count.
- engram_strengthen: Hebbian potentiation (+0.05, clamp 1.0,
bumps last_activated).
- engram_search, engram_scan_nodes: text search, paginated scan.
- engram_connect, engram_edge_between, engram_neighbors,
engram_neighbors_filtered, engram_edge_count.
- engram_activate: real spreading-activation algorithm.
BFS to depth, max-activation merge across paths, decay 0.7/hop,
multiplied by node confidence, filtered by epistemic_confidence
≥ 0.2 (refresh threshold), sorted desc.
- engram_save, engram_load: JSON snapshot persistence.
Batch 4 — LLM (5 builtins)
- llm_call, llm_call_system: Anthropic /v1/messages via libcurl.
ANTHROPIC_API_KEY from env. Default model claude-sonnet-4-5.
- llm_vision: adds image content block. URL / base64 / file path
detected by prefix.
- llm_models: returns the available model list.
- llm_call_agentic: stubbed with TODO (single-turn fallback to
llm_call_system); full tool_use loop is the next iteration.
Codegen fix: emit Float literals as `el_from_float(<v>)`. Without
the wrapper, C implicit conversion truncates 0.8 to 0 when passed to
a builtin expecting el_val_t. Float helpers moved to el_runtime.h
so generated programs can call them.
Compile-time
- cc -std=c11 -Wall -Wextra -c el_runtime.c → no errors, no warnings.
- Link requires -lcurl -lpthread (documented in header comment).
Verified end-to-end
- engram_node × 2, engram_connect, engram_activate("Hebbian", 2)
returns 2 activated nodes with correct epistemic confidence.
- http_get("https://httpbin.org/get") returns 259-byte JSON live.
- Self-host closure: stage1 vs stage2 byte-identical against the
new runtime.
- engram_save → engram_load round-trip preserves graph.
dist/platform/elc rebuilt against the new runtime (147 KB, up from
94 KB due to libcurl link). .prev2 preserves the prior binary.
The bytecode VM was the bootstrap path before C transpilation landed
(commit ede087e). With elc self-hosting, both elvm and the bootstrap
.elc artifacts are no longer on the critical path. Removing:
dist/elvm/elvm-aarch64-apple-darwin (4.3 MB legacy VM binary)
el-compiler/bootstrap/el-compiler.elc (111 KB bytecode bootstrap)
el-compiler/dist/el-compiler.elc (110 KB)
main.elc / main.map.json
llm_test.elc / llm_test.map.json
test_*.elc / test_*.map.json
The compiler is now: source.el → elc → C → cc → native binary.
One tier. No VM. No bytecode in the runtime path.
dist/platform/elc.legacy preserved as backup of the broken pre-fix
binary; will retire once we're confident in the new path.
Closes the known limitation from the self-host commit: `fn add(a:Int,
b:Int) { a + b }` now compiles to integer addition, not string concat.
Previously the codegen heuristic guessed string concat whenever both
operands were Idents with no literal anchor.
Mechanism
- parser captures the leading type identifier from `let x: T = ...`
bindings (new "type" field on Let) and from function parameter
annotations (new "type" field on each param).
- codegen maintains a per-function int-name set in process state via
state_set("__int_names", csv). cg_fn seeds it from typed parameters;
cg_stmt extends it from typed `let` bindings and from `let x = <Int
literal>` (literal inference).
- BinOp Plus: when both sides are Idents and both names are in the
int-name set, emit arithmetic; otherwise the existing literal-anchor
heuristic applies, with string concat as the fallback.
This is the first compiler change made entirely through the self-
hosting workflow — no Python bootstrap. Edit el source, run existing
elc on elc-combined.el, cc the output, test. Closure holds at the
new binary.
Tests
- add(40, 2) → 42
- count_to(10) → 45 (let i: Int / let total: Int rebinding)
- Regression suite (tiny/implret/whiletest/lextest) unchanged.
dist/platform/elc updated; .prev preserved.
Today's milestone: dist/platform/elc compiles itself byte-for-byte to
itself (stage1 == stage2 == stage3 verified). The compiler is now a
real binary in the world.
What landed
- Spec rewrite (language.md) to truth — every feature marked
implemented / planned / not-in-this-language with no fiction.
- C runtime extension: 51 new builtins. JSON parser + accessors,
time, UUID, env, in-process state K/V, float formatting + math,
string ops (index_of, split, char_at, char_code, pad_left/right,
format), list ops (push, push_front, join, range), bool_to_str.
Runtime grew 631 → 1611 lines, header 171 → 247.
- Codegen fix: transform_implicit_return lifts a function's bare
trailing expression into an explicit return. Without it, lex(),
parse(), and every other implicit-return function returned 0/nil
and the whole pipeline produced empty C output.
- Codegen fix: index expressions dispatch on AST kind. obj["literal"]
→ el_get_field (map), arr[i] → el_list_get (list). Same Index node
in the parser, two different runtime calls.
- Codegen fix: skip emitting fn main() (collides with C main()) and
honor parsed return-type annotations so Void functions don't get
return-wrapped (return println(x) is a C type error).
- Parser: capture return-type identifier from -> Ret annotations.
- Lexer: + vessel keyword, + % operator, + \r escape.
- Runtime fix: el_list_append now allocates a fresh list rather than
realloc'ing the input. Realloc moved blocks made caller pointers
dangle, which was inserting garbage values into declared lists and
causing strcmp segfaults. Persistent allocation eliminates the
whole class of use-after-free at modest memory cost.
Bootstrap path
- One-shot Python helper translated elc-combined.el to C and
produced stage1. Helper is disposable; not committed.
- stage1 compiles elc-combined.el → stage2.c which cc compiles to
stage2; stage2 compiles elc-combined.el → stage3.c. stage2.c and
stage3.c are byte-identical. Closure proven.
- New elc installed at dist/platform/elc; old broken binary
preserved as dist/platform/elc.legacy.
- dist/platform/elc.c is the canonical generated source.
- elvm and the bytecode pipeline are no longer on the critical path.
Known gap
- The `+` operator's heuristic dispatch still picks string concat
when both operands are Idents with no literal anchor. Self-hosting
works because the compiler source is careful, but `fn add(a:Int,
b:Int) { a + b }` will not do arithmetic until codegen reads the
parsed type annotations to dispatch. Fix is wiring; not done here.
Tested
- tiny / lextest / whiletest / map+field / array build all run.
- cgi-studio (1037 lines real El) compiles to C cleanly. Link fails
only because runtime is missing fs_list, json_encode, llm_*; those
are scheduled batches.
- Three-stage closure (stage1 vs stage2 vs stage3) byte-identical.
Sealed Rust genesis compiler source. Tarball preserved at
~/Archives/el-rust-bootstrap-20260430.tar.gz. Self-hosted El
compiler (dist/platform/elc) is the canonical compiler from here on.
Rewrites codegen.el to produce C source instead of JSON bytecode,
eliminating the ELVM interpreter as a runtime dependency.
- All El values use el_val_t (int64_t) as the universal type; integers
are stored directly, strings/pointers via uintptr_t cast
- String literals wrapped with EL_STR(), arithmetic works natively
- fn declarations become C functions returning el_val_t
- let bindings become el_val_t local variables
- if/else, while, for all map to native C control flow
- String + String uses el_str_concat(); numeric + uses C +
- strip_outer_parens() prevents double-paren warnings in if/while
- compiler.el updated to describe C output and correct CLI usage
Adds el-compiler/runtime/ with:
- el_runtime.h: declares all builtins using el_val_t
- el_runtime.c: implements I/O, strings, math, list, map, fs, JSON;
HTTP builtins are stubs (return empty string) pending libcurl
Compile El programs with:
cc -I<runtime-dir> -o hello hello.c el_runtime.c
The El compiler cannot parse app { config { KEY: Type = value } } syntax —
it's a declaration, not an expression. parse_app_block() already extracts
config/secrets/flags correctly; the remaining source just needs the block
removed before Compiler::compile() runs.
strip_app_block() replaces the block with blank lines (preserving line numbers)
and correctly skips occurrences inside // comments.
Fixes daemon startup: 'compile error: parse error: invalid expression starting with : at 632:23'
Full-featured terminal explorer for the Engram knowledge graph built
natively in El. Features:
- ANSI-colored TUI with box-drawing borders and salience bars
- All API endpoints: stats, nodes by type/tier, search, edges,
spreading activation, node detail with neighbor traversal
- Text report export via fs_write
- Offline/unreachable mode with helpful startup messages
- Interactive mode command reference
- ENGRAM_URL env var for connecting to non-default servers
- Uses json_get_raw for nested JSON object traversal
Adds json_get_raw(value, key) -> String that serializes any JSON field
(including nested objects and arrays) back to a JSON string. Existing
json_get_string only handles primitive string values. This enables
traversing nested JSON structures in El programs.
Also registers json_get_raw in is_builtin().
Round 14: Breadcrumb directory click — clicking a path segment in the breadcrumb expands/reveals that directory in the file tree
Round 15: El version in status bar — GET /api/status now returns el_version (via el --version), shown in status bar right side; EL_BINARY config env var
Round 16: Search improvements — case-sensitive, whole-word, regex toggles (Alt+C/W/R); project-wide replace-all in current file; backend SearchOpts struct for each mode
Round 17: Sticky scroll improvements — uses CM6 posAtCoords for accurate first-visible-line; clickable to jump to definition; sticky-name/sticky-goto styling
Round 18: File tree header — New File (+) button and Refresh (↺) button in file tree header panel
Round 19: Status bar diagnostics — error count (✕ N) and warning count (⚠ N) shown in status bar, clickable to jump to problems panel
Round 20: Polish — more El snippets (test, seed, assert, activate, parallel, deploy, import, with, retry, reason, trace), expanded command palette (11 new commands)
Registers canvas_image(path, x, y, w, h) in the type system and
implements it in the interpreter using the image crate — scales to
exact dimensions via Lanczos3 and alpha-composites onto the pixmap.
Removes wry/tao WebView layer. Adds a pixel-level canvas API so Engram
programs can drive the window directly — fill rects, draw text, handle
mouse/keyboard input — enabling the UI framework to be written in Engram.
Dependencies: winit 0.29 (rwh_05), softbuffer 0.3, tiny-skia 0.11, fontdue 0.8.
Adds five new builtins backed by wry 0.47 + tao 0.30:
- window_open(title, width, height) — configure window
- webview_load_url(url) — set URL to navigate to
- webview_load_html(html) — set HTML to display
- webview_eval(js) — queue JS to eval after load
- window_run() — open native OS window and run event loop (blocking, exits on close)
WebViewState is stored in a thread_local. window_run() calls
tao's event_loop.run() which never returns; process exits when
the window is closed via std::process::exit(0).
Engram can now power the Neuron CLI:
- readline(prompt) -> String: interactive terminal input via stdin
- http_post_auth(url, token, body) -> String: authenticated POST for daemon API
- color_cyan/green/red/yellow/bold/dim(s) -> String: ANSI color output
All registered in el-types type checker
- el build now resolves import "file.el" directives recursively (was only
done for el run-file and el check; project builds failed silently)
- Add .gitignore (target/, *.elc, *.sealed, *.map.json)
New builtins in dispatch_builtin:
- hmac_sha256(secret, data) -> String (hex)
- base64_url_encode(s) / base64_url_decode(s) -> String
- unix_timestamp() -> Int
- uuid_v4() -> String (alias for uuid_new)
- json_encode(v) / json_decode(s) -> polymorphic
- json_get_string(obj, key), json_get_int(obj, key), json_get_array(obj, key)
— work on both Value::Map and Value::Struct (json_parse returns Struct)
- http_get_auth(url, token), http_put_auth(url, token, body), http_delete_auth(url, token)
- string_split_last(s, delim) -> [before, after] on last occurrence
- array_get(arr, idx) -> element
http_serve upgraded to general-purpose router: passes all requests to
handle_request(method, path, body) — not just /axon/message. Method, path,
body stored in GLOBAL_STATE before calling callback; handle_request receives
them as positional args via initial_stack.
env() now returns "" (not Nil) when var is unset — enables `env("X") == ""`
comparisons in Engram code.
run_sub_interpreter_with_stack() added to support pre-pushing args onto the
call stack before entering a function.
Fix pre-existing non-exhaustive match errors in el-fmt, el-types, el-arch
for Retry, Deploy, With, Reason, Parallel, Trace, Activate AST nodes.
Register all builtins in TypeEnv::with_builtins() to eliminate type-checker
warnings for builtin calls.
Implements 8 new language features:
- |> pipe operator: a |> f desugars to f(a), left-associative chains
- with record update: let b = a with { field: val } — non-destructive struct update
- retry/fallback: retry N times { ... } fallback { ... } with counter-based loop codegen
- reason: AI inference primitive calling soma /v1/chat/completions at runtime
- parallel: concurrent execution block returning a Map of named results via threads
- trace: zero-cost observability block emitting TraceBegin/TraceEnd with ms timing
- requires: precondition annotation on fn, emits ContractCheck bytecode at entry
- deploy: deployment-as-syntax posting to soma /v1/deploy at runtime
All features thread through lexer → parser/AST → codegen → runtime interpreter.
- Neuron pair programming panel (right column, 300px): full conversation interface
with code-block rendering, "Insert at cursor" for AI suggestions, quick actions
(explain selection, find activations, analyze types), thinking indicator
- Live activation preview: floating inline widget when cursor is inside
`activate TypeName where "..."` expressions; queries /api/lsp/activate-preview
- Type graph activation highlighting: `activate` statements cause named types to
pulse with purple glow rings; animation loop runs while activations are present
- Knowledge Graph Explorer: new bottom panel tab; search the Engram DB for semantic
concepts, insert activate expressions, ask Neuron about nodes
- Semantic autocomplete: completions with score >= 0.8 get "⟁ semantic" prefix and
priority boost in the completion list
- Backend: /api/reason extended to support three modes: "hypothesis" (existing),
"pair" (proxies to soma /v1/chat/completions with full file context + system
prompt), "knowledge-search" (proxies to Engram DB search endpoint)
- Backend: new GET /api/lsp/activate-preview endpoint queries Engram for nodes
matching a partial activate expression
- Type graph node click now navigates editor to the type definition
- Resolved merge conflicts in engram-lang/crates (ast.rs, parser.rs, checker.rs,
types.rs): took union of HEAD and worktree-agent branches
- Register print/println/log/print_err in TypeEnv::with_builtins() with polymorphic (Unknown) param type so any value type is accepted without spurious warnings
- Add StructLit expr to AST + parser (uppercase-IDENT { ... } syntax), BuildStruct bytecode instruction + Struct value to runtime
- Add type_params to FnDef AST node and TypeParam variant to TypeExpr; parser parses <T, E> generics; type checker treats TypeParam as Unknown (universal type)
- Rewrite interpreter to support user-defined function calls via call stack (Frame + return_ip); dispatch_builtin handles print/println/log/print_err/__build_list__
- Fix engram_activate_search to unwrap { results: [...] } response envelope from /search; use std::net for sync HTTP to avoid reqwest dependency
- Add run-file command to CLI for single-file execution without el.toml
- Fix worktree engram-crypto path dep
Axum HTTP server (port 7771) serving a single-page IDE with CodeMirror 6
syntax highlighting for engram-lang, a force-directed type graph visualizer,
LSP (completions, hover, diagnostics), SSE-streamed build/run output, a
plugin host with five first-party plugins, and a reasoning panel that proxies
to engram-server. 28 tests across three crates, zero warnings.
Will Anderson