Three research-grounded improvements:
1. Tier-based temporal decay in el_runtime.c (engram_node_full, engram_node_layered):
Working=48h, Episodic=72h, Semantic=336h, Procedural=720h half-lives.
Grounded in ACT-R literature — differentiated decay by chunk type. The
temporal_decay_rate field existed but was always 0 (global 168h for everything).
New nodes now carry the correct half-life for their tier from creation.
2. Implement route_neuron_knowledge_promote in server.el (was a silent stub):
Reads existing node, creates promoted-tier copy with supersedes edge,
checkpoints. promote_knowledge MCP tool now has real effect.
3. ISE label extraction + offset support in route_neuron_state_events:
POST now extracts 'event' field from content JSON as label (heartbeat,
wm_promotion, etc.) instead of always writing 'state-event'. GET now
accepts ?offset= for pagination to reach recent ISEs.
BM25+ (k1=1.2, b=0.75, delta=1.0) now powers all search routes in EL.
No external dependencies in the activation/search path.
- bm25_tokenize/bm25_count_term/bm25_score_doc/bm25_search_json in server.el
- route_search, route_neuron_recall: engram_search_json -> bm25_search_json
- route_activate: BM25 pre-bias (strengthen top-10) before spreading activation
- Remove standalone /api/bm25/search endpoint (BM25 is the engine, not a feature)
- Fix zero-score filter: float comparison not string match
- Add + to tokenizer for URL-encoded query params
- Scan floor 200 nodes regardless of limit size
- Revert Ollama engram_embed_query from 9af2482 (no Ollama at query time)
- Add list_set and math_exp builtins to el_runtime.c
- Add list_set, math_exp, and float_add/sub/mul/div/gt/lt/eq/gte/lte builtins to
el_runtime.c + el_runtime.h (float arithmetic builtins needed because EL operators
+*/ operate on raw el_val_t bits, not IEEE 754 doubles)
- Remove engram_embed_query() and its forward declaration from el_runtime.c
- Remove Ollama cosine-similarity blend from activation scoring (reverts 9af2482):
drops query_emb/query_edim variables, bias *= (1 + 0.3 * sim) block, and all
free(query_emb) calls from the activation loop
- Implement BM25+ scoring in server.el (k1=1.2, b=0.75, delta=1.0):
bm25_tokenize, bm25_count_term, bm25_score_doc, bm25_search_json
V1 uses n_t=1 approximation (constant IDF per corpus size); acceptable as a
first pass without an inverted index
- Wire /api/bm25/search POST/GET route in server.el dispatcher
- Zero Ollama calls in the activation/search path; embeddings on nodes are
untouched (still written at node-creation time)
engram_cosine_sim() was defined and embeddings were computed per-node
via nomic-embed-text on write, but the function was never called during
activation scoring. The goal_bias computation used only lexical substring
matching, ignoring all stored embedding vectors.
This change adds engram_embed_query() to embed the query string at search
time (5s timeout so Ollama latency never blocks activation), then blends
cosine similarity into the working-memory bias with α=0.3:
bias_final = goal_bias(lexical) * (1 + 0.3 * max(0, cosine_sim))
Nodes with high semantic similarity to the query but low lexical overlap
now receive up to 30% bias boost into working memory promotion. Gracefully
degrades to pure lexical when Ollama is unavailable or node has no embedding.
- Add ML-KEM-1024 + AES-256-GCM binary persistence to el_runtime.c with
two-key scheme (Neuron master + user key); SHAKE-256 key derivation
- Add nomic-embed-text 768-dim float32 embeddings on every node write
via Ollama; graceful fallback when Ollama is not running
- Wire all /api/neuron/* MCP routes directly into Engram (server.el),
eliminating the Kotlin server as the MCP backend
- Set ENGRAM_CHECKPOINT_INTERVAL = 1 (write binary on every node write,
not every 50)
- Add el_runtime.h declarations for engram_write_binary_el and
engram_load_binary_el builtins
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.
engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
Dharma's EngramDB client calls /nodes/list to retrieve all nodes.
Add this as an alias for the existing /nodes (and /api/nodes) route
so downstream clients don't need to be updated when the API drifts.
Also update dist/engram.c to match server.el.
When the query string includes node_type, we route to the new
engram_scan_nodes_by_type_json builtin instead of the unfiltered
scan. Existing callers without the param get identical behaviour.
Smoke-tested live on the neuron engram (3,200+ nodes):
?node_type=Knowledge → all Knowledge
?node_type=BacklogItem → all BacklogItem
?node_type=Imprint → 1 Imprint (only one cultivated so far)
?node_type=DoesNotExist → []
Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.
src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
GET /health
GET /api/stats
POST /api/nodes (auth required)
GET /api/nodes
GET /api/nodes/:id
DELETE /api/nodes/:id (auth required)
POST /api/edges (auth required)
GET /api/neighbors/:id
POST /api/activate
GET /api/activate
POST /api/search
GET /api/search
POST /api/strengthen (auth required)
POST /api/save (auth required)
POST /api/load (auth required)
Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.
Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.
Build: dist/platform/elc src/server.el > dist/engram.c
cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
dist/engram.c <runtime>/el_runtime.c
Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.
Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
Memory is not stored and retrieved — it is activated and propagated.
Implements the spreading activation model with salience decay, typed edges,
four memory tiers, and flat cosine vector search over a sled embedded store.
will.anderson
Will Anderson