Three targeted improvements based on graph health analysis (29K nodes, 104 edges):
1. ENGRAM_WM_THRESHOLD 0.15 → 0.08: sparse graph means BFS paths carry weak
signals (0.05-0.12 range). Prior threshold gatekept too aggressively. Grounded
in TBRS* cognitive model (θ=0.05); 0.08 is conservative but effective.
2. ENGRAM_INHIBITION_FACTOR 0.1 → 0.2: factor=0.1 (90% suppression) on a sparse
graph almost always fully silences targeted nodes. Factor=0.2 (80% suppression)
maintains strong inhibition while allowing partially-suppressed nodes to remain
faintly active — consistent with partial inhibition in cognitive neuroscience.
3. engram_wm_avg_weight() builtin: computes mean working_memory_weight of all
promoted nodes. Returns float bits via el_from_float for EL float_to_str usage.
Makes activation quality directly observable in heartbeat ISEs, distinguishing
'many weak activations' (sparse, low avg) from 'few strong' (dense, high avg).
Rebuilt engram binary with new runtime.
Two changes:
1. el_runtime.c — engram_activation_dampen(): add floor of 0.35.
ISE nodes with ac=900+ had dampen=0.128, giving effective salience=0.038
which fell below the epist>=0.1 gate in engram_activate. This silently
killed curiosity seeds "self identity values" and "decision pattern lesson"
— the only corpus matches were high-ac ISEs that were then excluded from
results, causing activated=0 on 50% of proactive_curiosity scans.
Floor at 0.35 keeps salience=0.3 nodes at effective_bg=0.105, above the
visibility threshold, without disrupting relative ordering of content nodes.
2. server.el — route_create_node: replace stale inline auto-link with
auto_link_content_node(). The inline logic used the old engram_search_json
(substring, no ISE filter) while the better BM25-based auto_link_content_node
was added in 2026-05-28 and wired to /api/neuron/* routes but not to the
raw /api/nodes POST path. Removes ~40 lines of duplicated logic.
Three changes:
1. Fix checkpoint ISE temporal_decay_rate: engram_emit_ise_internal was
hardcoded to 0.0 (global 168h default) instead of 2.310 (Working-tier
48h). Result: checkpoint ISEs accumulated at 3.5x intended rate.
2. Raise CHECKPOINT_INTERVAL 1→10: checkpoint ISE fires on every single
node write, producing 2:1 checkpoint:content ratio in ISE stream.
MCP routes still call engram_write_binary_el explicitly after each
important write, so no knowledge durability is lost.
3. Add auto_link_content_node to server.el: route_neuron_memory and
route_neuron_knowledge_capture were creating nodes with zero edges —
invisible to BFS traversal, only reachable via lexical/semantic seed.
New helper runs BM25 over top-20 results, skips ISE nodes (which
dominate the 14K-node corpus), connects up to 3 related nodes.
Add engram_auto_link_semantic(): when a node is embedded, scan all
embedded nodes for cosine sim >= 0.72 and create bidirectional
"semantic-similar" edges to the top-3 matches. Runs once at write
time rather than at every query, converting the expensive O(N) scan
from live activation into durable graph structure.
Fixes the core connectivity problem: 63 edges / 5364 nodes (0.012
edges/node). Verified: new Knowledge nodes now auto-link at sim 0.77–
0.95 with correct threshold discrimination. One checkpoint per insert
(not one per edge) bounds the persistence overhead.
Excludes InternalStateEvents and inbox/outbox transient nodes to keep
semantic graph clean.
Two improvements:
1. BM25 search corpus coverage (server.el) — raised scan cap from 500 to
5000 nodes. On the 161K-node graph, 500 was 0.3% coverage; 5000 is 3%.
engram_scan_nodes_json returns nodes sorted by salience DESC, so ISEs
(salience 0.3) fall below Knowledge/Memory (0.5–0.8) naturally — the
effective corpus stays content-dense. Also updated stale comment on the
ISE route (no longer need high offset; recent-first ordering from May 23).
2. Traversal inference guard (el_runtime.c) — two changes:
- INFER_CAP reduced 256→32: proactive curiosity runs engram_activate
every ~30s. At 256 edges/call the soul daemon accumulated 107K edges
in 23h (5× BFS slowdown). At 32 the rate drops 8×.
- Edge count guard: skip inference entirely when snap_ec ≥ 40,000.
At that density most A→C paths are already explicit; marginal inference
value is low and the O(edge_count²) inner-loop cost is high. Self-limits
unbounded accumulation across restarts.
Two improvements:
1. ISE scan ordering — engram_scan_nodes_by_type_json now sorts InternalStateEvent
nodes by created_at DESC (most-recent-first) instead of salience DESC. Old
high-salience ISEs (session-start, wm-promotion) no longer dominate offset 0,
burying recent heartbeat and curiosity_scan events at offsets 20000+. New
behavior: ?limit=10 returns the 10 most recent ISEs regardless of salience.
All other node types retain existing salience-sorted behavior.
2. http_serve_async registered as elc builtin — added to builtin_arity() in both
codegen.el (EL compiler source) and dist/platform/elc.c (compiled C). Also
rebuilt elc binary from updated elc.c. This closes the fragile-patch gap from
2026-05-21: elc previously treated http_serve_async as an unknown identifier,
and the gap description noted elc would 'silently revert to blocking http_serve'
on next soul rebuild. Now http_serve_async has a proper 2-arg arity entry and
will survive all future soul recompiles without a manual neuron.c patch.
Semantic seed floor raised 0.65 → 0.70: field literature (SuperLocalMemory
arXiv:2604.04514) validates 0.70-0.75 as the noise floor for sentence-transformer
embeddings on non-trivial corpora. The 0.65 threshold was admitting false
positives that diluted BFS traversal quality on the 13K-node post-ingestor graph.
Top-30 cap retained — sparse graph (1.26 edges/node) needs more semantic entry
points than a dense graph would.
ACT-R Base-Level Learning insight applied to engram_temporal_decay: current
purely time-based formula treated a node activated 100 times identically to a
node never activated (same decay rate). ACT-R BLL (Anderson 1993) shows
frequently-accessed memories resist temporal decay. Fix: freq_resist multiplier
= 1.0 + 0.1 * log(1 + activation_count). count=0→1.0×, count=9→1.23×,
count=99→1.46×. Existing activation_dampening continues to reduce per-query
novelty for well-known nodes — the two mechanisms are complementary: resist
decay (durable in graph), dampen per-query (don't dominate any single turn).
Three improvements from daily review:
1. Add semantic seed supplement to Pass 1 activation (el_runtime.c).
Previously, engram_cosine_sim was only called in goal_bias (Pass 2) for
nodes that already matched lexically. Nodes semantically close but
lexically disjoint were completely invisible to activation. With 8K+
world-ingestor nodes added overnight, this was a critical gap. Now: after
lexical seeding, scan un-seeded nodes for cosine sim ≥ 0.65 and inject
top-30 as additional seeds. Sem seeds get 85% of full act to stay weaker
than exact lexical matches.
2. Lower WM promotion ISE threshold from >0.5 to >0.1 (el_runtime.c).
Only one wm-promotion ISE was ever logged — the 0.5 floor was too high.
Most practical Knowledge/Memory promotions are in the 0.1-0.5 range.
Lowering to 0.1 makes working memory activity visible in state events.
Three research-grounded improvements:
1. Tier-based temporal decay in el_runtime.c (engram_node_full, engram_node_layered):
Working=48h, Episodic=72h, Semantic=336h, Procedural=720h half-lives.
Grounded in ACT-R literature — differentiated decay by chunk type. The
temporal_decay_rate field existed but was always 0 (global 168h for everything).
New nodes now carry the correct half-life for their tier from creation.
2. Implement route_neuron_knowledge_promote in server.el (was a silent stub):
Reads existing node, creates promoted-tier copy with supersedes edge,
checkpoints. promote_knowledge MCP tool now has real effect.
3. ISE label extraction + offset support in route_neuron_state_events:
POST now extracts 'event' field from content JSON as label (heartbeat,
wm_promotion, etc.) instead of always writing 'state-event'. GET now
accepts ?offset= for pagination to reach recent ISEs.
BM25+ (k1=1.2, b=0.75, delta=1.0) now powers all search routes in EL.
No external dependencies in the activation/search path.
- bm25_tokenize/bm25_count_term/bm25_score_doc/bm25_search_json in server.el
- route_search, route_neuron_recall: engram_search_json -> bm25_search_json
- route_activate: BM25 pre-bias (strengthen top-10) before spreading activation
- Remove standalone /api/bm25/search endpoint (BM25 is the engine, not a feature)
- Fix zero-score filter: float comparison not string match
- Add + to tokenizer for URL-encoded query params
- Scan floor 200 nodes regardless of limit size
- Revert Ollama engram_embed_query from 9af2482 (no Ollama at query time)
- Add list_set and math_exp builtins to el_runtime.c
- Add list_set, math_exp, and float_add/sub/mul/div/gt/lt/eq/gte/lte builtins to
el_runtime.c + el_runtime.h (float arithmetic builtins needed because EL operators
+*/ operate on raw el_val_t bits, not IEEE 754 doubles)
- Remove engram_embed_query() and its forward declaration from el_runtime.c
- Remove Ollama cosine-similarity blend from activation scoring (reverts 9af2482):
drops query_emb/query_edim variables, bias *= (1 + 0.3 * sim) block, and all
free(query_emb) calls from the activation loop
- Implement BM25+ scoring in server.el (k1=1.2, b=0.75, delta=1.0):
bm25_tokenize, bm25_count_term, bm25_score_doc, bm25_search_json
V1 uses n_t=1 approximation (constant IDF per corpus size); acceptable as a
first pass without an inverted index
- Wire /api/bm25/search POST/GET route in server.el dispatcher
- Zero Ollama calls in the activation/search path; embeddings on nodes are
untouched (still written at node-creation time)
engram_cosine_sim() was defined and embeddings were computed per-node
via nomic-embed-text on write, but the function was never called during
activation scoring. The goal_bias computation used only lexical substring
matching, ignoring all stored embedding vectors.
This change adds engram_embed_query() to embed the query string at search
time (5s timeout so Ollama latency never blocks activation), then blends
cosine similarity into the working-memory bias with α=0.3:
bias_final = goal_bias(lexical) * (1 + 0.3 * max(0, cosine_sim))
Nodes with high semantic similarity to the query but low lexical overlap
now receive up to 30% bias boost into working memory promotion. Gracefully
degrades to pure lexical when Ollama is unavailable or node has no embedding.
- Add ML-KEM-1024 + AES-256-GCM binary persistence to el_runtime.c with
two-key scheme (Neuron master + user key); SHAKE-256 key derivation
- Add nomic-embed-text 768-dim float32 embeddings on every node write
via Ollama; graceful fallback when Ollama is not running
- Wire all /api/neuron/* MCP routes directly into Engram (server.el),
eliminating the Kotlin server as the MCP backend
- Set ENGRAM_CHECKPOINT_INTERVAL = 1 (write binary on every node write,
not every 50)
- Add el_runtime.h declarations for engram_write_binary_el and
engram_load_binary_el builtins
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.
engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
Dharma's EngramDB client calls /nodes/list to retrieve all nodes.
Add this as an alias for the existing /nodes (and /api/nodes) route
so downstream clients don't need to be updated when the API drifts.
Also update dist/engram.c to match server.el.
When the query string includes node_type, we route to the new
engram_scan_nodes_by_type_json builtin instead of the unfiltered
scan. Existing callers without the param get identical behaviour.
Smoke-tested live on the neuron engram (3,200+ nodes):
?node_type=Knowledge → all Knowledge
?node_type=BacklogItem → all BacklogItem
?node_type=Imprint → 1 Imprint (only one cultivated so far)
?node_type=DoesNotExist → []
Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.
src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
GET /health
GET /api/stats
POST /api/nodes (auth required)
GET /api/nodes
GET /api/nodes/:id
DELETE /api/nodes/:id (auth required)
POST /api/edges (auth required)
GET /api/neighbors/:id
POST /api/activate
GET /api/activate
POST /api/search
GET /api/search
POST /api/strengthen (auth required)
POST /api/save (auth required)
POST /api/load (auth required)
Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.
Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.
Build: dist/platform/elc src/server.el > dist/engram.c
cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
dist/engram.c <runtime>/el_runtime.c
Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.
Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
Memory is not stored and retrieved — it is activated and propagated.
Implements the spreading activation model with salience decay, typed edges,
four memory tiers, and flat cosine vector search over a sled embedded store.
will.anderson
Will Anderson