neuron/docs/architecture/02-components.md

Neuron — Component Detail

Per-subsystem detail: routing/dispatch, the cognitive API, the memory & activation engine, and the MCP transport chain. For the why behind these boundaries read 01-vbd-decomposition.md first; this doc is the what and how, grounded in file citations.

---

1. Routing / dispatch — routes.el

Responsibility: turn an inbound HTTP request into a handler call. One function does it.

• Entry point: handle_request(method, path, body) -> String (routes.el:358-753). Structure: branch by method (GET :384, POST :549, DELETE :726, PATCH :739), then an ordered sequence of exact (str_eq) and prefix (str_starts_with) tests against the cleaned path. First match wins. There is no route table and no register-route — this is a deliberate hand-written dispatcher.
• Path params are extracted manually with str_slice/str_index_of (session id :539-541, typed-node type :508-513).
• Query strings stripped up front by strip_query (:77-83); the raw path (with query) is still passed to handlers that read params.
• Pre-dispatch middleware (cross-cutting, inline): an activity timestamp (state_set("soul.last_activity_ts", …) :367) and rate limiting (rate_limit_check(ip, path) :38-75, :372-378) — a per-IP 60 req/min sliding window, /health exempt, loopback skipped, returns a 429 body.
• Auth: none in the dispatch path. See doc 01, Divergence 3.
• Fallbacks: err_404 / err_405.

Collaborators: delegates to neuron-api.el (/api/neuron/*), sessions.el (/api/sessions/*), chat.el (/api/chat, /dharma/recv), the Axon Accessor (axon_get/post for /api/backlog|artifacts|projects|memories|knowledge), connectd_* (/api/connectors*), studio.el (/), and engram builtins for the raw /api/graph* reads.

Route surface (grouped; full table with line numbers is in the survey notes):

Group	Representative routes	Handler home
Session/context	/api/neuron/session/begin, /api/neuron/ctx, /api/sessions*	neuron-api, sessions.el
Memory	/api/neuron/memory, /recall, /memory/{evolve,forget,delete,update}, /node/{create,update,delete}	neuron-api
Knowledge	/api/neuron/knowledge/{search,capture,evolve,promote}, /knowledge	neuron-api
Graph/activation	/api/neuron/graph, /graph/link, /api/graph*, /list/:type	neuron-api + engram builtins
Cultivation/self	/api/neuron/cultivate, /lineage, /imprint/*, /synthesize	neuron-api, routes.el
Processes/config	/api/neuron/processes{,/define}, /config{,/tune}	neuron-api
State/consolidate	/api/neuron/state-events, /consolidate	neuron-api
Backlog/artifacts	/api/backlog, /artifacts, /projects, /memories	Axon (HTTP)
Chat/NLG	/api/chat, /see, /elp/chat, /dharma*, /nlg*	chat.el, elp-input.el
Health/UI	/health, /lineage, /	routes.el, studio.el

---

2. The cognitive API — neuron-api.el

Responsibility: the /api/neuron/* handlers — the operations that read and write the engram as cognition (session, memory, knowledge, graph, config, processes, state, cultivation). The file header notes these were migrated out of the MCP wrapper's HTTP calls into in-process engram builtins (neuron-api.el:3-9) — so most handlers call the store directly, no HTTP round-trip.

Primary collaborators are the engram builtins (engram_node_full, engram_search_json, engram_activate_json, engram_scan_nodes_json, engram_scan_nodes_by_type_json, engram_neighbors_json, engram_connect, engram_get_node_json, engram_stats_json, engram_save) and memory.el for tombstoning.

Handler groups:

• Session / context — handle_api_begin_session (:273-301), handle_api_compile_ctx (:305-317). Pull engram_stats_json, run spreading activation (engram_activate_json, depth-1 for begin, depth-2 for ctx), scan recent InternalStateEvents, then project the result through the compaction helpers so the payload can't overflow the MCP client's context. This is Engine work (axis 1) inside a Manager-shaped entry point.

• Memory — handle_api_remember (:322-348): maps importance → salience, injects a project:<name> tag, writes a Memory/Episodic node, then read-back-verifies persistence (api_persisted). Deletes are tombstone, never hard delete — node_delete / memory_delete / forget all route through tombstone_node → mem_tombstone. Updates/evolves are immutable supersede — node_update (:397-429), evolve_memory (:711-735) create a new node and wire engram_connect(new, old, "supersedes").

• Knowledge — search_knowledge (:458-478, falls back to engram_activate_json(q,2) when lexical search returns nothing), browse_knowledge, capture_knowledge (:492-504), evolve_knowledge, promote_knowledge (:526-542, writes a canonical-tier node + supersede edge). Evolve/promote respect is_protected_node.

• Graph — handle_api_inspect_graph (:778-813): resolves a named anchor (self/neuron → kn-efeb4a5b…, values → kn-5b606390…) or an explicit id, then engram_neighbors_json(resolved, depth, "both"). By default this is a plain neighbor traversal (byte-identical to the old behavior, so the studio app is unaffected). When called with compact=1 (or true) it returns a relevance-ranked projection (:804-810): the neighborhood is ranked and the top k neighbors (default 12) keep a UTF-8-safe content snippet (default snip=600) via api_neigh_full, while the remainder collapse to lightweight {id,label,node_type,tier,edge,pointer:true} stubs via api_neigh_pointer. This bounds a high-fanout identity anchor (voice, writing-imprint, self-root) from ~670 KB to ~25 KB so the MCP transport no longer socket-closes on self-load. The MCP wrapper appends &compact=1 on its inspectGraph/fetch-by-id path; the studio app omits the flag and is unchanged. handle_api_link_entities (:818-…) creates edges but blocks edges into protected nodes.

• Cultivation — handle_api_cultivate (:781-839): dispatches on operation (evolve_knowledge / evolve_memory / forget / link_entities) and performs the same engram ops but skips is_protected_node — the sanctioned identity-write path, gated by convention to Will's explicit cultivation sessions.

• Config / processes / state-events / consolidate — config anchors + a ConfigEntry node search (:616-639), tune_config (:642-653), browse_processes / define_process (:547-568), state-event log/list (:575-610), and consolidate (:855-880, an engram_save snapshot plus an optional SessionSummary node).

The projection/compaction layer (a real, recurring concern) lives in api_compact_node (:132-148), api_compact_node_array (:152-165), api_compact_activated (:170-189), and api_utf8_trunc (:116-127). These cap array length and truncate each node to identity + a bounded UTF-8-safe content snippet. Their consumers are begin_session and compile_ctx. The design principle is the important part: the API returns a relevance-bounded projection of the graph, not the graph. That bounding started as length-capping + activation-ordering; it now also includes a relevance-ranked neighbor projection — api_compact_neighbors (:288-317), backed by api_neigh_better/api_neigh_rank (relevance ordering), api_neigh_full (top-K, snippet), api_neigh_pointer (the rest, stub), and api_float_or. This is committed fact, not an in-flight concern: it is the compact=1 path of handle_api_inspect_graph above, and it is what makes self-load survive the MCP transport. It is compiled into dist/soul.c (this PR regenerated the amalgamation so CI ships it — see doc 05).

---

3. Memory & activation engine

This subsystem spans three files in this repo (memory.el, awareness.el, soul.el) and one in foundation (el_runtime.c). The split matters: the math is in C; the El files orchestrate, persist, and instrument it.

3a. Memory access — memory.el (the Accessor)

The single isolation point over the engram FFI. Key functions:

Fn	Lines	Backing call	Notes
mem_store	5-28	engram_node_full + read-back	verified write
mem_remember	30-32	mem_store	label soul-memory
mem_recall	34-36	engram_activate_json(query, depth)	spreading-activation recall (mutates WM)
mem_search	38-40	engram_search_json	pure lexical scan (no WM side-effect)
mem_strengthen	42-44	engram_strengthen	salience bump
mem_tombstone	52-62	engram_node_full + engram_connect	the one canonical soft-delete
mem_forget	70-72	mem_tombstone	soft delete (no longer hard)
mem_consolidate	92-133	engram_wm_top_json, engram_strengthen	salience-evolution pass
mem_save / mem_load	135-148	engram_save/load	snapshot I/O

Note the distinction between recall and search: mem_recall fires spreading activation (and warms working memory as a side effect); mem_search is a passive lexical lookup. Tiers here are tier_working / tier_episodic / tier_canonical (memory.el:1-3) — see doc 03 for how these relate to the engine's tier field and to the MCP surface vocabulary.

3b. Autonomous cognition — awareness.el (the daemon)

awareness.el is the idle-cognition daemon plus observability, not emotional-state code. awareness_run() (:1097-1284) is the master loop, launched last from soul.el:627. Each tick (SOUL_TICK_MS, ~200ms):

1. one_cycle() (:1041-1095) — the cognitive step: perceive() (:900-924, gated on a soul-inbox-pending tag, then engram_activate_json) → attend() (:926-973, parse trigger content into an action verb: remember / search / activate / strengthen / forget / consolidate / respond) → respond() (:975-1029, dispatch to the mem_* fns) → record() (:1031-1039, emit an InternalStateEvent) → consume the trigger.
2. Heartbeat (every 60s): hebb_consolidate() then emit_heartbeat() then mem_save snapshot (:1189-1197).
3. Curiosity scan (every 30s when idle): proactive_curiosity() (:701-876) rotates 4 seed-domain sets, activates a seed, strengthens the top result only if it changed (novelty-gated), and derives an autobiographical seed from the top-10 working-memory nodes with stopword/IDF/tabu filtering.
4. Engram sync (every 10 min): GET /api/sync → engram_load_merge → telemetry prune.

Two functions carry most of the file's weight and volatility:

• hebb_consolidate() (:64-99) — the durable-learning write-back. It drains newly-formed co-activation edges (engram_hebb_drain_json(64)) and POSTs them as one batch to /api/edges/batch (:94). The comment block (:33-63) records that before this path existed the soul threw away ~1,198 learned edges per restart — the daemon is where essentially all co-activation happens, and this is how it survives.
• emit_heartbeat() (:201-549, ~350 lines) — assembles ~50 gauges (WM saturation/churn, Hebbian candidate/edge counts, embedding coverage, corpus health) into one ISE. Pure observability; a fat, churny Accessor/Utility mix.

A threat scorer (:1286-1419) is grafted onto the end — command/path/history additive scoring, ≥70 blocks a tool call. Cross-cutting agentic-safety policy, unrelated to memory mechanism.

3c. Identity & the request pipeline — soul.el

soul.el is the top-level program (cgi "neuron-soul", :12-17) and imports every other module (:1-10). It owns:

• The identity graph. init_soul_edges() (:19-92) hard-wires a self_root node linked by identity edges (weight 0.95) to family/origin/value nodes, plus a dense co-value mesh (weight 0.7) among 8 value nodes. ensure_self_canonical_bridge() (:101-110) links the public traversal-root anchor (kn-efeb4a5b) to the curated self node via canonical-self edges. load_identity_context() (:153-240) loads intellectual-DNA / values / memory-philosophy content into a state key for prompt injection.
• Boot orchestration (:508-627): load snapshot → optional first-boot seed (guarded) → identity context → persona-from-env → boot-count increment → session-start event → genesis-only edge init → http_serve_async(port, "handle_request") → awareness_run().
• The request pipeline. layered_cycle() (:382-506) — a 4-layer stack for user input: L1 safety screen (safety_screen) → L2a continuity/ behavioral (steward_session_check) → L2b mission alignment (steward_align) → L2c affective-context injection → L3 imprint_respond → L1 output validation (safety_validate). Hard-bell inputs bypass the upper layers. The JSON contract between these layers is pinned by tests/test_layer_contract.el.

3d. Where the activation math actually is

el_runtime.c implements the two-layer activation model (background_activation via BFS fan-out, then working_memory_weight via an executive filter), ACT-R base-level learning (per-node access-timestamp ring buffer), 768-dim semantic embeddings, and Hebbian eligibility traces. Retrieval is spreading activation, not query: strength = parent_strength × edge_weight × target_salience × cosine(query, target). The El files never compute this — they seed it (engram_activate_json), harvest it (engram_hebb_drain_json), and persist it. See 03-data-and-memory.md.

---

4. The MCP transport chain — mcp-proxy, mcp-wrapper

The chain exists because two boundaries vary independently: the client transport (stdio MCP JSON-RPC) and the soul's protocol (HTTP REST). Each hop absorbs one.

• mcp-proxy/src/main.el (listens :7779) — a byte-forwarder. It accepts the client connection, forwards to the wrapper, and adds resilience: retry, health-gating, and a well-formed error envelope so a downstream hiccup never surfaces to the client as a broken pipe. It holds no MCP semantics — pure transport orchestration.

• mcp-wrapper/src/main.el (listens :17779) — the protocol translator. It speaks MCP JSON-RPC to the client and REST to the soul (:7770), owns the MCP lifecycle (initialize, tools/list, tools/call), and carries the tool catalog (~90 tools) that clients enumerate. dispatch_tool_call maps each tool to a soul REST endpoint. It also fires a spread-activation side effect (fire_activation) — after relevant calls it issues a /recall to warm related nodes, so tool use itself nudges working memory. The tool schemas in the catalog are largely name-only stubs — flag as a place where richer schemas could live.

• Manifests (mcp-proxy/manifest.el, mcp-wrapper/manifest.el) declare the build entry and package metadata for each transport binary.

End-to-end (one tools/call): client → proxy (:7779, forward+retry) → wrapper (:17779, JSON-RPC→REST, catalog dispatch) → soul (:7770, handle_request → handle_api_*) → engram builtins → (HTTP :8742 when in HTTP mode). The response walks back up, and the wrapper may fire a /recall warm-up on the way. The full sequence is drawn in 04-runtime-and-deployment.md.

VBD reading: proxy and wrapper are Managers of transport; the wrapper is also the Accessor that isolates the MCP protocol boundary from the soul (the soul knows only HTTP). The multi-hop shape is justified: the client transport, the protocol translation, and the cognition each change for different reasons and are deployed/updated independently.