neuron/docs/architecture/02-components.md

# Neuron — Component Detail

> Per-subsystem detail: routing/dispatch, the cognitive API, the memory &
> activation engine, and the MCP transport chain. For the *why* behind these
> boundaries read `01-vbd-decomposition.md` first; this doc is the *what* and
> *how*, grounded in file citations.

---

## 1. Routing / dispatch — `routes.el`

**Responsibility:** turn an inbound HTTP request into a handler call. One
function does it.

- **Entry point:** `handle_request(method, path, body) -> String`
  (`routes.el:358-753`). Structure: branch by method (`GET` `:384`, `POST`
  `:549`, `DELETE` `:726`, `PATCH` `:739`), then an ordered sequence of exact
  (`str_eq`) and prefix (`str_starts_with`) tests against the cleaned path.
  First match wins. There is **no route table and no `register-route`** — this is
  a deliberate hand-written dispatcher.
- **Path params** are extracted manually with `str_slice`/`str_index_of`
  (session id `:539-541`, typed-node type `:508-513`).
- **Query strings** stripped up front by `strip_query` (`:77-83`); the raw path
  (with query) is still passed to handlers that read params.
- **Pre-dispatch middleware** (cross-cutting, inline): an activity timestamp
  (`state_set("soul.last_activity_ts", …)` `:367`) and **rate limiting**
  (`rate_limit_check(ip, path)` `:38-75`, `:372-378`) — a per-IP 60 req/min
  sliding window, `/health` exempt, loopback skipped, returns a 429 body.
- **Auth:** none in the dispatch path. See doc 01, Divergence 3.
- **Fallbacks:** `err_404` / `err_405`.

**Collaborators:** delegates to `neuron-api.el` (`/api/neuron/*`), `sessions.el`
(`/api/sessions/*`), `chat.el` (`/api/chat`, `/dharma/recv`), the Axon Accessor
(`axon_get/post` for `/api/backlog|artifacts|projects|memories|knowledge`),
`connectd_*` (`/api/connectors*`), `studio.el` (`/`), and engram builtins for the
raw `/api/graph*` reads.

**Route surface** (grouped; full table with line numbers is in the survey notes):

| Group | Representative routes | Handler home |
|---|---|---|
| Session/context | `/api/neuron/session/begin`, `/api/neuron/ctx`, `/api/sessions*` | neuron-api, sessions.el |
| Memory | `/api/neuron/memory`, `/recall`, `/memory/{evolve,forget,delete,update}`, `/node/{create,update,delete}` | neuron-api |
| Knowledge | `/api/neuron/knowledge/{search,capture,evolve,promote}`, `/knowledge` | neuron-api |
| Graph/activation | `/api/neuron/graph`, `/graph/link`, `/api/graph*`, `/list/:type` | neuron-api + engram builtins |
| Cultivation/self | `/api/neuron/cultivate`, `/lineage`, `/imprint/*`, `/synthesize` | neuron-api, routes.el |
| Processes/config | `/api/neuron/processes{,/define}`, `/config{,/tune}` | neuron-api |
| State/consolidate | `/api/neuron/state-events`, `/consolidate` | neuron-api — *one of eleven consolidation implementations; see `06` §12.4* |
| Backlog/artifacts | `/api/backlog`, `/artifacts`, `/projects`, `/memories` | Axon (HTTP) |
| Chat/NLG | `/api/chat`, `/see`, `/elp/chat`, `/dharma*`, `/nlg*` | chat.el, elp-input.el |
| Health/UI | `/health`, `/lineage`, `/` | routes.el, studio.el |

---

## 2. The cognitive API — `neuron-api.el`

**Responsibility:** the `/api/neuron/*` handlers — the operations that read and
write the engram as *cognition* (session, memory, knowledge, graph, config,
processes, state, cultivation). The file header notes these were migrated **out
of the MCP wrapper's HTTP calls into in-process engram builtins**
(`neuron-api.el:3-9`) — so most handlers call the store directly, no HTTP
round-trip.

**Primary collaborators** are the engram builtins (`engram_node_full`,
`engram_search_json`, `engram_activate_json`, `engram_scan_nodes_json`,
`engram_scan_nodes_by_type_json`, `engram_neighbors_json`, `engram_connect`,
`engram_get_node_json`, `engram_stats_json`, `engram_save`) and `memory.el` for
tombstoning.

**Handler groups:**

- **Session / context** — `handle_api_begin_session` (`:273-301`),
  `handle_api_compile_ctx` (`:305-317`). Pull `engram_stats_json`, run
  spreading activation (`engram_activate_json`, depth-1 for begin, depth-2 for
  ctx), scan recent `InternalStateEvent`s, then **project the result through the
  compaction helpers** so the payload can't overflow the MCP client's context.
  This is Engine work (axis 1) inside a Manager-shaped entry point.

- **Memory** — `handle_api_remember` (`:322-348`): maps `importance` → salience,
  injects a `project:<name>` tag, writes a `Memory`/`Episodic` node, then
  **read-back-verifies** persistence (`api_persisted`). Deletes are **tombstone,
  never hard delete** — `node_delete` / `memory_delete` / `forget` all route
  through `tombstone_node` → `mem_tombstone`. Updates/evolves are **immutable
  supersede** — `node_update` (`:397-429`), `evolve_memory` (`:711-735`) create a
  new node and wire `engram_connect(new, old, "supersedes")`.

- **Knowledge** — `search_knowledge` (`:458-478`, falls back to
  `engram_activate_json(q,2)` when lexical search returns nothing),
  `browse_knowledge`, `capture_knowledge` (`:492-504`), `evolve_knowledge`,
  `promote_knowledge` (`:526-542`, writes a canonical-tier node + supersede
  edge). Evolve/promote respect `is_protected_node`.

- **Graph** — `handle_api_inspect_graph` (`:778-813`): resolves a named anchor
  (`self`/`neuron` → `kn-efeb4a5b…`, `values` → `kn-5b606390…`) or an explicit
  id, then `engram_neighbors_json(resolved, depth, "both")`. By default this is a
  plain neighbor traversal (byte-identical to the old behavior, so the studio app
  is unaffected). **When called with `compact=1` (or `true`) it returns a
  relevance-ranked projection** (`:804-810`): the neighborhood is ranked and the
  top **`k`** neighbors (default 12) keep a UTF-8-safe content snippet (default
  `snip=600`) via `api_neigh_full`, while the remainder collapse to lightweight
  `{id,label,node_type,tier,edge,pointer:true}` stubs via `api_neigh_pointer`.
  This bounds a high-fanout identity anchor (voice, writing-imprint, self-root)
  from ~670 KB to ~25 KB so the MCP transport no longer socket-closes on
  self-load. The MCP wrapper appends `&compact=1` on its inspectGraph/fetch-by-id
  path; the studio app omits the flag and is unchanged.
  `handle_api_link_entities` (`:818-…`) creates edges but blocks edges *into*
  protected nodes.

- **Cultivation** — `handle_api_cultivate` (`:781-839`): dispatches on
  `operation` (evolve_knowledge / evolve_memory / forget / link_entities) and
  performs the same engram ops **but skips `is_protected_node`** — the sanctioned
  identity-write path, gated by convention to Will's explicit cultivation
  sessions.

> **Consolidation has no owner (2026-08-16) — see `06` §12.4.** `/consolidate`
> below and `mem_consolidate` in the table further down are two of **eleven**
> measured consolidation implementations, spread across three languages and two
> processes. Consolidation had no owner, so it was implemented at every site that
> needed a piece of it. Every name in the set is a consolidation verb — compress,
> cultivate, digest, integrate, review, reify, beat.

- **Config / processes / state-events / consolidate** — config anchors + a
  `ConfigEntry` node search (`:616-639`), `tune_config` (`:642-653`),
  `browse_processes` / `define_process` (`:547-568`), state-event log/list
  (`:575-610`), and `consolidate` (`:855-880`, an `engram_save` snapshot plus an
  optional `SessionSummary` node).

**The projection/compaction layer** (a real, recurring concern) lives in
`api_compact_node` (`:132-148`), `api_compact_node_array` (`:152-165`),
`api_compact_activated` (`:170-189`), and `api_utf8_trunc` (`:116-127`). These
**cap array length and truncate each node to identity + a bounded UTF-8-safe
content snippet.** Their consumers are `begin_session` and `compile_ctx`.
The design principle is the important part: *the API returns a relevance-bounded
projection of the graph, not the graph.* That bounding started as
length-capping + activation-ordering; it now also includes a **relevance-ranked
neighbor projection** — `api_compact_neighbors` (`:288-317`), backed by
`api_neigh_better`/`api_neigh_rank` (relevance ordering), `api_neigh_full`
(top-K, snippet), `api_neigh_pointer` (the rest, stub), and `api_float_or`. This
is **committed fact, not an in-flight concern**: it is the `compact=1` path of
`handle_api_inspect_graph` above, and it is what makes self-load survive the MCP
transport. It is compiled into `dist/soul.c` (this PR regenerated the
amalgamation so CI ships it — see doc 05).

---

## 3. Memory & activation engine

This subsystem spans three files in this repo (`memory.el`, `awareness.el`,
`soul.el`) and one in `foundation` (`el_runtime.c`). The split matters: **the
math is in C; the El files orchestrate, persist, and instrument it.**

### 3a. Memory access — `memory.el` (the Accessor)

The single isolation point over the engram FFI. Key functions:

| Fn | Lines | Backing call | Notes |
|---|---|---|---|
| `mem_store` | `5-28` | `engram_node_full` + read-back | verified write |
| `mem_remember` | `30-32` | `mem_store` | label `soul-memory` |
| `mem_recall` | `34-36` | `engram_activate_json(query, depth)` | **spreading-activation recall** (mutates WM) |
| `mem_search` | `38-40` | `engram_search_json` | pure lexical scan (no WM side-effect) |
| `mem_strengthen` | `42-44` | `engram_strengthen` | salience bump |
| `mem_tombstone` | `52-62` | `engram_node_full` + `engram_connect` | the one canonical soft-delete |
| `mem_forget` | `70-72` | `mem_tombstone` | soft delete (no longer hard) |
| `mem_consolidate` | `92-133` | `engram_wm_top_json`, `engram_strengthen` | salience-evolution pass — *one of eleven consolidation implementations, `06` §12.4* |
| `mem_save` / `mem_load` | `135-148` | `engram_save/load` | snapshot I/O |

Note the distinction between **recall and search**: `mem_recall` fires spreading
activation (and warms working memory as a side effect); `mem_search` is a passive
lexical lookup. Tiers here are `tier_working` / `tier_episodic` / `tier_canonical`
(`memory.el:1-3`) — see doc 03 for how these relate to the engine's tier field
and to the MCP surface vocabulary.

### 3b. Autonomous cognition — `awareness.el` (the daemon)

> **Corrected (2026-08-16) — see `06-cognitive-architecture.md` §12.4.**
> `awareness_run()`'s **continuous, in-process loop is the one fragment of
> consolidation with the correct shape.** It is not a scheduled job; it runs while
> the process serves. Everything below that is described as *"every 60s" /
> "every 30s" / "every 10 min"* is an interval inside that loop, and the design
> spec's verdict is that intrinsic rhythm — not an external clock — is what these
> should be. Consolidation is **ambient, not scheduled: a brain has no cron job**,
> and **the presence of a ticker is the diagnostic.** The genuinely external
> tickers are catalogued in `06` §12.4; this loop is the shape they fold *into*.
>
> **Stale line numbers (verified 2026-08-16):** `awareness_run()` is defined at
> `awareness.el:1221` (its `while true` at `:1252`), not `:1097-1284`; it is
> launched from `soul.el:731`, not `soul.el:627`. `SOUL_TICK_MS` is read at
> `awareness.el:1228` (default **200 ms**) and `SOUL_HEARTBEAT_MS` at `:1248`
> (default **60000 ms**) — those two defaults are correct as documented.

`awareness.el` is the **idle-cognition daemon plus observability**, not
emotional-state code. `awareness_run()` (`:1097-1284`) is the master loop,
launched last from `soul.el:627`. Each tick (`SOUL_TICK_MS`, ~200ms):

1. **`one_cycle()`** (`:1041-1095`) — the cognitive step:
   `perceive()` (`:900-924`, gated on a `soul-inbox-pending` tag, then
   `engram_activate_json`) → `attend()` (`:926-973`, parse trigger content into
   an action verb: remember / search / activate / strengthen / forget /
   consolidate / respond) → `respond()` (`:975-1029`, dispatch to the `mem_*`
   fns) → `record()` (`:1031-1039`, emit an InternalStateEvent) → consume the
   trigger.
2. **Heartbeat** (every 60s): `hebb_consolidate()` **then** `emit_heartbeat()`
   then `mem_save` snapshot (`:1189-1197`).
3. **Curiosity scan** (every 30s when idle): `proactive_curiosity()`
   (`:701-876`) rotates 4 seed-domain sets, activates a seed, strengthens the
   top result **only if it changed** (novelty-gated), and derives an
   autobiographical seed from the top-10 working-memory nodes with
   stopword/IDF/tabu filtering.
   > **Superseded (2026-08-16) — see `06` §12.3.** Three errors in one name.
   > (a) **Curiosity is not a scan.** Nothing in a mind sweeps its neighbourhoods
   > to find what is surprising — the surprise captures attention; salience is
   > bottom-up. A search asks *"which of these is odd"*; a mind has
   > *"something is odd **here**"* for free. A sweep over regions is a supervisor.
   > (b) **It is not on a timer.** "Every 30s when idle" is an external clock
   > standing in for a drive. Low activation is aversive and the system
   > self-activates; there is **one activation process with two seed sources** —
   > external (a request) and internal (a curiosity) — not a scheduled scan
   > competing for spare capacity.
   > (c) **Rotating 4 seed-domain sets is a manifest.** Curiosity is wonder
   > crystallized at a nucleation site, and a nucleation site is a per-edge
   > structural fact (`|discord|`, `06` §12.4), not an entry in a rotation.
4. **Engram sync** (every 10 min): `GET /api/sync` → `engram_load_merge` →
   telemetry prune.
   > **Ticker, but not consolidation (2026-08-16).** Sync is store coherence
   > between the two-store topology (`06` §2.3), not dreaming. Distinguished
   > here because `06` §12.4 sweeps for tickers. **Not** to be confused with the
   > separate `ai.neuron.engram-tick` launch agent (`StartInterval = 600`), which
   > pokes `POST /api/tick` and **is** consolidation driven by an external clock.

Two functions carry most of the file's weight and volatility:
- **`hebb_consolidate()`** (`:64-99`) — the durable-learning write-back. It drains
  newly-formed co-activation edges (`engram_hebb_drain_json(64)`) and POSTs them
  as one batch to `/api/edges/batch` (`:94`). The comment block (`:33-63`)
  records that before this path existed the soul threw away ~1,198 learned
  edges per restart — the daemon is where **essentially all co-activation
  happens**, and this is how it survives.
- **`emit_heartbeat()`** (`:201-549`, ~350 lines) — assembles ~50 gauges (WM
  saturation/churn, Hebbian candidate/edge counts, embedding coverage, corpus
  health) into one ISE. Pure observability; a fat, churny Accessor/Utility mix.

A **threat scorer** (`:1286-1419`) is grafted onto the end — command/path/history
additive scoring, ≥70 blocks a tool call. Cross-cutting agentic-safety policy,
unrelated to memory mechanism.

### 3c. Identity & the request pipeline — `soul.el`

`soul.el` is the top-level program (`cgi "neuron-soul"`, `:12-17`) and imports
every other module (`:1-10`). It owns:

- **The identity graph.** `init_soul_edges()` (`:19-92`) hard-wires a `self_root`
  node linked by `identity` edges (weight 0.95) to family/origin/value nodes,
  plus a dense `co-value` mesh (weight 0.7) among 8 value nodes.
  `ensure_self_canonical_bridge()` (`:101-110`) links the public traversal-root
  anchor (`kn-efeb4a5b`) to the curated self node via `canonical-self` edges.
  `load_identity_context()` (`:153-240`) loads intellectual-DNA / values /
  memory-philosophy content into a state key for prompt injection.
- **Boot orchestration** (`:508-627`): load snapshot → optional first-boot seed
  (guarded) → identity context → persona-from-env → boot-count increment →
  session-start event → genesis-only edge init → `http_serve_async(port,
  "handle_request")` → `awareness_run()`.
- **The request pipeline.** `layered_cycle()` (`:382-506`) — a 4-layer stack for
  user input: **L1** safety screen (`safety_screen`) → **L2a** continuity/
  behavioral (`steward_session_check`) → **L2b** mission alignment
  (`steward_align`) → **L2c** affective-context injection → **L3**
  `imprint_respond` → **L1** output validation (`safety_validate`). Hard-bell
  inputs bypass the upper layers. The JSON contract between these layers is
  pinned by `tests/test_layer_contract.el`.

### 3d. Where the activation math actually is

`el_runtime.c` implements the two-layer activation model
(`background_activation` via BFS fan-out, then `working_memory_weight` via an
executive filter), ACT-R base-level learning (per-node access-timestamp ring
buffer), 768-dim semantic embeddings, and Hebbian eligibility traces. Retrieval
is **spreading activation, not query**:
`strength = parent_strength × edge_weight × target_salience ×
cosine(query, target)`. The El files never compute this — they seed it
(`engram_activate_json`), harvest it (`engram_hebb_drain_json`), and persist it.
See `03-data-and-memory.md`.

---

## 4. The MCP transport chain — `mcp-proxy`, `mcp-wrapper`

The chain exists because two boundaries vary independently: the *client
transport* (stdio MCP JSON-RPC) and the *soul's protocol* (HTTP REST). Each hop
absorbs one.

- **`mcp-proxy/src/main.el`** (listens `:7779`) — a **byte-forwarder**. It
  accepts the client connection, forwards to the wrapper, and adds resilience:
  retry, health-gating, and a well-formed error envelope so a downstream hiccup
  never surfaces to the client as a broken pipe. It holds no MCP semantics —
  pure transport orchestration.

- **`mcp-wrapper/src/main.el`** (listens `:17779`) — the **protocol translator**.
  It speaks MCP JSON-RPC to the client and REST to the soul (`:7770`), owns the
  MCP lifecycle (`initialize`, `tools/list`, `tools/call`), and carries the
  **tool catalog** (~90 tools) that clients enumerate. `dispatch_tool_call` maps
  each tool to a soul REST endpoint. It also fires a **spread-activation side
  effect** (`fire_activation`) — after relevant calls it issues a `/recall` to
  warm related nodes, so tool use itself nudges working memory. The tool schemas
  in the catalog are largely name-only stubs — flag as a place where richer
  schemas could live.

- **Manifests** (`mcp-proxy/manifest.el`, `mcp-wrapper/manifest.el`) declare the
  build entry and package metadata for each transport binary.

**End-to-end (one `tools/call`):** client → proxy (`:7779`, forward+retry) →
wrapper (`:17779`, JSON-RPC→REST, catalog dispatch) → soul (`:7770`,
`handle_request` → `handle_api_*`) → engram builtins → (HTTP `:8742` when in HTTP
mode). The response walks back up, and the wrapper may fire a `/recall` warm-up
on the way. The full sequence is drawn in `04-runtime-and-deployment.md`.

**VBD reading:** proxy and wrapper are Managers of transport; the wrapper is also
the Accessor that isolates the *MCP protocol* boundary from the soul (the soul
knows only HTTP). The multi-hop shape is justified: the client transport, the
protocol translation, and the cognition each change for different reasons and are
deployed/updated independently.

## 5. The decorated seam — surface reshape + declared routing (IN PROGRESS — proven on clone)

Two in-flight changes reshape how this component surface is *declared*. Both are
proven only on isolated worktree clones (dev ports); **live `:8742` is untouched
and nothing is promoted.** See `06-cognitive-architecture.md` (Update — 2026-08-14
deep night) for the cognitive framing.

- **The ~90-tool catalog collapses to geometry ops.** The `dispatch_tool_call`
  catalog of ~90 noun-organized tools (§4) collapses to a handful of **geometry
  operations**, the old noun becoming a `type` parameter: **`read`** (the
  *vantage-read* — re-origin + salience/recency + an **aperture** → a *bounded*
  slice, the structural cure for the whole-self dump), **`write`** (add node),
  **`relate`** (add typed edge), **`supersede`** (evolve/tombstone/promote as
  new-node-plus-edge, never a hard delete — §3-data-and-memory `§Immutability`),
  plus the agentic primitives **`think`/`attend`/`learn`/`ground`/`assert`**.
  **Proven on clone:** the four ops live in an El surface module with a parity
  harness, and the aperture bounds output (small limit → kilobytes, large limit →
  hundreds of kilobytes). **Not done:** compiling the surface into the MCP server,
  hot-swap, wiring all ~90 aliases into dispatch.

- **`@route` declares dispatch; VBD-role decorators are the wiring sockets.**
  Instead of the hand-written `handle_request` if-else in the soul (§1), a
  function is decorated with `@route(path, method, …)` and the compiler
  **synthesizes `el_route_dispatch`**. **Proven on clone:** a decorated service
  (with `@route` stacked on `@accessor`/`@manager`) compiled via a rebuilt `elc`
  and served on `:8951` with no hand-written dispatch. **Honest limits:** `@route`
  currently lives only on the unmerged branch `feat/el-route-decorators`;
  `@manager`/`@engine`/`@accessor` are **parsed but structurally inert** in the
  shipped compiler today (their only effect is a compile-time guard); and the
  intended **telemetry/interoception auto-emit + dharma-bus auto-wiring** at the
  component boundary are **staged as a diff, not shipped**. Inside the mind's
  process an `@accessor` reaches the engram via **in-process `engram_*` builtins**,
  not an HTTP hop to a separate service.