neuron/docs/architecture/01-vbd-decomposition.md

# Neuron — VBD Decomposition

> This is the load-bearing document. It applies Volatility-Based Decomposition
> (VBD) to the *actual* neuron code, not an idealized version of it. VBD asks one
> question — **what changes, why, and how often** — and draws component
> boundaries around the answers so that a change lands inside one component
> instead of rippling across many.
>
> VBD's component taxonomy:
> - **Managers** — stable orchestrators. They sequence use-cases and delegate;
>   they change only when the *shape* of a workflow changes.
> - **Engines** — volatile business rules. The "how" that churns.
> - **Resource Accessors** — isolate an external dependency (a store, an API) so
>   its volatility can't leak inward.
> - **Utilities** — cross-cutting, low-volatility helpers.
>
> Communication ideal: Managers orchestrate Engines and Accessors; Managers
> prefer async/event coupling to each other; Engines are stateless-ish and never
> reach external I/O directly; Accessors hide all I/O. We note below where neuron
> honors this and where it doesn't.

## The axes of change

Before classifying modules, name the volatility. These are the axes along which
neuron actually changes, ranked by observed churn (dated self-review comments in
the source are the evidence — the code keeps a changelog in its own margins).

### 1. Context / payload shaping — *highest churn*
How much of the graph, and in what projected form, gets returned to a
bounded MCP response. The `begin_session` / `compile_ctx` handlers and the
`api_compact_*` helpers carry dense dated review comments (2026-07-30, -31)
documenting repeated rework after unbounded payloads closed the MCP client
socket (`neuron-api.el:90-317`). This changes because the *client's* context
budget and the *shape* of "what's relevant right now" keep moving. The newest
rework in this axis is the **relevance-ranked neighbor projection**
(`api_compact_neighbors` + `api_neigh_*`) behind `inspect_graph`'s `compact=1`
path — it is what keeps *self-load* (traversing the high-fanout identity anchors)
from closing the socket. It is committed source, compiled into `dist/soul.c`.

### 2. Autonomous-cognition policy
What the idle soul chooses to think about: seed-domain selection, curiosity
rotation, novelty gating, and the inbox verb-mapping in `attend()`. The
`proactive_curiosity` / `auto_term_try_slot` machinery
(`awareness.el:590-876`) has the deepest git-archaeology in the codebase
(comments spanning 2026-05 → 2026-08). This is where the *behavior* of the
agent is tuned.

> **Superseded (2026-08-16) — see `06-cognitive-architecture.md` §12.3.**
> The volatility this Engine encapsulates is **real churn around a wrong model**.
> "Seed-domain selection" and "curiosity rotation" are a maintained manifest of
> things to be curious about; a nucleation site is a **per-edge structural fact**
> (`|discord|` = `|z(semantic proximity) − z(association strength)|`, `06` §12.4),
> not an entry in a rotation. The deep git-archaeology cited here is itself
> evidence: an Engine that has been re-tuned continuously since 2026-05 is
> encapsulating volatility that the substrate should have made constant.
> **Curiosity does not search for nucleation sites; it goes where salience
> already is** — machinery that already exists (`salience`,
> `background_activation`, `working_memory_weight`, `wm_anchor`).

### 3. Epistemic & memory semantics
Tiers, salience mapping, promotion/consolidation, the immutability policy
(tombstone/supersede), and knowledge disposition. These evolve as the memory
*philosophy* matures — e.g. `mem_forget` becoming a soft delete
(`memory.el:70`), the salience-evolution pass in `mem_consolidate`
(`memory.el:92-133`), the supersede-edge pattern (`neuron-api.el:394-428`).

### 4. Safety & stewardship rules
Crisis bell thresholds, agentic threat scoring, mission alignment, CGI
continuity fingerprinting. `safety.el`, `stewardship.el`, and the threat
scorer grafted onto `awareness.el:1286-1419` change on behavioral/regulatory
pressure, independently of everything else.

### 5. API / route surface growth
New cognitive endpoints and their dispatch. `routes.el` grows structurally as
tools are added; the `handle_request` if/else chain (`routes.el:358-753`) is
edited on every surface change.

*(A sixth axis — the activation/Hebbian numeric math — is real and volatile but
is externalized to `el_runtime.c`. See "Divergences," point 6.)*

## The component map

Modules classified against the taxonomy, with the volatility that justifies each
placement. Paths are repo-relative unless noted `foundation/…`.

### Managers (stable orchestration)

| Module / function | File | Why a Manager |
|---|---|---|
| `handle_request` | `routes.el:358-753` | Top-level HTTP dispatcher. Pure method/path routing; delegates every body of work. Changes only when the *route surface* (axis 5) changes, not when logic changes. |
| Boot sequence | `soul.el:508-627` | Sequences load → seed → identity → serve → daemon. Highest stability; changes only on architecture shifts. |
| `layered_cycle` | `soul.el:382-506` | Request use-case pipeline: L1 safety → L2 stewardship (continuity, mission, affect) → L3 imprint → L1 output validation. Orchestrates Engines; holds no rules itself. |
| `awareness_run` / `one_cycle` | `awareness.el:1097-1284`, `1041-1095` | Daemon lifecycle + the perceive→attend→respond→record sequencer. Manager of the autonomous loop. |
| Session CRUD | `sessions.el` | Orchestrates the immutable delete-then-recreate dance for conversation sessions (chat product). Manager-flavored, but leaks store detail (see Divergences). |
| MCP proxy | `mcp-proxy/src/main.el` | Orchestrates transport: accept stdio, forward, retry, health-gate, wrap errors. |
| MCP wrapper | `mcp-wrapper/src/main.el` | Orchestrates the JSON-RPC ⇄ REST translation, tool catalog, lifecycle (`initialize`/`tools/list`/`tools/call`). |

### Engines (volatile business rules)

| Module / function | File | Volatility it absorbs |
|---|---|---|
| `api_compact_*`, `begin_session`, `compile_ctx` | `neuron-api.el:90-317` | Axis 1 — context/payload shaping. The single most-reworked logic on the API side. |
| `attend()` | `awareness.el:926-973` | Axis 2 — inbox content → action-verb ruleset. |
| `proactive_curiosity`, `auto_term_try_slot` | `awareness.el:590-876` | Axis 2 — seed selection, stopword/IDF gates, tabu ring. Textbook Engine: highest churn. **Superseded (2026-08-16): curiosity is not an Engine — see Axis 2 above and `06` §12.3.** |
| threat scoring | `awareness.el:1286-1419` | Axis 4 — additive command/path/history threat rules. |
| `safety.el` (crisis/harm/bell) | `safety.el` | Axis 4 — crisis screening, bell thresholds, output validation. |
| `stewardship.el` | `stewardship.el` | Axis 4 — mission alignment, CGI check, continuity fingerprint. |
| `imprint.el` | `imprint.el` | Axis 2/3 — persona response + knowledge/memory surfacing per imprint. |
| `mem_consolidate` | `memory.el:92-133` | Axis 3 — which nodes to strengthen; salience-evolution rules. |
| salience/importance mapping | `neuron-api.el` (repeated in `remember`, `node_create`, `evolve_memory`, `cultivate`) | Axis 3 — importance-enum → salience float mapping. |
| chat mode selection | `chat.el` (via `routes.el:433-440`, `597-604`) | plan / agentic / `layered_cycle` routing. |
| **activation + Hebbian math** | `foundation/.../el_runtime.c` | Axis 6 — the true cognitive Engine, externalized to C. |

### Resource Accessors (isolate external I/O)

| Accessor | File | Dependency isolated |
|---|---|---|
| `mem_*` | `memory.el` | The engram FFI/HTTP. **The** memory Accessor — clean, single isolation point; every forget routes through `mem_tombstone` (`memory.el:46`). |
| `engram_*` builtins + `server.el` | `el_runtime.c`, `foundation/el/engram/src/server.el` | The graph store over HTTP `:8742`. |
| `axon_get` / `axon_post` | `routes.el` | The Axon backend (backlog, artifacts, projects, memories, non-neuron knowledge). |
| `connectd_get` / `connectd_post` | `routes.el:303-324` | `neuron-connectd` bridge (`:7771`). |
| `llm_call_system` / `llm_call_agentic` | runtime builtins (used in `routes.el:115`, chat) | The LLM. |
| `ise_post`, `hebb_consolidate` | `awareness.el:101-148`, `64-99` | Durable engram HTTP (`/api/neuron/state-events`, `/api/edges/batch`). |
| `render_studio` | `studio.el` | The UI surface. |

### Utilities (cross-cutting, stable)

`flag_true`, `strip_query`, `err_404/405` (`routes.el:14-91`);
`api_json_escape`, `api_query_param/int`, `api_ok/err`, `api_nonempty`,
`api_utf8_trunc`, `api_persisted` (`neuron-api.el:45-201`); `idle_*`/`pulse_*`
counters, `elapsed_ms/human`, `make_action`, `embed_ok` (`awareness.el`);
`session_make_content`, `aff_try_slot`, JSON builders (`sessions.el`, `soul.el`).
Beneath all of these, the El runtime builtins (`json_*`, `http_*`, crypto, time)
are the utility substrate every module shares.

## Communication topology (as built)

```
   MCP client
      │ JSON-RPC
   proxy ──► wrapper ──► soul.handle_request ──► neuron-api.handle_api_*
                                     │                    │
                                     │ layered_cycle      │ engram_* builtins
                                     ▼                    ▼
                          safety / steward / imprint   memory.el (Accessor)
                          (Engines)                       │
                                                          ▼
                                                   el_runtime.c graph
                                                   engram HTTP :8742

   awareness_run (daemon) ──perceive──► engram inbox (soul-inbox-pending tag)
                          ──hebb_consolidate──► POST /api/edges/batch
```

Two things about coupling:

- **Manager → Engine/Accessor is in-process and synchronous** (direct El calls),
  which matches VBD: rules and I/O sit behind the Managers.
- **Manager ↔ Manager is *not* the VBD async-event ideal.** It is synchronous
  HTTP (soul → engram, soul → Axon) plus one genuine event-ish channel: the
  **engram inbox**. The awareness daemon `perceive()`s by polling a
  `soul-inbox-pending` tag and consumes trigger nodes
  (`awareness.el:900-924`, `1090-1093`), and modules communicate asynchronously
  by writing **InternalStateEvent** nodes. That is a partial actor/event
  pattern, realized through the graph rather than a message bus.

## Where reality diverges from VBD (call it out)

Honest deviations, so no one reads this doc as a conformance certificate:

1. **No route table.** Dispatch is a hand-written if/else chain in
   `handle_request` (`routes.el:358-753`); there is no `register-route`
   registry. Path params are sliced by hand (`str_slice` + `str_index_of`,
   `routes.el:508-513, 539-541`) — one site carries an inline offset bug-fix
   comment. Acceptable for a single dispatcher, but it means the "route surface"
   Manager is edited manually on every change.

2. **Store I/O leaks into Managers.** `routes.el` inlines engram export logic for
   `/api/graph/edges` (`routes.el:394-422`, with a 2026-08-07 comment about a
   read-route that corrupted the canonical snapshot). The `awareness_run` sync
   block inlines `http_get /api/sync` + `engram_load_merge`
   (`awareness.el:1219-1279`). `emit_heartbeat` (`awareness.el:201-549`, ~350
   lines) mixes Utility (formatting), Accessor (HTTP/FFI reads), and Manager
   (state-delta tracking) in one function. These are Accessor responsibilities
   living inside orchestration — the clearest VBD smell in the codebase.

3. **No authentication.** The only access control on the HTTP surface is per-IP
   rate limiting (`routes.el:38-75`) plus `is_protected_node` on 15 hardcoded
   identity IDs (`neuron-api.el:20-37`). There is no bearer/token check in the
   dispatch path. Security is a cross-cutting concern only partially realized;
   the deployment relies on a **single-trusted-client, internal-only** boundary
   assumption (the `neuron-mcp` Service is ClusterIP, no external LB — see doc 04).

4. **Immutability is enforced above the Accessor, not in it.** The engram store
   itself hard-deletes (`DELETE /api/nodes/:id` → `engram_forget`,
   `server.el:322`). The invariant "we never delete, we tombstone/supersede"
   is a *routing policy* in `memory.el` / `neuron-api.el`, not a property of the
   store. A caller that hits the raw engram HTTP bypasses it.

5. **Mutation via delete-then-recreate.** Because nodes are immutable,
   `sessions.el` mutates a session by deleting and recreating the node — flagged
   non-atomic in its own comments (`sessions.el:303-308`, `:456`).

6. **The volatile core is in the stable layer.** The activation, decay, and
   Hebbian co-activation math — genuinely high-volatility numeric policy — lives
   in `el_runtime.c`, the foundational runtime every binary links. The El files
   here are a Manager+Accessor shell around it. This inverts VBD's usual
   layering (volatile logic should sit *above* stable infrastructure) and is the
   single most important thing to understand before changing memory behavior:
   you often can't, from this repo, without touching `foundation/el`.

7. **Vocabulary mismatch across layers.** The MCP-facing memory vocabulary
   (tiers `note → lesson → canonical`, disposition
   `experimental → … → deprecated`, importance enum `low/normal/high/critical`)
   is **not** the engine's model. The engine uses cognitive tiers
   `Working / Episodic / Semantic / Canonical` (a `tier` string field) plus
   continuous `salience`/`importance`/`confidence` floats, and stores epistemic
   tier/disposition as **tags** (`tier:canonical`, `disposition:stable`), not as
   enforced state (`neuron-api.el:533`, `server.el:519-522`). The mapping is a
   convention, not a guarded state machine. See `03-data-and-memory.md`.

## Testing spiral (VBD heuristic, as observed)

VBD recommends testing Engines first (pure logic), then Accessors (mock I/O),
then Managers (integration). The repo has `tests/*.el` matching this instinct —
`test_safety.el`, `test_bell_safety.el` (Engines), `test_layer_contract.el`
(the Manager↔Engine JSON contract `layered_cycle` depends on), `test_soul_guard.el`
(the boot Manager's seed guard), `test_sessions.el`. **Flag:** CI compiles and
smoke-tests only (`dist/neuron --help`); it does **not** run these `.el` suites
(`ci.yaml`). Whether they gate merges elsewhere is unverified — see doc 05.