neuron/docs/architecture/01-vbd-decomposition.md

Neuron — VBD Decomposition

This is the load-bearing document. It applies Volatility-Based Decomposition (VBD) to the actual neuron code, not an idealized version of it. VBD asks one question — what changes, why, and how often — and draws component boundaries around the answers so that a change lands inside one component instead of rippling across many.

VBD's component taxonomy:

• Managers — stable orchestrators. They sequence use-cases and delegate; they change only when the shape of a workflow changes.
• Engines — volatile business rules. The "how" that churns.
• Resource Accessors — isolate an external dependency (a store, an API) so its volatility can't leak inward.
• Utilities — cross-cutting, low-volatility helpers.

Communication ideal: Managers orchestrate Engines and Accessors; Managers prefer async/event coupling to each other; Engines are stateless-ish and never reach external I/O directly; Accessors hide all I/O. We note below where neuron honors this and where it doesn't.

The axes of change

Before classifying modules, name the volatility. These are the axes along which neuron actually changes, ranked by observed churn (dated self-review comments in the source are the evidence — the code keeps a changelog in its own margins).

1. Context / payload shaping — highest churn

How much of the graph, and in what projected form, gets returned to a bounded MCP response. The begin_session / compile_ctx handlers and the api_compact_* helpers carry dense dated review comments (2026-07-30, -31) documenting repeated rework after unbounded payloads closed the MCP client socket (neuron-api.el:90-317). This changes because the client's context budget and the shape of "what's relevant right now" keep moving. The newest rework in this axis is the relevance-ranked neighbor projection (api_compact_neighbors + api_neigh_*) behind inspect_graph's compact=1 path — it is what keeps self-load (traversing the high-fanout identity anchors) from closing the socket. It is committed source, compiled into dist/soul.c.

2. Autonomous-cognition policy

What the idle soul chooses to think about: seed-domain selection, curiosity rotation, novelty gating, and the inbox verb-mapping in attend(). The proactive_curiosity / auto_term_try_slot machinery (awareness.el:590-876) has the deepest git-archaeology in the codebase (comments spanning 2026-05 → 2026-08). This is where the behavior of the agent is tuned.

Superseded (2026-08-16) — see 06-cognitive-architecture.md §12.3. The volatility this Engine encapsulates is real churn around a wrong model. "Seed-domain selection" and "curiosity rotation" are a maintained manifest of things to be curious about; a nucleation site is a per-edge structural fact (|discord| = |z(semantic proximity) − z(association strength)|, 06 §12.4), not an entry in a rotation. The deep git-archaeology cited here is itself evidence: an Engine that has been re-tuned continuously since 2026-05 is encapsulating volatility that the substrate should have made constant. Curiosity does not search for nucleation sites; it goes where salience already is — machinery that already exists (salience, background_activation, working_memory_weight, wm_anchor).

3. Epistemic & memory semantics

Tiers, salience mapping, promotion/consolidation, the immutability policy (tombstone/supersede), and knowledge disposition. These evolve as the memory philosophy matures — e.g. mem_forget becoming a soft delete (memory.el:70), the salience-evolution pass in mem_consolidate (memory.el:92-133), the supersede-edge pattern (neuron-api.el:394-428).

4. Safety & stewardship rules

Crisis bell thresholds, agentic threat scoring, mission alignment, CGI continuity fingerprinting. safety.el, stewardship.el, and the threat scorer grafted onto awareness.el:1286-1419 change on behavioral/regulatory pressure, independently of everything else.

5. API / route surface growth

New cognitive endpoints and their dispatch. routes.el grows structurally as tools are added; the handle_request if/else chain (routes.el:358-753) is edited on every surface change.

(A sixth axis — the activation/Hebbian numeric math — is real and volatile but is externalized to el_runtime.c. See "Divergences," point 6.)

The component map

Modules classified against the taxonomy, with the volatility that justifies each placement. Paths are repo-relative unless noted foundation/….

Managers (stable orchestration)

Module / function	File	Why a Manager
handle_request	routes.el:358-753	Top-level HTTP dispatcher. Pure method/path routing; delegates every body of work. Changes only when the route surface (axis 5) changes, not when logic changes.
Boot sequence	soul.el:508-627	Sequences load → seed → identity → serve → daemon. Highest stability; changes only on architecture shifts.
layered_cycle	soul.el:382-506	Request use-case pipeline: L1 safety → L2 stewardship (continuity, mission, affect) → L3 imprint → L1 output validation. Orchestrates Engines; holds no rules itself.
awareness_run / one_cycle	awareness.el:1097-1284, 1041-1095	Daemon lifecycle + the perceive→attend→respond→record sequencer. Manager of the autonomous loop.
Session CRUD	sessions.el	Orchestrates the immutable delete-then-recreate dance for conversation sessions (chat product). Manager-flavored, but leaks store detail (see Divergences).
MCP proxy	mcp-proxy/src/main.el	Orchestrates transport: accept stdio, forward, retry, health-gate, wrap errors.
MCP wrapper	mcp-wrapper/src/main.el	Orchestrates the JSON-RPC ⇄ REST translation, tool catalog, lifecycle (initialize/tools/list/tools/call).

Engines (volatile business rules)

Module / function	File	Volatility it absorbs
api_compact_*, begin_session, compile_ctx	neuron-api.el:90-317	Axis 1 — context/payload shaping. The single most-reworked logic on the API side.
attend()	awareness.el:926-973	Axis 2 — inbox content → action-verb ruleset.
proactive_curiosity, auto_term_try_slot	awareness.el:590-876	Axis 2 — seed selection, stopword/IDF gates, tabu ring. Textbook Engine: highest churn. Superseded (2026-08-16): curiosity is not an Engine — see Axis 2 above and 06 §12.3.
threat scoring	awareness.el:1286-1419	Axis 4 — additive command/path/history threat rules.
safety.el (crisis/harm/bell)	safety.el	Axis 4 — crisis screening, bell thresholds, output validation.
stewardship.el	stewardship.el	Axis 4 — mission alignment, CGI check, continuity fingerprint.
imprint.el	imprint.el	Axis 2/3 — persona response + knowledge/memory surfacing per imprint.
mem_consolidate	memory.el:92-133	Axis 3 — which nodes to strengthen; salience-evolution rules.
salience/importance mapping	neuron-api.el (repeated in remember, node_create, evolve_memory, cultivate)	Axis 3 — importance-enum → salience float mapping.
chat mode selection	chat.el (via routes.el:433-440, 597-604)	plan / agentic / layered_cycle routing.
activation + Hebbian math	foundation/.../el_runtime.c	Axis 6 — the true cognitive Engine, externalized to C.

Resource Accessors (isolate external I/O)

Accessor	File	Dependency isolated
mem_*	memory.el	The engram FFI/HTTP. The memory Accessor — clean, single isolation point; every forget routes through mem_tombstone (memory.el:46).
engram_* builtins + server.el	el_runtime.c, foundation/el/engram/src/server.el	The graph store over HTTP :8742.
axon_get / axon_post	routes.el	The Axon backend (backlog, artifacts, projects, memories, non-neuron knowledge).
connectd_get / connectd_post	routes.el:303-324	neuron-connectd bridge (:7771).
llm_call_system / llm_call_agentic	runtime builtins (used in routes.el:115, chat)	The LLM.
ise_post, hebb_consolidate	awareness.el:101-148, 64-99	Durable engram HTTP (/api/neuron/state-events, /api/edges/batch).
render_studio	studio.el	The UI surface.

Utilities (cross-cutting, stable)

flag_true, strip_query, err_404/405 (routes.el:14-91); api_json_escape, api_query_param/int, api_ok/err, api_nonempty, api_utf8_trunc, api_persisted (neuron-api.el:45-201); idle_*/pulse_* counters, elapsed_ms/human, make_action, embed_ok (awareness.el); session_make_content, aff_try_slot, JSON builders (sessions.el, soul.el). Beneath all of these, the El runtime builtins (json_*, http_*, crypto, time) are the utility substrate every module shares.

Communication topology (as built)

   MCP client
      │ JSON-RPC
   proxy ──► wrapper ──► soul.handle_request ──► neuron-api.handle_api_*
                                     │                    │
                                     │ layered_cycle      │ engram_* builtins
                                     ▼                    ▼
                          safety / steward / imprint   memory.el (Accessor)
                          (Engines)                       │
                                                          ▼
                                                   el_runtime.c graph
                                                   engram HTTP :8742

   awareness_run (daemon) ──perceive──► engram inbox (soul-inbox-pending tag)
                          ──hebb_consolidate──► POST /api/edges/batch

Two things about coupling:

• Manager → Engine/Accessor is in-process and synchronous (direct El calls), which matches VBD: rules and I/O sit behind the Managers.
• Manager ↔ Manager is not the VBD async-event ideal. It is synchronous HTTP (soul → engram, soul → Axon) plus one genuine event-ish channel: the engram inbox. The awareness daemon perceive()s by polling a soul-inbox-pending tag and consumes trigger nodes (awareness.el:900-924, 1090-1093), and modules communicate asynchronously by writing InternalStateEvent nodes. That is a partial actor/event pattern, realized through the graph rather than a message bus.

Where reality diverges from VBD (call it out)

Honest deviations, so no one reads this doc as a conformance certificate:

1. No route table. Dispatch is a hand-written if/else chain in handle_request (routes.el:358-753); there is no register-route registry. Path params are sliced by hand (str_slice + str_index_of, routes.el:508-513, 539-541) — one site carries an inline offset bug-fix comment. Acceptable for a single dispatcher, but it means the "route surface" Manager is edited manually on every change.

2. Store I/O leaks into Managers. routes.el inlines engram export logic for /api/graph/edges (routes.el:394-422, with a 2026-08-07 comment about a read-route that corrupted the canonical snapshot). The awareness_run sync block inlines http_get /api/sync + engram_load_merge (awareness.el:1219-1279). emit_heartbeat (awareness.el:201-549, ~350 lines) mixes Utility (formatting), Accessor (HTTP/FFI reads), and Manager (state-delta tracking) in one function. These are Accessor responsibilities living inside orchestration — the clearest VBD smell in the codebase.

3. No authentication. The only access control on the HTTP surface is per-IP rate limiting (routes.el:38-75) plus is_protected_node on 15 hardcoded identity IDs (neuron-api.el:20-37). There is no bearer/token check in the dispatch path. Security is a cross-cutting concern only partially realized; the deployment relies on a single-trusted-client, internal-only boundary assumption (the neuron-mcp Service is ClusterIP, no external LB — see doc 04).

4. Immutability is enforced above the Accessor, not in it. The engram store itself hard-deletes (DELETE /api/nodes/:id → engram_forget, server.el:322). The invariant "we never delete, we tombstone/supersede" is a routing policy in memory.el / neuron-api.el, not a property of the store. A caller that hits the raw engram HTTP bypasses it.

5. Mutation via delete-then-recreate. Because nodes are immutable, sessions.el mutates a session by deleting and recreating the node — flagged non-atomic in its own comments (sessions.el:303-308, :456).

6. The volatile core is in the stable layer. The activation, decay, and Hebbian co-activation math — genuinely high-volatility numeric policy — lives in el_runtime.c, the foundational runtime every binary links. The El files here are a Manager+Accessor shell around it. This inverts VBD's usual layering (volatile logic should sit above stable infrastructure) and is the single most important thing to understand before changing memory behavior: you often can't, from this repo, without touching foundation/el.

7. Vocabulary mismatch across layers. The MCP-facing memory vocabulary (tiers note → lesson → canonical, disposition experimental → … → deprecated, importance enum low/normal/high/critical) is not the engine's model. The engine uses cognitive tiers Working / Episodic / Semantic / Canonical (a tier string field) plus continuous salience/importance/confidence floats, and stores epistemic tier/disposition as tags (tier:canonical, disposition:stable), not as enforced state (neuron-api.el:533, server.el:519-522). The mapping is a convention, not a guarded state machine. See 03-data-and-memory.md.

Testing spiral (VBD heuristic, as observed)

VBD recommends testing Engines first (pure logic), then Accessors (mock I/O), then Managers (integration). The repo has tests/*.el matching this instinct — test_safety.el, test_bell_safety.el (Engines), test_layer_contract.el (the Manager↔Engine JSON contract layered_cycle depends on), test_soul_guard.el (the boot Manager's seed guard), test_sessions.el. Flag: CI compiles and smoke-tests only (dist/neuron --help); it does not run these .el suites (ci.yaml). Whether they gate merges elsewhere is unverified — see doc 05.