will.anderson
65dd2cf097
The architecture docs describe four things the design spec has since ruled out, and each one is a supervisor invented for something that should be a property of the substrate: grounding modelled as a subsystem rather than as the edge weight it already is; faculties modelled as parameters of a read when abduce is a write; wonder materialized as a maintained manifest when it is the boundary of the structure; and consolidation implemented eleven times behind tickers when a brain has no cron job. Left standing rather than deleted, per the repo's own supersession discipline — the trail of how the understanding matured is the point. Each stale passage is marked inline and points at a new 06 §12 that transcribes the corrections and records the measured consolidation inventory. Authority: foundation/el, branch design/correspondence-and-censorship, lang/spec/correspondence-and-censorship.md.
231 lines
14 KiB
Markdown
231 lines
14 KiB
Markdown
# Neuron — VBD Decomposition
|
||
|
||
> This is the load-bearing document. It applies Volatility-Based Decomposition
|
||
> (VBD) to the *actual* neuron code, not an idealized version of it. VBD asks one
|
||
> question — **what changes, why, and how often** — and draws component
|
||
> boundaries around the answers so that a change lands inside one component
|
||
> instead of rippling across many.
|
||
>
|
||
> VBD's component taxonomy:
|
||
> - **Managers** — stable orchestrators. They sequence use-cases and delegate;
|
||
> they change only when the *shape* of a workflow changes.
|
||
> - **Engines** — volatile business rules. The "how" that churns.
|
||
> - **Resource Accessors** — isolate an external dependency (a store, an API) so
|
||
> its volatility can't leak inward.
|
||
> - **Utilities** — cross-cutting, low-volatility helpers.
|
||
>
|
||
> Communication ideal: Managers orchestrate Engines and Accessors; Managers
|
||
> prefer async/event coupling to each other; Engines are stateless-ish and never
|
||
> reach external I/O directly; Accessors hide all I/O. We note below where neuron
|
||
> honors this and where it doesn't.
|
||
|
||
## The axes of change
|
||
|
||
Before classifying modules, name the volatility. These are the axes along which
|
||
neuron actually changes, ranked by observed churn (dated self-review comments in
|
||
the source are the evidence — the code keeps a changelog in its own margins).
|
||
|
||
### 1. Context / payload shaping — *highest churn*
|
||
How much of the graph, and in what projected form, gets returned to a
|
||
bounded MCP response. The `begin_session` / `compile_ctx` handlers and the
|
||
`api_compact_*` helpers carry dense dated review comments (2026-07-30, -31)
|
||
documenting repeated rework after unbounded payloads closed the MCP client
|
||
socket (`neuron-api.el:90-317`). This changes because the *client's* context
|
||
budget and the *shape* of "what's relevant right now" keep moving. The newest
|
||
rework in this axis is the **relevance-ranked neighbor projection**
|
||
(`api_compact_neighbors` + `api_neigh_*`) behind `inspect_graph`'s `compact=1`
|
||
path — it is what keeps *self-load* (traversing the high-fanout identity anchors)
|
||
from closing the socket. It is committed source, compiled into `dist/soul.c`.
|
||
|
||
### 2. Autonomous-cognition policy
|
||
What the idle soul chooses to think about: seed-domain selection, curiosity
|
||
rotation, novelty gating, and the inbox verb-mapping in `attend()`. The
|
||
`proactive_curiosity` / `auto_term_try_slot` machinery
|
||
(`awareness.el:590-876`) has the deepest git-archaeology in the codebase
|
||
(comments spanning 2026-05 → 2026-08). This is where the *behavior* of the
|
||
agent is tuned.
|
||
|
||
> **Superseded (2026-08-16) — see `06-cognitive-architecture.md` §12.3.**
|
||
> The volatility this Engine encapsulates is **real churn around a wrong model**.
|
||
> "Seed-domain selection" and "curiosity rotation" are a maintained manifest of
|
||
> things to be curious about; a nucleation site is a **per-edge structural fact**
|
||
> (`|discord|` = `|z(semantic proximity) − z(association strength)|`, `06` §12.4),
|
||
> not an entry in a rotation. The deep git-archaeology cited here is itself
|
||
> evidence: an Engine that has been re-tuned continuously since 2026-05 is
|
||
> encapsulating volatility that the substrate should have made constant.
|
||
> **Curiosity does not search for nucleation sites; it goes where salience
|
||
> already is** — machinery that already exists (`salience`,
|
||
> `background_activation`, `working_memory_weight`, `wm_anchor`).
|
||
|
||
### 3. Epistemic & memory semantics
|
||
Tiers, salience mapping, promotion/consolidation, the immutability policy
|
||
(tombstone/supersede), and knowledge disposition. These evolve as the memory
|
||
*philosophy* matures — e.g. `mem_forget` becoming a soft delete
|
||
(`memory.el:70`), the salience-evolution pass in `mem_consolidate`
|
||
(`memory.el:92-133`), the supersede-edge pattern (`neuron-api.el:394-428`).
|
||
|
||
### 4. Safety & stewardship rules
|
||
Crisis bell thresholds, agentic threat scoring, mission alignment, CGI
|
||
continuity fingerprinting. `safety.el`, `stewardship.el`, and the threat
|
||
scorer grafted onto `awareness.el:1286-1419` change on behavioral/regulatory
|
||
pressure, independently of everything else.
|
||
|
||
### 5. API / route surface growth
|
||
New cognitive endpoints and their dispatch. `routes.el` grows structurally as
|
||
tools are added; the `handle_request` if/else chain (`routes.el:358-753`) is
|
||
edited on every surface change.
|
||
|
||
*(A sixth axis — the activation/Hebbian numeric math — is real and volatile but
|
||
is externalized to `el_runtime.c`. See "Divergences," point 6.)*
|
||
|
||
## The component map
|
||
|
||
Modules classified against the taxonomy, with the volatility that justifies each
|
||
placement. Paths are repo-relative unless noted `foundation/…`.
|
||
|
||
### Managers (stable orchestration)
|
||
|
||
| Module / function | File | Why a Manager |
|
||
|---|---|---|
|
||
| `handle_request` | `routes.el:358-753` | Top-level HTTP dispatcher. Pure method/path routing; delegates every body of work. Changes only when the *route surface* (axis 5) changes, not when logic changes. |
|
||
| Boot sequence | `soul.el:508-627` | Sequences load → seed → identity → serve → daemon. Highest stability; changes only on architecture shifts. |
|
||
| `layered_cycle` | `soul.el:382-506` | Request use-case pipeline: L1 safety → L2 stewardship (continuity, mission, affect) → L3 imprint → L1 output validation. Orchestrates Engines; holds no rules itself. |
|
||
| `awareness_run` / `one_cycle` | `awareness.el:1097-1284`, `1041-1095` | Daemon lifecycle + the perceive→attend→respond→record sequencer. Manager of the autonomous loop. |
|
||
| Session CRUD | `sessions.el` | Orchestrates the immutable delete-then-recreate dance for conversation sessions (chat product). Manager-flavored, but leaks store detail (see Divergences). |
|
||
| MCP proxy | `mcp-proxy/src/main.el` | Orchestrates transport: accept stdio, forward, retry, health-gate, wrap errors. |
|
||
| MCP wrapper | `mcp-wrapper/src/main.el` | Orchestrates the JSON-RPC ⇄ REST translation, tool catalog, lifecycle (`initialize`/`tools/list`/`tools/call`). |
|
||
|
||
### Engines (volatile business rules)
|
||
|
||
| Module / function | File | Volatility it absorbs |
|
||
|---|---|---|
|
||
| `api_compact_*`, `begin_session`, `compile_ctx` | `neuron-api.el:90-317` | Axis 1 — context/payload shaping. The single most-reworked logic on the API side. |
|
||
| `attend()` | `awareness.el:926-973` | Axis 2 — inbox content → action-verb ruleset. |
|
||
| `proactive_curiosity`, `auto_term_try_slot` | `awareness.el:590-876` | Axis 2 — seed selection, stopword/IDF gates, tabu ring. Textbook Engine: highest churn. **Superseded (2026-08-16): curiosity is not an Engine — see Axis 2 above and `06` §12.3.** |
|
||
| threat scoring | `awareness.el:1286-1419` | Axis 4 — additive command/path/history threat rules. |
|
||
| `safety.el` (crisis/harm/bell) | `safety.el` | Axis 4 — crisis screening, bell thresholds, output validation. |
|
||
| `stewardship.el` | `stewardship.el` | Axis 4 — mission alignment, CGI check, continuity fingerprint. |
|
||
| `imprint.el` | `imprint.el` | Axis 2/3 — persona response + knowledge/memory surfacing per imprint. |
|
||
| `mem_consolidate` | `memory.el:92-133` | Axis 3 — which nodes to strengthen; salience-evolution rules. |
|
||
| salience/importance mapping | `neuron-api.el` (repeated in `remember`, `node_create`, `evolve_memory`, `cultivate`) | Axis 3 — importance-enum → salience float mapping. |
|
||
| chat mode selection | `chat.el` (via `routes.el:433-440`, `597-604`) | plan / agentic / `layered_cycle` routing. |
|
||
| **activation + Hebbian math** | `foundation/.../el_runtime.c` | Axis 6 — the true cognitive Engine, externalized to C. |
|
||
|
||
### Resource Accessors (isolate external I/O)
|
||
|
||
| Accessor | File | Dependency isolated |
|
||
|---|---|---|
|
||
| `mem_*` | `memory.el` | The engram FFI/HTTP. **The** memory Accessor — clean, single isolation point; every forget routes through `mem_tombstone` (`memory.el:46`). |
|
||
| `engram_*` builtins + `server.el` | `el_runtime.c`, `foundation/el/engram/src/server.el` | The graph store over HTTP `:8742`. |
|
||
| `axon_get` / `axon_post` | `routes.el` | The Axon backend (backlog, artifacts, projects, memories, non-neuron knowledge). |
|
||
| `connectd_get` / `connectd_post` | `routes.el:303-324` | `neuron-connectd` bridge (`:7771`). |
|
||
| `llm_call_system` / `llm_call_agentic` | runtime builtins (used in `routes.el:115`, chat) | The LLM. |
|
||
| `ise_post`, `hebb_consolidate` | `awareness.el:101-148`, `64-99` | Durable engram HTTP (`/api/neuron/state-events`, `/api/edges/batch`). |
|
||
| `render_studio` | `studio.el` | The UI surface. |
|
||
|
||
### Utilities (cross-cutting, stable)
|
||
|
||
`flag_true`, `strip_query`, `err_404/405` (`routes.el:14-91`);
|
||
`api_json_escape`, `api_query_param/int`, `api_ok/err`, `api_nonempty`,
|
||
`api_utf8_trunc`, `api_persisted` (`neuron-api.el:45-201`); `idle_*`/`pulse_*`
|
||
counters, `elapsed_ms/human`, `make_action`, `embed_ok` (`awareness.el`);
|
||
`session_make_content`, `aff_try_slot`, JSON builders (`sessions.el`, `soul.el`).
|
||
Beneath all of these, the El runtime builtins (`json_*`, `http_*`, crypto, time)
|
||
are the utility substrate every module shares.
|
||
|
||
## Communication topology (as built)
|
||
|
||
```
|
||
MCP client
|
||
│ JSON-RPC
|
||
proxy ──► wrapper ──► soul.handle_request ──► neuron-api.handle_api_*
|
||
│ │
|
||
│ layered_cycle │ engram_* builtins
|
||
▼ ▼
|
||
safety / steward / imprint memory.el (Accessor)
|
||
(Engines) │
|
||
▼
|
||
el_runtime.c graph
|
||
engram HTTP :8742
|
||
|
||
awareness_run (daemon) ──perceive──► engram inbox (soul-inbox-pending tag)
|
||
──hebb_consolidate──► POST /api/edges/batch
|
||
```
|
||
|
||
Two things about coupling:
|
||
|
||
- **Manager → Engine/Accessor is in-process and synchronous** (direct El calls),
|
||
which matches VBD: rules and I/O sit behind the Managers.
|
||
- **Manager ↔ Manager is *not* the VBD async-event ideal.** It is synchronous
|
||
HTTP (soul → engram, soul → Axon) plus one genuine event-ish channel: the
|
||
**engram inbox**. The awareness daemon `perceive()`s by polling a
|
||
`soul-inbox-pending` tag and consumes trigger nodes
|
||
(`awareness.el:900-924`, `1090-1093`), and modules communicate asynchronously
|
||
by writing **InternalStateEvent** nodes. That is a partial actor/event
|
||
pattern, realized through the graph rather than a message bus.
|
||
|
||
## Where reality diverges from VBD (call it out)
|
||
|
||
Honest deviations, so no one reads this doc as a conformance certificate:
|
||
|
||
1. **No route table.** Dispatch is a hand-written if/else chain in
|
||
`handle_request` (`routes.el:358-753`); there is no `register-route`
|
||
registry. Path params are sliced by hand (`str_slice` + `str_index_of`,
|
||
`routes.el:508-513, 539-541`) — one site carries an inline offset bug-fix
|
||
comment. Acceptable for a single dispatcher, but it means the "route surface"
|
||
Manager is edited manually on every change.
|
||
|
||
2. **Store I/O leaks into Managers.** `routes.el` inlines engram export logic for
|
||
`/api/graph/edges` (`routes.el:394-422`, with a 2026-08-07 comment about a
|
||
read-route that corrupted the canonical snapshot). The `awareness_run` sync
|
||
block inlines `http_get /api/sync` + `engram_load_merge`
|
||
(`awareness.el:1219-1279`). `emit_heartbeat` (`awareness.el:201-549`, ~350
|
||
lines) mixes Utility (formatting), Accessor (HTTP/FFI reads), and Manager
|
||
(state-delta tracking) in one function. These are Accessor responsibilities
|
||
living inside orchestration — the clearest VBD smell in the codebase.
|
||
|
||
3. **No authentication.** The only access control on the HTTP surface is per-IP
|
||
rate limiting (`routes.el:38-75`) plus `is_protected_node` on 15 hardcoded
|
||
identity IDs (`neuron-api.el:20-37`). There is no bearer/token check in the
|
||
dispatch path. Security is a cross-cutting concern only partially realized;
|
||
the deployment relies on a **single-trusted-client, internal-only** boundary
|
||
assumption (the `neuron-mcp` Service is ClusterIP, no external LB — see doc 04).
|
||
|
||
4. **Immutability is enforced above the Accessor, not in it.** The engram store
|
||
itself hard-deletes (`DELETE /api/nodes/:id` → `engram_forget`,
|
||
`server.el:322`). The invariant "we never delete, we tombstone/supersede"
|
||
is a *routing policy* in `memory.el` / `neuron-api.el`, not a property of the
|
||
store. A caller that hits the raw engram HTTP bypasses it.
|
||
|
||
5. **Mutation via delete-then-recreate.** Because nodes are immutable,
|
||
`sessions.el` mutates a session by deleting and recreating the node — flagged
|
||
non-atomic in its own comments (`sessions.el:303-308`, `:456`).
|
||
|
||
6. **The volatile core is in the stable layer.** The activation, decay, and
|
||
Hebbian co-activation math — genuinely high-volatility numeric policy — lives
|
||
in `el_runtime.c`, the foundational runtime every binary links. The El files
|
||
here are a Manager+Accessor shell around it. This inverts VBD's usual
|
||
layering (volatile logic should sit *above* stable infrastructure) and is the
|
||
single most important thing to understand before changing memory behavior:
|
||
you often can't, from this repo, without touching `foundation/el`.
|
||
|
||
7. **Vocabulary mismatch across layers.** The MCP-facing memory vocabulary
|
||
(tiers `note → lesson → canonical`, disposition
|
||
`experimental → … → deprecated`, importance enum `low/normal/high/critical`)
|
||
is **not** the engine's model. The engine uses cognitive tiers
|
||
`Working / Episodic / Semantic / Canonical` (a `tier` string field) plus
|
||
continuous `salience`/`importance`/`confidence` floats, and stores epistemic
|
||
tier/disposition as **tags** (`tier:canonical`, `disposition:stable`), not as
|
||
enforced state (`neuron-api.el:533`, `server.el:519-522`). The mapping is a
|
||
convention, not a guarded state machine. See `03-data-and-memory.md`.
|
||
|
||
## Testing spiral (VBD heuristic, as observed)
|
||
|
||
VBD recommends testing Engines first (pure logic), then Accessors (mock I/O),
|
||
then Managers (integration). The repo has `tests/*.el` matching this instinct —
|
||
`test_safety.el`, `test_bell_safety.el` (Engines), `test_layer_contract.el`
|
||
(the Manager↔Engine JSON contract `layered_cycle` depends on), `test_soul_guard.el`
|
||
(the boot Manager's seed guard), `test_sessions.el`. **Flag:** CI compiles and
|
||
smoke-tests only (`dist/neuron --help`); it does **not** run these `.el` suites
|
||
(`ci.yaml`). Whether they gate merges elsewhere is unverified — see doc 05.
|