will.anderson
4bff40fa4a
High-fanout identity anchors (voice, writing-imprint, self-root) have ~670KB
neighborhoods. inspect_graph returned the full traversal, which overflowed the
MCP client's context and socket-closed the wrapper mid self-load -- the soul
could not traverse its own identity graph.
handle_api_inspect_graph gains an opt-in `compact` projection (compact=1|true):
the neighborhood is relevance-ranked, the top K (default 12) keep a UTF-8-safe
content snippet (default snip=600), and the remainder collapse to lightweight
{id,label,node_type,tier,edge,pointer:true} stubs. This bounds the voice node
from 669,799B -> 25,353B (HTTP 200, valid JSON) and the wrapper's soul-load no
longer socket-closes. New helpers: api_compact_neighbors, api_neigh_full,
api_neigh_pointer, api_neigh_rank, api_neigh_better, api_float_or.
The flag is gated: ABSENT it, the response is byte-identical to the old plain
traversal, so the studio app (which never sends it) is unaffected. The MCP
wrapper (mcp-wrapper/src/main.el) appends &compact=1 on its inspectGraph and
fetch-by-id paths.
dist/soul.c is REGENERATED so CI ships the fix: CI compiles the committed
single-TU dist/soul.c directly (running elb/elc on the Linux runner OOM-kills
it), so an .el-only change would build the OLD behavior. Regenerated and verified
on macOS -- compiles with the CI cc line (0 errors) and, on a throwaway soul over
a copy of the live snapshot, serves compact ~25KB / non-compact ~670KB. The regen
also syncs the amalgamation to this branch's .el sources, which had drifted
several self-review commits ahead of the previously-committed soul.c.
Docs: docs/architecture/00-05 added; 01/02/05 corrected so the relevance-ranked
inspect_graph projection reads as committed source, not an in-flight concern.
146 lines
8.5 KiB
Markdown
146 lines
8.5 KiB
Markdown
# Neuron — Architecture Overview
|
|
|
|
> Status: living document. Grounded in the committed source of the `neuron`
|
|
> repository as of 2026-08-10. Every structural claim cites a real file. Where a
|
|
> statement is inferred rather than read directly, it is labelled *(inference)*
|
|
> or *(unverified/TODO)*.
|
|
|
|
## What Neuron is
|
|
|
|
Neuron is a **persistent CGI (Cultivated General Intelligence) runtime**. It is
|
|
not a chatbot and not a stateless API in front of an LLM. It is a long-lived
|
|
process that *remembers* — it carries an identity, a graph of memory and
|
|
knowledge, and an autonomous idle-cognition loop across restarts. The LLM is one
|
|
resource it calls; the durable part is the **engram** (the graph) and the
|
|
**soul** (the program that reasons over it).
|
|
|
|
Three things run together to make that true:
|
|
|
|
- **The soul** — the compiled El program in this repo. It owns the HTTP surface,
|
|
the cognitive API, the request pipeline (`layered_cycle`), and the autonomous
|
|
awareness daemon. Entry point `soul.el`, served by `handle_request`
|
|
(`routes.el:358`).
|
|
- **The engram** — the graph store. Node/edge model, spreading activation, and
|
|
Hebbian co-activation physically live in the shared El runtime
|
|
(`el_runtime.c`); `engram/src/server.el` is a thin HTTP face on `:8742`. The
|
|
engram is a *sibling* repo (`foundation/el/engram`), compiled and co-located at
|
|
runtime, not part of this repo's source tree.
|
|
- **The El runtime** — `el_runtime.c` / `el_runtime.h`. Every compiled El binary
|
|
links it. It implements all builtins (`engram_*`, `http_*`, `json_*`, LLM,
|
|
crypto) and *is* the database — "no SQL, no db layer, no SQLite"
|
|
(`../foundation/el/engram/src/server.el:4-6`).
|
|
|
|
Neuron persists memory itself — this repo is the memory system. Do not confuse
|
|
it with the Neuron desktop/UI application, which is **out of scope** here and is
|
|
only ever a *client* of the MCP surface described in this set.
|
|
|
|
## System context
|
|
|
|
```
|
|
┌────────────────────────────────────────────────────────────┐
|
|
│ MCP clients (Claude Code, Soma chat UI, agents) │
|
|
│ — talk MCP JSON-RPC over stdio, or HTTP to the soul │
|
|
└───────────────┬────────────────────────────────────────────┘
|
|
│ MCP JSON-RPC (stdio)
|
|
┌──────────▼──────────┐
|
|
│ mcp-proxy :7779 │ byte-forwarder + retry + health
|
|
└──────────┬──────────┘
|
|
│ MCP JSON-RPC (stdio→HTTP)
|
|
┌──────────▼──────────┐
|
|
│ mcp-wrapper :17779 │ JSON-RPC ⇄ soul REST; ~90-tool catalog
|
|
└──────────┬──────────┘
|
|
│ HTTP (REST)
|
|
┌──────────▼──────────┐ ┌──────────────────────────┐
|
|
│ soul :7770 │──HTTP──▶│ engram :8742 │
|
|
│ handle_request │ │ graph store (snapshot) │
|
|
│ layered_cycle │◀──────▶│ el_runtime.c = the DB │
|
|
│ awareness daemon │ └──────────────────────────┘
|
|
└──────────┬──────────┘
|
|
│ HTTP
|
|
┌───────────────┼───────────────┬───────────────┐
|
|
▼ ▼ ▼ ▼
|
|
Axon backend neuron-connectd LLM API (self-callback
|
|
:backlog/ :7771 connectors Anthropic NEURON_API_URL)
|
|
artifacts/ (MCP bridges) format
|
|
projects
|
|
```
|
|
|
|
*Ports/topology verified*: proxy `:7779` and wrapper `:17779`
|
|
(`mcp-proxy/src/main.el`, `mcp-wrapper/src/main.el`); soul `:7770`
|
|
(`NEURON_PORT`, k8s `deployment-blue.yaml`); engram `:8742` (`entrypoint.sh`,
|
|
`server.el:711`). The Axon backend, `neuron-connectd` (`:7771`), and the LLM are
|
|
external dependencies the soul reaches over HTTP (`routes.el` `axon_get/post`,
|
|
`connectd_get/post`).
|
|
|
|
## The two external interfaces
|
|
|
|
Neuron exposes exactly two surfaces, and it is worth being precise about the
|
|
difference because they drive the whole component split:
|
|
|
|
1. **The MCP surface** — the *tool* interface. MCP clients call tools
|
|
(`begin_session`, `remember`, `search_knowledge`, `inspect_graph`,
|
|
`cultivate`, …). This is the interface Claude Code and agents use. It is
|
|
delivered by the **proxy → wrapper** chain, which translates MCP JSON-RPC
|
|
into the soul's HTTP REST calls. The wrapper carries a catalog of ~90 tools
|
|
(`mcp-wrapper/src/main.el`).
|
|
|
|
2. **The HTTP API** — the *cognitive* interface. The soul serves REST on
|
|
`:7770`. `routes.el` dispatches; `neuron-api.el` handles the cognitive
|
|
endpoints (`/api/neuron/*`). This same surface backs the chat product
|
|
(`/api/chat`, `/api/sessions`) and the studio UI (`/`).
|
|
|
|
In production the MCP client connects to the soul's HTTP directly — the
|
|
`neuron-mcp` ClusterIP Service targets `:7770` (`service.yaml`) and the
|
|
proxy/wrapper chain is primarily the **local developer adapter** that lets a
|
|
stdio MCP client speak to an HTTP soul. See `04-runtime-and-deployment.md`.
|
|
|
|
## Component map (summary)
|
|
|
|
The full VBD classification is in `01-vbd-decomposition.md`. In one glance:
|
|
|
|
| Layer | Module(s) | Role |
|
|
|---|---|---|
|
|
| HTTP dispatch | `routes.el` | Manager — hand-written method/path dispatch |
|
|
| Cognitive API | `neuron-api.el` | Managers + Engines — session/memory/knowledge/graph/cultivation handlers |
|
|
| Request pipeline | `soul.el` `layered_cycle` | Manager — L1 safety → L2 stewardship → L3 imprint |
|
|
| Boot + identity | `soul.el` | Manager — compose layers, seed identity graph, start server + daemon |
|
|
| Autonomous cognition | `awareness.el` | Manager (`awareness_run`) + Engines (curiosity, attend, threat) |
|
|
| Memory access | `memory.el` | Resource Accessor over the engram FFI/HTTP |
|
|
| Store | `engram/server.el` + `el_runtime.c` | Accessor (HTTP) over the real graph engine |
|
|
| Request-layer rules | `safety.el`, `stewardship.el`, `imprint.el` | Engines |
|
|
| Conversation sessions | `sessions.el` | Manager (chat product) |
|
|
| MCP transport | `mcp-proxy`, `mcp-wrapper` | Managers/Accessors — protocol boundary |
|
|
| Build | `manifest.el`, `dist/soul.c`, El toolchain | amalgamation → `soul.c` → binary |
|
|
|
|
## Reading guide
|
|
|
|
- **`01-vbd-decomposition.md`** — the volatility analysis. Start here for *why*
|
|
the boundaries fall where they do. Contains the full Manager/Engine/Accessor/
|
|
Utility table and the honest list of where the real code diverges from VBD.
|
|
- **`02-components.md`** — per-subsystem detail: routing, the cognitive API, the
|
|
memory & activation engine, the MCP transport chain. Read after 01.
|
|
- **`03-data-and-memory.md`** — the engram graph model: node/edge structs,
|
|
layers, the two tier systems, write-protection, tombstone/supersede
|
|
immutability, persistence.
|
|
- **`04-runtime-and-deployment.md`** — process/port topology, the end-to-end MCP
|
|
request path, local vs GKE blue/green, secrets/config.
|
|
- **`05-el-and-build.md`** — the El language, the `elc`/`elb` toolchain, the
|
|
amalgamation → `soul.c` → binary pipeline, and the compile-time capability
|
|
gates.
|
|
|
|
## A note on honesty
|
|
|
|
Two facts shape everything below and are stated once here so the rest reads
|
|
straight:
|
|
|
|
1. **The most volatile logic — the activation and Hebbian math — lives in the
|
|
most stable-looking layer**, the C runtime (`el_runtime.c`). The El files in
|
|
this repo are largely a *Manager + Accessor shell* around that core. This
|
|
inverts the usual VBD expectation and is called out wherever it matters.
|
|
2. **The immutability guarantee lives above the store, not in it.** The engram
|
|
HTTP server will hard-delete a node (`DELETE /api/nodes/:id` →
|
|
`engram_forget`, `server.el:322`). Immutability holds only because the
|
|
neuron-api / MCP layer routes every user-facing delete through *tombstone*
|
|
instead (`memory.el:46`). The invariant is a policy, not a property of the
|
|
accessor.
|