Files
neuron/docs/architecture/00-overview.md
T
will.anderson 4bff40fa4a
Neuron Soul CI / build (pull_request) Failing after 14m5s
Neuron Soul CI / deploy (pull_request) Has been skipped
fix(api): bound inspect_graph with relevance-ranked projection; regen soul.c
High-fanout identity anchors (voice, writing-imprint, self-root) have ~670KB
neighborhoods. inspect_graph returned the full traversal, which overflowed the
MCP client's context and socket-closed the wrapper mid self-load -- the soul
could not traverse its own identity graph.

handle_api_inspect_graph gains an opt-in `compact` projection (compact=1|true):
the neighborhood is relevance-ranked, the top K (default 12) keep a UTF-8-safe
content snippet (default snip=600), and the remainder collapse to lightweight
{id,label,node_type,tier,edge,pointer:true} stubs. This bounds the voice node
from 669,799B -> 25,353B (HTTP 200, valid JSON) and the wrapper's soul-load no
longer socket-closes. New helpers: api_compact_neighbors, api_neigh_full,
api_neigh_pointer, api_neigh_rank, api_neigh_better, api_float_or.

The flag is gated: ABSENT it, the response is byte-identical to the old plain
traversal, so the studio app (which never sends it) is unaffected. The MCP
wrapper (mcp-wrapper/src/main.el) appends &compact=1 on its inspectGraph and
fetch-by-id paths.

dist/soul.c is REGENERATED so CI ships the fix: CI compiles the committed
single-TU dist/soul.c directly (running elb/elc on the Linux runner OOM-kills
it), so an .el-only change would build the OLD behavior. Regenerated and verified
on macOS -- compiles with the CI cc line (0 errors) and, on a throwaway soul over
a copy of the live snapshot, serves compact ~25KB / non-compact ~670KB. The regen
also syncs the amalgamation to this branch's .el sources, which had drifted
several self-review commits ahead of the previously-committed soul.c.

Docs: docs/architecture/00-05 added; 01/02/05 corrected so the relevance-ranked
inspect_graph projection reads as committed source, not an in-flight concern.
2026-08-10 10:28:50 -05:00

8.5 KiB

Neuron — Architecture Overview

Status: living document. Grounded in the committed source of the neuron repository as of 2026-08-10. Every structural claim cites a real file. Where a statement is inferred rather than read directly, it is labelled (inference) or (unverified/TODO).

What Neuron is

Neuron is a persistent CGI (Cultivated General Intelligence) runtime. It is not a chatbot and not a stateless API in front of an LLM. It is a long-lived process that remembers — it carries an identity, a graph of memory and knowledge, and an autonomous idle-cognition loop across restarts. The LLM is one resource it calls; the durable part is the engram (the graph) and the soul (the program that reasons over it).

Three things run together to make that true:

  • The soul — the compiled El program in this repo. It owns the HTTP surface, the cognitive API, the request pipeline (layered_cycle), and the autonomous awareness daemon. Entry point soul.el, served by handle_request (routes.el:358).
  • The engram — the graph store. Node/edge model, spreading activation, and Hebbian co-activation physically live in the shared El runtime (el_runtime.c); engram/src/server.el is a thin HTTP face on :8742. The engram is a sibling repo (foundation/el/engram), compiled and co-located at runtime, not part of this repo's source tree.
  • The El runtimeel_runtime.c / el_runtime.h. Every compiled El binary links it. It implements all builtins (engram_*, http_*, json_*, LLM, crypto) and is the database — "no SQL, no db layer, no SQLite" (../foundation/el/engram/src/server.el:4-6).

Neuron persists memory itself — this repo is the memory system. Do not confuse it with the Neuron desktop/UI application, which is out of scope here and is only ever a client of the MCP surface described in this set.

System context

        ┌────────────────────────────────────────────────────────────┐
        │  MCP clients (Claude Code, Soma chat UI, agents)             │
        │  — talk MCP JSON-RPC over stdio, or HTTP to the soul         │
        └───────────────┬────────────────────────────────────────────┘
                        │  MCP JSON-RPC (stdio)
             ┌──────────▼──────────┐
             │  mcp-proxy  :7779   │  byte-forwarder + retry + health
             └──────────┬──────────┘
                        │  MCP JSON-RPC (stdio→HTTP)
             ┌──────────▼──────────┐
             │  mcp-wrapper :17779 │  JSON-RPC ⇄ soul REST; ~90-tool catalog
             └──────────┬──────────┘
                        │  HTTP (REST)
             ┌──────────▼──────────┐        ┌──────────────────────────┐
             │   soul  :7770       │──HTTP──▶│  engram  :8742           │
             │   handle_request    │        │  graph store (snapshot)  │
             │   layered_cycle     │◀──────▶│  el_runtime.c = the DB   │
             │   awareness daemon  │        └──────────────────────────┘
             └──────────┬──────────┘
                        │ HTTP
        ┌───────────────┼───────────────┬───────────────┐
        ▼               ▼               ▼               ▼
   Axon backend    neuron-connectd    LLM API       (self-callback
   :backlog/       :7771 connectors   Anthropic      NEURON_API_URL)
   artifacts/      (MCP bridges)      format
   projects

Ports/topology verified: proxy :7779 and wrapper :17779 (mcp-proxy/src/main.el, mcp-wrapper/src/main.el); soul :7770 (NEURON_PORT, k8s deployment-blue.yaml); engram :8742 (entrypoint.sh, server.el:711). The Axon backend, neuron-connectd (:7771), and the LLM are external dependencies the soul reaches over HTTP (routes.el axon_get/post, connectd_get/post).

The two external interfaces

Neuron exposes exactly two surfaces, and it is worth being precise about the difference because they drive the whole component split:

  1. The MCP surface — the tool interface. MCP clients call tools (begin_session, remember, search_knowledge, inspect_graph, cultivate, …). This is the interface Claude Code and agents use. It is delivered by the proxy → wrapper chain, which translates MCP JSON-RPC into the soul's HTTP REST calls. The wrapper carries a catalog of ~90 tools (mcp-wrapper/src/main.el).

  2. The HTTP API — the cognitive interface. The soul serves REST on :7770. routes.el dispatches; neuron-api.el handles the cognitive endpoints (/api/neuron/*). This same surface backs the chat product (/api/chat, /api/sessions) and the studio UI (/).

In production the MCP client connects to the soul's HTTP directly — the neuron-mcp ClusterIP Service targets :7770 (service.yaml) and the proxy/wrapper chain is primarily the local developer adapter that lets a stdio MCP client speak to an HTTP soul. See 04-runtime-and-deployment.md.

Component map (summary)

The full VBD classification is in 01-vbd-decomposition.md. In one glance:

Layer Module(s) Role
HTTP dispatch routes.el Manager — hand-written method/path dispatch
Cognitive API neuron-api.el Managers + Engines — session/memory/knowledge/graph/cultivation handlers
Request pipeline soul.el layered_cycle Manager — L1 safety → L2 stewardship → L3 imprint
Boot + identity soul.el Manager — compose layers, seed identity graph, start server + daemon
Autonomous cognition awareness.el Manager (awareness_run) + Engines (curiosity, attend, threat)
Memory access memory.el Resource Accessor over the engram FFI/HTTP
Store engram/server.el + el_runtime.c Accessor (HTTP) over the real graph engine
Request-layer rules safety.el, stewardship.el, imprint.el Engines
Conversation sessions sessions.el Manager (chat product)
MCP transport mcp-proxy, mcp-wrapper Managers/Accessors — protocol boundary
Build manifest.el, dist/soul.c, El toolchain amalgamation → soul.c → binary

Reading guide

  • 01-vbd-decomposition.md — the volatility analysis. Start here for why the boundaries fall where they do. Contains the full Manager/Engine/Accessor/ Utility table and the honest list of where the real code diverges from VBD.
  • 02-components.md — per-subsystem detail: routing, the cognitive API, the memory & activation engine, the MCP transport chain. Read after 01.
  • 03-data-and-memory.md — the engram graph model: node/edge structs, layers, the two tier systems, write-protection, tombstone/supersede immutability, persistence.
  • 04-runtime-and-deployment.md — process/port topology, the end-to-end MCP request path, local vs GKE blue/green, secrets/config.
  • 05-el-and-build.md — the El language, the elc/elb toolchain, the amalgamation → soul.c → binary pipeline, and the compile-time capability gates.

A note on honesty

Two facts shape everything below and are stated once here so the rest reads straight:

  1. The most volatile logic — the activation and Hebbian math — lives in the most stable-looking layer, the C runtime (el_runtime.c). The El files in this repo are largely a Manager + Accessor shell around that core. This inverts the usual VBD expectation and is called out wherever it matters.
  2. The immutability guarantee lives above the store, not in it. The engram HTTP server will hard-delete a node (DELETE /api/nodes/:idengram_forget, server.el:322). Immutability holds only because the neuron-api / MCP layer routes every user-facing delete through tombstone instead (memory.el:46). The invariant is a policy, not a property of the accessor.