feat(neuron): rung-zero live - first conversation on pure-Neuron stack
- kernel/graph: nodes+edges append-only store - conversation: chained turns, memory node, fold-based reads - session/sessions: durable admission inbox, promotion, serialized runner, eager tool settlement, provider/dialect/transform injected at composition - provider/dialects: raw-wire anthropic + openai-compatible, registry - llm/stream, tools(bash/read/write), auth(local claims), boot composition - providers resolve from ~/.config/neuron/providers.json - no baked vendors
This commit is contained in:
@@ -0,0 +1,371 @@
|
||||
# Neuron Architecture — v2 Core
|
||||
|
||||
> Status: DESIGN AGREED (event-bus.md whiteboard session + v2 spec family,
|
||||
> 2026-08-22). This document lays out the whole architecture in one place.
|
||||
> The governing specs remain authoritative for detail:
|
||||
> `specs/event-bus.md`, `specs/v2/*.md`, and `neuron.txt`.
|
||||
|
||||
---
|
||||
|
||||
## 0. Thesis
|
||||
|
||||
**The message is the entire application.**
|
||||
|
||||
State is never stored; it is folded from append-only streams. Nothing is
|
||||
overwritten — messages append, headers inject, schemas coexist, artifacts
|
||||
supersede. No operation anywhere takes "the past" as input.
|
||||
|
||||
One idea, fractal at every scale: **state = fold(append-only sequence).**
|
||||
|
||||
---
|
||||
|
||||
## 1. The Kernel — bus, envelopes, ledger
|
||||
|
||||
### 1.1 Bus
|
||||
|
||||
Process-global dumb wire. Four verbs:
|
||||
|
||||
```ts
|
||||
publish(topic, type, payload?) // fire-and-forget
|
||||
emit(topic, type, payload?) // async, awaits handlers
|
||||
subscribe(match, handler) // predicate on envelopes
|
||||
next(match, signal?) // await next matching envelope
|
||||
```
|
||||
|
||||
Agents subscribe once and **filter on their address segments**
|
||||
(orchestration id + agent token). Routing, scoping, and fan-out are
|
||||
consequences of matching — not mechanisms.
|
||||
|
||||
### 1.2 Envelope (four schema planes, fractal at every scale)
|
||||
|
||||
| Plane | What | Versioning |
|
||||
|---|---|---|
|
||||
| 0 | Envelope format itself | pinned root `event-bus-envelope-v1` |
|
||||
| 1 | Header values | per-header ref |
|
||||
| 2 | Body contract | body-level ref |
|
||||
| 3 | Payloads | per message-type |
|
||||
|
||||
Headers are an **ordered inject-only list**: duplicates accumulate, order
|
||||
is causality, corrections are new injections with reasons. `current(key)`
|
||||
folds last-wins; `history(key)` returns evolution. Every envelope is
|
||||
content-addressed (`id = hash(canonical form)`).
|
||||
|
||||
Artifacts ride with the record: `{ ref, hash?, kind? }` — evidence never
|
||||
separated from the message that produced it.
|
||||
|
||||
### 1.3 Topics = held addresses
|
||||
|
||||
There are not two kinds of topics. There is **one stream mechanism**;
|
||||
"durable" means someone holds a reference constant to the address —
|
||||
`orchestration.{id}`, `agent.{token}`, `orchestration.{id}.agent.{token}`,
|
||||
`session.{id}` — so its envelopes persist and stay foldable. "Ephemeral"
|
||||
means nobody holds it: delivered to listeners, then air.
|
||||
|
||||
The hierarchy in addresses *is* the filtering scheme. Subscription by
|
||||
predicate replaces per-topic registries.
|
||||
|
||||
Durable set (the ledger): orchestration lifecycle/packages/clearance,
|
||||
agent plan/steps/control, session steers, conversations.
|
||||
Ephemeral set (the air): token streams, presence, progress gauges.
|
||||
|
||||
The durable set alone reconstructs everything; ephemeral loss is never
|
||||
data loss.
|
||||
|
||||
### 1.4 Ledger and folds
|
||||
|
||||
One owned SQLite store holds three things total: topic logs, artifact
|
||||
blobs, schema registry. Four query verbs: by-topic, by-type,
|
||||
by-time-range, walk-by-causality. Fold/replay is the only read that
|
||||
builds state. Crash recovery = silence; last folded envelope is the
|
||||
resume pointer.
|
||||
|
||||
---
|
||||
|
||||
## 2. The Graph — nodes and edges
|
||||
|
||||
In the database we deal with nodes and edges.
|
||||
|
||||
### 2.1 Everything is a node
|
||||
|
||||
Knowledge. A project. A backlog. A backlog item. A conversation. A turn.
|
||||
An agent. An orchestration. **Memory.** Anything worth talking about
|
||||
twice is a first-class node with identity and address.
|
||||
|
||||
Nodes do not hold state. A node is a stable identity; its current state
|
||||
is a fold over incident edges.
|
||||
|
||||
### 2.2 Edges are envelopes
|
||||
|
||||
Every edge is a message: authored, sequenced, carrying its causal parent.
|
||||
Edges are **typed** — the vocabulary grows by use (`prompted`, `produced`,
|
||||
`supersedes`, `regards`, `blocked-by`, `remembers`, ...). New relationship
|
||||
kinds need no migration: new vocabulary, then append.
|
||||
|
||||
Relationships are **dynamic**: read from history, never written as
|
||||
records. Roles get revised by later injection; both readings stay true.
|
||||
|
||||
Law: payloads hold entities; edges are envelopes; collections are always
|
||||
folds.
|
||||
|
||||
### 2.3 Conversations are chains, not buckets
|
||||
|
||||
A conversation does **not** accumulate records into a bucket. It has an
|
||||
**opening turn**, and each turn links to the previous:
|
||||
|
||||
```
|
||||
conversation ──▶ opening turn ──▶ turn ──▶ turn ──▶ ...
|
||||
```
|
||||
|
||||
Turns are nodes; links are typed causal edges. Forking = pointing a new
|
||||
link at an old turn. Truncating = pointing elsewhere. Transcript = walk
|
||||
the chain.
|
||||
|
||||
### 2.4 Memory is a node
|
||||
|
||||
Memory holds **the things you want to remember.** Not a behavior, not a
|
||||
pile: a node whose typed edges reach into knowledge nodes, turns,
|
||||
artifacts, decisions.
|
||||
|
||||
- Retrieval replaces recollection: walk memory edges.
|
||||
- Context death is trivial: open the memory node, follow links.
|
||||
- Forgetting is explicit: prune or supersede memory edges; history stays.
|
||||
|
||||
---
|
||||
|
||||
## 3. Session Core (v2)
|
||||
|
||||
### 3.1 Admission ≠ execution
|
||||
|
||||
`sessions.prompt()` admits a durable `session_input` inbox row with
|
||||
receipt semantics (exact-retry reconciliation by message ID), then wakes
|
||||
execution. Admitted input is invisible until the serialized runner
|
||||
promotes it at a safe boundary (`Prompted`). Interrupt preserves pending
|
||||
inbox rows; idle interruption is a no-op.
|
||||
|
||||
### 3.2 Serialized runner
|
||||
|
||||
One explicit `llm.stream(request)` per provider turn. Eager local tool
|
||||
execution: settle each call durably, start child execution immediately,
|
||||
await all settlements after stream closure, reload projected history once
|
||||
before continuation. Stale `running` tools from a prior process fail
|
||||
durably on boot — abandoned side effects are never silently replayed.
|
||||
|
||||
Execution routing starts from only the session ID:
|
||||
`SessionExecution.resume → SessionStore.get → LocationServiceMap.get →
|
||||
SessionRunner.run`. Process-global coordinator serializes per-session
|
||||
drains; different sessions run concurrently.
|
||||
|
||||
### 3.3 Context Epochs
|
||||
|
||||
The exact privileged system context shown to the model persists as an
|
||||
immutable baseline plus a model-hidden snapshot of independently observed
|
||||
**Context Sources** (environment facts, date, global/upward AGENTS.md
|
||||
aggregate, agent skill guidance). Sources compose through a Location-
|
||||
scoped registry with stable namespaced keys; loaders return coherent
|
||||
typed values or explicitly Unavailable (stale-while-revalidate).
|
||||
|
||||
Changes reconcile only at **Safe Provider-Turn Boundaries** and commit as
|
||||
chronological Mid-Conversation System Messages, atomically advancing the
|
||||
snapshot. Compaction or session movement starts a fresh epoch with a
|
||||
freshly rendered baseline.
|
||||
|
||||
This is provenance made durable: the record of what the model was told is
|
||||
part of history, diffable, never silently mutated.
|
||||
|
||||
### 3.4 Compaction protocol
|
||||
|
||||
Budget check before each turn (context window minus reserved headroom).
|
||||
Compaction writes hidden `session.next.compaction.started/ended` events;
|
||||
only completion projects visible state (fresh baseline on next attempt).
|
||||
Overflow after durable output gets exactly one rebuild attempt, then
|
||||
terminal failure — no loops, no partial replay.
|
||||
|
||||
### 3.5 Delivery vocabulary
|
||||
|
||||
`steer` promotes at the next safe boundary mid-drain; `queue` waits until
|
||||
idle, then promotes exactly one and reevaluates. Promotion resets the
|
||||
agent's provider-turn allowance (once per batch).
|
||||
|
||||
### 3.6 Event sourcing
|
||||
|
||||
Durable `session.next.*` events with aggregate sequences; replay-and-tail
|
||||
cursor API (`sessions.events(after)`); finite history endpoint
|
||||
(`/session/:id/history`); projections reconstructable from events alone.
|
||||
Live deltas are ephemeral and never advance cursors.
|
||||
|
||||
---
|
||||
|
||||
## 4. Tools
|
||||
|
||||
One opaque `Tool.make({ description, input, output, execute, toModelOutput? })`.
|
||||
Codecs self-contained; single executor; registration-scoped overlay
|
||||
(process app tools under Location tools; latest active wins; closing
|
||||
reveals prior). Trusted built-ins capture Location services and formulate
|
||||
permission requests via PermissionV2 (`assert` policy + approval); the
|
||||
registry never injects permission helpers.
|
||||
|
||||
Settlement pipeline: resolve effective registration → decode input →
|
||||
invoke with runner-supplied context (`sessionID, agent, assistantMessageID,
|
||||
toolCallID`) → encode output → project model content → bound size (managed
|
||||
retention files for oversized output). Stale registrations reject calls
|
||||
without executing. Interruption cancels; it is never a tool result.
|
||||
|
||||
Built-ins: bounded `read` first (path resolution, escape rejection,
|
||||
authorization, paging), `bash` (host authority, unsandboxed, explicit
|
||||
external-directory checks), `apply_patch` (hunk parse, preflight,
|
||||
sequential commit, partial-application report).
|
||||
|
||||
---
|
||||
|
||||
## 5. Providers and Models
|
||||
|
||||
Catalog owns vendor/model data; providers are rows, models nested by
|
||||
provider (ids unique only within provider). Endpoint union:
|
||||
`openai/responses | openai/completions (+reasoning flavor) |
|
||||
anthropic/messages | aisdk:* | unknown(resolved from provider)`.
|
||||
|
||||
Plugin hooks (`provider.update`, `model.update`) with deterministic order
|
||||
(modelsDev 0 → env 10 → account 20 → provider 30 → config 40 → discovery
|
||||
50) carry enablement and option transformation; core stays a container.
|
||||
|
||||
**AMENDED — no hooks.** Hook architectures are the injection surface this
|
||||
project exists to eliminate; v2 does not reintroduce them. The catalog is
|
||||
a dumb container (`get / all / available / default`). Enablement and
|
||||
options are data — env entries, stored auth, config rows — read directly
|
||||
at composition time and merged in one visible place. Vendor-specific
|
||||
behavior is explicit registration code in the boot composition, ordered
|
||||
by its author, reviewable in one file. There are no callbacks, no cancel
|
||||
flags, no drafts, no trigger order to reason about.
|
||||
|
||||
Runner adaptation surface is narrow and explicit — responses-over-HTTP,
|
||||
completions chat (OpenAI + compatible), anthropic messages, and the three
|
||||
ai-sdk routes — everything else fails with `UnsupportedEndpointError`
|
||||
rather than degrading silently. WebSocket transport must not downgrade.
|
||||
|
||||
Neuron's own wire dialects (see §7) implement the HTTP surface directly.
|
||||
|
||||
## 6. Config (v2 review outcomes)
|
||||
|
||||
Keep: `$schema` (read-only), `shell`, `autoupdate` (global-only),
|
||||
`instructions` (ambient context sources).
|
||||
Redesign: `skills` → discovery-source array (local roots + remote URLs);
|
||||
`reference` → plural `references`.
|
||||
Remove: `logLevel`, `server` block, `command` (named workflows belong to
|
||||
skills). Legacy prompt shell expansion and per-command overrides are not
|
||||
ported; such capabilities belong to their owning domains.
|
||||
|
||||
## 7. Wire Dialects (Neuron-native transport)
|
||||
|
||||
No vendor SDK participates. One dialect per wire protocol, pure HTTP:
|
||||
|
||||
```
|
||||
dialects/
|
||||
types.ts Dialect contract: request() + stateful parser()
|
||||
anthropic.ts POST /v1/messages · x-api-key · content-block SSE
|
||||
openai.ts POST /v1/responses · Bearer · response.* SSE
|
||||
openai-compatible.ts /v1/chat/completions — every other vendor is a baseURL row
|
||||
registry.ts catalog api-name → dialect, generic fallback
|
||||
```
|
||||
|
||||
Transport: `dialect.request(...) → fetch → SSE lines → parser → normalized
|
||||
events`. Every byte on the wire is visible in this folder. This implements
|
||||
the catalog's endpoint union natively and replaces vendored SDK adapters
|
||||
turn by turn.
|
||||
|
||||
## 8. Orchestration (Rung 1+)
|
||||
|
||||
Six beats: ASSIGN → DECLARE → CLEARANCE → EXECUTE → REPORT → RECOVER.
|
||||
Dependency DAG is the concurrency model — no locks; conflicts are missing
|
||||
edges caught at clearance. Agents spin up threads, never other agents
|
||||
(spawn authority central, enforced at API shape). Cancellation tokens are
|
||||
durable control-topic entries: never missable, idempotent on replay.
|
||||
Every step reports exact token burn at the boundary; fan-out only when
|
||||
work comfortably exceeds coordination tax.
|
||||
|
||||
Cross-cutting concerns are pipeline stages and wire-taps manufactured by
|
||||
the bus itself (telemetry, authz stamps, metering, retry/dead-letter).
|
||||
Decorators wire, never work. Anything that matters crosses the bus.
|
||||
|
||||
**Composition is decoration, not repetition.** Instrumentation, tracing,
|
||||
guards, and subscriptions are decorators applied at the declaration site
|
||||
of a function — `@metered @traced @guard(policy)` around the work,
|
||||
`@on(address, filter)` for message handlers. Nobody writes subscribe()
|
||||
calls or log statements by hand a thousand times across files: if you
|
||||
find yourself doing that, the decorator doesn't exist yet — write it
|
||||
once, decorate everywhere. Hand-scattered plumbing is the smell; a named
|
||||
stage reused by decoration is the pattern.
|
||||
|
||||
## 9. Rungs
|
||||
|
||||
- **Rung 0 — Seed**: EventBus + DurableTopicLog + four verbs +
|
||||
conversation mapping live. Deliverable: context death loses nothing.
|
||||
Few hundred lines; more is smuggling.
|
||||
- **Rung 1 — Self-hosting agents**: tokens, plans/DAGs, decorators; the
|
||||
ORCHESTRATE → EXECUTE → LEARN → BUILD → REFINE kernel. Built by running
|
||||
Rung 0 sessions.
|
||||
- **Rung 2 — Orchestrator**: decomposition, clearance review, fan-out,
|
||||
economics break-evens.
|
||||
- **Rung 3 — Conversion campaigns**: existing application becomes work
|
||||
packages; module-by-module conversion through the pipeline; old tables
|
||||
demoted to views over envelopes, then deleted.
|
||||
|
||||
Never build a rung before standing on the previous one.
|
||||
|
||||
## 10. Current implementation map
|
||||
|
||||
| Piece | Where | State |
|
||||
|---|---|---|
|
||||
| Spec: bus/rungs | `specs/event-bus.md` | agreed |
|
||||
| Spec: session core | `specs/v2/session.md` | agreed |
|
||||
| Spec: tools / providers / config / todo | `specs/v2/*.md` | agreed |
|
||||
| Envelope schemas (planes 0–3) | `packages/core/src/event-bus-schema.ts` | implemented |
|
||||
| Bus wire + topic log | `packages/core/src/{event-bus,topic-log}.ts` | implemented |
|
||||
| Seed facade | `packages/core/src/seed.ts` | implemented |
|
||||
| Rung 1 kernel (5-beat cycle) | `runs/rung1/kernel.ts` + `main.ts` | implemented, evolving |
|
||||
| Wire dialects | `packages/neuron/src/provider/dialects/` | implemented |
|
||||
| Stream transport | `packages/neuron/src/llm/stream.ts` | implemented |
|
||||
| Runner / context sketches | `packages/neuron/src/session/` | drafts — superseded by this architecture; rewrite over Seed |
|
||||
| Node/edge store | — | designed (§2), not yet built |
|
||||
| TUI | `packages/tui` (front end unchanged) + `neuron-tui` (renderer fork) | link pending server contract |
|
||||
|
||||
## 11. Laws index
|
||||
|
||||
1. State = fold(append-only sequence). Append instead of edit — always.
|
||||
2. Headers inject-only; corrections are new injections with reasons.
|
||||
3. Payloads hold entities; edges are envelopes; collections are folds.
|
||||
4. Conversations chain from an opening turn; they never accumulate buckets.
|
||||
5. Memory is a node; retrieval replaces recollection; forgetting is explicit.
|
||||
6. Durable = a held reference exists; ephemeral = air.
|
||||
7. One explicit llm.stream per provider turn; admission ≠ execution.
|
||||
8. Provenance is durable: what the model was told is part of history.
|
||||
9. Decorators wire, never work. Anything that matters crosses the bus.
|
||||
10. Never build a rung before standing on the previous one.
|
||||
11. Events observe; they never intercept. If code must influence a
|
||||
mutation, it is written explicitly at the composition site — never
|
||||
as a hook. Hooks are eventing's idiotic cousin: control flow
|
||||
wearing a notification costume, with `cancel` flags as the
|
||||
confession.
|
||||
12. Interception is explicit or it does not exist. Want to intercept?
|
||||
Write an interceptor. Want to filter? Write a filter. Tap, meter,
|
||||
validate, dead-letter — each is a named, first-class pipeline stage,
|
||||
typed against its envelopes, composed at the assembly site with
|
||||
order chosen by its author. Hidden registration from anywhere is
|
||||
the thing we tore out; visible stages in one place are the thing
|
||||
we keep.
|
||||
13. Authentication is one thing; authorization is claims-based.
|
||||
Authentication verifies identity once at the gate — a single
|
||||
mechanism, never a subsystem family. Authorization evaluates
|
||||
**claims** carried by the principal: declarative statements
|
||||
(`org:x`, `scope:fs.write`, `role:orchestrator`) issued by trusted
|
||||
parties, riding as envelope headers, decided wherever needed
|
||||
13. Authentication is one thing; authorization is claims-based.
|
||||
Authentication verifies identity once at the gate — a single
|
||||
mechanism, never a subsystem family. Authorization evaluates
|
||||
**claims** carried by the principal. Claims are not a brittle,
|
||||
enumerated set — all kinds can be made: identity (`I am who I say
|
||||
I am` — the first, central claim), membership, scope, role,
|
||||
clearance, delegation, `spending_limit:5000`, anything an issuer
|
||||
dares assert. The kernel does not define what claims mean; issuers
|
||||
issue them, stages decide which ones they honor, and the graph
|
||||
remembers both. Claims are data; authority travels with the
|
||||
principal; the wire records the claim and the decision.
|
||||
Reference in New Issue
Block a user