neuron/specs/event-bus.md

SPEC: The EventBus Architecture

Status: DESIGN AGREED 2026-08-22 (whiteboard session, Will + agent) Next: Rung 0 implementation ("The Seed")

---

0. Thesis

The message is the entire application.

State is never stored; it is folded from append-only streams. Nothing is ever overwritten - messages append, headers inject, schemas coexist, artifacts supersede. There is no operation anywhere in the grammar whose input is "the past."

One idea, fractal at every scale: state = fold(append-only sequence).

---

1. The Bus

One generic EventBus. Process-global, dumb wire.

publish(topic, type, payload?)        // fire-and-forget
emit(topic, type, payload?)           // async, awaits handlers
subscribe(topic, handler) -> unsub    // handler receives Envelope
next(topic, filter?, signal?)         // await next matching envelope

• Topics are opaque strings. Callers own naming: orchestration IDs, session IDs, token IDs.
• Delivery is best-effort in-process. Durability belongs to whoever writes durable rows BEFORE publishing. The bus is not a database; the message stream IS the system of record.
• Transport-pluggable by design: same envelopes ride in-proc calls, IPC, queues, cloud endpoints, edge devices. Envelopes carry zero transport assumptions.

2. The Envelope (wire format)

Four schema planes, each evolving independently, all content-addressed:

Plane	What	Versioning
0	Envelope format itself (meta-schema)	pinned root: event-bus-envelope-v1; amend loudly
1	Header values	per-header SchemaRef
2	Body contract (type + artifact rules)	body-level ref
3	Payloads	per message-type ref; nested objects too

Schema refs are content-addressed (id = hash(definition)), optionally inline for small payloads. Schemas are durable first-class citizens: written once, immutable, resolvable dynamically by any participant. Old messages always decode. Replay works across versions, codebases, decades.

Envelope {
  topic      // durable handle
  seq        // position in topic's append-only sequence
  parentSeq? // causal spine - what this responds to/extends
  id         // hash of canonical form
  author     // token id / orchestrator id / stage name
  at         // epoch ms

  headers: [ { key, value, author, at, schema? } ]  // ORDERED LIST
                                                    // inject-only
  body {
    type           // e.g. "plan.step.completed"
    payloadJSON?   // decoded via payloadSchema
    payloadSchema? // Plane 3
    schema?        // Plane 2 (body contract)
    artifacts[]    // { ref, hash?, kind? } - evidence rides with record
  }
}

Header law:

• Ordered LIST (duplicates accumulate; order = causality).
• Inject-only. Modification/removal structurally impossible by API shape.
• current(key) = last-wins fold. history(key) = full evolution.
• Corrections are new injections with reasons, never edits.
• Relationship semantics live here; they are NOT static in the world - roles get revised by later injection, both readings stay true.

3. Topics

Topics declare their own persistence contract at creation:

• DURABLE: every envelope appended; replayable; recoverable.
• EPHEMERAL: delivery-only, never persisted; optional retainLast; optional ttl after zero subscribers.

Decision rule: if losing one envelope corrupts state or breaks recovery, it is durable. If its value expires after delivery, it is ephemeral.

Durable set (the ledger):

• orchestration.{id}.lifecycle created/decomposed/completed/cancelled
• orchestration.{id}.packages decomposition + dependency DAG + resources
• orchestration.{id}.clearance GO/no-go rulings, renegotiations
• agent.{token}.plan declared step plans
• agent.{token}.steps started/artifact/completed/failed (+token counts) = checkpoint journal AND economics ledger
• agent.{token}.control pause/resume/cancel. DURABLE on purpose: a cancellation must never evaporate. Idempotent receivers make replay safe.
• session.{id}.steer prompts admitted during active drains
• conversation topics a conversation IS a durable topic (see section 8)

Ephemeral set (the air):

• stream.{requestId} LLM token streams (retainLast: no)
• presence.{token} heartbeats; absence = crash signal
• progress.{orchestration} smoothed UI gauges (retainLast: yes)
• scratch.{pair} transient sibling-thread hand-offs

The durable set alone reconstructs everything. Ephemeral loss is never data loss.

4. Persistence ontology (four layers)

1. ENVELOPE STREAM    the state. The only reality. All durability here.
2. FOLDED VIEWS       derived state. Always recomputable from (1).
3. ARTIFACTS          materialized projections, content-addressed.
                      Live in the SAME store as the stream.
4. FILES ON DISK      exports of projections. Printouts, not truth.

Loss below layer 1 is rendering loss, recoverable by re-derivation. Artifacts EVOLVE BY SUPERSESSION: no overwrites ever. Forks are legal (multiple children of one parent); settling a fork = one group- supersession append naming the family. Git semantics emerge as a theorem.

The store of record holds: topic logs + artifact blobs + schema registry. Three things total. Everything else is derived, cached, or ephemeral.

5. Orchestration protocol (six beats)

1. ASSIGN      orchestrator -> agent: package + handle
2. DECLARE     agent plans its own steps (plan.declared), WAITS
3. CLEARANCE   orchestrator lays all plans side by side:
               resource conflicts? duplicate work? contradicting
               dependencies? sane estimates?
               -> GO per agent, or renegotiate while it's still data
4. EXECUTE     cleared agents run, publishing transitions upward
5. REPORT      continuous: status, artifacts, token burn
6. RECOVER     crash = silence. Last folded envelope = resume pointer.
               Completed steps' artifacts survive. Re-run only the
               in-flight step (steps should be idempotent).

Key properties:

• The dependency graph IS the concurrency model. No locks anywhere. Runnable = all edges satisfied. Parallelism discovered, not configured.
• Resource conflicts are missing edges; caught at CLEARANCE while cheap, serialized by injecting an edge.
• Orchestrator is planner + economist + router + reviewer + accountant + coroner. Just the one subscriber that sees every topic and folds.
• Agents plan their OWN work within packages; orchestrator reviews the whole board before any of it moves.
• Pausing/waiting-for-siblings costs nothing: a subscription with no matching events yet. Same mechanism as steer and cancel.

Agent spawning rule

Agents may spin up THREADS, never other AGENTS. Spawn authority stays central with the orchestrator. Enforced at API shape (agent handles expose spawnThread, not spawnAgent) plus runtime check.

Cancellation tokens

Every agent carries one; control messages address tokens directly. Pause/resume/cancel individual agents mid-flight without killing the orchestration. Tokens are durable-topics subscribers; commands cannot be missed, only late.

6. Economics

Every step reports exact token burn (tokensIn/tokensOut/toolCalls) in its completion envelopes - written at the boundary, not scraped from provider logs afterward.

Fan-out decision per package node:

• delegate only if work W comfortably exceeds coordination tax C (context injection + scaffolding + report-back + synthesis)
• sequential-if-chained: independence must buy wall-clock time
• DO-IT-MYSELF INLINE is a legitimate third option

Budgets watched live via fold; descope/collapse/cancel interventions happen mid-flight while cheap. Historical step-cost table accumulates into an empirical planner: which work types fan out profitably, which never do.

7. AOP and the seam

The bus MANUFACTURES the universal seam as a side effect of existing. All coordination crosses it; therefore all coordination is interceptible.

Cross-cutting concerns become pipeline stages / wire-taps, not scattered code:

• telemetry = Wire Tap subscriber on topic:*
• auth = filter stage between publish and deliver; decisions stamped into headers ({authz: granted, principal}) so audit trail IS the wire
• metering = header injection at publish boundaries
• retry, dead-lettering, validation, error-mapping = stackable stages

Decorator discipline (the anti-WCF rule): DECORATORS WIRE, NEVER WORK. Six-word vocabulary target: @agent @orchestrator @plan @step @on(stage) plus pipeline stages (@metered @guard). If logic creeps into a decorator, push it into the handler or the pipeline. No config files, no parallel configuration universe - declaration lives on the thing it declares.

Participation rule: anything that matters crosses the bus; internals are free. Cross a boundary naked and you are unmeasured, unaudited, uncancellable.

8. Conversation is a topic

A conversation with an assistant is an orchestration:

• human publishes assignment envelopes (prompts = steers)
• agent publishes plan/step/artifact/completion envelopes
• tool calls are threads under the agent's token
• context window = ephemeral cache; the topic log = durable truth
• compaction = folding the log, not summarizing away history
• retrieval replaces recollection: fetch slices by topic/type/time/ causality instead of re-injecting whole conversations
• crash/context-death = silence; next instance folds and resumes

This makes sessions deathless: handoff rituals dissolve because the log already contains everything.

9. Backlogs and projects (fold patterns, not entities)

There are no entities with fields - only streams and agreed folds. A backlog item is a fold pattern over intention/work envelopes on project.{name}.backlog.

Reading conventions (vocabulary, not schema):

• proposed -> ready -> blocked -> active -> done | dropped (folded)
• dependsOn: read from envelopes; blocked iff any dependency not done
• artifacts: linked by publication + injected {item} headers; roles evolve by revision-injection (spec -> evidence etc.)
• graduation: activating an item mints an orchestration id; the item folds to done when that topic completes

Relationship law: RELATIONSHIPS ARE READ FROM HISTORY, NOT WRITTEN AS RECORDS. The causal spine is the graph. Edge types come from the vocabulary of envelope types and header keys (caused-by=parentSeq, authored-by=author, lives-in=topic, produced=artifacts, supersedes= supersede envelopes, regards=injected headers). New relationship kinds = new vocabulary + a fold. The graph grows by vocabulary, never migration.

Payload law: PAYLOADS HOLD ENTITIES; EDGES ARE ENVELOPES; COLLECTIONS ARE ALWAYS FOLDS. Any list-of-references field inside a payload is the smell.

10. Bootstrap ladder

RUNG 0 - THE SEED (first build; smallest thing two people can run)

• EventBus (exists) + DurableTopicLog: single owned SQLite store, content-addressed envelopes exactly per section 2, fold/replay, four query verbs: by-topic, by-type, by-time-range, walk-by-causality
• Conversation mapping live: our own working sessions run as orchestrations. Deliverable proof: context death loses nothing.
• Discipline: if Rung 0 needs more than a few hundred lines, it is smuggling Rung 1 concerns.

RUNG 1 - SELF-HOSTING AGENTS

• tokens, plans/DAGs, decorators. Built BY running Rung 0 sessions. Development history written in the pipeline from here on.

RUNG 2 - THE ORCHESTRATOR

• decomposition, clearance review, fan-out, token-economics break-evens. Built through Rung 1 sessions; multiple parallel workstreams coordinating their own construction.

RUNG 3 - CONVERSION CAMPAIGNS

• enumerate the existing application as work packages; convert module by module THROUGH the pipeline. Existing tables demoted to views over envelopes, then deleted. retry.ts deleted (aspect replaces it).

Discipline: never build a rung before standing on the previous one. No speculative features; each layer earns the next by being used.

11. First artifacts

• citizen zero: this spec, stored as an artifact in the Seed's own store at Rung 0 completion, superseded (never edited) thereafter.
• backlog seed: project.neuron items = Rungs 1-3; project.valley items = pinned investigation threads (ishikawa continuation, testimonies held open).

---

Appendix: lineage (nothing here is invented)

Email Received: chains -> inject-only headers Hohpe/Woolf EIP -> Process Manager = orchestrator, Scatter-Gather = fan-out+fold, Claim Check = artifact refs, Control Bus = cancellation tokens, Wire Tap = telemetry, Dead Letter Channel = failed agents WCF channel stack -> interceptible pipeline; kept the structure, refused the config church git -> fork/supersede artifact semantics Event sourcing -> the entire persistence ontology "The world changes; only liars rewrite. Append instead."