neuron/specs/event-bus.md

# SPEC: The EventBus Architecture
Status: DESIGN AGREED 2026-08-22 (whiteboard session, Will + agent)
Next: Rung 0 implementation ("The Seed")

---

## 0. Thesis

The message is the entire application.

State is never stored; it is folded from append-only streams. Nothing is
ever overwritten - messages append, headers inject, schemas coexist,
artifacts supersede. There is no operation anywhere in the grammar whose
input is "the past."

One idea, fractal at every scale: **state = fold(append-only sequence).**

---

## 1. The Bus

One generic EventBus. Process-global, dumb wire.

```ts
publish(topic, type, payload?)        // fire-and-forget
emit(topic, type, payload?)           // async, awaits handlers
subscribe(topic, handler) -> unsub    // handler receives Envelope
next(topic, filter?, signal?)         // await next matching envelope
```

- Topics are opaque strings. Callers own naming: orchestration IDs,
  session IDs, token IDs.
- Delivery is best-effort in-process. Durability belongs to whoever
  writes durable rows BEFORE publishing. The bus is not a database;
  the message stream IS the system of record.
- Transport-pluggable by design: same envelopes ride in-proc calls,
  IPC, queues, cloud endpoints, edge devices. Envelopes carry zero
  transport assumptions.

## 2. The Envelope (wire format)

Four schema planes, each evolving independently, all content-addressed:

| Plane | What | Versioning |
|---|---|---|
| 0 | Envelope format itself (meta-schema) | pinned root: `event-bus-envelope-v1`; amend loudly |
| 1 | Header values | per-header SchemaRef |
| 2 | Body contract (type + artifact rules) | body-level ref |
| 3 | Payloads | per message-type ref; nested objects too |

Schema refs are content-addressed (`id = hash(definition)`), optionally
inline for small payloads. Schemas are durable first-class citizens:
written once, immutable, resolvable dynamically by any participant.
Old messages always decode. Replay works across versions, codebases,
decades.

```ts
Envelope {
  topic      // durable handle
  seq        // position in topic's append-only sequence
  parentSeq? // causal spine - what this responds to/extends
  id         // hash of canonical form
  author     // token id / orchestrator id / stage name
  at         // epoch ms

  headers: [ { key, value, author, at, schema? } ]  // ORDERED LIST
                                                    // inject-only
  body {
    type           // e.g. "plan.step.completed"
    payloadJSON?   // decoded via payloadSchema
    payloadSchema? // Plane 3
    schema?        // Plane 2 (body contract)
    artifacts[]    // { ref, hash?, kind? } - evidence rides with record
  }
}
```

Header law:
- Ordered LIST (duplicates accumulate; order = causality).
- Inject-only. Modification/removal structurally impossible by API shape.
- `current(key)` = last-wins fold. `history(key)` = full evolution.
- Corrections are new injections with reasons, never edits.
- Relationship semantics live here; they are NOT static in the world -
  roles get revised by later injection, both readings stay true.

## 3. Topics

Topics declare their own persistence contract at creation:

- DURABLE: every envelope appended; replayable; recoverable.
- EPHEMERAL: delivery-only, never persisted; optional retainLast;
  optional ttl after zero subscribers.

Decision rule: if losing one envelope corrupts state or breaks recovery,
it is durable. If its value expires after delivery, it is ephemeral.

Durable set (the ledger):
- `orchestration.{id}.lifecycle`   created/decomposed/completed/cancelled
- `orchestration.{id}.packages`    decomposition + dependency DAG + resources
- `orchestration.{id}.clearance`   GO/no-go rulings, renegotiations
- `agent.{token}.plan`             declared step plans
- `agent.{token}.steps`            started/artifact/completed/failed
                                   (+token counts) = checkpoint journal
                                   AND economics ledger
- `agent.{token}.control`          pause/resume/cancel. DURABLE on purpose:
                                   a cancellation must never evaporate.
                                   Idempotent receivers make replay safe.
- `session.{id}.steer`             prompts admitted during active drains
- conversation topics              a conversation IS a durable topic
                                   (see section 8)

Ephemeral set (the air):
- `stream.{requestId}`             LLM token streams (retainLast: no)
- `presence.{token}`               heartbeats; absence = crash signal
- `progress.{orchestration}`       smoothed UI gauges (retainLast: yes)
- `scratch.{pair}`                 transient sibling-thread hand-offs

The durable set alone reconstructs everything. Ephemeral loss is never
data loss.

## 4. Persistence ontology (four layers)

```
1. ENVELOPE STREAM    the state. The only reality. All durability here.
2. FOLDED VIEWS       derived state. Always recomputable from (1).
3. ARTIFACTS          materialized projections, content-addressed.
                      Live in the SAME store as the stream.
4. FILES ON DISK      exports of projections. Printouts, not truth.
```

Loss below layer 1 is rendering loss, recoverable by re-derivation.
Artifacts EVOLVE BY SUPERSESSION: no overwrites ever. Forks are legal
(multiple children of one parent); settling a fork = one group-
supersession append naming the family. Git semantics emerge as a theorem.

The store of record holds: topic logs + artifact blobs + schema registry.
Three things total. Everything else is derived, cached, or ephemeral.

## 5. Orchestration protocol (six beats)

```
1. ASSIGN      orchestrator -> agent: package + handle
2. DECLARE     agent plans its own steps (plan.declared), WAITS
3. CLEARANCE   orchestrator lays all plans side by side:
               resource conflicts? duplicate work? contradicting
               dependencies? sane estimates?
               -> GO per agent, or renegotiate while it's still data
4. EXECUTE     cleared agents run, publishing transitions upward
5. REPORT      continuous: status, artifacts, token burn
6. RECOVER     crash = silence. Last folded envelope = resume pointer.
               Completed steps' artifacts survive. Re-run only the
               in-flight step (steps should be idempotent).
```

Key properties:
- The dependency graph IS the concurrency model. No locks anywhere.
  Runnable = all edges satisfied. Parallelism discovered, not configured.
- Resource conflicts are missing edges; caught at CLEARANCE while
  cheap, serialized by injecting an edge.
- Orchestrator is planner + economist + router + reviewer + accountant +
  coroner. Just the one subscriber that sees every topic and folds.
- Agents plan their OWN work within packages; orchestrator reviews the
  whole board before any of it moves.
- Pausing/waiting-for-siblings costs nothing: a subscription with no
  matching events yet. Same mechanism as steer and cancel.

### Agent spawning rule
Agents may spin up THREADS, never other AGENTS. Spawn authority stays
central with the orchestrator. Enforced at API shape (agent handles
expose spawnThread, not spawnAgent) plus runtime check.

### Cancellation tokens
Every agent carries one; control messages address tokens directly.
Pause/resume/cancel individual agents mid-flight without killing the
orchestration. Tokens are durable-topics subscribers; commands cannot
be missed, only late.

## 6. Economics

Every step reports exact token burn (tokensIn/tokensOut/toolCalls)
in its completion envelopes - written at the boundary, not scraped
from provider logs afterward.

Fan-out decision per package node:
- delegate only if work W comfortably exceeds coordination tax C
  (context injection + scaffolding + report-back + synthesis)
- sequential-if-chained: independence must buy wall-clock time
- DO-IT-MYSELF INLINE is a legitimate third option

Budgets watched live via fold; descope/collapse/cancel interventions
happen mid-flight while cheap. Historical step-cost table accumulates
into an empirical planner: which work types fan out profitably, which
never do.

## 7. AOP and the seam

The bus MANUFACTURES the universal seam as a side effect of existing.
All coordination crosses it; therefore all coordination is interceptible.

Cross-cutting concerns become pipeline stages / wire-taps, not scattered
code:
- telemetry   = Wire Tap subscriber on topic:*
- auth        = filter stage between publish and deliver; decisions
                stamped into headers ({authz: granted, principal}) so
                audit trail IS the wire
- metering    = header injection at publish boundaries
- retry, dead-lettering, validation, error-mapping = stackable stages

Decorator discipline (the anti-WCF rule): DECORATORS WIRE, NEVER WORK.
Six-word vocabulary target: @agent @orchestrator @plan @step @on(stage)
plus pipeline stages (@metered @guard). If logic creeps into a decorator,
push it into the handler or the pipeline. No config files, no parallel
configuration universe - declaration lives on the thing it declares.

Participation rule: anything that matters crosses the bus; internals are
free. Cross a boundary naked and you are unmeasured, unaudited,
uncancellable.

## 8. Conversation is a topic

A conversation with an assistant is an orchestration:
- human publishes assignment envelopes (prompts = steers)
- agent publishes plan/step/artifact/completion envelopes
- tool calls are threads under the agent's token
- context window = ephemeral cache; the topic log = durable truth
- compaction = folding the log, not summarizing away history
- retrieval replaces recollection: fetch slices by topic/type/time/
  causality instead of re-injecting whole conversations
- crash/context-death = silence; next instance folds and resumes

This makes sessions deathless: handoff rituals dissolve because the
log already contains everything.

## 9. Backlogs and projects (fold patterns, not entities)

There are no entities with fields - only streams and agreed folds.
A backlog item is a fold pattern over intention/work envelopes on
`project.{name}.backlog`.

Reading conventions (vocabulary, not schema):
- proposed -> ready -> blocked -> active -> done | dropped (folded)
- dependsOn: read from envelopes; blocked iff any dependency not done
- artifacts: linked by publication + injected {item} headers; roles
  evolve by revision-injection (spec -> evidence etc.)
- graduation: activating an item mints an orchestration id; the item
  folds to done when that topic completes

Relationship law: RELATIONSHIPS ARE READ FROM HISTORY, NOT WRITTEN AS
RECORDS. The causal spine is the graph. Edge types come from the
vocabulary of envelope types and header keys (caused-by=parentSeq,
authored-by=author, lives-in=topic, produced=artifacts, supersedes=
supersede envelopes, regards=injected headers). New relationship kinds =
new vocabulary + a fold. The graph grows by vocabulary, never migration.

Payload law: PAYLOADS HOLD ENTITIES; EDGES ARE ENVELOPES; COLLECTIONS
ARE ALWAYS FOLDS. Any list-of-references field inside a payload is the
smell.

## 10. Bootstrap ladder

RUNG 0 - THE SEED (first build; smallest thing two people can run)
- EventBus (exists) + DurableTopicLog: single owned SQLite store,
  content-addressed envelopes exactly per section 2, fold/replay,
  four query verbs: by-topic, by-type, by-time-range, walk-by-causality
- Conversation mapping live: our own working sessions run as
  orchestrations. Deliverable proof: context death loses nothing.
- Discipline: if Rung 0 needs more than a few hundred lines, it is
  smuggling Rung 1 concerns.

RUNG 1 - SELF-HOSTING AGENTS
- tokens, plans/DAGs, decorators. Built BY running Rung 0 sessions.
  Development history written in the pipeline from here on.

RUNG 2 - THE ORCHESTRATOR
- decomposition, clearance review, fan-out, token-economics break-evens.
  Built through Rung 1 sessions; multiple parallel workstreams
  coordinating their own construction.

RUNG 3 - CONVERSION CAMPAIGNS
- enumerate the existing application as work packages; convert module
  by module THROUGH the pipeline. Existing tables demoted to views over
  envelopes, then deleted. retry.ts deleted (aspect replaces it).

Discipline: never build a rung before standing on the previous one.
No speculative features; each layer earns the next by being used.

## 11. First artifacts

- citizen zero: this spec, stored as an artifact in the Seed's own
  store at Rung 0 completion, superseded (never edited) thereafter.
- backlog seed: project.neuron items = Rungs 1-3; project.valley items
  = pinned investigation threads (ishikawa continuation, testimonies
  held open).

---

## Appendix: lineage (nothing here is invented)

Email Received: chains            -> inject-only headers
Hohpe/Woolf EIP                   -> Process Manager = orchestrator,
                                     Scatter-Gather = fan-out+fold,
                                     Claim Check = artifact refs,
                                     Control Bus = cancellation tokens,
                                     Wire Tap = telemetry,
                                     Dead Letter Channel = failed agents
WCF channel stack                 -> interceptible pipeline; kept the
                                     structure, refused the config church
git                               -> fork/supersede artifact semantics
Event sourcing                    -> the entire persistence ontology
"The world changes; only liars rewrite. Append instead."