feat(core): event bus seed - envelope schema, durable topic log, OELBR kernel
- EventBus: dumb wire, publish/subscribe, failures surface - Envelope: four schema planes, inject-only headers, content-addressed - TopicLog: owned SQLite ledger, replay + by-type/time/causality - Seed facade: durable=ledger+wire, ephemeral=wire only, fold-for-state - Rung 1 kernel: OELBR loop, DAG plans, durable cancellation tokens - Neuron v0 executable spec source; specs/event-bus.md citizen zero - Standing orders: design-before-code doctrine installed
This commit is contained in:
@@ -0,0 +1,310 @@
|
||||
# SPEC: The EventBus Architecture
|
||||
Status: DESIGN AGREED 2026-08-22 (whiteboard session, Will + agent)
|
||||
Next: Rung 0 implementation ("The Seed")
|
||||
|
||||
---
|
||||
|
||||
## 0. Thesis
|
||||
|
||||
The message is the entire application.
|
||||
|
||||
State is never stored; it is folded from append-only streams. Nothing is
|
||||
ever overwritten - messages append, headers inject, schemas coexist,
|
||||
artifacts supersede. There is no operation anywhere in the grammar whose
|
||||
input is "the past."
|
||||
|
||||
One idea, fractal at every scale: **state = fold(append-only sequence).**
|
||||
|
||||
---
|
||||
|
||||
## 1. The Bus
|
||||
|
||||
One generic EventBus. Process-global, dumb wire.
|
||||
|
||||
```ts
|
||||
publish(topic, type, payload?) // fire-and-forget
|
||||
emit(topic, type, payload?) // async, awaits handlers
|
||||
subscribe(topic, handler) -> unsub // handler receives Envelope
|
||||
next(topic, filter?, signal?) // await next matching envelope
|
||||
```
|
||||
|
||||
- Topics are opaque strings. Callers own naming: orchestration IDs,
|
||||
session IDs, token IDs.
|
||||
- Delivery is best-effort in-process. Durability belongs to whoever
|
||||
writes durable rows BEFORE publishing. The bus is not a database;
|
||||
the message stream IS the system of record.
|
||||
- Transport-pluggable by design: same envelopes ride in-proc calls,
|
||||
IPC, queues, cloud endpoints, edge devices. Envelopes carry zero
|
||||
transport assumptions.
|
||||
|
||||
## 2. The Envelope (wire format)
|
||||
|
||||
Four schema planes, each evolving independently, all content-addressed:
|
||||
|
||||
| Plane | What | Versioning |
|
||||
|---|---|---|
|
||||
| 0 | Envelope format itself (meta-schema) | pinned root: `event-bus-envelope-v1`; amend loudly |
|
||||
| 1 | Header values | per-header SchemaRef |
|
||||
| 2 | Body contract (type + artifact rules) | body-level ref |
|
||||
| 3 | Payloads | per message-type ref; nested objects too |
|
||||
|
||||
Schema refs are content-addressed (`id = hash(definition)`), optionally
|
||||
inline for small payloads. Schemas are durable first-class citizens:
|
||||
written once, immutable, resolvable dynamically by any participant.
|
||||
Old messages always decode. Replay works across versions, codebases,
|
||||
decades.
|
||||
|
||||
```ts
|
||||
Envelope {
|
||||
topic // durable handle
|
||||
seq // position in topic's append-only sequence
|
||||
parentSeq? // causal spine - what this responds to/extends
|
||||
id // hash of canonical form
|
||||
author // token id / orchestrator id / stage name
|
||||
at // epoch ms
|
||||
|
||||
headers: [ { key, value, author, at, schema? } ] // ORDERED LIST
|
||||
// inject-only
|
||||
body {
|
||||
type // e.g. "plan.step.completed"
|
||||
payloadJSON? // decoded via payloadSchema
|
||||
payloadSchema? // Plane 3
|
||||
schema? // Plane 2 (body contract)
|
||||
artifacts[] // { ref, hash?, kind? } - evidence rides with record
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Header law:
|
||||
- Ordered LIST (duplicates accumulate; order = causality).
|
||||
- Inject-only. Modification/removal structurally impossible by API shape.
|
||||
- `current(key)` = last-wins fold. `history(key)` = full evolution.
|
||||
- Corrections are new injections with reasons, never edits.
|
||||
- Relationship semantics live here; they are NOT static in the world -
|
||||
roles get revised by later injection, both readings stay true.
|
||||
|
||||
## 3. Topics
|
||||
|
||||
Topics declare their own persistence contract at creation:
|
||||
|
||||
- DURABLE: every envelope appended; replayable; recoverable.
|
||||
- EPHEMERAL: delivery-only, never persisted; optional retainLast;
|
||||
optional ttl after zero subscribers.
|
||||
|
||||
Decision rule: if losing one envelope corrupts state or breaks recovery,
|
||||
it is durable. If its value expires after delivery, it is ephemeral.
|
||||
|
||||
Durable set (the ledger):
|
||||
- `orchestration.{id}.lifecycle` created/decomposed/completed/cancelled
|
||||
- `orchestration.{id}.packages` decomposition + dependency DAG + resources
|
||||
- `orchestration.{id}.clearance` GO/no-go rulings, renegotiations
|
||||
- `agent.{token}.plan` declared step plans
|
||||
- `agent.{token}.steps` started/artifact/completed/failed
|
||||
(+token counts) = checkpoint journal
|
||||
AND economics ledger
|
||||
- `agent.{token}.control` pause/resume/cancel. DURABLE on purpose:
|
||||
a cancellation must never evaporate.
|
||||
Idempotent receivers make replay safe.
|
||||
- `session.{id}.steer` prompts admitted during active drains
|
||||
- conversation topics a conversation IS a durable topic
|
||||
(see section 8)
|
||||
|
||||
Ephemeral set (the air):
|
||||
- `stream.{requestId}` LLM token streams (retainLast: no)
|
||||
- `presence.{token}` heartbeats; absence = crash signal
|
||||
- `progress.{orchestration}` smoothed UI gauges (retainLast: yes)
|
||||
- `scratch.{pair}` transient sibling-thread hand-offs
|
||||
|
||||
The durable set alone reconstructs everything. Ephemeral loss is never
|
||||
data loss.
|
||||
|
||||
## 4. Persistence ontology (four layers)
|
||||
|
||||
```
|
||||
1. ENVELOPE STREAM the state. The only reality. All durability here.
|
||||
2. FOLDED VIEWS derived state. Always recomputable from (1).
|
||||
3. ARTIFACTS materialized projections, content-addressed.
|
||||
Live in the SAME store as the stream.
|
||||
4. FILES ON DISK exports of projections. Printouts, not truth.
|
||||
```
|
||||
|
||||
Loss below layer 1 is rendering loss, recoverable by re-derivation.
|
||||
Artifacts EVOLVE BY SUPERSESSION: no overwrites ever. Forks are legal
|
||||
(multiple children of one parent); settling a fork = one group-
|
||||
supersession append naming the family. Git semantics emerge as a theorem.
|
||||
|
||||
The store of record holds: topic logs + artifact blobs + schema registry.
|
||||
Three things total. Everything else is derived, cached, or ephemeral.
|
||||
|
||||
## 5. Orchestration protocol (six beats)
|
||||
|
||||
```
|
||||
1. ASSIGN orchestrator -> agent: package + handle
|
||||
2. DECLARE agent plans its own steps (plan.declared), WAITS
|
||||
3. CLEARANCE orchestrator lays all plans side by side:
|
||||
resource conflicts? duplicate work? contradicting
|
||||
dependencies? sane estimates?
|
||||
-> GO per agent, or renegotiate while it's still data
|
||||
4. EXECUTE cleared agents run, publishing transitions upward
|
||||
5. REPORT continuous: status, artifacts, token burn
|
||||
6. RECOVER crash = silence. Last folded envelope = resume pointer.
|
||||
Completed steps' artifacts survive. Re-run only the
|
||||
in-flight step (steps should be idempotent).
|
||||
```
|
||||
|
||||
Key properties:
|
||||
- The dependency graph IS the concurrency model. No locks anywhere.
|
||||
Runnable = all edges satisfied. Parallelism discovered, not configured.
|
||||
- Resource conflicts are missing edges; caught at CLEARANCE while
|
||||
cheap, serialized by injecting an edge.
|
||||
- Orchestrator is planner + economist + router + reviewer + accountant +
|
||||
coroner. Just the one subscriber that sees every topic and folds.
|
||||
- Agents plan their OWN work within packages; orchestrator reviews the
|
||||
whole board before any of it moves.
|
||||
- Pausing/waiting-for-siblings costs nothing: a subscription with no
|
||||
matching events yet. Same mechanism as steer and cancel.
|
||||
|
||||
### Agent spawning rule
|
||||
Agents may spin up THREADS, never other AGENTS. Spawn authority stays
|
||||
central with the orchestrator. Enforced at API shape (agent handles
|
||||
expose spawnThread, not spawnAgent) plus runtime check.
|
||||
|
||||
### Cancellation tokens
|
||||
Every agent carries one; control messages address tokens directly.
|
||||
Pause/resume/cancel individual agents mid-flight without killing the
|
||||
orchestration. Tokens are durable-topics subscribers; commands cannot
|
||||
be missed, only late.
|
||||
|
||||
## 6. Economics
|
||||
|
||||
Every step reports exact token burn (tokensIn/tokensOut/toolCalls)
|
||||
in its completion envelopes - written at the boundary, not scraped
|
||||
from provider logs afterward.
|
||||
|
||||
Fan-out decision per package node:
|
||||
- delegate only if work W comfortably exceeds coordination tax C
|
||||
(context injection + scaffolding + report-back + synthesis)
|
||||
- sequential-if-chained: independence must buy wall-clock time
|
||||
- DO-IT-MYSELF INLINE is a legitimate third option
|
||||
|
||||
Budgets watched live via fold; descope/collapse/cancel interventions
|
||||
happen mid-flight while cheap. Historical step-cost table accumulates
|
||||
into an empirical planner: which work types fan out profitably, which
|
||||
never do.
|
||||
|
||||
## 7. AOP and the seam
|
||||
|
||||
The bus MANUFACTURES the universal seam as a side effect of existing.
|
||||
All coordination crosses it; therefore all coordination is interceptible.
|
||||
|
||||
Cross-cutting concerns become pipeline stages / wire-taps, not scattered
|
||||
code:
|
||||
- telemetry = Wire Tap subscriber on topic:*
|
||||
- auth = filter stage between publish and deliver; decisions
|
||||
stamped into headers ({authz: granted, principal}) so
|
||||
audit trail IS the wire
|
||||
- metering = header injection at publish boundaries
|
||||
- retry, dead-lettering, validation, error-mapping = stackable stages
|
||||
|
||||
Decorator discipline (the anti-WCF rule): DECORATORS WIRE, NEVER WORK.
|
||||
Six-word vocabulary target: @agent @orchestrator @plan @step @on(stage)
|
||||
plus pipeline stages (@metered @guard). If logic creeps into a decorator,
|
||||
push it into the handler or the pipeline. No config files, no parallel
|
||||
configuration universe - declaration lives on the thing it declares.
|
||||
|
||||
Participation rule: anything that matters crosses the bus; internals are
|
||||
free. Cross a boundary naked and you are unmeasured, unaudited,
|
||||
uncancellable.
|
||||
|
||||
## 8. Conversation is a topic
|
||||
|
||||
A conversation with an assistant is an orchestration:
|
||||
- human publishes assignment envelopes (prompts = steers)
|
||||
- agent publishes plan/step/artifact/completion envelopes
|
||||
- tool calls are threads under the agent's token
|
||||
- context window = ephemeral cache; the topic log = durable truth
|
||||
- compaction = folding the log, not summarizing away history
|
||||
- retrieval replaces recollection: fetch slices by topic/type/time/
|
||||
causality instead of re-injecting whole conversations
|
||||
- crash/context-death = silence; next instance folds and resumes
|
||||
|
||||
This makes sessions deathless: handoff rituals dissolve because the
|
||||
log already contains everything.
|
||||
|
||||
## 9. Backlogs and projects (fold patterns, not entities)
|
||||
|
||||
There are no entities with fields - only streams and agreed folds.
|
||||
A backlog item is a fold pattern over intention/work envelopes on
|
||||
`project.{name}.backlog`.
|
||||
|
||||
Reading conventions (vocabulary, not schema):
|
||||
- proposed -> ready -> blocked -> active -> done | dropped (folded)
|
||||
- dependsOn: read from envelopes; blocked iff any dependency not done
|
||||
- artifacts: linked by publication + injected {item} headers; roles
|
||||
evolve by revision-injection (spec -> evidence etc.)
|
||||
- graduation: activating an item mints an orchestration id; the item
|
||||
folds to done when that topic completes
|
||||
|
||||
Relationship law: RELATIONSHIPS ARE READ FROM HISTORY, NOT WRITTEN AS
|
||||
RECORDS. The causal spine is the graph. Edge types come from the
|
||||
vocabulary of envelope types and header keys (caused-by=parentSeq,
|
||||
authored-by=author, lives-in=topic, produced=artifacts, supersedes=
|
||||
supersede envelopes, regards=injected headers). New relationship kinds =
|
||||
new vocabulary + a fold. The graph grows by vocabulary, never migration.
|
||||
|
||||
Payload law: PAYLOADS HOLD ENTITIES; EDGES ARE ENVELOPES; COLLECTIONS
|
||||
ARE ALWAYS FOLDS. Any list-of-references field inside a payload is the
|
||||
smell.
|
||||
|
||||
## 10. Bootstrap ladder
|
||||
|
||||
RUNG 0 - THE SEED (first build; smallest thing two people can run)
|
||||
- EventBus (exists) + DurableTopicLog: single owned SQLite store,
|
||||
content-addressed envelopes exactly per section 2, fold/replay,
|
||||
four query verbs: by-topic, by-type, by-time-range, walk-by-causality
|
||||
- Conversation mapping live: our own working sessions run as
|
||||
orchestrations. Deliverable proof: context death loses nothing.
|
||||
- Discipline: if Rung 0 needs more than a few hundred lines, it is
|
||||
smuggling Rung 1 concerns.
|
||||
|
||||
RUNG 1 - SELF-HOSTING AGENTS
|
||||
- tokens, plans/DAGs, decorators. Built BY running Rung 0 sessions.
|
||||
Development history written in the pipeline from here on.
|
||||
|
||||
RUNG 2 - THE ORCHESTRATOR
|
||||
- decomposition, clearance review, fan-out, token-economics break-evens.
|
||||
Built through Rung 1 sessions; multiple parallel workstreams
|
||||
coordinating their own construction.
|
||||
|
||||
RUNG 3 - CONVERSION CAMPAIGNS
|
||||
- enumerate the existing application as work packages; convert module
|
||||
by module THROUGH the pipeline. Existing tables demoted to views over
|
||||
envelopes, then deleted. retry.ts deleted (aspect replaces it).
|
||||
|
||||
Discipline: never build a rung before standing on the previous one.
|
||||
No speculative features; each layer earns the next by being used.
|
||||
|
||||
## 11. First artifacts
|
||||
|
||||
- citizen zero: this spec, stored as an artifact in the Seed's own
|
||||
store at Rung 0 completion, superseded (never edited) thereafter.
|
||||
- backlog seed: project.neuron items = Rungs 1-3; project.valley items
|
||||
= pinned investigation threads (ishikawa continuation, testimonies
|
||||
held open).
|
||||
|
||||
---
|
||||
|
||||
## Appendix: lineage (nothing here is invented)
|
||||
|
||||
Email Received: chains -> inject-only headers
|
||||
Hohpe/Woolf EIP -> Process Manager = orchestrator,
|
||||
Scatter-Gather = fan-out+fold,
|
||||
Claim Check = artifact refs,
|
||||
Control Bus = cancellation tokens,
|
||||
Wire Tap = telemetry,
|
||||
Dead Letter Channel = failed agents
|
||||
WCF channel stack -> interceptible pipeline; kept the
|
||||
structure, refused the config church
|
||||
git -> fork/supersede artifact semantics
|
||||
Event sourcing -> the entire persistence ontology
|
||||
"The world changes; only liars rewrite. Append instead."
|
||||
Reference in New Issue
Block a user