Land immutability arc + CI soul-contract gate on main #83

Merged
will.anderson merged 4 commits from feat/immutable-engram-deletes into main 2026-07-18 18:33:12 +00:00
Owner

What

Land the complete fixed soul on main: all of the July hotfix/elc-source-typos work (soul.c regen, route/safety fixes) plus the full immutability arc, and gate the Linux soul in CI so a stale/destructive soul can never publish again.

main (42bbadc, June 22) is a clean ancestor of this branch — conflict-free.

Included

  • hotfix July work (via the shared base @ 1011d8e): corrected dist/soul.c, route/safety fixes, Track B.
  • Immutability arc (d337fcb, f3034d2, 40d8001): update supersedes-with-edge; delete/forget tombstone (keep node + edges, hide from default reads, recoverable via ?include_deleted=1) — across memory/delete, node/delete, node/update, the generic /memory/forget, the cultivate forget op, the autonomous awareness forget action, and the MCP forget tool. No forget path calls engram_forget anymore. Both ship paths regenerated: dist/soul.c (single-TU) + per-module dist/{memory,awareness,neuron-api}.c.
  • CI gate (290a637): ci.yaml now runs scripts/verify-soul-contract.sh against dist/neuron as a hard block between the build and Publish — PRESENCE (every app route answered) + IMMUTABILITY (no hard-delete). Non-zero fails the build, so neuron-soul can't publish to foundation-prod or blue-green deploy to GKE with a broken soul.

Verified

macOS build of this exact soul.c passes the gate: PRESENCE all 27, IMMUTABILITY 5/5 KEPT (memory-update, memory-delete, node-update, node-delete, memory-forget). The Linux gate will run in this PR's CI.

Merging triggers the Linux soul rebuild + GKE blue-green redeploy of neuron-mcp (expected/approved). Touches only the neuron-mcp path — not the marketing/download site (separate Cloud Run).

## What Land the complete fixed soul on `main`: all of the July `hotfix/elc-source-typos` work (soul.c regen, route/safety fixes) **plus** the full immutability arc, and gate the Linux soul in CI so a stale/destructive soul can never publish again. `main` (42bbadc, June 22) is a clean ancestor of this branch — conflict-free. ## Included - **hotfix July work** (via the shared base @ 1011d8e): corrected `dist/soul.c`, route/safety fixes, Track B. - **Immutability arc** (d337fcb, f3034d2, 40d8001): `update` supersedes-with-edge; `delete`/`forget` **tombstone** (keep node + edges, hide from default reads, recoverable via `?include_deleted=1`) — across `memory/delete`, `node/delete`, `node/update`, the generic `/memory/forget`, the cultivate `forget` op, the autonomous awareness forget action, and the MCP `forget` tool. No forget path calls `engram_forget` anymore. Both ship paths regenerated: `dist/soul.c` (single-TU) + per-module `dist/{memory,awareness,neuron-api}.c`. - **CI gate** (290a637): `ci.yaml` now runs `scripts/verify-soul-contract.sh` against `dist/neuron` as a **hard block between the build and Publish** — PRESENCE (every app route answered) + IMMUTABILITY (no hard-delete). Non-zero fails the build, so `neuron-soul` can't publish to `foundation-prod` or blue-green deploy to GKE with a broken soul. ## Verified macOS build of this exact soul.c passes the gate: PRESENCE all 27, IMMUTABILITY 5/5 KEPT (memory-update, memory-delete, node-update, node-delete, memory-forget). The Linux gate will run in this PR's CI. Merging triggers the Linux soul rebuild + GKE blue-green redeploy of neuron-mcp (expected/approved). Touches only the neuron-mcp path — not the marketing/download site (separate Cloud Run).
will.anderson added 4 commits 2026-07-18 18:32:35 +00:00
The soul was hard-deleting engram nodes: memory/delete and node/delete
called engram_forget (frees the node and drops its incident edges), and
node/update created a replacement then forgot the original with no link
back. That violates the day-one rule that engram nodes are immutable —
memory could be silently, irrecoverably destroyed via the API.

Convert the three destructive handlers to Will's immutability semantics,
mirroring the pattern memory/update and knowledge evolve/promote already
use:

- node/update  -> create the new node, wire a "supersedes" edge new->old,
  KEEP the original. No engram_forget. (Was: create + forget old, no edge.)
- memory/delete and node/delete -> TOMBSTONE: keep the node AND its edges,
  create a Tombstone marker node (content = target id, label
  "tombstone:<id>") wired with a "tombstones" edge. Never engram_forget.
  Default bounded list reads (handle_api_list_typed / the memory list) hide
  tombstoned nodes and the markers; ?include_deleted=1 returns them, and
  internal cognition + /api/graph/nodes still traverse them. memory/update
  was already correct and is unchanged.

Full-graph hiding on /api/graph/nodes is deliberately NOT done at the el
layer: json_array_get is O(index), so filtering that endpoint (called with
limit up to 999999) would be O(n^2). That hide needs a runtime scan filter
and is a separate follow-up; nodes there remain traversable, tagged
status:deleted via the marker edge.

Regenerated dist/soul.c from these sources (flat single-TU amalgamation,
compiled under a 3GB physical-RSS watchdog, peak ~32MB). Verified with
scripts/verify-soul-contract.sh in neuron-ui: PRESENCE passes (all 27
routes) and IMMUTABILITY passes (all four mutation routes KEPT; deletes
produce a real tombstone marker and hide from the default list).
soul.c (the macOS single-TU amalgamation) already carries the
tombstone/supersede change, but the Windows/Linux build path compiles
the per-module dist/*.c instead (build-soul-windows.sh excludes soul.c).
That path was still pulling the stale, destructive dist/neuron-api.c —
so without this the cross-compiled neuron.exe would keep hard-deleting
engram nodes even though the source is fixed.

Regenerated with `elc --emit-header neuron-api.el`: no engram_forget in
memory_delete/node_delete (tombstone), supersedes edge in node_update.
Portable C — identical for macOS/Windows/Linux; only the runtime it links
against differs.
Phase 1 tombstoned memory/delete + node/delete, but the generic forget
path was still destructive. This routes every forget through the same
tombstone semantics so nothing in the daemon can hard-delete an engram
node anymore.

Canonical helper: mem_tombstone (memory.el, imported first so every module
can call it) — keep the node + its edges, record a Tombstone marker, never
engram_forget. neuron-api's tombstone_node now delegates to it (single
source of truth).

Per-path before -> after:
- memory.el `mem_forget`      hard delete (engram_forget) -> tombstone. This
  alone fixes both callers: the /api/neuron/memory/forget route
  (handle_api_forget) and the cultivate op=="forget".
- awareness.el autonomous `forget` action  engram_forget -> mem_tombstone.
  The soul can no longer autonomously hard-delete a memory.
- mcp-wrapper tool_forget  was a FAKE no-op that returned {"ok","deleted"}
  without deleting OR tombstoning -> now routes to the soul's tombstoning
  /api/neuron/memory/delete; tool description fixed to say it
  supersedes/tombstones (recoverable), not "Remove a node".

Left intentionally as hard deletes (internal GC / lifecycle, not user
memory, all call engram_forget directly): session-summary replace
(chat.el), mem_consolidate dedup (memory.el), session-start telemetry
pruning (soul.el), session lifecycle (sessions.el).

Regenerated both ship paths under a 3GB physical-RSS watchdog (peak ~32MB):
dist/soul.c (single-TU amalgamation, macOS/Linux) and the per-module
dist/{memory,awareness,neuron-api}.c (Windows/Linux build). Verified: no
forget handler calls engram_forget; a forget leaves the node present +
tombstone marker. Gate (neuron-ui verify-soul-contract.sh, new memory-forget
row): PRESENCE + IMMUTABILITY PASS.
ci: gate the Linux soul on the contract before publishing
Neuron Soul CI / build (pull_request) Failing after 12m4s
Neuron Soul CI / deploy (pull_request) Has been skipped
290a637883
Wire the soul contract gate into ci.yaml as a hard block between the cc
build and the Publish-to-Artifact-Registry step. A non-zero gate fails the
build, so a stale (route-404ing) or memory-destroying (hard-deleting) soul
can never publish neuron-soul to foundation-prod or blue-green deploy to
GKE — the same class-fix now guarding the desktop builds, extended to prod.

Vendors scripts/verify-soul-contract.sh (copied from neuron-ui; the route
contract is baked in, so it's portable POSIX bash/curl with no neuron-ui
source dependency). It boots dist/neuron on a throwaway port with a
throwaway HOME/engram/cgi — never touching ~/.neuron or any live service —
and checks PRESENCE (every app route answered) + IMMUTABILITY (no engram
write route hard-deletes; deletes/forgets tombstone). Adds curl to the
build deps for the probe.
will.anderson merged commit c199a13a7f into main 2026-07-18 18:33:11 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#83