neuron-technologies/neuron
2
0
Fork 0
Code Issues 22 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Make engram deletes/updates immutable (tombstone + supersede) #82

Open
will.anderson wants to merge 4 commits from feat/immutable-engram-deletes into hotfix/elc-source-typos
pull from: feat/immutable-engram-deletes
merge into: :hotfix/elc-source-typos
Conversation 0 Commits 4 Files Changed 10
+778 -269
will.anderson commented 2026-07-17 21:43:20 +00:00
Owner
Copy Link
Copy Source

What

Make the soul's engram delete and update routes immutable, and regenerate dist/soul.c.

Why

Three handlers were destroying engram nodes:

  • memory/delete, node/delete → engram_forget (frees the node and drops its incident edges)
  • node/update → created a replacement then engram_forget'd the original, with no link back

That violates the day-one rule that engram nodes are immutable — API callers could silently, irrecoverably destroy memory. memory/update was already correct (supersede-with-edge); the others were not.

The fix (mirrors memory/update and knowledge evolve/promote)

  • node/update → create the new node, wire a supersedes edge new→old, keep the original. No engram_forget.
  • memory/delete + node/delete → tombstone: keep the node and all its edges; create a Tombstone marker node (content = target id, label tombstone:<id>) wired with a tombstones edge. Never engram_forget. Default bounded list reads (handle_api_list_typed / the memory list) hide tombstoned nodes and the markers; ?include_deleted=1 returns them, and internal cognition + /api/graph/nodes still traverse them.

Deliberately deferred (documented in code)

Full-graph hiding on /api/graph/nodes is not done at the el layer: json_array_get is O(index), so filtering that endpoint (the app calls it with limit up to 999999) would be O(n²). That hide needs a runtime scan filter (skip tombstoned in engram_scan_nodes_json) and is a separate follow-up. Tombstoned nodes remain present and traversable there, tagged status:deleted via the marker edge.

Regen + verification

  • dist/soul.c regenerated from these sources as a flat single-TU amalgamation, compiled under a 3GB physical-RSS watchdog (macOS ignores ulimit -v), peak ~32MB. Diff is exactly neuron-api.el + dist/soul.c.
  • Base is hotfix/elc-source-typos (where the shipped soul.c / neuron-ui PR #144 came from) so the diff stays minimal.
  • Verified with neuron-ui/scripts/verify-soul-contract.sh against the freshly-built binary on a throwaway port:
SECTION A PRESENCE:    PASS — all 27 routes answered
SECTION B IMMUTABILITY:
  memory-update  KEPT   (supersede edge)
  memory-delete  KEPT   (tombstoned + hidden from default list)
  node-update    KEPT   (supersede edge)
  node-delete    KEPT   (tombstoned + hidden from default list)
GATE: PASS

The stale/destructive binary fails the same gate (memory-delete / node-update / node-delete → DESTROYED), so the gate discriminates.

Never touched the live soul (:7770), engram (:8742), ~/.neuron, or the running :7799 sandbox — all build + test on throwaway paths/ports.

Do not merge — going live is a controlled-deploy call.

## What Make the soul's engram `delete` and `update` routes **immutable**, and regenerate `dist/soul.c`. ## Why Three handlers were destroying engram nodes: - `memory/delete`, `node/delete` → `engram_forget` (frees the node **and** drops its incident edges) - `node/update` → created a replacement then `engram_forget`'d the original, with no link back That violates the day-one rule that engram nodes are immutable — API callers could silently, irrecoverably destroy memory. `memory/update` was already correct (supersede-with-edge); the others were not. ## The fix (mirrors `memory/update` and knowledge evolve/promote) - **`node/update`** → create the new node, wire a `supersedes` edge new→old, **keep** the original. No `engram_forget`. - **`memory/delete` + `node/delete`** → **tombstone**: keep the node and all its edges; create a `Tombstone` marker node (content = target id, label `tombstone:<id>`) wired with a `tombstones` edge. Never `engram_forget`. Default bounded list reads (`handle_api_list_typed` / the memory list) hide tombstoned nodes and the markers; `?include_deleted=1` returns them, and internal cognition + `/api/graph/nodes` still traverse them. ### Deliberately deferred (documented in code) Full-graph hiding on `/api/graph/nodes` is **not** done at the el layer: `json_array_get` is O(index), so filtering that endpoint (the app calls it with `limit` up to 999999) would be O(n²). That hide needs a runtime scan filter (skip tombstoned in `engram_scan_nodes_json`) and is a separate follow-up. Tombstoned nodes remain present and traversable there, tagged `status:deleted` via the marker edge. ## Regen + verification - `dist/soul.c` regenerated from these sources as a flat single-TU amalgamation, compiled under a **3GB physical-RSS watchdog** (macOS ignores `ulimit -v`), peak ~32MB. Diff is exactly `neuron-api.el` + `dist/soul.c`. - Base is `hotfix/elc-source-typos` (where the shipped `soul.c` / neuron-ui PR #144 came from) so the diff stays minimal. - Verified with `neuron-ui/scripts/verify-soul-contract.sh` against the freshly-built binary on a throwaway port: ``` SECTION A PRESENCE: PASS — all 27 routes answered SECTION B IMMUTABILITY: memory-update KEPT (supersede edge) memory-delete KEPT (tombstoned + hidden from default list) node-update KEPT (supersede edge) node-delete KEPT (tombstoned + hidden from default list) GATE: PASS ``` The stale/destructive binary fails the same gate (memory-delete / node-update / node-delete → DESTROYED), so the gate discriminates. Never touched the live soul (:7770), engram (:8742), `~/.neuron`, or the running :7799 sandbox — all build + test on throwaway paths/ports. **Do not merge** — going live is a controlled-deploy call.
will.anderson added 1 commit 2026-07-17 21:43:25 +00:00
Make engram deletes/updates immutable (tombstone + supersede) d337fcb265 
The soul was hard-deleting engram nodes: memory/delete and node/delete
called engram_forget (frees the node and drops its incident edges), and
node/update created a replacement then forgot the original with no link
back. That violates the day-one rule that engram nodes are immutable —
memory could be silently, irrecoverably destroyed via the API.

Convert the three destructive handlers to Will's immutability semantics,
mirroring the pattern memory/update and knowledge evolve/promote already
use:

- node/update  -> create the new node, wire a "supersedes" edge new->old,
  KEEP the original. No engram_forget. (Was: create + forget old, no edge.)
- memory/delete and node/delete -> TOMBSTONE: keep the node AND its edges,
  create a Tombstone marker node (content = target id, label
  "tombstone:<id>") wired with a "tombstones" edge. Never engram_forget.
  Default bounded list reads (handle_api_list_typed / the memory list) hide
  tombstoned nodes and the markers; ?include_deleted=1 returns them, and
  internal cognition + /api/graph/nodes still traverse them. memory/update
  was already correct and is unchanged.

Full-graph hiding on /api/graph/nodes is deliberately NOT done at the el
layer: json_array_get is O(index), so filtering that endpoint (called with
limit up to 999999) would be O(n^2). That hide needs a runtime scan filter
and is a separate follow-up; nodes there remain traversable, tagged
status:deleted via the marker edge.

Regenerated dist/soul.c from these sources (flat single-TU amalgamation,
compiled under a 3GB physical-RSS watchdog, peak ~32MB). Verified with
scripts/verify-soul-contract.sh in neuron-ui: PRESENCE passes (all 27
routes) and IMMUTABILITY passes (all four mutation routes KEPT; deletes
produce a real tombstone marker and hide from the default list).
will.anderson added 1 commit 2026-07-17 21:58:05 +00:00
Regenerate per-module dist/neuron-api.c for the immutability fix f3034d23f7 
soul.c (the macOS single-TU amalgamation) already carries the
tombstone/supersede change, but the Windows/Linux build path compiles
the per-module dist/*.c instead (build-soul-windows.sh excludes soul.c).
That path was still pulling the stale, destructive dist/neuron-api.c —
so without this the cross-compiled neuron.exe would keep hard-deleting
engram nodes even though the source is fixed.

Regenerated with `elc --emit-header neuron-api.el`: no engram_forget in
memory_delete/node_delete (tombstone), supersedes edge in node_update.
Portable C — identical for macOS/Windows/Linux; only the runtime it links
against differs.
will.anderson added 1 commit 2026-07-18 18:23:03 +00:00
Tombstone the remaining forget paths (close the immutability arc) 40d800195a 
Phase 1 tombstoned memory/delete + node/delete, but the generic forget
path was still destructive. This routes every forget through the same
tombstone semantics so nothing in the daemon can hard-delete an engram
node anymore.

Canonical helper: mem_tombstone (memory.el, imported first so every module
can call it) — keep the node + its edges, record a Tombstone marker, never
engram_forget. neuron-api's tombstone_node now delegates to it (single
source of truth).

Per-path before -> after:
- memory.el `mem_forget`      hard delete (engram_forget) -> tombstone. This
  alone fixes both callers: the /api/neuron/memory/forget route
  (handle_api_forget) and the cultivate op=="forget".
- awareness.el autonomous `forget` action  engram_forget -> mem_tombstone.
  The soul can no longer autonomously hard-delete a memory.
- mcp-wrapper tool_forget  was a FAKE no-op that returned {"ok","deleted"}
  without deleting OR tombstoning -> now routes to the soul's tombstoning
  /api/neuron/memory/delete; tool description fixed to say it
  supersedes/tombstones (recoverable), not "Remove a node".

Left intentionally as hard deletes (internal GC / lifecycle, not user
memory, all call engram_forget directly): session-summary replace
(chat.el), mem_consolidate dedup (memory.el), session-start telemetry
pruning (soul.el), session lifecycle (sessions.el).

Regenerated both ship paths under a 3GB physical-RSS watchdog (peak ~32MB):
dist/soul.c (single-TU amalgamation, macOS/Linux) and the per-module
dist/{memory,awareness,neuron-api}.c (Windows/Linux build). Verified: no
forget handler calls engram_forget; a forget leaves the node present +
tombstone marker. Gate (neuron-ui verify-soul-contract.sh, new memory-forget
row): PRESENCE + IMMUTABILITY PASS.
will.anderson added 1 commit 2026-07-18 18:32:08 +00:00
ci: gate the Linux soul on the contract before publishing
Neuron Soul CI / build (pull_request) Failing after 12m4s
Details
Neuron Soul CI / deploy (pull_request) Has been skipped
Details
290a637883 
Wire the soul contract gate into ci.yaml as a hard block between the cc
build and the Publish-to-Artifact-Registry step. A non-zero gate fails the
build, so a stale (route-404ing) or memory-destroying (hard-deleting) soul
can never publish neuron-soul to foundation-prod or blue-green deploy to
GKE — the same class-fix now guarding the desktop builds, extended to prod.

Vendors scripts/verify-soul-contract.sh (copied from neuron-ui; the route
contract is baked in, so it's portable POSIX bash/curl with no neuron-ui
source dependency). It boots dist/neuron on a throwaway port with a
throwaway HOME/engram/cgi — never touching ~/.neuron or any live service —
and checks PRESENCE (every app route answered) + IMMUTABILITY (no engram
write route hard-deletes; deletes/forgets tombstone). Adds curl to the
build deps for the probe.
Some required checks failed
Neuron Soul CI / build (pull_request) Failing after 12m4s
Details
Neuron Soul CI / deploy (pull_request) Has been skipped
Details
This pull request has changes conflicting with the target branch.
  • dist/soul.c
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/immutable-engram-deletes:feat/immutable-engram-deletes
git checkout feat/immutable-engram-deletes
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#82
Delete Branch "feat/immutable-engram-deletes"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?