Add reversal doc for §5 geometry operators EL cutover

2026-08-13 00:51:03 -05:00
parent 4965600d65
commit e0bc303139
1 changed files with 102 additions and 0 deletions
@@ -0,0 +1,102 @@
+# Reversal / Decisions — §5 Geometry Operators EL Cutover
+
+**Date:** 2026-08-13
+**Branch:** `engram-tiered-storage` (worktree `/tmp/engram-tiered-wt`)
+**Parent commit:** `5336cfe` (M9 §5 geometry operators as C functions + EL builtins, staged)
+**Scope:** make the six engram geometry operators callable from a compiled `.el`
+program, and demonstrate it on real store data. Staged, reversible. NOT pushed,
+NOT tagged. Live `:8742` daemon and `~/.neuron/engram` never touched.
+
+---
+
+## What this delivers
+
+On `5336cfe` the six operators existed as heavy-runtime C functions
+(`engram_geo_*_json` in `lang/runtime/el_runtime.c:12287-12385`, declared in
+`el_runtime.h:627-632`) but the EL call surface was deferred. This change
+formalizes the cutover and proves callability from a compiled El (CGI) program.
+
+### Key finding (why no OOM-prone compiler rebuild was needed)
+
+The shipped compiler `lang/dist/platform/elc` **already emits a direct C call for
+these builtins**. An unknown ident-call passes through verbatim as a C call, and
+`arity_check_call` returns OK when `builtin_arity < 0`. So a compiled `.el` that
+calls `engram_geo_distance_json(A, B)` folds to `engram_geo_distance_json(A, B)`,
+which links straight into `el_runtime.c`. No self-host fold of `elc-cli.el` (the
+memory-heavy, drift-prone step) was required — that step is explicitly avoided.
+
+---
+
+## Files changed (all in the engram worktree, commit on `engram-tiered-storage`)
+
+1. **`lang/el-compiler/src/codegen.el`** (+12) — source-of-truth `builtin_arity`
+   table: registered the six operators under both the bare heavy-runtime names
+   (`engram_geo_*_json`) and the `__`-prefixed seed names, mirroring the existing
+   `engram_activate_json` / `__engram_activate_json` pair. Effect: a future
+   legitimately-rebuilt elc validates arg counts. No effect on the shipped binary.
+
+2. **`lang/elc.c`** (+36) — the folded-C mirror of the same table, kept in sync
+   with `codegen.el`. (`lang/elc.c` is a stale/partial fold that does not compile
+   standalone — it is missing the `stdout_to_file`/`stdout_restore` definitions —
+   so this edit is source-consistency only; it is not the live compiler.)
+
+3. **`lang/runtime/engram.el`** (+31) — six module wrappers
+   `engram_geo_*_json(...) -> String { return __engram_geo_*_json(...) }`,
+   mirroring the existing `engram_activate_json` wrapper. Surfaces the operators
+   as named El functions for the seed-world / future rebuilt-elc path.
+
+4. **`lang/runtime/engram_geometry.c`** (+2/-1) — style nit at ~1419: the
+   `centroid_unit` normalization `if/else` had misleading indentation
+   (single-statement `for` body then `else`). Braced the `if` arm. Behavior
+   identical; not a numerical change.
+
+---
+
+## Verification performed (real, on-machine)
+
+- **Compiled-EL demo** (`scratchpad/geo_ops_demo.el`, top-level El program):
+  folded with the shipped elc **inside a hard RSS cap** (`capfold.sh` monitor,
+  peak RSS ~4MB), cc-linked against `el_runtime.c + engram_store.c +
+  engram_geometry.c + engram_vindex.c`, run against a **COPY** of the store
+  (`demostore/neuron.egm` from `real_copy.egm`, 13,036 nodes, throwaway `HOME`,
+  no server, not `:8742`). Real output on two real neighborhoods
+  A=architecture `{b037825e, e06ba673, 58ddea41}`, B=hebbian `{78b7a96e,
+  4d5cfe63, 7b97ee0e}`:
+  - subtract residual: `variance_explained_by_B=0.447564, residual_scale=0.304879,
+    removed_dims=3, residual_n_axes=8, centroid_diff_mag=0.125119`
+  - subtract setdiff: `n_only=43, removed=72, centroid_diff_mag=0.125119`
+  - distance: `centroid_distance=0.125119, centroid_cosine=0.778572,
+    wasserstein2=0.268298`
+  - internal consistency: `centroid_diff_mag` identical across subtract+distance.
+- **C unit suite** `test_geo_ops.c`: 20/20 checks pass, ASan+UBSan clean, after
+  the `engram_geometry.c` edit. No regression.
+
+---
+
+## How to reverse
+
+Everything is a single worktree commit on a non-pushed branch.
+
+- **Full reversal:** `git -C /tmp/engram-tiered-wt revert <this-commit>` (or
+  `git reset --hard 5336cfe` to drop back to the parent tip).
+- **Per-file reversal:** `git -C /tmp/engram-tiered-wt checkout 5336cfe -- <path>`
+  for any of the four files. Each edit is additive/local:
+  - The arity entries (`codegen.el`, `elc.c`) are inert unless elc is rebuilt.
+  - The `engram.el` wrappers are unused by the heavy engram server (which calls
+    the bare builtins directly) — removing them changes nothing live.
+  - The `engram_geometry.c` brace change is behavior-neutral.
+- **No runtime/deploy reversal needed:** nothing was deployed. `:8742`, the
+  launch agent, and `~/.neuron/engram` were never modified. No tag, no push.
+
+---
+
+## Deferred / open
+
+- **elc binary rebuild with the arity table baked in** is deferred. The canonical
+  rebuild path (`elc elc-cli.el > elc-new.c`; AGENTS.md) is the self-host fold —
+  the memory-heavy, compiler-revision-drift step. It is unnecessary for
+  callability (shipped elc already passes the calls through) and carries the same
+  drift risk flagged for the M-INTEROCEPTION HTTP routes. Do it only as part of a
+  deliberate, capped compiler-cutover.
+- **HTTP routes** for the operators (server.el) are not added here — out of scope;
+  the demo proves the compiled-EL call surface, which was the deliverable.