This repository has been archived on 2026-08-20. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
el-retired/docs/runbooks/2026-08-13-reasoning-operators-decisions-reversal.md
will.anderson a3358dfc95 Reasoning layer: analogy/induction/abduction/causal/planning over §5 geometry ops
Compose the live relational-neighborhood geometry OPERATORS into five reasoning
modes as pure, read-only C (engram_reason.{h,c}); each is proven with closed-form
constructed tests before it ships, not declared.

- ANALOGY  (Procrustes R + residual translation, apply to C, rank candidates)
- INDUCTION (combine-pooled rule geometry + point-to-manifold membership)
- ABDUCTION (best-explaining structure by point-to-manifold fit)
- CAUSAL   (centroid-cosine correlation vs directed influence: temporal
            precedence + association surviving confounder control via subtract;
            emits a correlation-vs-causation flag)
- PLANNING (geo-distance edges + Dijkstra → discrete geodesic path)

A shared point-to-manifold fit primitive underlies induction membership and
abduction ranking. engram/test/run_reason_tests.sh: 33/33 checks on both PERF
and ASan/UBSan passes; macOS leaks 0/0.

ANALOGY is surfaced as an el builtin (engram_reason_analogy_json) via the same
pass-through the §5 operators use — demonstrated callable from compiled El with a
container-capped fold (no self-host fold). The other four are C-layer only: their
set/point/timestamp inputs do not map to the flat-CSV el ABI without touching
codegen (deferred). engram_reason.c must join the server link line beside
engram_geometry.c at cutover. See docs/runbooks/2026-08-13-reasoning-operators-*.
2026-08-13 01:33:14 -05:00

7.1 KiB
Raw Permalink Blame History

Reasoning Operators — Decisions & Reversal

Date: 2026-08-13 Branch: engram-tiered-storage (worktree /tmp/engram-tiered-wt) Status: staged locally — NOT pushed, NOT tagged, NOT merged. Live :8742 untouched.

What this adds

A REASONING layer built as pure C compositions over the already-live §5 geometry OPERATORS (engram_geometry.{h,c}: overlap, subtract, setdiff, combine, distance, analogy). Where the operators are a relational algebra over neighborhood descriptors, these are reasoning modes built by chaining that algebra. New files:

  • lang/runtime/engram_reason.h — public API for the five modes + a shared point-to-manifold fit primitive.
  • lang/runtime/engram_reason.c — implementations. READ-ONLY over descriptor inputs, stdlib + libm only, touches no store / index / activation. All geometry is delegated to engram_geo_*; this file only composes.
  • engram/test/test_reason.c + engram/test/run_reason_tests.sh — closed-form constructed tests (hand-built descriptors with known answers), PERF + ASan/UBSan.

El-exposure (pass-through, no self-host fold):

  • lang/runtime/el_runtime.c+#include "engram_reason.h" and the builtin engram_reason_analogy_json(a_csv,b_csv,c_csv).
  • lang/runtime/el_runtime.h — its declaration.
  • lang/runtime/el_seed.c — native __engram_reason_analogy_json wrapper (same C-table wiring as the §5 ops).

The five modes — signatures & composition

Mode C entry point Composes
ANALOGY A:B :: C:? engram_reason_analogy(A,B,C,candidates,n,out) engram_geo_analogy (Procrustes R) + engram_geo_analogy_apply + centroid L2. Learns R_{A→B} = engram_geo_analogy(B,A) (that op returns R with apply(R, Y-axis)≈X-axis), reconstructs the residual translation t = c_B R·c_A, maps mapped = R·c_C + t, ranks candidates by distance.
INDUCTION {E_i}→rule engram_reason_induce(examples,n,top_axes,ext_floor,out) + engram_reason_membership engram_geo_combine folded left→right → pooled "rule" descriptor; shared subspace surfaces as the dominant pooled axes. Membership = point-to-manifold fit.
ABDUCTION x→best H engram_reason_abduce(obs,dim,hyps,n,ext_floor,out) shared engram_reason_point_fit against each hypothesis; argmax fit score; full ranking.
CAUSAL x?y | Z,t engram_reason_causal(x,y,confounders,nZ,t_x,t_y,drop_frac,out) centroid cosine (raw correlation) + engram_geo_subtract residual-centroid (control for each confounder, take the strongest single explainer) + temporal precedence. Verdict DIRECTED / CONFOUNDED / NONE + a confounded flag.
PLANNING start→goal engram_reason_plan(nodes,n,start,goal,radius,use_w,out) engram_geo_distance as edge weights over neighborhoods within radius; O(n²) Dijkstra → discrete geodesic path.

Shared primitive engram_reason_point_fit splits (x centroid) into an in-subspace Mahalanobis distance (scaled by axis extents) and an orthogonal off-model residual; it is the single engine under INDUCTION's membership test and ABDUCTION's ranking.

Proof (DONE-WITH-PROOF)

engram/test/run_reason_tests.sh: 33/33 checks, 0 failures on BOTH passes (PERF -O2, and ASan+UBSan). macOS leaks --atExit: 0 leaks / 0 total leaked bytes.

Per-mode closed-form assertions actually exercised:

  • ANALOGY — A→B = +90° rotation in e0-e1 plane + a +5 shift in e2; Procrustes residual ~0; predicted point (0,2,5,0) recovered exactly; nearest candidate = the planted true D (index 1), distance ~0.
  • INDUCTION — 3 examples sharing span(e0,e1) (extents 1.0 / 0.8) each with a small idiosyncratic axis (e2 or e3); induced top-2 axes lie in span(e0,e1) (extents recovered ~1.0 / ~0.8); held-out in-plane point fits (membership 0.885), off-subspace point rejected (0.100), in-plane-but-far point rejected (0.039).
  • ABDUCTION — observation planted inside H1 among {H0,H1,H2}; best = H1, rank[0] = H1, H1 smallest distance.
  • CAUSAL — chain A→B→C along e0 (t 1<2<3) + confounder Z(e1) that leaks into A and drives D(t=4): A→B and B→C flagged DIRECTED with correct precedence and association that survives control; AD CONFOUNDED (raw |cos|=0.707 collapses to 0.0 under control) with confounded=1; BD NONE (no association).
  • PLANNING — 6 neighborhoods on a semicircle (r=10); neighbor_radius=7 admits only consecutive hops; plan = [0,1,2,3,4,5] (the arc), cost 30.90 (> the 20-unit chord, confirming it is the geodesic through the manifold, not a straight jump); a too-small radius correctly yields reached=0.

El-exposure status

  • ANALOGY is el-callable via the same pass-through the §5 operators use. Proof: a container-capped fold (capfold.sh, peak ~0 GB) of a demo .el through the shipped lang/dist/platform/elc emits a direct C call engram_reason_analogy_json(A,B,A) (no registration, no self-host fold); the generated C links against el_runtime.c + engram_reason.c + geometry/store/vindex and runs end-to-end. (The standalone demo's store copy boots 0 nodes — a pre-existing quirk that hits the shipped geo demo identically — so the call returns {"error":"geometry unavailable"}; this still proves the compiled El → C reasoning symbol → JSON chain executes. Numeric correctness on real data is covered by the C test.) This compile also confirms el_runtime.c + engram_reason.c compile and link clean.
  • INDUCTION / ABDUCTION / CAUSAL / PLANNING are C-layer only for now. Their inputs are candidate sets, raw points, and timestamps that do not map to the flat comma- separated-seed El ABI. A richer marshalling surface would touch the codegen/registration path and risk an uncapped fold — explicitly deferred per the hard rail. The C functions are fully proven and callable from any C caller today.

Build wiring (for the later cutover/durability pass)

engram_reason.c must be added to the engram server link line alongside engram_geometry.c (the heavy-runtime path cc dist/engram.c el_runtime.c engram_store.c engram_geometry.c engram_vindex.c …). el_runtime.c now #includes engram_reason.h and references engram_reason_analogy_json, so a build that omits engram_reason.c will fail to link that symbol. One-line addition, same as how engram_geometry.c was originally added.

Reversal

Fully additive; nothing existing was modified in behavior. To revert:

  1. Delete lang/runtime/engram_reason.h, lang/runtime/engram_reason.c, engram/test/test_reason.c, engram/test/run_reason_tests.sh, and this doc.
  2. In lang/runtime/el_runtime.c: remove #include "engram_reason.h" and the engram_reason_analogy_json function.
  3. In lang/runtime/el_runtime.h: remove the engram_reason_analogy_json declaration.
  4. In lang/runtime/el_seed.c: remove the __engram_reason_analogy_json wrapper.
  5. Remove engram_reason.c from any server link line if the cutover added it.

No store, schema, config, WAL, or on-disk format was touched; no data migration exists, so reversal is a pure code removal with no state to undo.