9 Commits

Author	SHA1	Message	Date
bigmerge	8c2406ff6b	runtime: the link set is multi-file — name it once, ship all of it ... el_runtime.c was created 2026-05-03 as an explicitly temporary build shim. It was deleted that afternoon ("runtime is 100% native El") and restored 25 minutes later "UNTIL the compiler is updated to emit #include el_seed.h". The `until` never came. 3.5 months on it is 20,527 lines, and nothing was ever set up to notice — a file scheduled for deletion gets no owner, no budget, no boundary. What kept it growing is not inertia, it is an instruction. lang/AGENTS.md said el_runtime.c "is the authoritative single-file link target ... THIS IS WHERE A NEW C BUILTIN'S IMPLEMENTATION MUST CURRENTLY LIVE TO BE LINKABLE", and made it step 1 of the add-a-builtin recipe. That is false. Placement is a link-time concern: builtin_arity maps NAME -> ARITY INT only, the El name is emitted as the exact C symbol, and `ld` resolves it — the compiler cannot tell which .c a symbol came from. `nm lang/dist/platform/elc` on the shipped compiler already shows T _engram_geo_reify_index_new, T _vindex_insert, T _engram_think, T _engram_reason_abduce: it is linked from ten translation units today. In a repo where agents write most of the code, a false instruction in the instruction file is the forcing function. The file grew because the recipe said to grow it. The multi-file runtime is therefore already real, and the docs and the distribution never caught up — which left a live, shipped bug: * Linking el_runtime.c alone FAILS at `ld` (undefined engram_ground_json, engram_activate_inner, eg_find_relation, cog_assert_two_axis, ...) because el_runtime.c #includes six engram headers and calls into all six siblings. * sdk-release.yaml shipped el_runtime.c/.h + engram_store.c/.h and none of the other five required .c files, so downstream consumers of the el-runtime-c Artifact Registry package and of install.sh got a lib/ that cannot link. * .githooks/pre-commit linked el_runtime.c alone with stderr to /dev/null, so it reported all 13 native suites as FAILED with the real ld error invisible. * AGENTS.md's self-host recipe compiled el-compiler/runtime/el_runtime.c — a path the same file's "DO NOT EDIT" list names as a lagging fork. The root fix is to stop writing the list down eight times: * lang/runtime/SOURCES — the canonical link set, in one place, in link order. * scripts/el-runtime-sources.sh — prints it, optionally prefixed; --check fails loudly on a missing file, --headers for the shipped headers. * Every link line in AGENTS.md, lang/AGENTS.md, DESIGN.md, lang/spec/language.md, the three workflows and the pre-commit hook now reads that one list. * Adding a concern's .c is one line in SOURCES, so a new builtin no longer has to be appended to el_runtime.c just because appending was the cheaper edit. Distribution: ship the siblings rather than amalgamate. Amalgamation needs a new tool and contradicts DESIGN.md's compile-once-link-many; the siblings are already independently authored and independently tested (engram/test/*.sh link subsets directly), and engram_store.c was already shipped, so this completes a mechanism that existed rather than inventing one. Source is also a superset: a consumer that wants one file can concatenate, one that wants separate TUs cannot undo an amalgamation. el-runtime-c/-h stay for backward compatibility; el-runtime-src is added carrying the complete set plus SOURCES. lang/AGENTS.md now points new C builtins at the concern-owning .c and states plainly that the compiler cannot tell which .c a symbol came from, with the nm evidence. AGENTS.md's "reconcile which is canonical (verify)" note is resolved: neither file supersedes the other, the canonical unit is the set. Verified locally (the bar; not CI): * engram/src/server.el compiles and links against the SOURCES set. * Compile-once-link-many into libel.a links the same program. * elb builds from the corrected recipe. * Self-host fixpoint byte-identical (11,110 lines, stage2 == stage3) built with the SOURCES-driven link line. * pre-commit hook: 0 of 13 native suites passing -> 8 of 13. The 5 still-failing suites are PRE-EXISTING and untouched here: test_fs (fs_list_json undeclared), test_state (state_has, state_get_or undeclared), test_json (json_build_array/json_build_object/json_escape_string undefined), test_time (now_ns undefined), test_env (1 assertion). Builtins registered in builtin_arity with no implementation or no declaration anywhere — the same recipe defect, now visible because the linker error is no longer suppressed. Not attempted: making elc emit #include el_seed.h and dropping elb's hardcoded runtime path. That is the correct long-term fix and finishes the 2026-05-03 migration, but it touches codegen and self-hosting and belongs in its own change.	2026-08-16 16:44:26 -05:00
bigmerge	caa1206af5	docs: the nine-op surface shipped, and two of its primitives are the wrong shape ... lang/AGENTS.md said the collapse was 'not yet compiled into the MCP server'. Verified against the live tool surface: it is exactly the nine ops. Noted that think's faculty parameter and ground's minted edge are both documented as the wrong shape.	2026-08-16 15:49:44 -05:00
Neuron	fe820928b0	docs: the builtin recipe never required a test ... lang/AGENTS.md:71-77 gives four steps for adding a C builtin and ends at 'confirm the self-host fixpoint is byte-identical'. No step asks for a test. The only 'verify' in the file is that fixpoint, which proves the COMPILER REPRODUCES ITSELF and says nothing about whether the builtin works — so the recipe reads as complete while having checked nothing about the thing just added. Measured on 2026-08-16: engram_node_set_emb, engram_curiosity_json and dream_set_handler were all added in a single session with zero tests, by an agent following this recipe. Separately a UTF-8 fix was written and tested and THE TEST PASSED ON THE UNPATCHED BUILD — the real defect was elsewhere, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified. Adds step 5 with the two failure shapes actually encountered: a test that never exercises the change (a route default bypassed the code under test), and an induction that loses a race (curl --max-time left BOTH builds alive; only SO_LINGER 0, a real RST, reproduced it). Plus the port-binding check, because a stale instance answering has silently produced false results here more than once and pkill -f does not reliably match argv './engram'. Documentation only. Does not touch the (a) split-the-C / (b) close-the- compiler-gap question, which is a separate decision.	2026-08-16 13:53:08 -05:00
bigmerge	09dade0613	Merge remote-tracking branch 'origin/pr/103' into HEAD ... # Conflicts: # lang/AGENTS.md # lang/runtime/el_runtime.h	2026-08-15 18:26:57 -05:00
bigmerge	bacaf3d39c	engram: reconcile M8 HNSW vindex (#109) onto current dev, restore 3 fixes the branch predated ... Lands feat/reframe-region-setop (PR #109: native set-based reframe_region, decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's actual current HEAD, plus engram-tiered-storage's still-unique test suite. RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/ verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/ select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two branches; #109's copy is a strict superset (adds vindex_harvest_from_store, used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and engram_verify.c are also byte-identical. #109's own branch point already carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing left to merge into #109 for those files. The one thing engram-tiered-storage had that #109's tree dropped: its full test suite (test_vindex.c, test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh harnesses) — ported over here unchanged. WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch forked from dev on 2026-08-14 15:40 (before restructure-adjacent history diverged the file's merge-base for `git merge` — it presented as an add/add conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but it silently reverted THREE dev fixes landed on 2026-08-14/15, after the branch point, that the PR's diff had no way to know about: 1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of the query-aware propagation gate dropped the shift-and-floor rescale about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate 0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi] is computed) is orthogonal to WHAT it gates on and both are kept. 2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global). PR's tree predates this and dropped all three counters + their JSON stats fields. Restored declarations, all 4 direct increment sites, the eg_wm_carry_over bll increment, and the act-stats JSON fields -- alongside (not instead of) the PR's own P4 afferent / API-reshape counters already in that same struct/JSON. 3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev selects the STRONGEST qualifying candidate for consolidation each call; PR's tree predates this and reverted to hash-slot order (arbitrary wrt association strength) for edge formation -- the one path that writes PERMANENT structure. Restored the strongest-candidate while-loop, keeping the PR's own genuine improvement at that site (engram_adj_on_edge_added incremental-index append instead of a bare adj_dirty=1 full-rebuild flag). engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env flags, /api/nodes connected-count in responses) were pure additive: dev's side was empty, PR's side added the feature. Took PR's side whole. VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): - cc -std=c11 -O2, clean link against the real engram/src/server.el via elc, zero errors. - vindex_bench (built standalone, read-only harvest) against the real production store clone (13,671 embedded nodes, 768-dim nomic-embed-text): recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs 2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s for the full 13,671-node set -- see the flagged risk below. - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned snapshot of the live store, 13,424 nodes / 37,656 edges) and called /api/activate for real: first call after boot 41.5s (pays the one-time HNSW build inline -- matches the standalone bench), second/third calls 356ms/605ms, no crash, correct results, act-stats JSON (including the restored evict_floor/cap/bll fields) reads correctly. KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync builds the HNSW index synchronously, inline, on the first engram_activate() call after every process start (or index invalidation). On the real node count that is a ~46s blocking stall on a single-threaded server -- the first request after every restart (or its concurrent siblings) waits the full build. Recommend a background/incremental build (or a bounded per-call build budget) before this ever reaches the live daemon. See PR description / final report for the fuller writeup.	2026-08-15 16:46:44 -05:00
bigmerge	08cbcef5d9	engram: fix lazy-embed index gap (#20) and make activate's cosine scan lazy; extract vindex harvest primitive with a bench/oracle harness ... Adds an O(1) "seen" bitmap so lazily-embedded older nodes get picked up incrementally instead of only on a full rebuild (embed-gap #20). Replaces engram_activate's O(N*D) cosine prescan with a lazy-memoized cosine cache (eg_cosq_at), proven bit-identical to the old path. Extracts a clean vindex_harvest_from_store primitive (read-only vector harvest, careful malloc/ownership/error-path handling) reused by both index-build and the new vindex_bench.c — a read-only proof harness comparing brute-force vs HNSW recall/latency on both the real store and synthetic data. .nsbx-env intentionally excluded — local sandbox config (ports, paths, dev-only placeholder key), not checked in.	2026-08-15 14:26:16 -05:00
bigmerge	1010185978	Add op_assert grounded-envelope primitive and purview-bounded mutation wrappers ... Adds engram_assert_json — a grounded "assertion envelope" primitive for a realizer/op_assert seam (per backlog bl-53/#57) — plus purview-scoped mutation wrappers engram_node_full_in/engram_connect_in, which refuse non-default purviews rather than silently mutating the live store. Threads through el_seed.c/h wrappers and the codegen.el arity table per the project's existing C-builtin recipe. Also rewrites lang/AGENTS.md build docs with verified (2026-08-15) findings that el_seed.c does not compile standalone.	2026-08-15 14:24:11 -05:00
will.anderson	0a72fced28	engram: WAL persistence + integrity hardening + single canonical runtime ... Establish lang/runtime/ as the ONE canonical el runtime (from the active runtime that carries hebb/emb persistence + the new WAL); repoint the el CI publish, engram build, elb default, and in-repo build scripts to it; delete the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh drift guard. Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian edge weights — learned co-activation was wiped on every restart. Publishing from canonical ships the stranded 'learning that cannot outlive the process' fix. WAL storage engine + integrity fixes (DELETE->tombstone + store-layer protection, safe data-dir default) ride in behind ENGRAM_WAL (default off = byte-identical to today). Verified: engram elb per-module build clean, WAL gate 66/66, native smoke ok, drift-guard green.	2026-08-11 21:31:37 -05:00
Will Anderson	1ae68962cf	restructure: move el compiler content into lang/	2026-05-05 01:38:51 -05:00