bacaf3d39c
El SDK CI - dev / build-and-test (pull_request) Failing after 4m49s
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region, decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's actual current HEAD, plus engram-tiered-storage's still-unique test suite. RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/ verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/ select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two branches; #109's copy is a strict superset (adds vindex_harvest_from_store, used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and engram_verify.c are also byte-identical. #109's own branch point already carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing left to merge into #109 for those files. The one thing engram-tiered-storage had that #109's tree dropped: its full test suite (test_vindex.c, test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh harnesses) — ported over here unchanged. WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch forked from dev on 2026-08-14 15:40 (before restructure-adjacent history diverged the file's merge-base for `git merge` — it presented as an add/add conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but it silently reverted THREE dev fixes landed on 2026-08-14/15, after the branch point, that the PR's diff had no way to know about: 1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of the query-aware propagation gate dropped the shift-and-floor rescale about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate 0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi] is computed) is orthogonal to WHAT it gates on and both are kept. 2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global). PR's tree predates this and dropped all three counters + their JSON stats fields. Restored declarations, all 4 direct increment sites, the eg_wm_carry_over bll increment, and the act-stats JSON fields -- alongside (not instead of) the PR's own P4 afferent / API-reshape counters already in that same struct/JSON. 3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev selects the STRONGEST qualifying candidate for consolidation each call; PR's tree predates this and reverted to hash-slot order (arbitrary wrt association strength) for edge formation -- the one path that writes PERMANENT structure. Restored the strongest-candidate while-loop, keeping the PR's own genuine improvement at that site (engram_adj_on_edge_added incremental-index append instead of a bare adj_dirty=1 full-rebuild flag). engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env flags, /api/nodes connected-count in responses) were pure additive: dev's side was empty, PR's side added the feature. Took PR's side whole. VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): - cc -std=c11 -O2, clean link against the real engram/src/server.el via elc, zero errors. - vindex_bench (built standalone, read-only harvest) against the real production store clone (13,671 embedded nodes, 768-dim nomic-embed-text): recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs 2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s for the full 13,671-node set -- see the flagged risk below. - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned snapshot of the live store, 13,424 nodes / 37,656 edges) and called /api/activate for real: first call after boot 41.5s (pays the one-time HNSW build inline -- matches the standalone bench), second/third calls 356ms/605ms, no crash, correct results, act-stats JSON (including the restored evict_floor/cap/bll fields) reads correctly. KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync builds the HNSW index synchronously, inline, on the first engram_activate() call after every process start (or index invalidation). On the real node count that is a ~46s blocking stall on a single-threaded server -- the first request after every restart (or its concurrent siblings) waits the full build. Recommend a background/incremental build (or a bounded per-call build budget) before this ever reaches the live daemon. See PR description / final report for the fuller writeup.
160 lines
8.1 KiB
C
160 lines
8.1 KiB
C
/* test_geometry.c — build + RUN gate for the M9 FOUNDATION geometry descriptor
|
|
* (engram_geometry.{c,h}). Self-contained: synthesizes a store with two KNOWN
|
|
* embedding clusters + intra-cluster hebb edges, then verifies the descriptor
|
|
* recovers the shape — centroid near the seeded cluster, skeleton = the strong
|
|
* intra-cluster edges, membership gradient, radius, positive co-registration.
|
|
*
|
|
* Pure C11; links engram_geometry.c + engram_store.c + engram_vindex.c; -lm.
|
|
* ASan/UBSan clean. Needs no live data.
|
|
*/
|
|
#include "engram_geometry.h"
|
|
#include "engram_store.h"
|
|
#include "engram_vindex.h"
|
|
#include <stdio.h>
|
|
#include <stdlib.h>
|
|
#include <string.h>
|
|
#include <math.h>
|
|
#include <stdint.h>
|
|
#include <unistd.h>
|
|
|
|
#define DIM 64
|
|
static int g_fail=0;
|
|
#define CHECK(c,m) do{ if(!(c)){printf(" FAIL: %s\n",m); g_fail=1;} else printf(" ok: %s\n",m);}while(0)
|
|
|
|
static uint64_t rs=0x1234abcdULL;
|
|
static uint64_t xr(void){ uint64_t z=(rs+=0x9E3779B97F4A7C15ULL);
|
|
z=(z^(z>>30))*0xBF58476D1CE4E5B9ULL; z=(z^(z>>27))*0x94D049BB133111EBULL; return z^(z>>31); }
|
|
static float jitter(void){ return (float)(((double)(xr()>>11)*(1.0/9007199254740992.0))-0.5)*0.15f; }
|
|
|
|
/* two clusters: A centered on axis 0, B centered on axis 1. NA+NB nodes. */
|
|
#define NA 40
|
|
#define NB 40
|
|
|
|
int main(void){
|
|
printf("=== engram_geometry (M9 foundation) test suite ===\n");
|
|
char path[256]; snprintf(path,sizeof path,"/tmp/geo_test_store_%d.egm",(int)getpid());
|
|
unlink(path);
|
|
EngramPagedStore* st=store_create(path);
|
|
if(!st){ printf("FAIL: store_create\n"); return 1; }
|
|
|
|
char aids[NA][16], bids[NB][16];
|
|
/* cluster A: near +e0 ; cluster B: near +e1 */
|
|
for(int i=0;i<NA;i++){
|
|
StoreNode n; memset(&n,0,sizeof n);
|
|
snprintf(aids[i],16,"A%d",i); n.id=aids[i]; n.node_type="Concept"; n.tier="Semantic";
|
|
n.content="cluster-A"; n.salience=0.5+0.01*i;
|
|
float v[DIM]; for(int d=0;d<DIM;d++) v[d]=jitter(); v[0]=1.0f+jitter();
|
|
n.emb=v; n.emb_dim=DIM; store_put_node(st,&n);
|
|
}
|
|
for(int i=0;i<NB;i++){
|
|
StoreNode n; memset(&n,0,sizeof n);
|
|
snprintf(bids[i],16,"B%d",i); n.id=bids[i]; n.node_type="Concept"; n.tier="Semantic";
|
|
n.content="cluster-B"; n.salience=0.3;
|
|
float v[DIM]; for(int d=0;d<DIM;d++) v[d]=jitter(); v[1]=1.0f+jitter();
|
|
n.emb=v; n.emb_dim=DIM; store_put_node(st,&n);
|
|
}
|
|
/* strong intra-A hebb edges (a chain + hub), weaker cross edges A0<->B0 */
|
|
int ei=0;
|
|
for(int i=1;i<NA;i++){
|
|
StoreEdge e; memset(&e,0,sizeof e); char id[24]; snprintf(id,24,"eA%d",ei++);
|
|
e.id=id; e.from_id=aids[0]; e.to_id=aids[i]; e.relation="assoc"; e.weight=0.9; e.hebb=0.4;
|
|
store_put_edge(st,&e);
|
|
}
|
|
for(int i=1;i<NB;i++){
|
|
StoreEdge e; memset(&e,0,sizeof e); char id[24]; snprintf(id,24,"eB%d",ei++);
|
|
e.id=id; e.from_id=bids[0]; e.to_id=bids[i]; e.relation="assoc"; e.weight=0.9; e.hebb=0.4;
|
|
store_put_edge(st,&e);
|
|
}
|
|
{ StoreEdge e; memset(&e,0,sizeof e); e.id=(char*)"eX"; e.from_id=aids[0]; e.to_id=bids[0];
|
|
e.relation="assoc"; e.weight=0.5; e.hebb=0.0; store_put_edge(st,&e); }
|
|
store_close(st);
|
|
|
|
VIndex* ix=vindex_create(DIM,0,0);
|
|
char** ids=NULL; int nids=0;
|
|
int ins=vindex_build_from_store(ix, path, &ids, &nids);
|
|
CHECK(ins==NA+NB, "vindex built over all embedded nodes");
|
|
|
|
GeoParams P; engram_geo_default_params(&P); P.ann_k=20; P.max_members=0;
|
|
|
|
/* seed inside cluster A -> expect an A-dominated neighborhood */
|
|
st=store_open(path);
|
|
/* global-mean cache over the embedded set: the centering offset */
|
|
GeoMeanCache* mc=engram_geo_mean_build(st);
|
|
const float* gm=engram_geo_mean_vec(mc);
|
|
CHECK(mc!=NULL && engram_geo_mean_dim(mc)==DIM, "global-mean cache built over embedded set");
|
|
CHECK(engram_geo_mean_count(mc)==(uint64_t)(NA+NB), "global mean averaged all embedded nodes");
|
|
const char* seeds[1]={aids[0]};
|
|
/* CENTERED descriptor: pass the global mean so geometry runs in isotropic space */
|
|
GeoDescriptor* g=engram_geometry_descriptor(st, ix, ids, nids, seeds, 1, &P, gm);
|
|
CHECK(g!=NULL, "descriptor computed");
|
|
if(g){
|
|
printf(" members=%d embedded=%d edges=%d k_core=%d radius=%.4f co_reg=%.3f n_axes=%d\n",
|
|
g->n_members,g->n_embedded,g->n_edges,g->k_core,g->radius,g->co_registration,g->n_axes);
|
|
|
|
/* geometry ran in CENTERED space: g->centroid is the centered centroid,
|
|
* g->global_mean the applied offset. Reconstruct the raw prototype
|
|
* (centroid + global_mean) and check it sits on cluster-A's axis. */
|
|
CHECK(g->global_mean!=NULL, "descriptor recorded the centering offset (centered mode)");
|
|
int argmax=0; float best=-1.f;
|
|
for(int d=0;d<g->dim;d++){ float raw=g->centroid[d]+(g->global_mean?g->global_mean[d]:0.f);
|
|
if(fabsf(raw)>best){ best=fabsf(raw); argmax=d; } }
|
|
printf(" raw-prototype dominant axis = %d (expect 0); centered c[0]=%.3f c[1]=%.3f\n",
|
|
argmax, g->centroid[0], g->centroid[1]);
|
|
CHECK(argmax==0, "raw prototype sits on cluster-A's axis (near members)");
|
|
/* centering pushes A off cluster-B's axis: centered c[0] > c[1] */
|
|
CHECK(g->centroid[0] > g->centroid[1], "centered centroid leans off B's axis (isotropy)");
|
|
|
|
/* hub should be A0 (the intra-A hub with NA-1 strong edges) */
|
|
CHECK(g->hub_id && strcmp(g->hub_id,"A0")==0, "hub = the relational center A0");
|
|
|
|
/* membership: seed A0 == 1.0; A-members strong, B-members (if any) weaker */
|
|
double seedw=-1, minA=2, maxB=-1; int na=0,nb=0;
|
|
for(int i=0;i<g->n_members;i++){
|
|
const char* id=g->members[i].id; double w=g->members[i].membership;
|
|
if(strcmp(id,"A0")==0) seedw=w;
|
|
if(id[0]=='A'){ na++; if(w<minA)minA=w; }
|
|
if(id[0]=='B'){ nb++; if(w>maxB)maxB=w; }
|
|
}
|
|
printf(" A-members=%d B-members=%d seedw=%.3f\n", na,nb,seedw);
|
|
CHECK(fabs(seedw-1.0)<1e-9, "seed membership == 1.0");
|
|
CHECK(na>=NA-1, "neighborhood recovers cluster A");
|
|
|
|
/* skeleton = the strong intra-A edges: every edge eff_weight>=threshold,
|
|
* and edges connect A-nodes (co-registration should be positive: wired
|
|
* pairs are semantically near). */
|
|
int allstrong=1, allA=1;
|
|
for(int e=0;e<g->n_edges;e++){
|
|
if(g->edges[e].eff_weight < P.edge_min_weight) allstrong=0;
|
|
const char* a=g->members[g->edges[e].a].id, *b=g->members[g->edges[e].b].id;
|
|
if(!(a[0]=='A'&&b[0]=='A')) { /* the lone eX cross edge is allowed */
|
|
if(!((strcmp(a,"A0")==0&&strcmp(b,"B0")==0)||(strcmp(a,"B0")==0&&strcmp(b,"A0")==0))) allA=0; }
|
|
}
|
|
CHECK(allstrong, "skeleton holds only above-threshold (strong) edges");
|
|
CHECK(allA, "skeleton backbone is the intra-cluster wiring");
|
|
CHECK(g->co_registration>0.0, "co-registration positive (wired pairs are semantically near)");
|
|
|
|
/* principal axes: extents strictly non-increasing */
|
|
int mono=1; for(int i=1;i<g->n_axes;i++) if(g->axes[i].extent>g->axes[i-1].extent+1e-9) mono=0;
|
|
CHECK(g->n_axes>0 && mono, "principal axes sorted by descending extent");
|
|
CHECK(g->radius>0, "radius positive");
|
|
}
|
|
engram_geo_free(g);
|
|
|
|
/* edge cases: NULL store, no seeds, relational-only (NULL vindex) */
|
|
CHECK(engram_geometry_descriptor(NULL,ix,ids,nids,seeds,1,&P,gm)==NULL, "NULL store -> NULL");
|
|
CHECK(engram_geometry_descriptor(st,ix,ids,nids,seeds,0,&P,gm)==NULL, "zero seeds -> NULL");
|
|
GeoDescriptor* g2=engram_geometry_descriptor(st, NULL, NULL, 0, seeds, 1, &P, gm);
|
|
CHECK(g2!=NULL && g2->n_members>=NA-1, "relational-only path (no vindex) works");
|
|
engram_geo_free(g2);
|
|
/* raw (uncentered) mode still supported: global_mean=NULL -> no offset recorded */
|
|
GeoDescriptor* g3=engram_geometry_descriptor(st, ix, ids, nids, seeds, 1, &P, NULL);
|
|
CHECK(g3!=NULL && g3->global_mean==NULL, "raw mode (global_mean=NULL) leaves offset unset");
|
|
engram_geo_free(g3);
|
|
|
|
engram_geo_mean_free(mc);
|
|
for(int i=0;i<nids;i++) free(ids[i]); free(ids);
|
|
vindex_free(ix); store_close(st); unlink(path);
|
|
printf("\n=== %s ===\n", g_fail?"FAILURES PRESENT":"ALL TESTS PASSED");
|
|
return g_fail;
|
|
}
|