runtime: transduction is a language concern, so move it into the language
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
#141 let signal enter as geometry and it worked, but it was placed at the CONSUMER and said so in its own commit message. This is the correction. Three defects, all of them placement: 1. It sat in the engram. Ingest is a LANGUAGE concern — every el program touching any modality needs it, and the engram is merely one el program that happens to hold a graph. The geometry surface is now defined in el_runtime.c immediately ABOVE the engram section and depends on nothing inside it. Delete the entire engram and geometry still enters el. 2. It marshalled the vector as a hex STRING, because el had no first-class geometry value — which reintroduced text as the TRANSPORT medium one layer below the problem being fixed. Geometry is now an el value: a magic-tagged heap object carried in el_val_t, same discipline as List/Map. Hex survives only as an adapter at the edge, which is all an encoding should ever be. 3. It needed an arbitrary `dim <= 8192` bound purely to size an allocation from a caller's CLAIM about a string's length. A value carries its own width, so the width is derived and never asserted. The bound is gone, not raised — there is nothing left to validate. Language surface, none of it engram-prefixed: geometry_new / _dim / _is / _get / _set / _norm / _free, geometry_from_f32le_hex + geometry_to_f32le_hex as the wire adapters, realizer_register(modality, fn_name), realizer_has, and transduce(signal, modality) -> Geometry. REALIZERS ARE DECLARABLE IN EL. This is the part that makes the move real rather than nominal: registration resolves a name with dlsym against the running binary, the identical mechanism http_set_handler already relies on, because every el `fn name(...)` compiles to a global C symbol with that exact name. So an ordinary el function IS a realizer and a new modality needs no runtime patch. Verified end to end in lang/examples/transduce.el: an el-defined tone_realizer is registered by name, transduce dispatches to it, and the signal demonstrably reaches it (distinct signals produce distinct geometry). A modality with no realizer transduces to NOTHING. There is deliberately no built-in realizer, not even for text — silently embedding a description of a signal and calling that perception is the exact defect this ends. engram/src/server.el is migrated: POST /api/nodes decodes "emb" hex exactly once, at the edge, into a Geometry, and everything below that line moves geometry. The wire is unchanged because production clients speak it. "dim" is now an ASSERTION about the vector, not the source of its width; disagreement is a rejected ingest, not a silent reinterpretation. #141's engram_node_set_emb becomes a DEPRECATED WRAPPER over geometry_from_f32le_hex + node_attach_geometry — kept only because the runtime ships as an SDK asset and a downstream binary may link the symbol. Its exact contract, negative cases included, is preserved and re-verified. ingest.el's `fn transduce` is renamed transduce_manifold. Mechanically it had to yield the name (duplicate C symbol, a hard compile error, measured). But it was never signal->geometry: it chunks already-extracted content into a node+edge manifold, one layer up, and had taken the name belonging to the primitive underneath it. Behaviour unchanged. PROPERTIES FROM #141 PRESERVED, each re-measured on a scratch engram (:8971, never prod :8742): - off-dimension vectors stored but NOT indexed — the HNSW build loop still filters on n->emb_dim == dim at four sites, so a 64-dim voice vector is durable and addressable without perturbing the 768-dim canonical index - geometry makes a node ineligible for embed_backfill: after backfill the 64-dim voice node was still 64-dim while the text control acquired 768 - the create response reports whether geometry landed, and the node document always emits emb_dim and embedded Read-back with control and negatives, all verified against a PID-confirmed fresh binary: geometry node emb_dim=64 embedded=true / emb_set=1; text-only control emb_dim=0 embedded=false / emb_set=0; malformed hex, ragged length, and dim-disagreement each emb_set=0. Two compiler landmines found by reading the generated C rather than trusting a successful build, both documented at their sites: elc lowers `a == b` to str_eq unless both operand NAMES are in the per-function int-name set (which does NOT propagate into nested if-expression blocks — the first cut would have strcmp'd two integers as pointers on the first geometry-bearing request), and `+` lowers to string concat when either operand is a user-defined call.
This commit is contained in:
+52
-16
@@ -247,6 +247,24 @@ fn persist_bulk() -> Int {
|
||||
return persist_canonical()
|
||||
}
|
||||
|
||||
// COMPILER LANDMINE, measured 2026-08-16 — do not inline this back into the
|
||||
// caller. elc lowers `a == b` to numeric comparison only when both operand
|
||||
// NAMES are in the per-function int-name set, which `let x: Int` populates.
|
||||
// That registration does NOT propagate into a nested if-expression block: the
|
||||
// first cut of the geometry-ingest path wrote `let claimed: Int = ...` and
|
||||
// `let got: Int = ...` inside the else-arm and `claimed == got` came out of
|
||||
// codegen as `str_eq(claimed, got)` — strcmp on two integers reinterpreted as
|
||||
// pointers, i.e. a segfault on the first geometry-bearing request. Read back
|
||||
// out of the generated C, not guessed. Function PARAMETERS annotated `: Int`
|
||||
// do register reliably (verified: `if (claimed == actual)`), so the comparison
|
||||
// lives in a function of its own. Note also the explicit `return`s — a trailing
|
||||
// if-EXPRESSION at a function tail emits as a statement and the function
|
||||
// returns 0 regardless, which is the same probe's second finding.
|
||||
fn width_agrees(claimed: Int, actual: Int) -> Int {
|
||||
if claimed == actual { return 1 }
|
||||
return 0
|
||||
}
|
||||
|
||||
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
|
||||
// label, importance, tier, and tags — engram_node() defaults label to content
|
||||
// and importance to 0.5, so every node created over HTTP lost its metadata.
|
||||
@@ -288,26 +306,44 @@ fn route_create_node(method: String, path: String, body: String) -> String {
|
||||
salience, importance, confidence,
|
||||
tier, tags
|
||||
)
|
||||
// GEOMETRY INGEST (2026-08-16 self-review): this route accepted an "emb"
|
||||
// field, returned 200 with a fresh id, and stored NOTHING — engram_node_full
|
||||
// has no vector parameter, so the caller's geometry was silently discarded
|
||||
// and the node came back emb_dim=None / embedded:false. Measured live while
|
||||
// trying to admit a voice signal. The consequence was structural, not
|
||||
// cosmetic: text was the only entry medium, so any non-text modality had to
|
||||
// be DESCRIBED in prose and what we then reasoned over was the geometry of
|
||||
// the description, not of the signal.
|
||||
// GEOMETRY INGEST — geometry-valued end to end (2026-08-16).
|
||||
//
|
||||
// "emb" is little-endian float32 hex (dim*8 chars) — the encoding the
|
||||
// perception vessel's /voice/embed already emits, so a realizer's output
|
||||
// moves in with no float-array round trip. "dim" defaults to the vector's
|
||||
// implied width. Off-dimension vectors are stored but not inserted into the
|
||||
// resident index (its build loop filters on emb_dim), so a modality vector
|
||||
// is durable and addressable without perturbing the canonical index.
|
||||
// The defect this route originally had: it accepted an "emb" field,
|
||||
// returned 200 with a fresh id, and stored NOTHING, because engram_node_full
|
||||
// has no vector parameter. The consequence was structural, not cosmetic —
|
||||
// text was the only entry medium, so any non-text modality had to be
|
||||
// DESCRIBED in prose, and what we then reasoned over was the geometry of the
|
||||
// description, not of the signal.
|
||||
//
|
||||
// #141 fixed the drop but marshalled the vector as a hex STRING through
|
||||
// engram_node_set_emb, which put text back as the TRANSPORT medium one layer
|
||||
// below the problem being fixed. This is that correction: hex is decoded
|
||||
// exactly ONCE, here at the edge, into a first-class Geometry, and every
|
||||
// step below this line moves geometry rather than text. An encoding at the
|
||||
// boundary is what an encoding is for.
|
||||
//
|
||||
// The WIRE is deliberately unchanged — "emb" is still little-endian float32
|
||||
// hex (8 chars per component), the encoding the perception vessel's
|
||||
// /voice/embed already emits — because production clients speak it. What
|
||||
// changed is underneath it.
|
||||
//
|
||||
// "dim" is now treated as an ASSERTION about the vector the caller sent, not
|
||||
// as the source of its width: a Geometry carries its own width. A stated dim
|
||||
// that disagrees is a REJECTED ingest, not a silent reinterpretation. Omitting
|
||||
// "dim" is fine and means "trust the vector", which is the honest default.
|
||||
//
|
||||
// Off-dimension vectors remain stored but not inserted into the resident HNSW
|
||||
// index (its build loop filters on emb_dim), so a 64-dim voice geometry is
|
||||
// durable and addressable without perturbing the 768-dim canonical index.
|
||||
let emb_hex: String = json_get_string(body, "emb")
|
||||
let emb_set: Int = if str_eq(emb_hex, "") { 0 } else {
|
||||
let g: Geometry = geometry_from_f32le_hex(emb_hex)
|
||||
let got: Int = geometry_dim(g)
|
||||
let dim_raw: String = json_get_raw(body, "dim")
|
||||
let dim: Int = if str_eq(dim_raw, "") { str_len(emb_hex) / 8 } else { json_get_int(body, "dim") }
|
||||
engram_node_set_emb(id, emb_hex, dim)
|
||||
let claimed: Int = if str_eq(dim_raw, "") { got } else { json_get_int(body, "dim") }
|
||||
let landed: Int = if width_agrees(claimed, got) > 0 { node_attach_geometry(id, g) } else { 0 }
|
||||
let freed: Int = geometry_free(g)
|
||||
landed
|
||||
}
|
||||
let saved: Int = persist_node(id)
|
||||
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): connect the fresh node to its
|
||||
|
||||
Reference in New Issue
Block a user