runtime: transduction is a language concern, so move it into the language
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
#141 let signal enter as geometry and it worked, but it was placed at the CONSUMER and said so in its own commit message. This is the correction. Three defects, all of them placement: 1. It sat in the engram. Ingest is a LANGUAGE concern — every el program touching any modality needs it, and the engram is merely one el program that happens to hold a graph. The geometry surface is now defined in el_runtime.c immediately ABOVE the engram section and depends on nothing inside it. Delete the entire engram and geometry still enters el. 2. It marshalled the vector as a hex STRING, because el had no first-class geometry value — which reintroduced text as the TRANSPORT medium one layer below the problem being fixed. Geometry is now an el value: a magic-tagged heap object carried in el_val_t, same discipline as List/Map. Hex survives only as an adapter at the edge, which is all an encoding should ever be. 3. It needed an arbitrary `dim <= 8192` bound purely to size an allocation from a caller's CLAIM about a string's length. A value carries its own width, so the width is derived and never asserted. The bound is gone, not raised — there is nothing left to validate. Language surface, none of it engram-prefixed: geometry_new / _dim / _is / _get / _set / _norm / _free, geometry_from_f32le_hex + geometry_to_f32le_hex as the wire adapters, realizer_register(modality, fn_name), realizer_has, and transduce(signal, modality) -> Geometry. REALIZERS ARE DECLARABLE IN EL. This is the part that makes the move real rather than nominal: registration resolves a name with dlsym against the running binary, the identical mechanism http_set_handler already relies on, because every el `fn name(...)` compiles to a global C symbol with that exact name. So an ordinary el function IS a realizer and a new modality needs no runtime patch. Verified end to end in lang/examples/transduce.el: an el-defined tone_realizer is registered by name, transduce dispatches to it, and the signal demonstrably reaches it (distinct signals produce distinct geometry). A modality with no realizer transduces to NOTHING. There is deliberately no built-in realizer, not even for text — silently embedding a description of a signal and calling that perception is the exact defect this ends. engram/src/server.el is migrated: POST /api/nodes decodes "emb" hex exactly once, at the edge, into a Geometry, and everything below that line moves geometry. The wire is unchanged because production clients speak it. "dim" is now an ASSERTION about the vector, not the source of its width; disagreement is a rejected ingest, not a silent reinterpretation. #141's engram_node_set_emb becomes a DEPRECATED WRAPPER over geometry_from_f32le_hex + node_attach_geometry — kept only because the runtime ships as an SDK asset and a downstream binary may link the symbol. Its exact contract, negative cases included, is preserved and re-verified. ingest.el's `fn transduce` is renamed transduce_manifold. Mechanically it had to yield the name (duplicate C symbol, a hard compile error, measured). But it was never signal->geometry: it chunks already-extracted content into a node+edge manifold, one layer up, and had taken the name belonging to the primitive underneath it. Behaviour unchanged. PROPERTIES FROM #141 PRESERVED, each re-measured on a scratch engram (:8971, never prod :8742): - off-dimension vectors stored but NOT indexed — the HNSW build loop still filters on n->emb_dim == dim at four sites, so a 64-dim voice vector is durable and addressable without perturbing the 768-dim canonical index - geometry makes a node ineligible for embed_backfill: after backfill the 64-dim voice node was still 64-dim while the text control acquired 768 - the create response reports whether geometry landed, and the node document always emits emb_dim and embedded Read-back with control and negatives, all verified against a PID-confirmed fresh binary: geometry node emb_dim=64 embedded=true / emb_set=1; text-only control emb_dim=0 embedded=false / emb_set=0; malformed hex, ragged length, and dim-disagreement each emb_set=0. Two compiler landmines found by reading the generated C rather than trusting a successful build, both documented at their sites: elc lowers `a == b` to str_eq unless both operand NAMES are in the per-function int-name set (which does NOT propagate into nested if-expression blocks — the first cut would have strcmp'd two integers as pointers on the first geometry-bearing request), and `+` lowers to string concat when either operand is a user-defined call.
This commit is contained in:
@@ -586,6 +586,60 @@ void el_runtime_dharma_event_arrive(const char* event_type,
|
||||
const char* payload,
|
||||
const char* source);
|
||||
|
||||
/* ── Geometry: signal as a first-class El value ──────────────────────────────
|
||||
*
|
||||
* A Geometry is an opaque, magic-tagged heap value carried in an el_val_t —
|
||||
* the same discipline as List/Map. It holds a width and a float32 payload,
|
||||
* and it is the medium a non-text modality enters in. Declared HERE, above
|
||||
* the engram block, because transduction is a LANGUAGE concern: every El
|
||||
* program touching any modality needs it, and the engram is merely one El
|
||||
* program that happens to hold a graph. See el_runtime.c ("Geometry: signal
|
||||
* as a first-class el value") for the full rationale.
|
||||
*
|
||||
* El-side type annotation is simply `Geometry` — an opaque boxed pointer,
|
||||
* exactly like Instant / Calendar / Rhythm. No codegen change is required.
|
||||
*
|
||||
* OWNERSHIP: a Geometry is owned by the El caller and released with
|
||||
* geometry_free. node_attach_geometry COPIES, so a node and the caller's
|
||||
* value have independent lifetimes. */
|
||||
|
||||
el_val_t geometry_new(el_val_t dim); /* zero-filled; 0 on failure */
|
||||
el_val_t geometry_dim(el_val_t g); /* width, 0 if not a Geometry */
|
||||
el_val_t geometry_is(el_val_t g); /* 1 if a live Geometry */
|
||||
el_val_t geometry_get(el_val_t g, el_val_t i); /* Float component */
|
||||
el_val_t geometry_set(el_val_t g, el_val_t i, el_val_t x); /* 1 ok / 0 out of range */
|
||||
el_val_t geometry_norm(el_val_t g); /* Float L2 — lets a caller
|
||||
* check a realizer emitted
|
||||
* signal, not zeros */
|
||||
el_val_t geometry_free(el_val_t g); /* 1 if freed, 0 if not a Geometry.
|
||||
* Returns a value (not void) so it
|
||||
* is safe in any El expression
|
||||
* position without a codegen
|
||||
* void-builtin table entry. */
|
||||
|
||||
/* Wire ADAPTERS — the only place an encoding appears, and only at the edge.
|
||||
* `f32le hex` is little-endian float32, 8 hex chars per component: the
|
||||
* encoding the perception vessel's /voice/embed already emits. The width is
|
||||
* DERIVED from the input length, never supplied by a caller — which is why
|
||||
* there is no max-dim constant here to validate a claimed length against. */
|
||||
el_val_t geometry_from_f32le_hex(el_val_t hex); /* 0 on empty/odd-length/non-hex */
|
||||
el_val_t geometry_to_f32le_hex(el_val_t g); /* "" if not a Geometry */
|
||||
|
||||
/* ── Realizers + transduce ───────────────────────────────────────────────────
|
||||
* A REALIZER maps one modality into geometry. Registration is by NAME, so a
|
||||
* new modality never requires a runtime patch: every El `fn name(...)`
|
||||
* compiles to a global C symbol with that exact name, and the registry
|
||||
* resolves it with dlsym against the running binary — the same mechanism
|
||||
* http_set_handler already relies on.
|
||||
*
|
||||
* fn tone_realizer(signal: String) -> Geometry { ... }
|
||||
* realizer_register("tone", "tone_realizer")
|
||||
* let g: Geometry = transduce(sample, "tone")
|
||||
*/
|
||||
el_val_t realizer_register(el_val_t modality, el_val_t fn_name); /* 1 ok / 0 unresolved */
|
||||
el_val_t realizer_has(el_val_t modality); /* 1 if a realizer is registered */
|
||||
el_val_t transduce(el_val_t signal, el_val_t modality); /* Geometry, or 0 if no organ */
|
||||
|
||||
/* ── Engram local graph primitives ───────────────────────────────────────────
|
||||
* Operate on the CGI's local Engram knowledge graph.
|
||||
* `engram_activate` queries the local graph only; `dharma_activate` is
|
||||
@@ -613,10 +667,22 @@ void engram_strengthen(el_val_t node_id);
|
||||
void engram_forget(el_val_t node_id);
|
||||
el_val_t engram_prune_telemetry(el_val_t older_than_ms);
|
||||
el_val_t engram_node_count(void);
|
||||
/* Attach geometry to an existing node. `hex` is little-endian float32,
|
||||
* exactly dim*8 hex chars — the encoding realizers already emit. Lets a
|
||||
* non-text modality enter as geometry instead of being described in prose
|
||||
* and embedded as its description. Returns 1 on success, 0 otherwise. */
|
||||
/* Attach a Geometry to an existing node, and read the attached width back.
|
||||
* Named for the operation, not the store: a node acquires geometry. This is
|
||||
* the geometry-valued ingest path — nothing about it is hex, and nothing
|
||||
* about it assumes the caller's vector matches the canonical text-embedding
|
||||
* width. node_geometry_dim exists so an attach is VERIFIED by reading it
|
||||
* back rather than by trusting a success return. */
|
||||
el_val_t node_attach_geometry(el_val_t node_id, el_val_t g); /* 1 ok / 0 otherwise */
|
||||
el_val_t node_geometry_dim(el_val_t node_id); /* width, 0 if none */
|
||||
|
||||
/* DEPRECATED (shipped in #141, superseded 2026-08-16). Equivalent to
|
||||
* geometry_from_f32le_hex + node_attach_geometry, and now implemented as
|
||||
* exactly that. Kept only so anything built against the #141 runtime keeps
|
||||
* linking; `dim` is accepted but treated as an assertion about the vector's
|
||||
* width rather than as its source. New code should not call this — a hex
|
||||
* string is a wire encoding, not a way to move geometry between two pieces
|
||||
* of El. Returns 1 on success, 0 otherwise. */
|
||||
el_val_t engram_node_set_emb(el_val_t id, el_val_t hex, el_val_t dim);
|
||||
el_val_t engram_search(el_val_t query, el_val_t limit);
|
||||
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
|
||||
|
||||
Reference in New Issue
Block a user