runtime: transduction is a language concern, so move it into the language
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
El SDK CI - dev / build-and-test (pull_request) Failing after 14m58s
#141 let signal enter as geometry and it worked, but it was placed at the CONSUMER and said so in its own commit message. This is the correction. Three defects, all of them placement: 1. It sat in the engram. Ingest is a LANGUAGE concern — every el program touching any modality needs it, and the engram is merely one el program that happens to hold a graph. The geometry surface is now defined in el_runtime.c immediately ABOVE the engram section and depends on nothing inside it. Delete the entire engram and geometry still enters el. 2. It marshalled the vector as a hex STRING, because el had no first-class geometry value — which reintroduced text as the TRANSPORT medium one layer below the problem being fixed. Geometry is now an el value: a magic-tagged heap object carried in el_val_t, same discipline as List/Map. Hex survives only as an adapter at the edge, which is all an encoding should ever be. 3. It needed an arbitrary `dim <= 8192` bound purely to size an allocation from a caller's CLAIM about a string's length. A value carries its own width, so the width is derived and never asserted. The bound is gone, not raised — there is nothing left to validate. Language surface, none of it engram-prefixed: geometry_new / _dim / _is / _get / _set / _norm / _free, geometry_from_f32le_hex + geometry_to_f32le_hex as the wire adapters, realizer_register(modality, fn_name), realizer_has, and transduce(signal, modality) -> Geometry. REALIZERS ARE DECLARABLE IN EL. This is the part that makes the move real rather than nominal: registration resolves a name with dlsym against the running binary, the identical mechanism http_set_handler already relies on, because every el `fn name(...)` compiles to a global C symbol with that exact name. So an ordinary el function IS a realizer and a new modality needs no runtime patch. Verified end to end in lang/examples/transduce.el: an el-defined tone_realizer is registered by name, transduce dispatches to it, and the signal demonstrably reaches it (distinct signals produce distinct geometry). A modality with no realizer transduces to NOTHING. There is deliberately no built-in realizer, not even for text — silently embedding a description of a signal and calling that perception is the exact defect this ends. engram/src/server.el is migrated: POST /api/nodes decodes "emb" hex exactly once, at the edge, into a Geometry, and everything below that line moves geometry. The wire is unchanged because production clients speak it. "dim" is now an ASSERTION about the vector, not the source of its width; disagreement is a rejected ingest, not a silent reinterpretation. #141's engram_node_set_emb becomes a DEPRECATED WRAPPER over geometry_from_f32le_hex + node_attach_geometry — kept only because the runtime ships as an SDK asset and a downstream binary may link the symbol. Its exact contract, negative cases included, is preserved and re-verified. ingest.el's `fn transduce` is renamed transduce_manifold. Mechanically it had to yield the name (duplicate C symbol, a hard compile error, measured). But it was never signal->geometry: it chunks already-extracted content into a node+edge manifold, one layer up, and had taken the name belonging to the primitive underneath it. Behaviour unchanged. PROPERTIES FROM #141 PRESERVED, each re-measured on a scratch engram (:8971, never prod :8742): - off-dimension vectors stored but NOT indexed — the HNSW build loop still filters on n->emb_dim == dim at four sites, so a 64-dim voice vector is durable and addressable without perturbing the 768-dim canonical index - geometry makes a node ineligible for embed_backfill: after backfill the 64-dim voice node was still 64-dim while the text control acquired 768 - the create response reports whether geometry landed, and the node document always emits emb_dim and embedded Read-back with control and negatives, all verified against a PID-confirmed fresh binary: geometry node emb_dim=64 embedded=true / emb_set=1; text-only control emb_dim=0 embedded=false / emb_set=0; malformed hex, ragged length, and dim-disagreement each emb_set=0. Two compiler landmines found by reading the generated C rather than trusting a successful build, both documented at their sites: elc lowers `a == b` to str_eq unless both operand NAMES are in the per-function int-name set (which does NOT propagate into nested if-expression blocks — the first cut would have strcmp'd two integers as pointers on the first geometry-bearing request), and `+` lowers to string concat when either operand is a user-defined call.
This commit is contained in:
bigmerge
+373
-55
@@ -5959,6 +5959,308 @@ void el_cgi_init(el_val_t name, el_val_t dharma_id, el_val_t principal,
|
||||
}
|
||||
|
||||
|
||||
/* ── Geometry: signal as a first-class el value ──────────────────────────────
|
||||
*
|
||||
* WHY THIS IS IN THE LANGUAGE, AND WHY IT IS DEFINED HERE (2026-08-16).
|
||||
*
|
||||
* Until yesterday no El ingest path could carry a vector. Nodes took text,
|
||||
* and geometry was DERIVED from that text by engram_embed_backfill. Text was
|
||||
* therefore the mandatory entry medium: any non-text modality — audio, image,
|
||||
* sensor — had to be DESCRIBED in prose first, so the geometry we then
|
||||
* reasoned over was the geometry OF THE DESCRIPTION, not of the signal. That
|
||||
* is faking it. The architecture is: geometry in, always; we do not fake it,
|
||||
* we project.
|
||||
*
|
||||
* The first fix (#141, engram_node_set_emb) proved the path end to end but
|
||||
* placed it wrong in three ways, each of which this section corrects:
|
||||
*
|
||||
* 1. It sat at the CONSUMER. Transduction is a LANGUAGE concern — every El
|
||||
* program touching any modality needs it, not just the one that happens
|
||||
* to hold a graph. So this section is defined HERE, immediately above
|
||||
* the engram block, and depends on nothing inside it. The engram is a
|
||||
* client of this surface, not its owner. That ordering is the point:
|
||||
* you can delete the entire engram and geometry still enters El.
|
||||
*
|
||||
* 2. It marshalled the vector as a hex STRING, because El had no
|
||||
* first-class geometry value — which reintroduced text as the TRANSPORT
|
||||
* medium one layer below the problem being fixed. Geometry is now a
|
||||
* value. Hex survives only as a wire ADAPTER at the edge
|
||||
* (geometry_from/to_f32le_hex), which is all an encoding should ever be.
|
||||
*
|
||||
* 3. It needed an arbitrary `dim <= 8192` bound, purely to check a
|
||||
* caller-supplied dim against a string's length before allocating. A
|
||||
* real geometry value CARRIES its own width, so here the width is
|
||||
* derived and never asserted, and there is nothing left to validate.
|
||||
* The bound is gone rather than merely raised — the only thing that can
|
||||
* fail is the allocation itself, which is an honest failure.
|
||||
*
|
||||
* REPRESENTATION: magic-tagged heap object (see "Refcounted heap objects"),
|
||||
* carried in an el_val_t. The payload is a separate allocation so the header
|
||||
* never moves. The magic word is >= 0x80 in its MSB so the string/small-int
|
||||
* sniffing in looks_like_heap_obj can never confuse a Geometry for either.
|
||||
*
|
||||
* OWNERSHIP: a Geometry is owned by the El caller and released with
|
||||
* geometry_free. node_attach_geometry COPIES its payload into the node, so a
|
||||
* node and the caller's value have independent lifetimes and freeing one
|
||||
* never touches the other. Geometry deliberately does NOT participate in
|
||||
* el_retain/el_release: the shipped elc emits neither on let-bindings
|
||||
* (measured), so hooking it there would be dead code that could only ever
|
||||
* free a live vector early.
|
||||
*/
|
||||
|
||||
#define EL_MAGIC_GEOM 0xE1608E01u
|
||||
|
||||
typedef struct {
|
||||
ElHeader hdr;
|
||||
int32_t dim;
|
||||
float* v;
|
||||
} ElGeometry;
|
||||
|
||||
/* Resolve an el_val_t to a live Geometry, or NULL. Every accessor goes
|
||||
* through this, so a stale/foreign/zero value is a clean 0-return rather
|
||||
* than a dereference. */
|
||||
static ElGeometry* geom_of(el_val_t g) {
|
||||
if (!looks_like_heap_obj(g)) return NULL;
|
||||
ElGeometry* p = (ElGeometry*)(uintptr_t)g;
|
||||
if (p->hdr.magic != EL_MAGIC_GEOM) return NULL;
|
||||
return p;
|
||||
}
|
||||
|
||||
el_val_t geometry_new(el_val_t dim) {
|
||||
int32_t d = (int32_t)(int64_t)dim;
|
||||
if (d <= 0) return (el_val_t)0;
|
||||
ElGeometry* g = (ElGeometry*)malloc(sizeof(ElGeometry));
|
||||
if (!g) return (el_val_t)0;
|
||||
g->v = (float*)calloc((size_t)d, sizeof(float));
|
||||
if (!g->v) { free(g); return (el_val_t)0; }
|
||||
g->hdr.magic = EL_MAGIC_GEOM;
|
||||
g->hdr.refcount = 1;
|
||||
g->dim = d;
|
||||
return (el_val_t)(uintptr_t)g;
|
||||
}
|
||||
|
||||
el_val_t geometry_dim(el_val_t g) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
return p ? (el_val_t)p->dim : (el_val_t)0;
|
||||
}
|
||||
|
||||
el_val_t geometry_is(el_val_t g) {
|
||||
return geom_of(g) ? (el_val_t)1 : (el_val_t)0;
|
||||
}
|
||||
|
||||
el_val_t geometry_get(el_val_t g, el_val_t i) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
int64_t k = (int64_t)i;
|
||||
if (!p || k < 0 || k >= (int64_t)p->dim) return el_from_float(0.0);
|
||||
return el_from_float((double)p->v[k]);
|
||||
}
|
||||
|
||||
el_val_t geometry_set(el_val_t g, el_val_t i, el_val_t x) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
int64_t k = (int64_t)i;
|
||||
if (!p || k < 0 || k >= (int64_t)p->dim) return (el_val_t)0;
|
||||
p->v[k] = (float)el_to_float(x);
|
||||
return (el_val_t)1;
|
||||
}
|
||||
|
||||
el_val_t geometry_norm(el_val_t g) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
if (!p) return el_from_float(0.0);
|
||||
double s = 0.0;
|
||||
for (int32_t i = 0; i < p->dim; i++) s += (double)p->v[i] * (double)p->v[i];
|
||||
return el_from_float(sqrt(s));
|
||||
}
|
||||
|
||||
el_val_t geometry_free(el_val_t g) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
if (!p) return (el_val_t)0;
|
||||
free(p->v);
|
||||
p->hdr.magic = 0; /* poison so use-after-free is detected, as List/Map do */
|
||||
free(p);
|
||||
return (el_val_t)1;
|
||||
}
|
||||
|
||||
/* geometry_from_f32le_hex — decode little-endian float32 hex INTO geometry.
|
||||
*
|
||||
* This is the ONE place hex appears, and it appears as what it actually is:
|
||||
* an encoding at the boundary, not the medium El reasons in. The width is
|
||||
* DERIVED from the input length (8 hex chars per float32) and never supplied
|
||||
* by the caller — which is precisely why #141's arbitrary `dim <= 8192`
|
||||
* bound has no counterpart here. There is nothing to validate.
|
||||
*
|
||||
* Returns 0 on empty input, a length that is not a multiple of 8, or any
|
||||
* non-hex character. */
|
||||
el_val_t geometry_from_f32le_hex(el_val_t hex) {
|
||||
const char* s = EL_CSTR(hex);
|
||||
if (!s) return (el_val_t)0;
|
||||
size_t n = strlen(s);
|
||||
if (n == 0 || (n % 8u) != 0) return (el_val_t)0;
|
||||
size_t d = n / 8u;
|
||||
if (d > (size_t)INT32_MAX) return (el_val_t)0;
|
||||
|
||||
el_val_t gv = geometry_new((el_val_t)(int64_t)d);
|
||||
ElGeometry* g = geom_of(gv);
|
||||
if (!g) return (el_val_t)0;
|
||||
|
||||
for (size_t i = 0; i < d; i++) {
|
||||
uint32_t w = 0;
|
||||
for (int k = 0; k < 8; k++) {
|
||||
char c = s[i * 8u + (size_t)k];
|
||||
uint32_t nib;
|
||||
if (c >= '0' && c <= '9') nib = (uint32_t)(c - '0');
|
||||
else if (c >= 'a' && c <= 'f') nib = (uint32_t)(c - 'a' + 10);
|
||||
else if (c >= 'A' && c <= 'F') nib = (uint32_t)(c - 'A' + 10);
|
||||
else { geometry_free(gv); return (el_val_t)0; }
|
||||
w = (w << 4) | nib;
|
||||
}
|
||||
/* Hex is emitted little-endian byte order; rebuild the word. */
|
||||
uint32_t le = ((w & 0x000000FFu) << 24) | ((w & 0x0000FF00u) << 8) |
|
||||
((w & 0x00FF0000u) >> 8) | ((w & 0xFF000000u) >> 24);
|
||||
float f;
|
||||
memcpy(&f, &le, sizeof(f));
|
||||
g->v[i] = f;
|
||||
}
|
||||
return gv;
|
||||
}
|
||||
|
||||
/* geometry_to_f32le_hex — the egress adapter, exact inverse of the above.
|
||||
* Present so a program that must hand geometry to a non-El peer over a text
|
||||
* wire can do so explicitly, at the edge, instead of the language pretending
|
||||
* text was the medium all along. */
|
||||
el_val_t geometry_to_f32le_hex(el_val_t g) {
|
||||
ElGeometry* p = geom_of(g);
|
||||
if (!p) return EL_STR("");
|
||||
static const char* HEXD = "0123456789abcdef";
|
||||
size_t n = (size_t)p->dim * 8u;
|
||||
char* out = el_strbuf(n); /* arena-tracked; allocates n+1, exits on OOM */
|
||||
for (int32_t i = 0; i < p->dim; i++) {
|
||||
uint32_t w;
|
||||
memcpy(&w, &p->v[i], sizeof(w));
|
||||
/* Emit little-endian byte order: low byte first. */
|
||||
for (int b = 0; b < 4; b++) {
|
||||
uint32_t byte = (w >> (8 * b)) & 0xFFu;
|
||||
out[(size_t)i * 8u + (size_t)b * 2u] = HEXD[(byte >> 4) & 0xF];
|
||||
out[(size_t)i * 8u + (size_t)b * 2u + 1] = HEXD[byte & 0xF];
|
||||
}
|
||||
}
|
||||
out[n] = '\0';
|
||||
return (el_val_t)(uintptr_t)out;
|
||||
}
|
||||
|
||||
/* ── Realizers: transduction declared in El, not patched into the runtime ────
|
||||
*
|
||||
* A REALIZER maps one modality into geometry. The whole reason transduction
|
||||
* belongs in the language is that ADDING A MODALITY MUST NOT REQUIRE A
|
||||
* RUNTIME PATCH — otherwise "the realizers are in the engram" just becomes
|
||||
* "the realizers are in the runtime" and nothing has actually moved. So
|
||||
* realizers are declared in El and registered by NAME:
|
||||
*
|
||||
* fn tone_realizer(signal: String) -> Geometry {
|
||||
* let g: Geometry = geometry_new(8)
|
||||
* ... geometry_set(g, i, x) ...
|
||||
* g
|
||||
* }
|
||||
*
|
||||
* realizer_register("tone", "tone_realizer")
|
||||
* let g: Geometry = transduce(sample, "tone")
|
||||
*
|
||||
* The name→symbol step rides the identical, already load-bearing mechanism
|
||||
* http_set_handler uses (see "HTTP server"): every El `fn name(...)` compiles
|
||||
* to a global C symbol with that exact name, so dlsym(RTLD_DEFAULT, name)
|
||||
* against the running binary resolves an El-defined function. No codegen
|
||||
* change, no first-class function references, no runtime edit per modality.
|
||||
* A realizer written in El is a first-class realizer.
|
||||
*
|
||||
* A realizer may equally be a C symbol linked into the program; the registry
|
||||
* cannot tell the difference and has no reason to care.
|
||||
*/
|
||||
|
||||
typedef el_val_t (*el_realizer_fn)(el_val_t);
|
||||
|
||||
typedef struct {
|
||||
char* modality;
|
||||
el_realizer_fn fn;
|
||||
} ElRealizer;
|
||||
|
||||
static ElRealizer _realizers[64];
|
||||
static size_t _realizer_count = 0;
|
||||
static pthread_mutex_t _realizer_mu = PTHREAD_MUTEX_INITIALIZER;
|
||||
|
||||
static el_realizer_fn realizer_lookup(const char* m) {
|
||||
el_realizer_fn out = NULL;
|
||||
pthread_mutex_lock(&_realizer_mu);
|
||||
for (size_t i = 0; i < _realizer_count; i++) {
|
||||
if (strcmp(_realizers[i].modality, m) == 0) { out = _realizers[i].fn; break; }
|
||||
}
|
||||
pthread_mutex_unlock(&_realizer_mu);
|
||||
return out;
|
||||
}
|
||||
|
||||
el_val_t realizer_register(el_val_t modality, el_val_t fn_name) {
|
||||
const char* m = EL_CSTR(modality);
|
||||
const char* fn = EL_CSTR(fn_name);
|
||||
if (!m || !*m || !fn || !*fn) return (el_val_t)0;
|
||||
|
||||
/* An unresolvable name is a REGISTRATION FAILURE, reported as 0 — not a
|
||||
* silent no-op that only surfaces later as "this modality produces
|
||||
* nothing". Distinguishing "no organ" from "broken organ" at the moment
|
||||
* of wiring is the lesson #141 was written to enforce. */
|
||||
void* sym = dlsym(RTLD_DEFAULT, fn);
|
||||
if (!sym) return (el_val_t)0;
|
||||
|
||||
pthread_mutex_lock(&_realizer_mu);
|
||||
for (size_t i = 0; i < _realizer_count; i++) {
|
||||
if (strcmp(_realizers[i].modality, m) == 0) {
|
||||
_realizers[i].fn = (el_realizer_fn)sym; /* re-registration replaces */
|
||||
pthread_mutex_unlock(&_realizer_mu);
|
||||
return (el_val_t)1;
|
||||
}
|
||||
}
|
||||
if (_realizer_count < sizeof(_realizers) / sizeof(_realizers[0])) {
|
||||
/* _persist, NOT el_strdup: the registry outlives any request, and an
|
||||
* arena-tracked copy would be freed at el_request_end — leaving a
|
||||
* dangling modality name if a program registers a realizer from
|
||||
* inside a handler rather than at startup. */
|
||||
_realizers[_realizer_count].modality = el_strdup_persist(m);
|
||||
_realizers[_realizer_count].fn = (el_realizer_fn)sym;
|
||||
_realizer_count++;
|
||||
pthread_mutex_unlock(&_realizer_mu);
|
||||
return (el_val_t)1;
|
||||
}
|
||||
pthread_mutex_unlock(&_realizer_mu);
|
||||
return (el_val_t)0;
|
||||
}
|
||||
|
||||
el_val_t realizer_has(el_val_t modality) {
|
||||
const char* m = EL_CSTR(modality);
|
||||
if (!m || !*m) return (el_val_t)0;
|
||||
return realizer_lookup(m) ? (el_val_t)1 : (el_val_t)0;
|
||||
}
|
||||
|
||||
/* transduce — THE primitive: signal in, geometry out.
|
||||
*
|
||||
* Dispatches to the realizer registered for `modality`. Returns 0 (not a
|
||||
* Geometry) when no realizer is registered, and geometry_is() on the result
|
||||
* is the check.
|
||||
*
|
||||
* There is deliberately NO built-in realizer, not even for text. A modality
|
||||
* the program has declared no organ for is one it genuinely cannot sense,
|
||||
* and returning nothing is more honest than quietly embedding a description
|
||||
* of the signal and calling that perception — which is the exact failure
|
||||
* this whole change exists to end.
|
||||
*
|
||||
* The result is validated to actually BE a Geometry before it is handed
|
||||
* back, so a realizer that returns something else transduced nothing rather
|
||||
* than handing a caller a value that will misbehave far from here. */
|
||||
el_val_t transduce(el_val_t signal, el_val_t modality) {
|
||||
const char* m = EL_CSTR(modality);
|
||||
if (!m || !*m) return (el_val_t)0;
|
||||
el_realizer_fn fn = realizer_lookup(m);
|
||||
if (!fn) return (el_val_t)0;
|
||||
el_val_t g = fn(signal);
|
||||
return geom_of(g) ? g : (el_val_t)0;
|
||||
}
|
||||
|
||||
/* ── Batch 3: Engram in-process graph store ──────────────────────────────── */
|
||||
/*
|
||||
* Single global EngramStore allocated lazily on first call. All node and
|
||||
@@ -8563,80 +8865,96 @@ el_val_t engram_node_count(void) {
|
||||
return (el_val_t)engram_get()->node_count;
|
||||
}
|
||||
|
||||
/* engram_node_set_emb — attach GEOMETRY to an existing node.
|
||||
/* node_attach_geometry — a node acquires geometry.
|
||||
*
|
||||
* WHY THIS EXISTS (2026-08-16). Until now no ingest path could carry a
|
||||
* vector. engram_node / engram_node_full / engram_node_layered take text
|
||||
* only, and the sole way a node acquired an embedding was
|
||||
* engram_embed_backfill DERIVING one from n->content. That made text the
|
||||
* mandatory entry medium: any non-text modality (audio, image, sensor)
|
||||
* had to be described in prose first, and the geometry we then reasoned
|
||||
* over was the geometry OF THE DESCRIPTION, not of the signal. Measured
|
||||
* consequence: POST /api/nodes accepted an "emb" field, returned 200 with
|
||||
* a fresh id, and stored emb_dim=None / embedded:false — the vector was
|
||||
* silently discarded because no parameter existed to receive it.
|
||||
* Named for the operation, not for the store that happens to hold the node.
|
||||
* This is the geometry-valued ingest path that replaces #141's hex-string
|
||||
* one: nothing here parses text, and nothing here takes a caller's word for
|
||||
* how wide the vector is. The Geometry carries its own width.
|
||||
*
|
||||
* `hex` is little-endian float32, the encoding the perception vessel's
|
||||
* /voice/embed already emits, so a realizer's output moves in without a
|
||||
* JSON float-array round trip. Length must be exactly dim*8 hex chars.
|
||||
* The payload is COPIED into the node, so the node and the caller's Geometry
|
||||
* have independent lifetimes — the caller may geometry_free() immediately
|
||||
* after, and a later free of the node's emb never touches the El value.
|
||||
*
|
||||
* DIMENSION POLICY: dim need NOT equal the canonical text-embedding dim.
|
||||
* A modality vector of a different width is stored and is simply not
|
||||
* inserted into the resident HNSW index, whose build loop already filters
|
||||
* on `n->emb_dim == dim`. So off-dimension geometry is durable and
|
||||
* addressable without perturbing the canonical index.
|
||||
* DIMENSION POLICY (measured in #141, load-bearing — do not regress): dim
|
||||
* need NOT equal the canonical text-embedding width. An off-dimension vector
|
||||
* is stored and is simply not inserted into the resident HNSW index, whose
|
||||
* build loop already filters on `n->emb_dim == dim`. So a 64-dim voice
|
||||
* geometry is durable and addressable without perturbing the 768-dim
|
||||
* canonical index.
|
||||
*
|
||||
* Setting emb also makes the node ineligible for embed_backfill (which
|
||||
* only fills nodes with no emb), so a realizer's vector is never
|
||||
* Attaching geometry also makes the node ineligible for embed_backfill
|
||||
* (which fills only nodes with no emb), so a realizer's vector is never
|
||||
* overwritten by a text-derived one.
|
||||
*
|
||||
* Returns 1 on success, 0 on unknown id / malformed hex / bad dim. */
|
||||
el_val_t engram_node_set_emb(el_val_t id, el_val_t hex, el_val_t dim) {
|
||||
const char* sid = EL_CSTR(id);
|
||||
const char* sh = EL_CSTR(hex);
|
||||
int32_t d = (int32_t)(int64_t)dim;
|
||||
/* Bound the allocation. No max-dim constant existed because no caller
|
||||
* could supply a dim before this function; 8192 is generous for any
|
||||
* realizer (canonical text embeddings are 768, MFCC voice stats 64)
|
||||
* while keeping a malformed `dim` from requesting an unbounded malloc. */
|
||||
if (!sid || !*sid || !sh || d <= 0 || d > 8192) return (el_val_t)0;
|
||||
* Returns 1 on success, 0 on unknown id or a value that is not a Geometry. */
|
||||
el_val_t node_attach_geometry(el_val_t node_id, el_val_t g) {
|
||||
const char* sid = EL_CSTR(node_id);
|
||||
if (!sid || !*sid) return (el_val_t)0;
|
||||
|
||||
size_t need = (size_t)d * 8u; /* 4 bytes → 8 hex chars per float */
|
||||
if (strlen(sh) != need) return (el_val_t)0;
|
||||
ElGeometry* p = geom_of(g);
|
||||
if (!p || p->dim <= 0) return (el_val_t)0;
|
||||
|
||||
EngramNode* n = engram_find_node(sid);
|
||||
if (!n) return (el_val_t)0;
|
||||
|
||||
float* v = (float*)malloc(sizeof(float) * (size_t)d);
|
||||
float* v = (float*)malloc(sizeof(float) * (size_t)p->dim);
|
||||
if (!v) return (el_val_t)0;
|
||||
|
||||
for (int32_t i = 0; i < d; i++) {
|
||||
uint32_t w = 0;
|
||||
for (int k = 0; k < 8; k++) {
|
||||
char c = sh[(size_t)i * 8u + (size_t)k];
|
||||
uint32_t nib;
|
||||
if (c >= '0' && c <= '9') nib = (uint32_t)(c - '0');
|
||||
else if (c >= 'a' && c <= 'f') nib = (uint32_t)(c - 'a' + 10);
|
||||
else if (c >= 'A' && c <= 'F') nib = (uint32_t)(c - 'A' + 10);
|
||||
else { free(v); return (el_val_t)0; }
|
||||
w = (w << 4) | nib;
|
||||
}
|
||||
/* Hex is emitted little-endian byte order; rebuild the word. */
|
||||
uint32_t le = ((w & 0x000000FFu) << 24) | ((w & 0x0000FF00u) << 8) |
|
||||
((w & 0x00FF0000u) >> 8) | ((w & 0xFF000000u) >> 24);
|
||||
float f;
|
||||
memcpy(&f, &le, sizeof(f));
|
||||
v[i] = f;
|
||||
}
|
||||
memcpy(v, p->v, sizeof(float) * (size_t)p->dim);
|
||||
|
||||
free(n->emb);
|
||||
n->emb = v;
|
||||
n->emb_dim = d;
|
||||
n->emb = v;
|
||||
n->emb_dim = p->dim;
|
||||
n->updated_at = engram_now_ms();
|
||||
if (engram_store_enabled()) eg_store_put_node(n);
|
||||
return (el_val_t)1;
|
||||
}
|
||||
|
||||
/* node_geometry_dim — read the attached width back, 0 if the node carries
|
||||
* none. Exists so an attach is VERIFIED by reading it back rather than by
|
||||
* trusting a success return. That is not a nicety: #141 was misdiagnosed for
|
||||
* an hour precisely because a genuine ingest drop and a mere reporting gap
|
||||
* were indistinguishable from the outside. */
|
||||
el_val_t node_geometry_dim(el_val_t node_id) {
|
||||
const char* sid = EL_CSTR(node_id);
|
||||
if (!sid || !*sid) return (el_val_t)0;
|
||||
EngramNode* n = engram_find_node(sid);
|
||||
if (!n || !n->emb) return (el_val_t)0;
|
||||
return (el_val_t)n->emb_dim;
|
||||
}
|
||||
|
||||
/* engram_node_set_emb — DEPRECATED. Shipped in #141; superseded 2026-08-16
|
||||
* by geometry_from_f32le_hex + node_attach_geometry, and now implemented as
|
||||
* literally that.
|
||||
*
|
||||
* It is kept, rather than removed, for one reason only: the runtime is
|
||||
* published as an SDK asset, so a downstream binary may already be linking
|
||||
* this symbol. It is NOT kept because a hex string is an acceptable way to
|
||||
* move geometry between two pieces of El — it isn't, and that was the
|
||||
* placement defect. New code calls transduce() or geometry_from_f32le_hex()
|
||||
* plus node_attach_geometry().
|
||||
*
|
||||
* The #141 contract is preserved exactly, including its negative cases, so
|
||||
* this remains a drop-in: `dim` <= 0 rejects, malformed hex rejects, and a
|
||||
* `dim` that disagrees with the vector's actual width rejects. The
|
||||
* difference is that `dim` is now an ASSERTION checked against a width the
|
||||
* Geometry already knows, rather than the authority the allocation trusted —
|
||||
* which is why #141's arbitrary `dim <= 8192` guard has no counterpart here.
|
||||
* There is no longer an unbounded-malloc hazard to guard against. */
|
||||
el_val_t engram_node_set_emb(el_val_t id, el_val_t hex, el_val_t dim) {
|
||||
int32_t want = (int32_t)(int64_t)dim;
|
||||
if (want <= 0) return (el_val_t)0;
|
||||
|
||||
el_val_t gv = geometry_from_f32le_hex(hex);
|
||||
ElGeometry* p = geom_of(gv);
|
||||
if (!p) return (el_val_t)0; /* empty / malformed hex */
|
||||
if (p->dim != want) { geometry_free(gv); return (el_val_t)0; } /* length mismatch */
|
||||
|
||||
el_val_t ok = node_attach_geometry(id, gv);
|
||||
geometry_free(gv);
|
||||
return ok;
|
||||
}
|
||||
|
||||
/* ── Telemetry retention ────────────────────────────────────────────────────
|
||||
* (2026-07-16 self-review) InternalStateEvent nodes are append-only telemetry
|
||||
* (heartbeat, curiosity_scan, engram_sync) written ~3/min by the awareness
|
||||
|
||||
@@ -586,6 +586,60 @@ void el_runtime_dharma_event_arrive(const char* event_type,
|
||||
const char* payload,
|
||||
const char* source);
|
||||
|
||||
/* ── Geometry: signal as a first-class El value ──────────────────────────────
|
||||
*
|
||||
* A Geometry is an opaque, magic-tagged heap value carried in an el_val_t —
|
||||
* the same discipline as List/Map. It holds a width and a float32 payload,
|
||||
* and it is the medium a non-text modality enters in. Declared HERE, above
|
||||
* the engram block, because transduction is a LANGUAGE concern: every El
|
||||
* program touching any modality needs it, and the engram is merely one El
|
||||
* program that happens to hold a graph. See el_runtime.c ("Geometry: signal
|
||||
* as a first-class el value") for the full rationale.
|
||||
*
|
||||
* El-side type annotation is simply `Geometry` — an opaque boxed pointer,
|
||||
* exactly like Instant / Calendar / Rhythm. No codegen change is required.
|
||||
*
|
||||
* OWNERSHIP: a Geometry is owned by the El caller and released with
|
||||
* geometry_free. node_attach_geometry COPIES, so a node and the caller's
|
||||
* value have independent lifetimes. */
|
||||
|
||||
el_val_t geometry_new(el_val_t dim); /* zero-filled; 0 on failure */
|
||||
el_val_t geometry_dim(el_val_t g); /* width, 0 if not a Geometry */
|
||||
el_val_t geometry_is(el_val_t g); /* 1 if a live Geometry */
|
||||
el_val_t geometry_get(el_val_t g, el_val_t i); /* Float component */
|
||||
el_val_t geometry_set(el_val_t g, el_val_t i, el_val_t x); /* 1 ok / 0 out of range */
|
||||
el_val_t geometry_norm(el_val_t g); /* Float L2 — lets a caller
|
||||
* check a realizer emitted
|
||||
* signal, not zeros */
|
||||
el_val_t geometry_free(el_val_t g); /* 1 if freed, 0 if not a Geometry.
|
||||
* Returns a value (not void) so it
|
||||
* is safe in any El expression
|
||||
* position without a codegen
|
||||
* void-builtin table entry. */
|
||||
|
||||
/* Wire ADAPTERS — the only place an encoding appears, and only at the edge.
|
||||
* `f32le hex` is little-endian float32, 8 hex chars per component: the
|
||||
* encoding the perception vessel's /voice/embed already emits. The width is
|
||||
* DERIVED from the input length, never supplied by a caller — which is why
|
||||
* there is no max-dim constant here to validate a claimed length against. */
|
||||
el_val_t geometry_from_f32le_hex(el_val_t hex); /* 0 on empty/odd-length/non-hex */
|
||||
el_val_t geometry_to_f32le_hex(el_val_t g); /* "" if not a Geometry */
|
||||
|
||||
/* ── Realizers + transduce ───────────────────────────────────────────────────
|
||||
* A REALIZER maps one modality into geometry. Registration is by NAME, so a
|
||||
* new modality never requires a runtime patch: every El `fn name(...)`
|
||||
* compiles to a global C symbol with that exact name, and the registry
|
||||
* resolves it with dlsym against the running binary — the same mechanism
|
||||
* http_set_handler already relies on.
|
||||
*
|
||||
* fn tone_realizer(signal: String) -> Geometry { ... }
|
||||
* realizer_register("tone", "tone_realizer")
|
||||
* let g: Geometry = transduce(sample, "tone")
|
||||
*/
|
||||
el_val_t realizer_register(el_val_t modality, el_val_t fn_name); /* 1 ok / 0 unresolved */
|
||||
el_val_t realizer_has(el_val_t modality); /* 1 if a realizer is registered */
|
||||
el_val_t transduce(el_val_t signal, el_val_t modality); /* Geometry, or 0 if no organ */
|
||||
|
||||
/* ── Engram local graph primitives ───────────────────────────────────────────
|
||||
* Operate on the CGI's local Engram knowledge graph.
|
||||
* `engram_activate` queries the local graph only; `dharma_activate` is
|
||||
@@ -613,10 +667,22 @@ void engram_strengthen(el_val_t node_id);
|
||||
void engram_forget(el_val_t node_id);
|
||||
el_val_t engram_prune_telemetry(el_val_t older_than_ms);
|
||||
el_val_t engram_node_count(void);
|
||||
/* Attach geometry to an existing node. `hex` is little-endian float32,
|
||||
* exactly dim*8 hex chars — the encoding realizers already emit. Lets a
|
||||
* non-text modality enter as geometry instead of being described in prose
|
||||
* and embedded as its description. Returns 1 on success, 0 otherwise. */
|
||||
/* Attach a Geometry to an existing node, and read the attached width back.
|
||||
* Named for the operation, not the store: a node acquires geometry. This is
|
||||
* the geometry-valued ingest path — nothing about it is hex, and nothing
|
||||
* about it assumes the caller's vector matches the canonical text-embedding
|
||||
* width. node_geometry_dim exists so an attach is VERIFIED by reading it
|
||||
* back rather than by trusting a success return. */
|
||||
el_val_t node_attach_geometry(el_val_t node_id, el_val_t g); /* 1 ok / 0 otherwise */
|
||||
el_val_t node_geometry_dim(el_val_t node_id); /* width, 0 if none */
|
||||
|
||||
/* DEPRECATED (shipped in #141, superseded 2026-08-16). Equivalent to
|
||||
* geometry_from_f32le_hex + node_attach_geometry, and now implemented as
|
||||
* exactly that. Kept only so anything built against the #141 runtime keeps
|
||||
* linking; `dim` is accepted but treated as an assertion about the vector's
|
||||
* width rather than as its source. New code should not call this — a hex
|
||||
* string is a wire encoding, not a way to move geometry between two pieces
|
||||
* of El. Returns 1 on success, 0 otherwise. */
|
||||
el_val_t engram_node_set_emb(el_val_t id, el_val_t hex, el_val_t dim);
|
||||
el_val_t engram_search(el_val_t query, el_val_t limit);
|
||||
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
|
||||
|
||||
Reference in New Issue
Block a user