Compare commits
45 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 688f24b4c1 | |||
| d777936ee4 | |||
| 4a57b4faa8 | |||
| 0ee82d9e91 | |||
| 9526bda507 | |||
| 0389bf9363 | |||
| fe820928b0 | |||
| 385c18442d | |||
| cace6a5ebf | |||
| 7a1501d097 | |||
| d41645388a | |||
| 8a307dfd42 | |||
| 616815b2ab | |||
| 1a8a966cb3 | |||
| 1f70b9fa18 | |||
| 317466e8f7 | |||
| eb3e6d7c1f | |||
| 88e3008735 | |||
| 26af149aa1 | |||
| c18abf799c | |||
| b305b49f40 | |||
| 8ae163e8e5 | |||
| 3fcc36c2f1 | |||
| a6cef4b983 | |||
| 8e9d88fc01 | |||
| e99a4640e2 | |||
| bdc1f99fb9 | |||
| 44b621e551 | |||
| ded6ca546f | |||
| b5b96c05ed | |||
| c79033b749 | |||
| 1119295238 | |||
| 9c07970943 | |||
| 0832865952 | |||
| e0b2c0ea54 | |||
| cf060adbfd | |||
| 63fe8a766d | |||
| b5a0a729e6 | |||
| b26dd47aef | |||
| b55e6bfd53 | |||
| dbb06f6ee4 | |||
| 6a6b589ba0 | |||
| b5d1e53902 | |||
| 6291a35bb9 | |||
| 4c3414072b |
@@ -61,6 +61,13 @@ Per test file, current build model:
|
|||||||
| `cc` test .c → .o | 0.02s |
|
| `cc` test .c → .o | 0.02s |
|
||||||
| link | 0.02s |
|
| link | 0.02s |
|
||||||
|
|
||||||
|
> **STALE as of el #132 — re-measured 2026-08-16.** The `test_compiler` figure below was
|
||||||
|
> *entirely* the `strlen`-per-character quadratic, now fixed. Re-measured on the same host:
|
||||||
|
> **3.58s → 0.03s (119x)**, and the 422 KB compiler concatenation likewise compiles in 0.03s.
|
||||||
|
> The table is retained only as the historical record that motivated the gate. The remaining
|
||||||
|
> per-file cost is the redundant `el_runtime.c` rebuild, which §9's compile-once architecture
|
||||||
|
> addresses.
|
||||||
|
|
||||||
Per-file `elc` time across the existing suite:
|
Per-file `elc` time across the existing suite:
|
||||||
|
|
||||||
| File | Bytes | elc time |
|
| File | Bytes | elc time |
|
||||||
@@ -416,6 +423,48 @@ Wall-clock needs statistics. **Allocation counts do not.** They are perfectly de
|
|||||||
> a level. That is why the gate fits a curve across a sweep instead of comparing one number to a
|
> a level. That is why the gate fits a curve across a sweep instead of comparing one number to a
|
||||||
> threshold.
|
> threshold.
|
||||||
|
|
||||||
|
> **Second correction, same day — THE ALLOCATION GATE ALONE WOULD HAVE MISSED THE REAL BUG.**
|
||||||
|
>
|
||||||
|
> el #132 found the actual elc quadratic: `strlen()` called inside `str_char_code()` and
|
||||||
|
> `str_slice()`, so the lexer rescanned the remaining input on every character. Pure CPU.
|
||||||
|
> **Zero allocation.** `str_char_code` is a bounds check and an index — it allocates nothing.
|
||||||
|
>
|
||||||
|
> Measured on three controlled specimens (`lang/.work/fitprobe.el`), growth ratio per doubling of
|
||||||
|
> n across n = 200/400/800/1600:
|
||||||
|
>
|
||||||
|
> | specimen | allocs | bytes | time | what it proves |
|
||||||
|
> |---|---|---|---|---|
|
||||||
|
> | `linear` — one alloc per item | 2.00 2.00 2.00 → **O(n)** | 2.16 2.07 2.23 → **O(n)** | 0.83 2.00 2.05 → **O(n)** | clean baseline |
|
||||||
|
> | `accum` — rebuilds accumulator | 2.00 2.00 2.00 → **O(n)** | 3.97 3.99 3.99 → **O(n²)** | noisy | count misses, **bytes catches** |
|
||||||
|
> | `compute` — n scans over n chars | 0 → **FLAT** | 0 → **FLAT** | 3.93 4.01 3.96 → **O(n²)** | **both alloc signals blind; only time catches** |
|
||||||
|
>
|
||||||
|
> `compute` is el #132's shape exactly. A gate fitting only allocation count and bytes classifies
|
||||||
|
> it as FLAT and passes it. **The gate as originally specified would not have caught the defect it
|
||||||
|
> was created for.**
|
||||||
|
>
|
||||||
|
> Therefore the gate fits **THREE** signals and fails if ANY exceeds its declared curve:
|
||||||
|
>
|
||||||
|
> ```
|
||||||
|
> bench "elc_compile" over n in [...] expect time O(n) allocs O(n) bytes O(n) { ... }
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> - **allocs (count)** — deterministic, zero-noise. Catches per-item allocation growth.
|
||||||
|
> - **allocs (bytes)** — deterministic, zero-noise. Catches accumulator-rebuild quadratics that
|
||||||
|
> count cannot see.
|
||||||
|
> - **time** — noisy, needs the sweep and statistics. The ONLY signal that sees pure-compute
|
||||||
|
> complexity regressions. Gate on the fitted *exponent*, never on absolute duration, so CI
|
||||||
|
> hardware variance scales the coefficient and leaves the classification intact.
|
||||||
|
>
|
||||||
|
> The deterministic signals remain preferable where they apply — they need no statistics and are
|
||||||
|
> correct on the first run. They are simply not sufficient.
|
||||||
|
>
|
||||||
|
> **`black_box` is mandatory, and consuming the result is NOT enough.** The first version of
|
||||||
|
> `compute` accumulated `total + 1` in a nested loop and reported **0 µs at every n** while
|
||||||
|
> returning a numerically correct n². Clang recognised the idiom and closed the loop to a
|
||||||
|
> multiply. Feeding the result into output did not prevent it. Only making the inner operation an
|
||||||
|
> opaque external call restored the real curve. A benchmark harness that trusts the user to defeat
|
||||||
|
> the optimiser will silently measure nothing — and report success while doing it.
|
||||||
|
|
||||||
Instrument the runtime with allocation counters and fit *those* against n instead of time:
|
Instrument the runtime with allocation counters and fit *those* against n instead of time:
|
||||||
|
|
||||||
```el
|
```el
|
||||||
|
|||||||
+197
-38
@@ -10,10 +10,60 @@
|
|||||||
// cc -std=c11 -O2 -lcurl -lpthread -o engram server.c el_runtime.c
|
// cc -std=c11 -O2 -lcurl -lpthread -o engram server.c el_runtime.c
|
||||||
// ./engram
|
// ./engram
|
||||||
//
|
//
|
||||||
// Configuration via environment:
|
// Configuration is DECLARED, not scattered. See the `program` block below:
|
||||||
// ENGRAM_BIND — host:port (default :8742)
|
// every knob's type and default lives there and nowhere else, is resolved from
|
||||||
// ENGRAM_API_KEY — bearer auth (optional)
|
// the environment (env wins, declaration is the fallback) and validated before
|
||||||
// ENGRAM_DATA_DIR — snapshot location (default ~/.neuron/engram)
|
// any statement of this file runs. Read one with config("NAME") -> String.
|
||||||
|
//
|
||||||
|
// The one deliberate exception is ENGRAM_DATA_DIR — see the note in the block.
|
||||||
|
|
||||||
|
// ── Program declaration (cross-cutting concerns) ──────────────────────────────
|
||||||
|
//
|
||||||
|
// singleton: two engram processes against one data dir is data loss, not a
|
||||||
|
// warning. The runtime takes an exclusive flock at startup and a second start
|
||||||
|
// is refused loudly with the holder's pid.
|
||||||
|
//
|
||||||
|
// NOT declared here, on purpose: ENGRAM_DATA_DIR. Its resolution is owned by
|
||||||
|
// engram_resolve_data_dir() (el_runtime.c), which defaults to $HOME/.neuron/engram
|
||||||
|
// and fails LOUD rather than silently persisting to an ephemeral directory.
|
||||||
|
// Declaring a default for it here as well would put the data dir's fallback in
|
||||||
|
// two places — which is precisely the defect this migration removes (until
|
||||||
|
// 2026-08-15 the reseed backup path carried its own "/tmp/engram" default that
|
||||||
|
// disagreed with the resolver, so the pre-destructive safety copy landed in /tmp).
|
||||||
|
// HOME is likewise not declared: it is a genuine environment read, not a knob.
|
||||||
|
program "engram" {
|
||||||
|
singleton: "engram"
|
||||||
|
|
||||||
|
// ── Core server ──
|
||||||
|
env ENGRAM_BIND: String = ":8742"
|
||||||
|
// Default "" leaves auth DISABLED (check_auth_ok short-circuits to true on an
|
||||||
|
// empty key). That is the pre-existing behaviour and is deliberately preserved
|
||||||
|
// here; making this `required` is the obvious hardening follow-up, but it is a
|
||||||
|
// behaviour change and out of scope for this migration.
|
||||||
|
env ENGRAM_API_KEY: String = ""
|
||||||
|
|
||||||
|
// ── Feature flags (bool-ish Strings; the predicate fns below own truthiness) ──
|
||||||
|
env ENGRAM_STORE: String = "off"
|
||||||
|
env ENGRAM_WAL: String = "off"
|
||||||
|
env ENGRAM_AUTOCONNECT: String = "off"
|
||||||
|
env ENGRAM_ISE_OFFGRAPH: String = "off"
|
||||||
|
|
||||||
|
// ── ISE telemetry ──
|
||||||
|
env ENGRAM_ISE_RETENTION_MS: Int = "172800000"
|
||||||
|
|
||||||
|
// ── Guide (local Qwen3 via llama-server) ──
|
||||||
|
env GUIDE_ENABLE: String = "off"
|
||||||
|
env GUIDE_TIER_FORCE: String = ""
|
||||||
|
env GUIDE_CACHE_DIR: String = ""
|
||||||
|
env GUIDE_RAM_GB_4B: Int = "16"
|
||||||
|
env GUIDE_RAM_GB_1P7B: Int = "8"
|
||||||
|
env GUIDE_BACKEND: String = "llama-server"
|
||||||
|
env GUIDE_HOST: String = "127.0.0.1"
|
||||||
|
env GUIDE_PORT: Int = "8771"
|
||||||
|
env GUIDE_LLAMA_SERVER_BIN: String = "llama-server"
|
||||||
|
env GUIDE_NGL: Int = "99"
|
||||||
|
env GUIDE_CTX: Int = "4096"
|
||||||
|
}
|
||||||
|
|
||||||
// ── Helpers ───────────────────────────────────────────────────────────────────
|
// ── Helpers ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -133,7 +183,7 @@ fn route_text_health(method: String, path: String, body: String) -> String {
|
|||||||
// engram_store_enabled() in el_runtime.c EXACTLY (1 / on / true). Default off →
|
// engram_store_enabled() in el_runtime.c EXACTLY (1 / on / true). Default off →
|
||||||
// every persistence path below is byte-for-byte the historical snapshot behavior.
|
// every persistence path below is byte-for-byte the historical snapshot behavior.
|
||||||
fn store_on() -> Bool {
|
fn store_on() -> Bool {
|
||||||
let v: String = env("ENGRAM_STORE")
|
let v: String = config("ENGRAM_STORE")
|
||||||
if str_eq(v, "1") { return true }
|
if str_eq(v, "1") { return true }
|
||||||
if str_eq(v, "on") { return true }
|
if str_eq(v, "on") { return true }
|
||||||
if str_eq(v, "true") { return true }
|
if str_eq(v, "true") { return true }
|
||||||
@@ -162,7 +212,6 @@ fn persist_canonical() -> Int {
|
|||||||
if store_on() {
|
if store_on() {
|
||||||
return engram_store_checkpoint()
|
return engram_store_checkpoint()
|
||||||
}
|
}
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
|
||||||
let dir: String = engram_resolve_data_dir()
|
let dir: String = engram_resolve_data_dir()
|
||||||
// (2026-08-10 self-review) This returned a hardcoded 1, which made every
|
// (2026-08-10 self-review) This returned a hardcoded 1, which made every
|
||||||
// caller's `let saved: Int = persist_canonical()` a dead variable — six
|
// caller's `let saved: Int = persist_canonical()` a dead variable — six
|
||||||
@@ -176,7 +225,7 @@ fn persist_canonical() -> Int {
|
|||||||
// per-write full-snapshot behavior. When ON, structural mutations append O(1)
|
// per-write full-snapshot behavior. When ON, structural mutations append O(1)
|
||||||
// WAL records instead of rewriting the whole graph, with threshold compaction.
|
// WAL records instead of rewriting the whole graph, with threshold compaction.
|
||||||
fn wal_on() -> Bool {
|
fn wal_on() -> Bool {
|
||||||
str_eq(env("ENGRAM_WAL"), "on")
|
str_eq(config("ENGRAM_WAL"), "on")
|
||||||
}
|
}
|
||||||
|
|
||||||
// autoconnect_on — ENGRAM_AUTOCONNECT. Will's rule: "we shouldn't be inserting
|
// autoconnect_on — ENGRAM_AUTOCONNECT. Will's rule: "we shouldn't be inserting
|
||||||
@@ -184,7 +233,7 @@ fn wal_on() -> Bool {
|
|||||||
// edge (kNN over embeddings) so no content node enters the graph edgeless.
|
// edge (kNN over embeddings) so no content node enters the graph edgeless.
|
||||||
// Default OFF -> byte-identical to prior behavior (node created, no auto edges).
|
// Default OFF -> byte-identical to prior behavior (node created, no auto edges).
|
||||||
fn autoconnect_on() -> Bool {
|
fn autoconnect_on() -> Bool {
|
||||||
let v: String = env("ENGRAM_AUTOCONNECT")
|
let v: String = config("ENGRAM_AUTOCONNECT")
|
||||||
if str_eq(v, "1") { return true }
|
if str_eq(v, "1") { return true }
|
||||||
if str_eq(v, "on") { return true }
|
if str_eq(v, "on") { return true }
|
||||||
if str_eq(v, "true") { return true }
|
if str_eq(v, "true") { return true }
|
||||||
@@ -197,7 +246,7 @@ fn autoconnect_on() -> Bool {
|
|||||||
// separate state-event log tier instead of the node graph. Default OFF -> ISEs
|
// separate state-event log tier instead of the node graph. Default OFF -> ISEs
|
||||||
// remain graph nodes exactly as before (with 48h prune).
|
// remain graph nodes exactly as before (with 48h prune).
|
||||||
fn ise_offgraph_on() -> Bool {
|
fn ise_offgraph_on() -> Bool {
|
||||||
let v: String = env("ENGRAM_ISE_OFFGRAPH")
|
let v: String = config("ENGRAM_ISE_OFFGRAPH")
|
||||||
if str_eq(v, "1") { return true }
|
if str_eq(v, "1") { return true }
|
||||||
if str_eq(v, "on") { return true }
|
if str_eq(v, "on") { return true }
|
||||||
if str_eq(v, "true") { return true }
|
if str_eq(v, "true") { return true }
|
||||||
@@ -247,6 +296,24 @@ fn persist_bulk() -> Int {
|
|||||||
return persist_canonical()
|
return persist_canonical()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// COMPILER LANDMINE, measured 2026-08-16 — do not inline this back into the
|
||||||
|
// caller. elc lowers `a == b` to numeric comparison only when both operand
|
||||||
|
// NAMES are in the per-function int-name set, which `let x: Int` populates.
|
||||||
|
// That registration does NOT propagate into a nested if-expression block: the
|
||||||
|
// first cut of the geometry-ingest path wrote `let claimed: Int = ...` and
|
||||||
|
// `let got: Int = ...` inside the else-arm and `claimed == got` came out of
|
||||||
|
// codegen as `str_eq(claimed, got)` — strcmp on two integers reinterpreted as
|
||||||
|
// pointers, i.e. a segfault on the first geometry-bearing request. Read back
|
||||||
|
// out of the generated C, not guessed. Function PARAMETERS annotated `: Int`
|
||||||
|
// do register reliably (verified: `if (claimed == actual)`), so the comparison
|
||||||
|
// lives in a function of its own. Note also the explicit `return`s — a trailing
|
||||||
|
// if-EXPRESSION at a function tail emits as a statement and the function
|
||||||
|
// returns 0 regardless, which is the same probe's second finding.
|
||||||
|
fn width_agrees(claimed: Int, actual: Int) -> Int {
|
||||||
|
if claimed == actual { return 1 }
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
|
// INCOMPLETE-ROUTE FIX (2026-07-24 self-review): this route silently dropped
|
||||||
// label, importance, tier, and tags — engram_node() defaults label to content
|
// label, importance, tier, and tags — engram_node() defaults label to content
|
||||||
// and importance to 0.5, so every node created over HTTP lost its metadata.
|
// and importance to 0.5, so every node created over HTTP lost its metadata.
|
||||||
@@ -288,6 +355,45 @@ fn route_create_node(method: String, path: String, body: String) -> String {
|
|||||||
salience, importance, confidence,
|
salience, importance, confidence,
|
||||||
tier, tags
|
tier, tags
|
||||||
)
|
)
|
||||||
|
// GEOMETRY INGEST — geometry-valued end to end (2026-08-16).
|
||||||
|
//
|
||||||
|
// The defect this route originally had: it accepted an "emb" field,
|
||||||
|
// returned 200 with a fresh id, and stored NOTHING, because engram_node_full
|
||||||
|
// has no vector parameter. The consequence was structural, not cosmetic —
|
||||||
|
// text was the only entry medium, so any non-text modality had to be
|
||||||
|
// DESCRIBED in prose, and what we then reasoned over was the geometry of the
|
||||||
|
// description, not of the signal.
|
||||||
|
//
|
||||||
|
// #141 fixed the drop but marshalled the vector as a hex STRING through
|
||||||
|
// engram_node_set_emb, which put text back as the TRANSPORT medium one layer
|
||||||
|
// below the problem being fixed. This is that correction: hex is decoded
|
||||||
|
// exactly ONCE, here at the edge, into a first-class Geometry, and every
|
||||||
|
// step below this line moves geometry rather than text. An encoding at the
|
||||||
|
// boundary is what an encoding is for.
|
||||||
|
//
|
||||||
|
// The WIRE is deliberately unchanged — "emb" is still little-endian float32
|
||||||
|
// hex (8 chars per component), the encoding the perception vessel's
|
||||||
|
// /voice/embed already emits — because production clients speak it. What
|
||||||
|
// changed is underneath it.
|
||||||
|
//
|
||||||
|
// "dim" is now treated as an ASSERTION about the vector the caller sent, not
|
||||||
|
// as the source of its width: a Geometry carries its own width. A stated dim
|
||||||
|
// that disagrees is a REJECTED ingest, not a silent reinterpretation. Omitting
|
||||||
|
// "dim" is fine and means "trust the vector", which is the honest default.
|
||||||
|
//
|
||||||
|
// Off-dimension vectors remain stored but not inserted into the resident HNSW
|
||||||
|
// index (its build loop filters on emb_dim), so a 64-dim voice geometry is
|
||||||
|
// durable and addressable without perturbing the 768-dim canonical index.
|
||||||
|
let emb_hex: String = json_get_string(body, "emb")
|
||||||
|
let emb_set: Int = if str_eq(emb_hex, "") { 0 } else {
|
||||||
|
let g: Geometry = geometry_from_f32le_hex(emb_hex)
|
||||||
|
let got: Int = geometry_dim(g)
|
||||||
|
let dim_raw: String = json_get_raw(body, "dim")
|
||||||
|
let claimed: Int = if str_eq(dim_raw, "") { got } else { json_get_int(body, "dim") }
|
||||||
|
let landed: Int = if width_agrees(claimed, got) > 0 { node_attach_geometry(id, g) } else { 0 }
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
landed
|
||||||
|
}
|
||||||
let saved: Int = persist_node(id)
|
let saved: Int = persist_node(id)
|
||||||
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): connect the fresh node to its
|
// ORPHAN PREVENTION (ENGRAM_AUTOCONNECT): connect the fresh node to its
|
||||||
// nearest embedded neighbors so it never enters the graph edgeless.
|
// nearest embedded neighbors so it never enters the graph edgeless.
|
||||||
@@ -298,7 +404,11 @@ fn route_create_node(method: String, path: String, body: String) -> String {
|
|||||||
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
|
if added > 0 { let sv2: Int = persist_edges_since(ec0) }
|
||||||
added
|
added
|
||||||
} else { 0 }
|
} else { 0 }
|
||||||
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\",\"connected\":" + int_to_str(connected) + "}"
|
// Report whether the supplied geometry actually landed. The old response
|
||||||
|
// was success-shaped regardless — 200 with an id while the vector was
|
||||||
|
// discarded — which is how the drop went unnoticed. A caller can now
|
||||||
|
// assert on emb_set instead of trusting the status code.
|
||||||
|
"{\"id\":\"" + id + "\",\"content\":\"" + content + "\",\"node_type\":\"" + node_type + "\",\"connected\":" + int_to_str(connected) + ",\"emb_set\":" + int_to_str(emb_set) + "}"
|
||||||
}
|
}
|
||||||
|
|
||||||
fn route_get_node(method: String, path: String, body: String) -> String {
|
fn route_get_node(method: String, path: String, body: String) -> String {
|
||||||
@@ -333,7 +443,6 @@ fn route_scan_nodes(method: String, path: String, body: String) -> String {
|
|||||||
// process ever booted with a partial/empty store, the first read request
|
// process ever booted with a partial/empty store, the first read request
|
||||||
// clobbered the good snapshot. Read routes must never write the canonical path.)
|
// clobbered the good snapshot. Read routes must never write the canonical path.)
|
||||||
fn route_scan_edges(method: String, path: String, body: String) -> String {
|
fn route_scan_edges(method: String, path: String, body: String) -> String {
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
|
||||||
let dir: String = engram_resolve_data_dir()
|
let dir: String = engram_resolve_data_dir()
|
||||||
let snap_path: String = dir + "/.scan-export.json"
|
let snap_path: String = dir + "/.scan-export.json"
|
||||||
engram_save(snap_path)
|
engram_save(snap_path)
|
||||||
@@ -494,7 +603,6 @@ fn route_forget(method: String, path: String, body: String) -> String {
|
|||||||
|
|
||||||
fn route_save(method: String, path: String, body: String) -> String {
|
fn route_save(method: String, path: String, body: String) -> String {
|
||||||
let p_raw: String = json_get_string(body, "path")
|
let p_raw: String = json_get_string(body, "path")
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
|
||||||
let dir: String = engram_resolve_data_dir()
|
let dir: String = engram_resolve_data_dir()
|
||||||
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
||||||
// (2026-08-10 self-review) engram_save returns 0 on an empty path and the
|
// (2026-08-10 self-review) engram_save returns 0 on an empty path and the
|
||||||
@@ -578,7 +686,6 @@ fn route_drift(method: String, path: String, body: String) -> String {
|
|||||||
|
|
||||||
fn route_load(method: String, path: String, body: String) -> String {
|
fn route_load(method: String, path: String, body: String) -> String {
|
||||||
let p_raw: String = json_get_string(body, "path")
|
let p_raw: String = json_get_string(body, "path")
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
|
||||||
let dir: String = engram_resolve_data_dir()
|
let dir: String = engram_resolve_data_dir()
|
||||||
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
let p: String = if str_eq(p_raw, "") { dir + "/snapshot.json" } else { p_raw }
|
||||||
// (2026-08-10 self-review) This was a stub response over the single most
|
// (2026-08-10 self-review) This was a stub response over the single most
|
||||||
@@ -649,7 +756,6 @@ fn route_embed_backfill(method: String, path: String, body: String) -> String {
|
|||||||
// (it skips nodes already present by ID). Auth-exempt: same-host internal call.
|
// (it skips nodes already present by ID). Auth-exempt: same-host internal call.
|
||||||
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
|
// (2026-06-27 self-review: added this route to fix silent 10-min sync failures)
|
||||||
fn route_sync(method: String, path: String, body: String) -> String {
|
fn route_sync(method: String, path: String, body: String) -> String {
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
|
||||||
let dir: String = engram_resolve_data_dir()
|
let dir: String = engram_resolve_data_dir()
|
||||||
// 2026-07-21 self-review: export to a scratch path, never the canonical
|
// 2026-07-21 self-review: export to a scratch path, never the canonical
|
||||||
// snapshot.json — read routes must not be able to clobber the good snapshot.
|
// snapshot.json — read routes must not be able to clobber the good snapshot.
|
||||||
@@ -725,8 +831,12 @@ fn route_reseed_nodes(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(p, "") { return err_json("path is required") }
|
if str_eq(p, "") { return err_json("path is required") }
|
||||||
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
|
if str_eq(fs_read(p), "") { return err_json("file missing or empty") }
|
||||||
|
|
||||||
let dir_raw: String = env("ENGRAM_DATA_DIR")
|
// (2026-08-15) This site carried its own "/tmp/engram" fallback, which
|
||||||
let dir: String = if str_eq(dir_raw, "") { "/tmp/engram" } else { dir_raw }
|
// DISAGREED with engram_resolve_data_dir() ($HOME/.neuron/engram, fail-loud).
|
||||||
|
// The consumer is the pre-destructive backup below, so with ENGRAM_DATA_DIR
|
||||||
|
// unset the safety copy taken before a reseed landed in an ephemeral /tmp
|
||||||
|
// while the store it was protecting lived elsewhere. One owner, one answer.
|
||||||
|
let dir: String = engram_resolve_data_dir()
|
||||||
let backup: String = dir + "/.reseed-backup.json"
|
let backup: String = dir + "/.reseed-backup.json"
|
||||||
|
|
||||||
let replace_raw: String = json_get_raw(body, "replace")
|
let replace_raw: String = json_get_raw(body, "replace")
|
||||||
@@ -818,8 +928,7 @@ fn route_emit_ise(method: String, path: String, body: String) -> String {
|
|||||||
sal, imp, conf,
|
sal, imp, conf,
|
||||||
"Episodic", "[\"internal-state\",\"InternalStateEvent\"]"
|
"Episodic", "[\"internal-state\",\"InternalStateEvent\"]"
|
||||||
)
|
)
|
||||||
let ret_raw: String = env("ENGRAM_ISE_RETENTION_MS")
|
let ret_ms: Int = str_to_int(config("ENGRAM_ISE_RETENTION_MS"))
|
||||||
let ret_ms: Int = if str_eq(ret_raw, "") { 172800000 } else { str_to_int(ret_raw) }
|
|
||||||
let pruned: Int = engram_prune_telemetry(ret_ms)
|
let pruned: Int = engram_prune_telemetry(ret_ms)
|
||||||
"{\"ok\":true,\"id\":\"" + id + "\",\"pruned\":" + int_to_str(pruned) + "}"
|
"{\"ok\":true,\"id\":\"" + id + "\",\"pruned\":" + int_to_str(pruned) + "}"
|
||||||
}
|
}
|
||||||
@@ -916,6 +1025,22 @@ fn route_similarity(method: String, path: String, body: String) -> String {
|
|||||||
// nothing on request. NOTE: the offline reify WRITER (engram_geo_reify_store) is
|
// nothing on request. NOTE: the offline reify WRITER (engram_geo_reify_store) is
|
||||||
// currently unwired, so on the live store the resident index is empty and the
|
// currently unwired, so on the live store the resident index is empty and the
|
||||||
// list returns [] until reification runs — see the cutover report.
|
// list returns [] until reification runs — see the cutover report.
|
||||||
|
// route_scan_emb — GET /api/nodes/emb?limit=&offset= — read the raw geometry.
|
||||||
|
//
|
||||||
|
// engram_scan_nodes_emb_json has existed as a builtin with NO ROUTE, so the
|
||||||
|
// embeddings — the actual positions every distance, angle, membership and
|
||||||
|
// grounding is computed from — were unreadable from outside the process. You
|
||||||
|
// cannot verify a coordinate system you cannot see, and every claim about the
|
||||||
|
// frame (isotropy, centering, what the origin is) was therefore unfalsifiable
|
||||||
|
// from the API. Read-only.
|
||||||
|
fn route_scan_emb(method: String, path: String, body: String) -> String {
|
||||||
|
let l_raw: String = query_param(path, "limit")
|
||||||
|
let o_raw: String = query_param(path, "offset")
|
||||||
|
let l: Int = if str_eq(l_raw, "") { 200 } else { str_to_int(l_raw) }
|
||||||
|
let o: Int = if str_eq(o_raw, "") { 0 } else { str_to_int(o_raw) }
|
||||||
|
return engram_scan_nodes_emb_json(l, o)
|
||||||
|
}
|
||||||
|
|
||||||
fn route_neighborhoods(method: String, path: String, body: String) -> String {
|
fn route_neighborhoods(method: String, path: String, body: String) -> String {
|
||||||
engram_geo_reify_list_json()
|
engram_geo_reify_list_json()
|
||||||
}
|
}
|
||||||
@@ -1009,6 +1134,11 @@ fn route_faculty(path: String, faculty: String) -> String {
|
|||||||
fn route_boundary_proof(method: String, path: String, body: String) -> String {
|
fn route_boundary_proof(method: String, path: String, body: String) -> String {
|
||||||
return "{\"op\":\"boundary_proof\",\"body_instrumentation\":\"none\",\"seam\":\"@manager -> engram_boundary_beat auto-injected\"}"
|
return "{\"op\":\"boundary_proof\",\"body_instrumentation\":\"none\",\"seam\":\"@manager -> engram_boundary_beat auto-injected\"}"
|
||||||
}
|
}
|
||||||
|
// ── GROUNDING: an attribute of the RELATION, and the relation's weight is a
|
||||||
|
// VECTOR (factual, relational, associative, polarity, provenance, timestamp).
|
||||||
|
// /api/ground READS it — it never writes. /api/ground/record is the write,
|
||||||
|
// named as one, and it consolidates only on a consequential + salient move.
|
||||||
|
// /api/ground/trajectory reads the supersession chain as a time series.
|
||||||
fn route_ground(method: String, path: String, body: String) -> String {
|
fn route_ground(method: String, path: String, body: String) -> String {
|
||||||
let claim: String = json_get_string(body, "claim")
|
let claim: String = json_get_string(body, "claim")
|
||||||
let evidence: String = json_get_string(body, "evidence")
|
let evidence: String = json_get_string(body, "evidence")
|
||||||
@@ -1017,12 +1147,31 @@ fn route_ground(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(evidence, "") { return err_json("missing evidence") }
|
if str_eq(evidence, "") { return err_json("missing evidence") }
|
||||||
return engram_ground_json(claim, evidence, for_whom)
|
return engram_ground_json(claim, evidence, for_whom)
|
||||||
}
|
}
|
||||||
|
fn route_ground_record(method: String, path: String, body: String) -> String {
|
||||||
|
let claim: String = json_get_string(body, "claim")
|
||||||
|
let evidence: String = json_get_string(body, "evidence")
|
||||||
|
let provenance: String = json_get_string(body, "provenance")
|
||||||
|
let floor: String = json_get_string(body, "floor")
|
||||||
|
if str_eq(claim, "") { return err_json("missing claim") }
|
||||||
|
if str_eq(evidence, "") { return err_json("missing evidence") }
|
||||||
|
return engram_ground_record_json(claim, evidence, provenance, floor)
|
||||||
|
}
|
||||||
|
fn route_ground_trajectory(method: String, path: String, body: String) -> String {
|
||||||
|
let claim: String = query_param(path, "claim")
|
||||||
|
let evidence: String = query_param(path, "evidence")
|
||||||
|
if str_eq(claim, "") { return err_json("missing claim") }
|
||||||
|
if str_eq(evidence, "") { return err_json("missing evidence") }
|
||||||
|
return engram_ground_trajectory_json(claim, evidence)
|
||||||
|
}
|
||||||
fn route_assert(method: String, path: String, body: String) -> String {
|
fn route_assert(method: String, path: String, body: String) -> String {
|
||||||
let claim: String = query_param(path, "claim")
|
let claim: String = query_param(path, "claim")
|
||||||
if str_eq(claim, "") { return err_json("missing claim") }
|
if str_eq(claim, "") { return err_json("missing claim") }
|
||||||
let for_whom: String = query_param(path, "for_whom")
|
let for_whom: String = query_param(path, "for_whom")
|
||||||
let floor: String = query_param(path, "floor")
|
let floor: String = query_param(path, "floor")
|
||||||
return engram_assert_json(claim, for_whom, floor)
|
// Both floors. A well-evidenced claim does not earn the right to be asserted
|
||||||
|
// regardless of whether it means the right thing. rel_floor defaults to floor.
|
||||||
|
let rel_floor: String = query_param(path, "rel_floor")
|
||||||
|
return engram_assert_json(claim, for_whom, floor, rel_floor)
|
||||||
}
|
}
|
||||||
fn route_attend(method: String, path: String, body: String) -> String {
|
fn route_attend(method: String, path: String, body: String) -> String {
|
||||||
let node: String = json_get_string(body, "node")
|
let node: String = json_get_string(body, "node")
|
||||||
@@ -1068,14 +1217,12 @@ fn route_correspondence_beat(method: String, path: String, body: String) -> Stri
|
|||||||
// turns native thinking ON: the response carries reasoning_content (the thinking)
|
// turns native thinking ON: the response carries reasoning_content (the thinking)
|
||||||
// alongside content (the answer).
|
// alongside content (the answer).
|
||||||
|
|
||||||
fn guide_env_or(key: String, dflt: String) -> String {
|
// (2026-08-15) guide_env_or(key, dflt) lived here. Its whole job was supplying a
|
||||||
let v: String = env(key)
|
// per-call-site default, which is now the program block's job — every GUIDE_* knob
|
||||||
if str_eq(v, "") { return dflt }
|
// is declared once at the top of this file and read straight through config().
|
||||||
return v
|
|
||||||
}
|
|
||||||
|
|
||||||
fn guide_enabled() -> Bool {
|
fn guide_enabled() -> Bool {
|
||||||
let v: String = env("GUIDE_ENABLE")
|
let v: String = config("GUIDE_ENABLE")
|
||||||
if str_eq(v, "1") { return true }
|
if str_eq(v, "1") { return true }
|
||||||
if str_eq(v, "on") { return true }
|
if str_eq(v, "on") { return true }
|
||||||
if str_eq(v, "true") { return true }
|
if str_eq(v, "true") { return true }
|
||||||
@@ -1120,15 +1267,15 @@ fn guide_probe_metal() -> Bool {
|
|||||||
|
|
||||||
// ── 2. Tier selection (config-driven thresholds, spec-autoselected) ────────────
|
// ── 2. Tier selection (config-driven thresholds, spec-autoselected) ────────────
|
||||||
fn guide_threshold_4b() -> Int {
|
fn guide_threshold_4b() -> Int {
|
||||||
return str_to_int(guide_env_or("GUIDE_RAM_GB_4B", "16"))
|
return str_to_int(config("GUIDE_RAM_GB_4B"))
|
||||||
}
|
}
|
||||||
fn guide_threshold_1p7b() -> Int {
|
fn guide_threshold_1p7b() -> Int {
|
||||||
return str_to_int(guide_env_or("GUIDE_RAM_GB_1P7B", "8"))
|
return str_to_int(config("GUIDE_RAM_GB_1P7B"))
|
||||||
}
|
}
|
||||||
|
|
||||||
// GUIDE_TIER_FORCE overrides the spec autoselect (used to prove cheaply on 0.6b).
|
// GUIDE_TIER_FORCE overrides the spec autoselect (used to prove cheaply on 0.6b).
|
||||||
fn guide_select_tier(ram_gb: Int) -> String {
|
fn guide_select_tier(ram_gb: Int) -> String {
|
||||||
let forced: String = env("GUIDE_TIER_FORCE")
|
let forced: String = config("GUIDE_TIER_FORCE")
|
||||||
if !str_eq(forced, "") { return forced }
|
if !str_eq(forced, "") { return forced }
|
||||||
if ram_gb >= guide_threshold_4b() { return "4b" }
|
if ram_gb >= guide_threshold_4b() { return "4b" }
|
||||||
if ram_gb >= guide_threshold_1p7b() { return "1.7b" }
|
if ram_gb >= guide_threshold_1p7b() { return "1.7b" }
|
||||||
@@ -1148,8 +1295,10 @@ fn guide_file(tier: String) -> String {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn guide_cache_dir() -> String {
|
fn guide_cache_dir() -> String {
|
||||||
let c: String = env("GUIDE_CACHE_DIR")
|
let c: String = config("GUIDE_CACHE_DIR")
|
||||||
if !str_eq(c, "") { return c }
|
if !str_eq(c, "") { return c }
|
||||||
|
// HOME stays a raw env() read: it is the ambient environment, not a knob of
|
||||||
|
// this program, and it is deliberately absent from the program block.
|
||||||
let home: String = env("HOME")
|
let home: String = env("HOME")
|
||||||
if !str_eq(home, "") { return home + "/.neuron/guide/models" }
|
if !str_eq(home, "") { return home + "/.neuron/guide/models" }
|
||||||
return engram_resolve_data_dir() + "/guide-models"
|
return engram_resolve_data_dir() + "/guide-models"
|
||||||
@@ -1190,9 +1339,9 @@ fn guide_fetch(tier: String) -> Bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ── 4/5. Backend abstraction + BIND as an engageable interlocutor ──────────────
|
// ── 4/5. Backend abstraction + BIND as an engageable interlocutor ──────────────
|
||||||
fn guide_backend() -> String { return guide_env_or("GUIDE_BACKEND", "llama-server") }
|
fn guide_backend() -> String { return config("GUIDE_BACKEND") }
|
||||||
fn guide_host() -> String { return guide_env_or("GUIDE_HOST", "127.0.0.1") }
|
fn guide_host() -> String { return config("GUIDE_HOST") }
|
||||||
fn guide_port() -> String { return guide_env_or("GUIDE_PORT", "8771") }
|
fn guide_port() -> String { return config("GUIDE_PORT") }
|
||||||
fn guide_base_url() -> String { return "http://" + guide_host() + ":" + guide_port() }
|
fn guide_base_url() -> String { return "http://" + guide_host() + ":" + guide_port() }
|
||||||
|
|
||||||
// guide_healthy — is the guide present and answering? llama-server's /health
|
// guide_healthy — is the guide present and answering? llama-server's /health
|
||||||
@@ -1210,9 +1359,9 @@ fn guide_healthy() -> Bool {
|
|||||||
fn guide_load(tier: String) -> Bool {
|
fn guide_load(tier: String) -> Bool {
|
||||||
if guide_healthy() { return true }
|
if guide_healthy() { return true }
|
||||||
let path: String = guide_model_path(tier)
|
let path: String = guide_model_path(tier)
|
||||||
let bin: String = guide_env_or("GUIDE_LLAMA_SERVER_BIN", "llama-server")
|
let bin: String = config("GUIDE_LLAMA_SERVER_BIN")
|
||||||
let ngl: String = guide_env_or("GUIDE_NGL", "99")
|
let ngl: String = config("GUIDE_NGL")
|
||||||
let ctx: String = guide_env_or("GUIDE_CTX", "4096")
|
let ctx: String = config("GUIDE_CTX")
|
||||||
let logf: String = guide_cache_dir() + "/llama-server." + guide_port() + ".log"
|
let logf: String = guide_cache_dir() + "/llama-server." + guide_port() + ".log"
|
||||||
let cmd: String = bin + " -m '" + path + "' --host " + guide_host() + " --port " + guide_port() + " -c " + ctx + " -ngl " + ngl + " --jinja >> '" + logf + "' 2>&1"
|
let cmd: String = bin + " -m '" + path + "' --host " + guide_host() + " --port " + guide_port() + " -c " + ctx + " -ngl " + ngl + " --jinja >> '" + logf + "' 2>&1"
|
||||||
let pid: String = exec_bg(cmd)
|
let pid: String = exec_bg(cmd)
|
||||||
@@ -1608,7 +1757,7 @@ fn route_supersede(method: String, path: String, body: String) -> String {
|
|||||||
// ── Auth ──────────────────────────────────────────────────────────────────────
|
// ── Auth ──────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
fn check_auth_ok(method: String, body: String) -> Bool {
|
fn check_auth_ok(method: String, body: String) -> Bool {
|
||||||
let key: String = env("ENGRAM_API_KEY")
|
let key: String = config("ENGRAM_API_KEY")
|
||||||
if str_eq(key, "") { return true }
|
if str_eq(key, "") { return true }
|
||||||
// Read-only methods don't require auth. Until http_serve surfaces
|
// Read-only methods don't require auth. Until http_serve surfaces
|
||||||
// request headers we can't accept a Bearer token cleanly; mutating
|
// request headers we can't accept a Bearer token cleanly; mutating
|
||||||
@@ -1687,6 +1836,9 @@ fn handle_request(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(method, "GET") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
|
if str_eq(method, "GET") && (str_eq(clean, "/api/edges") || str_eq(clean, "/edges")) {
|
||||||
return route_scan_edges(method, path, body)
|
return route_scan_edges(method, path, body)
|
||||||
}
|
}
|
||||||
|
if str_eq(method, "GET") && (str_eq(clean, "/api/nodes/emb") || str_eq(clean, "/nodes/emb")) {
|
||||||
|
return route_scan_emb(method, path, body)
|
||||||
|
}
|
||||||
if str_eq(method, "GET") && str_starts_with(clean, "/api/nodes/") {
|
if str_eq(method, "GET") && str_starts_with(clean, "/api/nodes/") {
|
||||||
return route_get_node(method, path, body)
|
return route_get_node(method, path, body)
|
||||||
}
|
}
|
||||||
@@ -1776,6 +1928,14 @@ fn handle_request(method: String, path: String, body: String) -> String {
|
|||||||
if str_eq(method, "GET") && str_starts_with(clean, "/api/plan") {
|
if str_eq(method, "GET") && str_starts_with(clean, "/api/plan") {
|
||||||
return route_faculty(path, "plan")
|
return route_faculty(path, "plan")
|
||||||
}
|
}
|
||||||
|
// Order matters: the more specific paths must be tested before the /api/ground
|
||||||
|
// prefix match below, which would otherwise swallow them.
|
||||||
|
if str_eq(method, "POST") && str_starts_with(clean, "/api/ground/record") {
|
||||||
|
return route_ground_record(method, path, body)
|
||||||
|
}
|
||||||
|
if str_eq(method, "GET") && str_starts_with(clean, "/api/ground/trajectory") {
|
||||||
|
return route_ground_trajectory(method, path, body)
|
||||||
|
}
|
||||||
if str_eq(method, "POST") && str_starts_with(clean, "/api/ground") {
|
if str_eq(method, "POST") && str_starts_with(clean, "/api/ground") {
|
||||||
return route_ground(method, path, body)
|
return route_ground(method, path, body)
|
||||||
}
|
}
|
||||||
@@ -1871,8 +2031,7 @@ fn handle_request(method: String, path: String, body: String) -> String {
|
|||||||
|
|
||||||
// ── Entry ─────────────────────────────────────────────────────────────────────
|
// ── Entry ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
let bind_raw: String = env("ENGRAM_BIND")
|
let bind_str: String = config("ENGRAM_BIND")
|
||||||
let bind_str: String = if str_eq(bind_raw, "") { ":8742" } else { bind_raw }
|
|
||||||
let port: Int = parse_port(bind_str)
|
let port: Int = parse_port(bind_str)
|
||||||
|
|
||||||
// On startup, try to load any existing snapshot (best effort).
|
// On startup, try to load any existing snapshot (best effort).
|
||||||
|
|||||||
Executable
+40
@@ -0,0 +1,40 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Build + RUN the §7 GROUNDING-VECTOR tests (engram_cognition.c): the one decay
|
||||||
|
# model, the consequence gate, and the stored/derived split. Closed-form
|
||||||
|
# constructed cases — no server, no store, no network. Pure C11 (stdlib + libm).
|
||||||
|
# Standalone — NOT folded through elc. Two passes:
|
||||||
|
# 1. PERF — optimised (-O2, no sanitizer): the functional gate.
|
||||||
|
# 2. SAFETY — ASan + UBSan on the same suite.
|
||||||
|
#
|
||||||
|
# NEGATIVE CONTROL (invariant §8.6 — no test without one). Every symbol this
|
||||||
|
# suite exercises (cog_decay_factor, cog_grounding_significant,
|
||||||
|
# cog_significance_inherent, CogGrounding, CogProvClass) is introduced by the
|
||||||
|
# change under test, so the suite does not COMPILE against the pre-change source.
|
||||||
|
# To reproduce:
|
||||||
|
# git show origin/dev:lang/runtime/engram_cognition.h > /tmp/pre/engram_cognition.h
|
||||||
|
# git show origin/dev:lang/runtime/engram_cognition.c > /tmp/pre/engram_cognition.c
|
||||||
|
# cc -I/tmp/pre engram/test/test_grounding_vector.c /tmp/pre/engram_cognition.c ...
|
||||||
|
# => error: unknown type name 'CogGrounding'; no binary produced.
|
||||||
|
set -e
|
||||||
|
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||||
|
RT="$HERE/../../lang/runtime"
|
||||||
|
CC=${CC:-cc}
|
||||||
|
SRC="$HERE/test_grounding_vector.c $RT/engram_cognition.c $RT/engram_reason.c $RT/engram_geometry.c $RT/engram_store.c $RT/engram_vindex.c"
|
||||||
|
WARN="-std=c11 -Wall -Wextra"
|
||||||
|
# engram_store.c declares emit_log as a WEAK symbol and null-checks it, which is
|
||||||
|
# how a test links the store without the EL runtime. Darwin's ld does not resolve
|
||||||
|
# an undefined weak symbol at static-link time, so it must be allowed explicitly.
|
||||||
|
# (The pre-existing runners in this directory — run_verify_tests.sh among them —
|
||||||
|
# do not do this and therefore fail to link on macOS. Unrelated to this change.)
|
||||||
|
LDX=""
|
||||||
|
[ "$(uname -s)" = "Darwin" ] && LDX="-Wl,-U,_emit_log"
|
||||||
|
TMP=$(mktemp -d)
|
||||||
|
|
||||||
|
echo "### PASS 1: PERF (optimised, un-sanitised) — functional gate"
|
||||||
|
$CC $WARN -O2 -I"$RT" $SRC -lm -lpthread $LDX -o "$TMP/perf"
|
||||||
|
"$TMP/perf"
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "### PASS 2: SAFETY (ASan/UBSan)"
|
||||||
|
$CC $WARN -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer -I"$RT" $SRC -lm -lpthread $LDX -o "$TMP/safe"
|
||||||
|
ASAN_OPTIONS=${ASAN_OPTIONS:-detect_leaks=0} UBSAN_OPTIONS=halt_on_error=1 "$TMP/safe"
|
||||||
Executable
+107
@@ -0,0 +1,107 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# run_vindex_concurrency_tests.sh — regression harness for the 2026-08-16 soul crash.
|
||||||
|
#
|
||||||
|
# Four halves. The SET is the point: it separates two hazards the original two-half
|
||||||
|
# version conflated, and which have fixes in different files.
|
||||||
|
#
|
||||||
|
# 1. single ASan+UBSan, one thread. MUST be clean. Hard failure.
|
||||||
|
#
|
||||||
|
# 2. readers TSan, N readers, NO writer. Hazard (a): the visited set used
|
||||||
|
# to live on the index, so two pure READS stamped each other's
|
||||||
|
# epoch. Fixed in engram_vindex.c (frame-owned VVisit +
|
||||||
|
# `const VIndex*` search). MUST be clean. Hard failure.
|
||||||
|
#
|
||||||
|
# 3. unsynchronized TSan, writer + reader on a BARE index. Hazard (b): in-place
|
||||||
|
# HNSW insert rewires existing elements' neighbour lists and
|
||||||
|
# reallocs elems[]. EXPECTED TO RACE, PERMANENTLY. This is not
|
||||||
|
# a bug to fix inside engram_vindex.c — it is the executable
|
||||||
|
# proof that a publication boundary must exist above it.
|
||||||
|
# Not a failure. If it ever goes CLEAN, the test stopped
|
||||||
|
# interleaving and half 4 is no longer meaningful either.
|
||||||
|
#
|
||||||
|
# 4. published TSan, owner + N readers through a publication boundary
|
||||||
|
# (rwlock: readers shared, owner exclusive) mirroring
|
||||||
|
# eg_vindex_view / eg_vindex_maintain in lang/runtime/el_runtime.c.
|
||||||
|
# MUST be clean, and all inserts must land. Hard failure.
|
||||||
|
#
|
||||||
|
# See test_vindex_concurrency.c for the full story (SIGSEGV at ASCII address
|
||||||
|
# "gramNode", heap corruption in xzm_realloc, etc).
|
||||||
|
#
|
||||||
|
# usage: run_vindex_concurrency_tests.sh
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
RUNTIME="$(cd "$HERE/../../lang/runtime" && pwd)"
|
||||||
|
WORK="$(mktemp -d)"
|
||||||
|
trap 'rm -rf "$WORK"' EXIT
|
||||||
|
|
||||||
|
SRC="$HERE/test_vindex_concurrency.c"
|
||||||
|
VINDEX="$RUNTIME/engram_vindex.c"
|
||||||
|
|
||||||
|
fail=0
|
||||||
|
|
||||||
|
echo "== [1/4] single-threaded control under AddressSanitizer =="
|
||||||
|
cc -std=c11 -g -O1 -fsanitize=address,undefined -fno-omit-frame-pointer \
|
||||||
|
-I"$RUNTIME" -o "$WORK/single" "$SRC" "$VINDEX" -lm || { echo "BUILD FAILED"; exit 2; }
|
||||||
|
if ASAN_OPTIONS=detect_leaks=0 "$WORK/single" single; then
|
||||||
|
echo " -> OK"
|
||||||
|
else
|
||||||
|
echo " -> FAIL: the single-threaded control must always be clean."
|
||||||
|
echo " If this fails the bug is NOT (only) concurrency — look for a real"
|
||||||
|
echo " out-of-bounds or lifetime error in engram_vindex.c."
|
||||||
|
fail=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
cc -std=c11 -g -O1 -fsanitize=thread -fno-omit-frame-pointer \
|
||||||
|
-I"$RUNTIME" -o "$WORK/conc" "$SRC" "$VINDEX" -lm || { echo "BUILD FAILED"; exit 2; }
|
||||||
|
|
||||||
|
# run_tsan <mode> <logfile>; echoes nothing, sets $tsan_raced
|
||||||
|
run_tsan() {
|
||||||
|
TSAN_OPTIONS="halt_on_error=0" "$WORK/conc" "$1" >"$2" 2>&1
|
||||||
|
tsan_rc=$?
|
||||||
|
if grep -q "ThreadSanitizer: data race" "$2"; then tsan_raced=1; else tsan_raced=0; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "== [2/4] concurrent READERS, no writer (visited-set gate) =="
|
||||||
|
run_tsan readers "$WORK/readers.log"
|
||||||
|
if [ "$tsan_raced" = "1" ]; then
|
||||||
|
echo " -> REGRESSION: two concurrent reads still race."
|
||||||
|
grep -m1 -A6 "ThreadSanitizer: data race" "$WORK/readers.log" | sed 's/^/ /'
|
||||||
|
echo " The visited set was supposed to be owned by the call frame."
|
||||||
|
fail=1
|
||||||
|
else
|
||||||
|
echo " -> clean (concurrent reads are safe)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "== [3/4] writer+reader on a BARE index (expected-race probe) =="
|
||||||
|
run_tsan unsynchronized "$WORK/unsync.log"
|
||||||
|
if [ "$tsan_raced" = "1" ]; then
|
||||||
|
echo " -> RACE DETECTED, as expected:"
|
||||||
|
grep -m1 -A4 "ThreadSanitizer: data race" "$WORK/unsync.log" | sed 's/^/ /'
|
||||||
|
echo " In-place HNSW insert mutates existing elements. Not fixable inside"
|
||||||
|
echo " engram_vindex.c — this is why the publication boundary exists."
|
||||||
|
else
|
||||||
|
echo " -> NOTE: no race reported. The probe did not interleave; half 4's"
|
||||||
|
echo " clean result proves less than it should. Investigate."
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "== [4/4] owner+readers through the publication boundary (boundary gate) =="
|
||||||
|
run_tsan published "$WORK/pub.log"
|
||||||
|
if [ "$tsan_raced" = "1" ]; then
|
||||||
|
echo " -> REGRESSION: the publication boundary did not serialize the owner."
|
||||||
|
grep -m1 -A6 "ThreadSanitizer: data race" "$WORK/pub.log" | sed 's/^/ /'
|
||||||
|
fail=1
|
||||||
|
elif [ "$tsan_rc" != "0" ]; then
|
||||||
|
echo " -> FAIL: boundary clean under TSan but the run failed:"
|
||||||
|
tail -3 "$WORK/pub.log" | sed 's/^/ /'
|
||||||
|
fail=1
|
||||||
|
else
|
||||||
|
echo " -> clean (readers project concurrently; the owner's inserts all landed)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
[ "$fail" -eq 0 ] && echo "RESULT: PASS" || echo "RESULT: FAIL"
|
||||||
|
exit "$fail"
|
||||||
@@ -0,0 +1,176 @@
|
|||||||
|
/* test_grounding_vector.c — deterministic tests for §7: the one decay model, the
|
||||||
|
* consequence gate, and the stored/derived split. Links engram_cognition.c
|
||||||
|
* directly; no server, no store, no network. See run_grounding_vector_tests.sh.
|
||||||
|
*
|
||||||
|
* NEGATIVE CONTROL (invariant §8.6). Every symbol exercised here —
|
||||||
|
* cog_decay_factor, cog_grounding_significant, cog_significance_inherent,
|
||||||
|
* CogGrounding, CogProvClass — is introduced by the change under test, so this
|
||||||
|
* suite does not COMPILE against the pre-change source, let alone pass. The
|
||||||
|
* runner documents the exact reproduction.
|
||||||
|
*/
|
||||||
|
#include "engram_cognition.h"
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <math.h>
|
||||||
|
|
||||||
|
static int fails = 0;
|
||||||
|
static void ok(int cond, const char* what) {
|
||||||
|
printf(" %-62s %s\n", what, cond ? "PASS" : "*** FAIL ***");
|
||||||
|
if (!cond) fails++;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* The decay formula exactly as el_runtime.c carried it before the move, so the
|
||||||
|
* refactor can be shown to be bit-identical rather than merely similar. */
|
||||||
|
static double old_engram_temporal_decay(long long age_ms, long long activation_count,
|
||||||
|
double temporal_decay_rate) {
|
||||||
|
if (age_ms <= 0) return 1.0;
|
||||||
|
double lambda = (temporal_decay_rate > 0.0) ? temporal_decay_rate : 0.693147;
|
||||||
|
double age_hours = (double)age_ms / 3600000.0;
|
||||||
|
double t_half = 168.0 * (1.0 + log(1.0 + (double)activation_count));
|
||||||
|
double factor = exp(-lambda * age_hours / t_half);
|
||||||
|
if (factor < 0.25) factor = 0.25;
|
||||||
|
return factor;
|
||||||
|
}
|
||||||
|
|
||||||
|
static CogGrounding base(void) {
|
||||||
|
CogGrounding g; memset(&g, 0, sizeof g);
|
||||||
|
g.present = 1;
|
||||||
|
g.factual = 0.60; g.relational = 0.60;
|
||||||
|
g.factual_now = 0.60; g.relational_now = 0.60;
|
||||||
|
g.associative = 0.1; g.polarity = 1.0;
|
||||||
|
g.prov = COG_PROV_TOLD;
|
||||||
|
g.fac_proj = 1.0; g.rel_proj = 1.0;
|
||||||
|
g.cos_angle = 0.9; g.agreement = 1;
|
||||||
|
g.ts = 1000; g.seq = 1; g.reinforcements = 3;
|
||||||
|
return g;
|
||||||
|
}
|
||||||
|
|
||||||
|
int main(void) {
|
||||||
|
const double F = 0.5, R = 0.5;
|
||||||
|
|
||||||
|
printf("\n== 1. DECAY IS THE ONE MODEL, AND IT IS BIT-IDENTICAL TO WHAT IT REPLACED ==\n");
|
||||||
|
{
|
||||||
|
long long ages[] = {0, 3600000LL, 86400000LL, 7*86400000LL, 30*86400000LL, 365*86400000LL};
|
||||||
|
int allsame = 1;
|
||||||
|
for (int i = 0; i < 6; i++)
|
||||||
|
for (int ac = 0; ac < 4; ac++) {
|
||||||
|
long long acs[] = {0, 1, 10, 1000};
|
||||||
|
double a = cog_decay_factor(ages[i], (double)acs[ac], 0.0);
|
||||||
|
double b = old_engram_temporal_decay(ages[i], acs[ac], 0.0);
|
||||||
|
if (a != b) allsame = 0;
|
||||||
|
}
|
||||||
|
ok(allsame, "cog_decay_factor == the pre-move engram_temporal_decay (24 pts)");
|
||||||
|
ok(cog_decay_factor(0, 0, 0.0) == 1.0, "age 0 -> no decay");
|
||||||
|
}
|
||||||
|
printf("\n DECAY OVER ELAPSED TIME (reinforcements = 0, default rate):\n");
|
||||||
|
printf(" %10s %10s\n", "elapsed", "decay");
|
||||||
|
{
|
||||||
|
struct { const char* label; long long ms; } pts[] = {
|
||||||
|
{"0", 0LL},
|
||||||
|
{"1 hour", 3600000LL},
|
||||||
|
{"1 day", 86400000LL},
|
||||||
|
{"3 days", 3LL*86400000LL},
|
||||||
|
{"7 days", 7LL*86400000LL},
|
||||||
|
{"14 days", 14LL*86400000LL},
|
||||||
|
{"30 days", 30LL*86400000LL},
|
||||||
|
{"90 days", 90LL*86400000LL},
|
||||||
|
};
|
||||||
|
double prev = 2.0; int monotone = 1;
|
||||||
|
for (unsigned i = 0; i < sizeof pts / sizeof pts[0]; i++) {
|
||||||
|
double d = cog_decay_factor(pts[i].ms, 0, 0.0);
|
||||||
|
printf(" %10s %10.6f\n", pts[i].label, d);
|
||||||
|
if (d > prev) monotone = 0;
|
||||||
|
prev = d;
|
||||||
|
}
|
||||||
|
ok(monotone, "decay is monotone non-increasing in elapsed time");
|
||||||
|
ok(fabs(cog_decay_factor(7LL*86400000LL, 0, 0.0) - 0.5) < 1e-6,
|
||||||
|
"7 days at zero reinforcements == exactly one half-life (0.5)");
|
||||||
|
ok(cog_decay_factor(7LL*86400000LL, 100, 0.0) > cog_decay_factor(7LL*86400000LL, 0, 0.0),
|
||||||
|
"reinforcement slows ageing (Lindy term)");
|
||||||
|
ok(cog_decay_factor(3650LL*86400000LL, 0, 0.0) == 0.25,
|
||||||
|
"floor is a preference not a cliff: bottoms out at 0.25");
|
||||||
|
}
|
||||||
|
|
||||||
|
printf("\n== 2. CONSEQUENCE GATE: EVERY TRIGGER, AND NO EPSILON ANYWHERE ==\n");
|
||||||
|
{
|
||||||
|
CogGrounding p = base(), n = base();
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_NONE,
|
||||||
|
"identical vectors -> NONE (a re-read must not consolidate)");
|
||||||
|
|
||||||
|
n = base(); n.factual = 0.9999; n.factual_now = 0.9999;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_NONE,
|
||||||
|
"factual 0.60 -> 0.9999 without crossing the floor -> NONE");
|
||||||
|
|
||||||
|
n = base(); n.relational = 0.5001; n.relational_now = 0.5001;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_NONE,
|
||||||
|
"relational 0.60 -> 0.5001, still above floor -> NONE");
|
||||||
|
|
||||||
|
n = base(); n.factual_now = 0.4999;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_FACTUAL_FLOOR,
|
||||||
|
"a 0.1001 drop that CROSSES the floor -> FACTUAL_FLOOR");
|
||||||
|
|
||||||
|
n = base(); n.relational_now = 0.4999;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_RELATIONAL_FLOOR,
|
||||||
|
"relational crossing its floor -> RELATIONAL_FLOOR");
|
||||||
|
|
||||||
|
n = base(); n.cos_angle = -0.05; n.agreement = -1;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_AGREEMENT_FLIP,
|
||||||
|
"agreement +1 -> -1 -> AGREEMENT_FLIP");
|
||||||
|
|
||||||
|
n = base(); n.fac_proj = -0.2;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_DIRECTION_REVERSAL,
|
||||||
|
"factual gradient reverses -> DIRECTION_REVERSAL");
|
||||||
|
|
||||||
|
n = base(); n.rel_proj = -0.2;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_DIRECTION_REVERSAL,
|
||||||
|
"relational gradient reverses -> DIRECTION_REVERSAL");
|
||||||
|
|
||||||
|
n = base(); n.polarity = -1.0;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_POLARITY_FLIP,
|
||||||
|
"support -> contradiction -> POLARITY_FLIP (inherent)");
|
||||||
|
|
||||||
|
n = base(); n.polarity = 0.0;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_POLARITY_FLIP,
|
||||||
|
"support -> ignorance (zero) -> POLARITY_FLIP: not the same state");
|
||||||
|
|
||||||
|
n = base(); n.prov = COG_PROV_OBSERVED;
|
||||||
|
ok(cog_grounding_significant(&p, &n, F, R) == COG_SIG_PROVENANCE_CHANGE,
|
||||||
|
"told -> observed -> PROVENANCE_CHANGE (inherent)");
|
||||||
|
|
||||||
|
CogGrounding fresh; memset(&fresh, 0, sizeof fresh);
|
||||||
|
ok(cog_grounding_significant(&fresh, &n, F, R) == COG_SIG_FIRST_RECORD,
|
||||||
|
"no prior version -> FIRST_RECORD");
|
||||||
|
}
|
||||||
|
|
||||||
|
printf("\n== 3. INHERENT MOVES BYPASS THE SALIENCE GATE ==\n");
|
||||||
|
ok(cog_significance_inherent(COG_SIG_POLARITY_FLIP), "polarity flip is inherent");
|
||||||
|
ok(cog_significance_inherent(COG_SIG_PROVENANCE_CHANGE), "provenance change is inherent");
|
||||||
|
ok(cog_significance_inherent(COG_SIG_FIRST_RECORD), "first record is inherent");
|
||||||
|
ok(!cog_significance_inherent(COG_SIG_FACTUAL_FLOOR), "a floor crossing is NOT inherent");
|
||||||
|
ok(!cog_significance_inherent(COG_SIG_NONE), "NONE is not inherent");
|
||||||
|
|
||||||
|
printf("\n== 4. THE STORED/DERIVED SPLIT: DERIVED VALUES ARE NEVER SERIALIZED ==\n");
|
||||||
|
{
|
||||||
|
CogGrounding g = base();
|
||||||
|
g.decay = 0.3333; g.factual_now = 0.1234; g.relational_now = 0.2345;
|
||||||
|
g.associative_now = 0.4567; g.age_ms = 999999; g.stale = 1;
|
||||||
|
char* m = cog_grounding_metadata("pre-existing=keepme", &g);
|
||||||
|
ok(m != NULL, "serializer returns a document");
|
||||||
|
ok(m && strstr(m, "pre-existing=keepme"), "pre-existing edge metadata preserved verbatim");
|
||||||
|
ok(m && strstr(m, "GRD1"), "GRD1 magic present");
|
||||||
|
ok(m && !strstr(m, "0.3333"), "decay is NOT stored");
|
||||||
|
ok(m && !strstr(m, "0.1234"), "factual_now is NOT stored");
|
||||||
|
ok(m && !strstr(m, "0.2345"), "relational_now is NOT stored");
|
||||||
|
ok(m && !strstr(m, "0.4567"), "associative_now is NOT stored");
|
||||||
|
ok(m && !strstr(m, "999999"), "age is NOT stored");
|
||||||
|
ok(m && strstr(m, "told"), "provenance class IS stored");
|
||||||
|
ok(m && strstr(m, "0.6"), "the factual/relational dimensions ARE stored");
|
||||||
|
if (m) { printf("\n --- serialized GRD1 block ---\n%s -----------------------------\n", m); }
|
||||||
|
free(m);
|
||||||
|
}
|
||||||
|
|
||||||
|
printf("\n%s (%d failure%s)\n\n", fails ? "SOME TESTS FAILED" : "ALL TESTS PASSED",
|
||||||
|
fails, fails == 1 ? "" : "s");
|
||||||
|
return fails ? 1 : 0;
|
||||||
|
}
|
||||||
@@ -0,0 +1,251 @@
|
|||||||
|
/* test_vindex_concurrency.c — regression test for the 2026-08-16 soul crash.
|
||||||
|
*
|
||||||
|
* WHAT BROKE: the soul daemon crash-looped (5 crashes in ~100s) with SIGSEGV in
|
||||||
|
* search_layer <- vindex_insert <- eg_vindex_sync, a SIGABRT, and a fault inside
|
||||||
|
* xzm_realloc's own freelist — i.e. heap corruption. The SIGSEGV address
|
||||||
|
* 0x65646f4e6d617267 is little-endian ASCII "gramNode": string bytes being
|
||||||
|
* dereferenced as an Elem vector pointer.
|
||||||
|
*
|
||||||
|
* ROOT CAUSE: VIndex owns its traversal scratch (visited[] + visit_epoch), and
|
||||||
|
* search_layer mutates it via visited_reset(). So the index is unsafe for ANY
|
||||||
|
* concurrent use — including two concurrent READS. soul.el starts http_serve_async
|
||||||
|
* (a thread per connection) and then runs awareness_run() on the main thread, which
|
||||||
|
* reaches the same global index through engram_activate; nothing serialized them.
|
||||||
|
*
|
||||||
|
* Neither hnswlib nor FAISS puts the visited set on the index: hnswlib checks one
|
||||||
|
* out of a VisitedListPool per query, FAISS uses a thread_local VisitedTable.
|
||||||
|
*
|
||||||
|
* THE ORIGINAL `concurrent` HALF CONFLATED TWO DISTINCT HAZARDS (2026-08-16). It ran
|
||||||
|
* a writer against a reader on one bare index, so it could not tell apart:
|
||||||
|
*
|
||||||
|
* (a) READ/READ corruption — two searches stamping each other's visited epoch.
|
||||||
|
* A defect INSIDE engram_vindex.c, fixable there, and now fixed: the visited
|
||||||
|
* set moved to the call frame and vindex_search takes a `const VIndex*`.
|
||||||
|
*
|
||||||
|
* (b) WRITE/READ corruption — vindex_insert rewires the neighbour lists of
|
||||||
|
* EXISTING elements and reallocs elems[], so an insert is a mutation of the
|
||||||
|
* whole structure. This is NOT fixable inside engram_vindex.c at any price:
|
||||||
|
* it is inherent to in-place HNSW. It requires a publication boundary ABOVE
|
||||||
|
* the data structure (el_runtime.c: eg_vindex_view / eg_vindex_maintain).
|
||||||
|
*
|
||||||
|
* Conflating them made the suite unfailable-then-unpassable: fixing (a) left (b)
|
||||||
|
* still racing, which reads as "the fix did not work" when in fact a different,
|
||||||
|
* correctly-located fix is what (b) needs. So the halves are now separate:
|
||||||
|
*
|
||||||
|
* single N clustered vectors, ONE thread, ASan. The CONTROL. Must always
|
||||||
|
* be clean. When this passes and a concurrent half fails, the defect
|
||||||
|
* is concurrency, not an out-of-bounds/logic error in the graph code.
|
||||||
|
* (On 2026-08-16 this control cleared all 13,820 real dim-768 store
|
||||||
|
* vectors under ASan, which DISPROVED an inspection-derived hypothesis
|
||||||
|
* about an out-of-bounds reverse-link write at engram_vindex.c:340.)
|
||||||
|
*
|
||||||
|
* readers N reader threads, NO writer, one shared index, TSan. This is
|
||||||
|
* hazard (a) in isolation. It RACED before the visited set moved off
|
||||||
|
* the index struct and must be CLEAN now. Hard gate.
|
||||||
|
*
|
||||||
|
* unsynchronized writer + reader on a bare index, TSan. Hazard (b) in isolation.
|
||||||
|
* EXPECTED TO RACE, permanently — it is the executable proof that
|
||||||
|
* the index cannot be made safe from the inside, and therefore that
|
||||||
|
* the publication boundary in el_runtime.c has to exist. If this
|
||||||
|
* ever goes clean, the test stopped interleaving; do not celebrate.
|
||||||
|
*
|
||||||
|
* published writer + readers through a publication boundary that mirrors
|
||||||
|
* eg_vindex_view / eg_vindex_maintain (rwlock: readers shared,
|
||||||
|
* the single owner exclusive), TSan. Must be CLEAN. Hard gate.
|
||||||
|
* This is what proves the shape of the runtime fix, in the same
|
||||||
|
* process, rather than asserting it.
|
||||||
|
*
|
||||||
|
* Absence of a crash does NOT mean absence of a race — always read the sanitizer
|
||||||
|
* verdict, never just the exit code.
|
||||||
|
*
|
||||||
|
* Build/run: engram/test/run_vindex_concurrency_tests.sh
|
||||||
|
*/
|
||||||
|
#include "engram_vindex.h"
|
||||||
|
|
||||||
|
#include <pthread.h>
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <stdint.h>
|
||||||
|
|
||||||
|
#define DIM 128
|
||||||
|
#define NVEC 3000
|
||||||
|
#define SEED_N 50
|
||||||
|
|
||||||
|
static VIndex* g_ix;
|
||||||
|
static float* g_vecs;
|
||||||
|
|
||||||
|
/* Deterministic filler. Real embeddings are strongly correlated, not uniform noise;
|
||||||
|
* clustering keeps many candidates near-equidistant, which exercises the diversity
|
||||||
|
* heuristic and the visited set far harder than random vectors do. */
|
||||||
|
static void fill_vectors(void) {
|
||||||
|
g_vecs = (float*)malloc((size_t)NVEC * DIM * sizeof(float));
|
||||||
|
if (!g_vecs) { fprintf(stderr, "OOM\n"); exit(1); }
|
||||||
|
for (int i = 0; i < NVEC; i++) {
|
||||||
|
int cluster = i % 8;
|
||||||
|
for (int d = 0; d < DIM; d++)
|
||||||
|
g_vecs[(size_t)i * DIM + d] =
|
||||||
|
(float)(((d + cluster * 7) % 13) / 13.0) +
|
||||||
|
(float)(((i * 2654435761u + (unsigned)d) % 97) / 9700.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void* writer_fn(void* arg) {
|
||||||
|
(void)arg;
|
||||||
|
for (int i = SEED_N; i < NVEC; i++)
|
||||||
|
(void)vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void* reader_fn(void* arg) {
|
||||||
|
(void)arg;
|
||||||
|
uint64_t ids[8]; float ds[8];
|
||||||
|
for (int i = 0; i < 20000; i++)
|
||||||
|
(void)vindex_search(g_ix, g_vecs + (size_t)(i % NVEC) * DIM, 8, 0, ids, ds);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
static int run_single(void) {
|
||||||
|
printf("[single] inserting %d vectors on one thread (ASan control)\n", NVEC);
|
||||||
|
g_ix = vindex_create(DIM, 0, 0);
|
||||||
|
if (!g_ix) { fprintf(stderr, "[single] vindex_create failed\n"); return 1; }
|
||||||
|
for (int i = 0; i < NVEC; i++) {
|
||||||
|
if (vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM) != 0) {
|
||||||
|
fprintf(stderr, "[single] insert %d failed\n", i); return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (vindex_size(g_ix) != (size_t)NVEC) {
|
||||||
|
fprintf(stderr, "[single] size %zu != %d\n", vindex_size(g_ix), NVEC); return 1;
|
||||||
|
}
|
||||||
|
uint64_t ids[16]; float ds[16];
|
||||||
|
for (int q = 0; q < 200; q++) {
|
||||||
|
int k = vindex_search(g_ix, g_vecs + (size_t)((q * 7) % NVEC) * DIM, 16, 0, ids, ds);
|
||||||
|
if (k < 0) { fprintf(stderr, "[single] search failed at q=%d\n", q); return 1; }
|
||||||
|
}
|
||||||
|
vindex_free(g_ix); g_ix = NULL;
|
||||||
|
printf("[single] PASS — no memory error (this must ALWAYS pass)\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Hazard (b) in isolation: writer + reader on a BARE index, no boundary. */
|
||||||
|
static int run_unsynchronized(void) {
|
||||||
|
printf("[unsynchronized] 1 writer + 1 reader on a BARE index (TSan probe)\n");
|
||||||
|
printf("[unsynchronized] a race here is EXPECTED and PERMANENT — in-place HNSW\n");
|
||||||
|
printf("[unsynchronized] insert rewires existing elements. This is the proof that\n");
|
||||||
|
printf("[unsynchronized] the publication boundary must live ABOVE engram_vindex.c.\n");
|
||||||
|
g_ix = vindex_create(DIM, 0, 0);
|
||||||
|
if (!g_ix) { fprintf(stderr, "[unsynchronized] vindex_create failed\n"); return 1; }
|
||||||
|
for (int i = 0; i < SEED_N; i++)
|
||||||
|
(void)vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM);
|
||||||
|
|
||||||
|
pthread_t w, r;
|
||||||
|
if (pthread_create(&w, NULL, writer_fn, NULL) ||
|
||||||
|
pthread_create(&r, NULL, reader_fn, NULL)) {
|
||||||
|
fprintf(stderr, "[unsynchronized] pthread_create failed\n"); return 1;
|
||||||
|
}
|
||||||
|
pthread_join(w, NULL);
|
||||||
|
pthread_join(r, NULL);
|
||||||
|
vindex_free(g_ix); g_ix = NULL;
|
||||||
|
printf("[unsynchronized] completed — CHECK THE SANITIZER VERDICT, not this line.\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── hazard (a) in isolation: concurrent READS only ───────────────────────────
|
||||||
|
* This is what the frame-owned visited set fixes. Before that change, two
|
||||||
|
* vindex_search calls on one index wrote each other's epoch stamp; TSan reported
|
||||||
|
* the race at visited_reset and the traversal then walked bogus element indices. */
|
||||||
|
#define NREADERS 4
|
||||||
|
|
||||||
|
static int run_readers(void) {
|
||||||
|
printf("[readers] %d concurrent readers, NO writer, one shared index (TSan)\n", NREADERS);
|
||||||
|
printf("[readers] this is the visited-set regression gate — must be CLEAN.\n");
|
||||||
|
g_ix = vindex_create(DIM, 0, 0);
|
||||||
|
if (!g_ix) { fprintf(stderr, "[readers] vindex_create failed\n"); return 1; }
|
||||||
|
for (int i = 0; i < NVEC; i++)
|
||||||
|
(void)vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM);
|
||||||
|
|
||||||
|
pthread_t t[NREADERS];
|
||||||
|
for (int i = 0; i < NREADERS; i++)
|
||||||
|
if (pthread_create(&t[i], NULL, reader_fn, NULL)) {
|
||||||
|
fprintf(stderr, "[readers] pthread_create failed\n"); return 1;
|
||||||
|
}
|
||||||
|
for (int i = 0; i < NREADERS; i++) pthread_join(t[i], NULL);
|
||||||
|
vindex_free(g_ix); g_ix = NULL;
|
||||||
|
printf("[readers] completed — CHECK THE SANITIZER VERDICT, not this line.\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── the publication boundary, mirroring el_runtime.c ─────────────────────────
|
||||||
|
* Readers take the boundary SHARED and hold it across the whole search; the one
|
||||||
|
* owner takes it EXCLUSIVE to extend. Same shape as eg_vindex_view /
|
||||||
|
* eg_vindex_maintain. Note the reader's index pointer is `const VIndex*` — the
|
||||||
|
* compiler, not this comment, is what stops a reader inserting. */
|
||||||
|
static pthread_rwlock_t g_pub = PTHREAD_RWLOCK_INITIALIZER;
|
||||||
|
|
||||||
|
static void* pub_writer_fn(void* arg) {
|
||||||
|
(void)arg;
|
||||||
|
for (int i = SEED_N; i < NVEC; i++) {
|
||||||
|
pthread_rwlock_wrlock(&g_pub);
|
||||||
|
(void)vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM);
|
||||||
|
pthread_rwlock_unlock(&g_pub);
|
||||||
|
}
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void* pub_reader_fn(void* arg) {
|
||||||
|
(void)arg;
|
||||||
|
uint64_t ids[8]; float ds[8];
|
||||||
|
for (int i = 0; i < 5000; i++) {
|
||||||
|
pthread_rwlock_rdlock(&g_pub);
|
||||||
|
const VIndex* view = g_ix; /* immutable view */
|
||||||
|
(void)vindex_search(view, g_vecs + (size_t)(i % NVEC) * DIM, 8, 0, ids, ds);
|
||||||
|
pthread_rwlock_unlock(&g_pub);
|
||||||
|
}
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
static int run_published(void) {
|
||||||
|
printf("[published] 1 owner + %d readers through a publication boundary (TSan)\n", NREADERS);
|
||||||
|
printf("[published] this is the eg_vindex_view/eg_vindex_maintain gate — must be CLEAN.\n");
|
||||||
|
g_ix = vindex_create(DIM, 0, 0);
|
||||||
|
if (!g_ix) { fprintf(stderr, "[published] vindex_create failed\n"); return 1; }
|
||||||
|
for (int i = 0; i < SEED_N; i++)
|
||||||
|
(void)vindex_insert(g_ix, (uint64_t)i, g_vecs + (size_t)i * DIM);
|
||||||
|
|
||||||
|
pthread_t w, r[NREADERS];
|
||||||
|
if (pthread_create(&w, NULL, pub_writer_fn, NULL)) {
|
||||||
|
fprintf(stderr, "[published] pthread_create failed\n"); return 1;
|
||||||
|
}
|
||||||
|
for (int i = 0; i < NREADERS; i++)
|
||||||
|
if (pthread_create(&r[i], NULL, pub_reader_fn, NULL)) {
|
||||||
|
fprintf(stderr, "[published] pthread_create failed\n"); return 1;
|
||||||
|
}
|
||||||
|
pthread_join(w, NULL);
|
||||||
|
for (int i = 0; i < NREADERS; i++) pthread_join(r[i], NULL);
|
||||||
|
if (vindex_size(g_ix) != (size_t)NVEC) {
|
||||||
|
fprintf(stderr, "[published] size %zu != %d — the owner lost inserts\n",
|
||||||
|
vindex_size(g_ix), NVEC);
|
||||||
|
vindex_free(g_ix); g_ix = NULL; return 1;
|
||||||
|
}
|
||||||
|
vindex_free(g_ix); g_ix = NULL;
|
||||||
|
printf("[published] all %d inserts landed; CHECK THE SANITIZER VERDICT too.\n", NVEC);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
int main(int argc, char** argv) {
|
||||||
|
const char* mode = (argc > 1) ? argv[1] : "single";
|
||||||
|
fill_vectors();
|
||||||
|
int rc;
|
||||||
|
if (!strcmp(mode, "single")) rc = run_single();
|
||||||
|
else if (!strcmp(mode, "readers")) rc = run_readers();
|
||||||
|
else if (!strcmp(mode, "unsynchronized")) rc = run_unsynchronized();
|
||||||
|
else if (!strcmp(mode, "published")) rc = run_published();
|
||||||
|
/* back-compat: the pre-split name meant the bare writer+reader probe. */
|
||||||
|
else if (!strcmp(mode, "concurrent")) rc = run_unsynchronized();
|
||||||
|
else {
|
||||||
|
fprintf(stderr, "usage: %s [single|readers|unsynchronized|published]\n", argv[0]);
|
||||||
|
rc = 2;
|
||||||
|
}
|
||||||
|
free(g_vecs);
|
||||||
|
return rc;
|
||||||
|
}
|
||||||
+56
-12
@@ -13,7 +13,7 @@
|
|||||||
// relations add edges. Every node enters with PROVENANCE + grounding-level
|
// relations add edges. Every node enters with PROVENANCE + grounding-level
|
||||||
// + stewardship class from the moment of entry.
|
// + stewardship class from the moment of entry.
|
||||||
//
|
//
|
||||||
// transduce() is THE single mechanism — one function, polymorphic, with no
|
// transduce_bytes() is THE single mechanism — one function, polymorphic, with no
|
||||||
// content-type branch inside it. It does not ask whether a payload is
|
// content-type branch inside it. It does not ask whether a payload is
|
||||||
// prose, structured data, or raw/opaque bytes (audio, or anything else);
|
// prose, structured data, or raw/opaque bytes (audio, or anything else);
|
||||||
// it runs one boundary-scan-with-fixed-window-fallback chunking algorithm
|
// it runs one boundary-scan-with-fixed-window-fallback chunking algorithm
|
||||||
@@ -401,10 +401,54 @@ fn head80(s: String) -> String {
|
|||||||
// truncates at the first embedded NUL, which is routine in real binary
|
// truncates at the first embedded NUL, which is routine in real binary
|
||||||
// bytes) is a MECHANICAL fidelity concern that belongs to whatever produced
|
// bytes) is a MECHANICAL fidelity concern that belongs to whatever produced
|
||||||
// `source` (see ingest_file's file_source_string below) — not a
|
// `source` (see ingest_file's file_source_string below) — not a
|
||||||
// content-type judgment made in here. transduce() never learns whether a
|
// content-type judgment made in here. transduce_bytes() never learns whether a
|
||||||
// chunk is plain text or a base64-encoded raw-byte window; every chunk is
|
// chunk is plain text or a base64-encoded raw-byte window; every chunk is
|
||||||
// handled identically either way.
|
// handled identically either way.
|
||||||
fn transduce(nodes: [String], edges: [String], source: String,
|
// NAMING, CORRECTED 2026-08-16 (second pass). This function was renamed
|
||||||
|
// `transduce` -> `transduce_bytes` earlier the same day, on the reasoning
|
||||||
|
// that it "was never signal->geometry — it chunks already-extracted content
|
||||||
|
// and PACKS it into a node+edge manifold, one layer up, and it had taken the
|
||||||
|
// name that belongs to the primitive underneath it."
|
||||||
|
//
|
||||||
|
// THAT REASONING WAS BACKWARDS, and it is worth recording why rather than
|
||||||
|
// quietly re-renaming. Producing a node+edge manifold is not a layer above
|
||||||
|
// transduction — it IS transduction. Transduction is not conversion. When you
|
||||||
|
// take in music you do not store the song as one discrete geometry; you break
|
||||||
|
// it into its component parts and store the geometry of each along with the
|
||||||
|
// relations between them. The song is the structure of those relations.
|
||||||
|
// Signal -> one vector is the operation UNDERNEATH transduction, and its name
|
||||||
|
// is encoding, or geometry. So the layer that was doing it right got renamed
|
||||||
|
// out of the way so the layer doing it wrong could have the name.
|
||||||
|
//
|
||||||
|
// The primitive has since been corrected: `transduce(signal, modality)` now
|
||||||
|
// returns a Manifold — components plus relations — not a Geometry
|
||||||
|
// (el_runtime.c, "Manifold"). The two layers are therefore doing the SAME KIND
|
||||||
|
// of thing, and the inversion dissolves rather than needing to be re-argued.
|
||||||
|
//
|
||||||
|
// What is left is a real distinction, and it is about MODALITY, not layering:
|
||||||
|
//
|
||||||
|
// * `transduce(signal, modality)` dispatches to a realizer that KNOWS the
|
||||||
|
// modality and can name its components — for audio: pitch, interval,
|
||||||
|
// rhythm, harmonic function.
|
||||||
|
// * `transduce_bytes` below is the OPAQUE-BYTES realizer: the decomposition
|
||||||
|
// available to a reader that knows nothing about what it is reading. It
|
||||||
|
// still yields components and relations (chunk nodes; contains / precedes
|
||||||
|
// / section_of edges), which is why it is transduction and not packing. It
|
||||||
|
// just cuts on the only structure visible without understanding — byte
|
||||||
|
// boundaries — so its components are positional rather than meaningful.
|
||||||
|
// That is a LIMITATION of this realizer, not the definition of the
|
||||||
|
// operation.
|
||||||
|
//
|
||||||
|
// The name is suffixed by its modality, not demoted to a lesser layer. Keeping
|
||||||
|
// a distinct symbol is also still mechanically required: every El `fn name`
|
||||||
|
// compiles to a global C symbol, so reusing `transduce` here is a hard
|
||||||
|
// `conflicting types` error the moment ingest.c links el_runtime.c.
|
||||||
|
//
|
||||||
|
// WHERE THIS SHOULD GO: this function should become a registered realizer
|
||||||
|
// returning a real Manifold, so ingest rides the same primitive as every other
|
||||||
|
// modality instead of carrying a parallel implementation. Not done here.
|
||||||
|
// Nothing about this function's behaviour changed in this pass.
|
||||||
|
fn transduce_bytes(nodes: [String], edges: [String], source: String,
|
||||||
prov: String, ground: String, steward: String,
|
prov: String, ground: String, steward: String,
|
||||||
root_lid: String, root_title: String) -> [String] {
|
root_lid: String, root_title: String) -> [String] {
|
||||||
let tagbase: String = "prov:" + prov + " ground:" + ground + " steward:" + steward
|
let tagbase: String = "prov:" + prov + " ground:" + ground + " steward:" + steward
|
||||||
@@ -531,8 +575,8 @@ fn default_steward() -> String {
|
|||||||
// trustworthy verbatim. When they don't (silent truncation happened),
|
// trustworthy verbatim. When they don't (silent truncation happened),
|
||||||
// rebuild the payload as base64-encoded fixed-size windows read directly
|
// rebuild the payload as base64-encoded fixed-size windows read directly
|
||||||
// off disk (fs_read_b64_chunk — binary-safe in C), joined with the same
|
// off disk (fs_read_b64_chunk — binary-safe in C), joined with the same
|
||||||
// "\n\n" boundary marker transduce()'s generic scan already looks for, so
|
// "\n\n" boundary marker transduce_bytes()'s generic scan already looks for, so
|
||||||
// transduce() sees one ordinary boundary-delimited payload and runs its one
|
// transduce_bytes() sees one ordinary boundary-delimited payload and runs its one
|
||||||
// algorithm on it exactly as it would on prose — it never learns that a
|
// algorithm on it exactly as it would on prose — it never learns that a
|
||||||
// fidelity problem occurred upstream, let alone why.
|
// fidelity problem occurred upstream, let alone why.
|
||||||
fn file_source_string(path: String, text: String, real_size: Int) -> String {
|
fn file_source_string(path: String, text: String, real_size: Int) -> String {
|
||||||
@@ -541,7 +585,7 @@ fn file_source_string(path: String, text: String, real_size: Int) -> String {
|
|||||||
// 3072 raw bytes -> 4096 base64 chars (3 divides evenly into base64's
|
// 3072 raw bytes -> 4096 base64 chars (3 divides evenly into base64's
|
||||||
// 3-byte/4-char ratio); keeps each resulting node's content a clean,
|
// 3-byte/4-char ratio); keeps each resulting node's content a clean,
|
||||||
// bounded, low-kilobytes unit, same order of magnitude as the fixed
|
// bounded, low-kilobytes unit, same order of magnitude as the fixed
|
||||||
// fallback window in transduce() itself.
|
// fallback window in transduce_bytes() itself.
|
||||||
let win: Int = 3072
|
let win: Int = 3072
|
||||||
let out: String = ""
|
let out: String = ""
|
||||||
let off: Int = 0
|
let off: Int = 0
|
||||||
@@ -561,7 +605,7 @@ fn file_source_string(path: String, text: String, real_size: Int) -> String {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ingest one file -> report JSON. Uniform for every file regardless of
|
// ingest one file -> report JSON. Uniform for every file regardless of
|
||||||
// extension or content — transduce() decides nothing about content-type, so
|
// extension or content — transduce_bytes() decides nothing about content-type, so
|
||||||
// neither does this function; it only decides whether the raw bytes made it
|
// neither does this function; it only decides whether the raw bytes made it
|
||||||
// through the read intact (file_source_string), which is a fidelity
|
// through the read intact (file_source_string), which is a fidelity
|
||||||
// question, not a format one.
|
// question, not a format one.
|
||||||
@@ -573,14 +617,14 @@ fn ingest_file(path: String) -> String {
|
|||||||
return "{\"error\":\"empty or unreadable\",\"path\":" + j_q(path) + "}"
|
return "{\"error\":\"empty or unreadable\",\"path\":" + j_q(path) + "}"
|
||||||
}
|
}
|
||||||
let prov: String = "file:" + path
|
let prov: String = "file:" + path
|
||||||
let packed: [String] = transduce(el_list_empty(), el_list_empty(),
|
let packed: [String] = transduce_bytes(el_list_empty(), el_list_empty(),
|
||||||
source, prov, default_ground(), default_steward(),
|
source, prov, default_ground(), default_steward(),
|
||||||
"doc:" + basename(path), basename(path))
|
"doc:" + basename(path), basename(path))
|
||||||
return merge_packed(packed)
|
return merge_packed(packed)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ingest a directory: walk one level, ingest every file found, aggregate.
|
// ingest a directory: walk one level, ingest every file found, aggregate.
|
||||||
// No extension filter — transduce() handles any payload uniformly now, so
|
// No extension filter — transduce_bytes() handles any payload uniformly now, so
|
||||||
// there is no content-type gate at the directory boundary either.
|
// there is no content-type gate at the directory boundary either.
|
||||||
fn ingest_dir(path: String) -> String {
|
fn ingest_dir(path: String) -> String {
|
||||||
let entries: [String] = fs_list(path)
|
let entries: [String] = fs_list(path)
|
||||||
@@ -615,7 +659,7 @@ fn ingest_dir(path: String) -> String {
|
|||||||
fn ingest_url(url: String) -> String {
|
fn ingest_url(url: String) -> String {
|
||||||
let body: String = http_get(url)
|
let body: String = http_get(url)
|
||||||
if str_eq(body, "") { return "{\"error\":\"empty fetch\",\"url\":" + j_q(url) + "}" }
|
if str_eq(body, "") { return "{\"error\":\"empty fetch\",\"url\":" + j_q(url) + "}" }
|
||||||
let packed: [String] = transduce(el_list_empty(), el_list_empty(),
|
let packed: [String] = transduce_bytes(el_list_empty(), el_list_empty(),
|
||||||
body, "url:" + url, "extracted", "public-web",
|
body, "url:" + url, "extracted", "public-web",
|
||||||
"url:" + url, url)
|
"url:" + url, url)
|
||||||
return merge_packed(packed)
|
return merge_packed(packed)
|
||||||
@@ -630,7 +674,7 @@ fn ingest_llm(query: String) -> String {
|
|||||||
let resp: String = http_post_json("http://127.0.0.1:11434/api/generate", body)
|
let resp: String = http_post_json("http://127.0.0.1:11434/api/generate", body)
|
||||||
let answer: String = json_get_string(resp, "response")
|
let answer: String = json_get_string(resp, "response")
|
||||||
if str_eq(answer, "") { return "{\"error\":\"no model response\"}" }
|
if str_eq(answer, "") { return "{\"error\":\"no model response\"}" }
|
||||||
let packed: [String] = transduce(el_list_empty(), el_list_empty(),
|
let packed: [String] = transduce_bytes(el_list_empty(), el_list_empty(),
|
||||||
answer, "llm:" + model + ":" + query, "candidate-provisional", "guide-provisional",
|
answer, "llm:" + model + ":" + query, "candidate-provisional", "guide-provisional",
|
||||||
"llm:" + query, "guide answer: " + query)
|
"llm:" + query, "guide answer: " + query)
|
||||||
return merge_packed(packed)
|
return merge_packed(packed)
|
||||||
@@ -682,7 +726,7 @@ fn ingest_stream(path: String) -> String {
|
|||||||
// It is NOT a content-type flag: it says nothing about what's inside the
|
// It is NOT a content-type flag: it says nothing about what's inside the
|
||||||
// bytes once fetched, and none of the five ingest_* functions it selects
|
// bytes once fetched, and none of the five ingest_* functions it selects
|
||||||
// among interpret their payload differently by content shape anymore —
|
// among interpret their payload differently by content shape anymore —
|
||||||
// they all hand off to the single, format-agnostic transduce(). The old
|
// they all hand off to the single, format-agnostic transduce_bytes(). The old
|
||||||
// "structured" value (a caller-declared alias for "file", used only to hint
|
// "structured" value (a caller-declared alias for "file", used only to hint
|
||||||
// the now-removed JSON-vs-prose branch) is gone along with that branch.
|
// the now-removed JSON-vs-prose branch) is gone along with that branch.
|
||||||
let kind: String = env("INGEST_KIND")
|
let kind: String = env("INGEST_KIND")
|
||||||
|
|||||||
@@ -73,6 +73,17 @@ When you add a C builtin (verbatim-emit recipe — the El name is emitted as the
|
|||||||
2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.
|
2. Add a `__`-prefixed thin wrapper in `el_seed.c` and declare it in `el_seed.h`.
|
||||||
3. Add the name to `builtin_arity` in `el-compiler/src/codegen.el` — add **both** the plain and `__`-prefixed spellings.
|
3. Add the name to `builtin_arity` in `el-compiler/src/codegen.el` — add **both** the plain and `__`-prefixed spellings.
|
||||||
4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical.
|
4. Rebuild the elc binary (see below) and confirm the self-host fixpoint is byte-identical.
|
||||||
|
5. **Prove it with a NEGATIVE CONTROL.** Show the test FAILING on a build without your change, then passing with it. A test that has never been seen to fail has proven nothing.
|
||||||
|
|
||||||
|
> **Step 5 is not optional, and step 4 does not cover it.** The fixpoint proves the *compiler reproduces itself*. It says nothing whatsoever about whether your builtin works. A recipe ending at "byte-identical" reads as complete while having verified nothing about the thing just added — which is why this file, until 2026-08-16, produced builtins with no tests at all.
|
||||||
|
>
|
||||||
|
> Measured cost of the omission (2026-08-16): `engram_node_set_emb`, `engram_curiosity_json` and `dream_set_handler` were all added in one session with zero tests. Separately, a UTF-8 fix was written, tested, and **the test passed on the unpatched build too** — the defect was elsewhere entirely, and only building the pre-fix binary exposed it. Without a negative control that fix would have merged as verified.
|
||||||
|
>
|
||||||
|
> Two shapes that pass while proving nothing, both hit the same day:
|
||||||
|
> - A test that never exercises your change (the route supplied a default that bypassed the code under test).
|
||||||
|
> - An induction that loses a race. `curl --max-time` on a large response left *both* builds alive; only `SO_LINGER 0` — a genuine RST, so the peer is provably gone — reproduced the failure. Six of ten attempts is not a control.
|
||||||
|
>
|
||||||
|
> Before every probe, confirm **your** process bound the port (`lsof -nP -iTCP:<port>`, match the PID). A stale instance answering on the port has silently produced false results here more than once, and `pkill -f` does not reliably match an argv like `./engram`.
|
||||||
|
|
||||||
Worked example: the `engram_assert_json` (op_assert seam) and `engram_node_full_in`/`engram_connect_in` (purview write-side) primitives added 2026-08-15 follow exactly this recipe.
|
Worked example: the `engram_assert_json` (op_assert seam) and `engram_node_full_in`/`engram_connect_in` (purview write-side) primitives added 2026-08-15 follow exactly this recipe.
|
||||||
|
|
||||||
|
|||||||
Vendored
BIN
Binary file not shown.
@@ -862,10 +862,23 @@ fn cg_expr(expr: Map<String, Any>) -> String {
|
|||||||
// arithmetic BinOp (or vice-versa). Without this check the
|
// arithmetic BinOp (or vice-versa). Without this check the
|
||||||
// fallthrough to str_eq produces str_eq(int_value, int_value)
|
// fallthrough to str_eq produces str_eq(int_value, int_value)
|
||||||
// which reads the integer as a char* and segfaults.
|
// which reads the integer as a char* and segfaults.
|
||||||
|
// EITHER side provably Int is enough. Requiring BOTH meant a call
|
||||||
|
// whose return type codegen cannot infer poisoned the operator:
|
||||||
|
// getint(5) == a -> str_eq(getint(5), a)
|
||||||
|
// even with `a` declared Int. str_eq then reads an integer as a
|
||||||
|
// char* and segfaults. Only an integer LITERAL on one side forced
|
||||||
|
// the numeric form, so the bug was invisible in the common case.
|
||||||
|
//
|
||||||
|
// Loosening to OR is strictly safer: when one side is a known Int,
|
||||||
|
// str_eq is always wrong (it dereferences that int), while numeric
|
||||||
|
// comparison is at worst a wrong answer on an already ill-typed
|
||||||
|
// program. When neither side is Int nothing changes, so string
|
||||||
|
// comparison is untouched.
|
||||||
if is_int_expr(left) {
|
if is_int_expr(left) {
|
||||||
if is_int_expr(right) {
|
|
||||||
return "(" + left_c + " == " + right_c + ")"
|
return "(" + left_c + " == " + right_c + ")"
|
||||||
}
|
}
|
||||||
|
if is_int_expr(right) {
|
||||||
|
return "(" + left_c + " == " + right_c + ")"
|
||||||
}
|
}
|
||||||
// Float literal or negative float literal: use plain == (bit-equal
|
// Float literal or negative float literal: use plain == (bit-equal
|
||||||
// el_val_t comparison). This handles `r0 == 3.0`, `neg == -3.0`, etc.
|
// el_val_t comparison). This handles `r0 == 3.0`, `neg == -3.0`, etc.
|
||||||
@@ -921,10 +934,12 @@ fn cg_expr(expr: Map<String, Any>) -> String {
|
|||||||
}
|
}
|
||||||
// Same mixed Ident/BinOp fix as EqEq: use is_int_expr to detect
|
// Same mixed Ident/BinOp fix as EqEq: use is_int_expr to detect
|
||||||
// integer-typed operands before falling through to !str_eq.
|
// integer-typed operands before falling through to !str_eq.
|
||||||
|
// Either side Int is enough — see the EqEq note above.
|
||||||
if is_int_expr(left) {
|
if is_int_expr(left) {
|
||||||
if is_int_expr(right) {
|
|
||||||
return "(" + left_c + " != " + right_c + ")"
|
return "(" + left_c + " != " + right_c + ")"
|
||||||
}
|
}
|
||||||
|
if is_int_expr(right) {
|
||||||
|
return "(" + left_c + " != " + right_c + ")"
|
||||||
}
|
}
|
||||||
// Float-typed operands use plain != (bit-equal comparison).
|
// Float-typed operands use plain != (bit-equal comparison).
|
||||||
if is_float_expr(left) {
|
if is_float_expr(left) {
|
||||||
@@ -1495,6 +1510,11 @@ fn cg_stmt(stmt: Map<String, Any>, indent: String, declared: [String]) -> [Strin
|
|||||||
if str_eq(ltype, "Int") {
|
if str_eq(ltype, "Int") {
|
||||||
add_int_name(name)
|
add_int_name(name)
|
||||||
}
|
}
|
||||||
|
// Same as params: Bool is an int in the value model. Without this a
|
||||||
|
// `let ok: Bool = ...` compared to another Bool lowered to str_eq.
|
||||||
|
if str_eq(ltype, "Bool") {
|
||||||
|
add_int_name(name)
|
||||||
|
}
|
||||||
if str_eq(ltype, "Float") {
|
if str_eq(ltype, "Float") {
|
||||||
add_float_name(name)
|
add_float_name(name)
|
||||||
}
|
}
|
||||||
@@ -2887,6 +2907,7 @@ fn builtin_arity(name: String) -> Int {
|
|||||||
if str_eq(name, "el_alloc_count") { return 0 }
|
if str_eq(name, "el_alloc_count") { return 0 }
|
||||||
if str_eq(name, "el_alloc_bytes") { return 0 }
|
if str_eq(name, "el_alloc_bytes") { return 0 }
|
||||||
if str_eq(name, "el_peak_rss") { return 0 }
|
if str_eq(name, "el_peak_rss") { return 0 }
|
||||||
|
if str_eq(name, "el_black_box") { return 1 }
|
||||||
if str_eq(name, "engram_neighbors_json") { return 3 }
|
if str_eq(name, "engram_neighbors_json") { return 3 }
|
||||||
if str_eq(name, "engram_activate_json") { return 2 }
|
if str_eq(name, "engram_activate_json") { return 2 }
|
||||||
if str_eq(name, "engram_stats_json") { return 0 }
|
if str_eq(name, "engram_stats_json") { return 0 }
|
||||||
@@ -3112,6 +3133,15 @@ fn build_int_names_for_params(params: [Map<String, Any>]) -> Bool {
|
|||||||
if str_eq(ptype, "Int") {
|
if str_eq(ptype, "Int") {
|
||||||
add_int_name(pname)
|
add_int_name(pname)
|
||||||
}
|
}
|
||||||
|
// Bool is an integer in the value model (type_to_c maps Bool -> "int";
|
||||||
|
// el_runtime.h: "Bool -> el_val_t (0 = false, nonzero = true)"), but
|
||||||
|
// Bool names were registered nowhere. So `cond == want` between two
|
||||||
|
// Bool params fell through to str_eq and dereferenced 0 or 1 as a
|
||||||
|
// char* — an immediate segfault. Track them as int-like, which is what
|
||||||
|
// they are.
|
||||||
|
if str_eq(ptype, "Bool") {
|
||||||
|
add_int_name(pname)
|
||||||
|
}
|
||||||
if str_eq(ptype, "Float") {
|
if str_eq(ptype, "Float") {
|
||||||
add_float_name(pname)
|
add_float_name(pname)
|
||||||
}
|
}
|
||||||
@@ -3235,6 +3265,7 @@ fn is_top_level_decl(stmt: Map<String, Any>) -> Bool {
|
|||||||
if kind == "EnumDef" { return true }
|
if kind == "EnumDef" { return true }
|
||||||
if kind == "Import" { return true }
|
if kind == "Import" { return true }
|
||||||
if kind == "CgiBlock" { return true }
|
if kind == "CgiBlock" { return true }
|
||||||
|
if kind == "ProgramBlock" { return true }
|
||||||
if kind == "ExternFn" { return true }
|
if kind == "ExternFn" { return true }
|
||||||
false
|
false
|
||||||
}
|
}
|
||||||
@@ -3247,6 +3278,55 @@ fn cgi_arg(value: String, has_value: Bool) -> String {
|
|||||||
return "EL_NULL"
|
return "EL_NULL"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// -- Program block: cross-cutting concerns injected at the process boundary ----
|
||||||
|
//
|
||||||
|
// emit_program_init — emit the `static void __el_program_init(void)` that
|
||||||
|
// carries a program's declared cross-cutting concerns. Called from main()
|
||||||
|
// BEFORE any user statement runs, so the guarantees hold for the whole process
|
||||||
|
// rather than depending on each call site remembering to ask for them.
|
||||||
|
//
|
||||||
|
// This is emitted at the point the `program` block is encountered, not buffered
|
||||||
|
// until main(). The streaming backend emits in source order and cannot hold a
|
||||||
|
// declaration's entry list alive until main(); emitting a named function here
|
||||||
|
// and calling it from main() means only a single bool has to survive.
|
||||||
|
//
|
||||||
|
// Order matters and is deliberate:
|
||||||
|
// 1. singleton FIRST — if another instance already holds the lock, refuse and
|
||||||
|
// exit before touching configuration, ports, or any data directory.
|
||||||
|
// 2. config declarations — resolve env-or-default, one declaration per entry.
|
||||||
|
// 3. validate LAST — report EVERY missing/ill-typed entry at once, then exit.
|
||||||
|
fn el_bool_arg(b: Bool) -> String {
|
||||||
|
if b { return "EL_INT(1)" }
|
||||||
|
return "EL_INT(0)"
|
||||||
|
}
|
||||||
|
|
||||||
|
fn emit_program_init(stmt: Map<String, Any>) -> Void {
|
||||||
|
let pname: String = stmt["name"]
|
||||||
|
emit_line("static void __el_program_init(void) {")
|
||||||
|
let has_singleton: Bool = stmt["has_singleton"]
|
||||||
|
if has_singleton {
|
||||||
|
let sid: String = stmt["singleton"]
|
||||||
|
emit_line(" el_singleton_acquire(EL_STR(" + c_str_lit(sid) + "));")
|
||||||
|
}
|
||||||
|
let entries = stmt["entries"]
|
||||||
|
let n: Int = native_list_len(entries)
|
||||||
|
let i = 0
|
||||||
|
while i < n {
|
||||||
|
let e = native_list_get(entries, i)
|
||||||
|
let ename: String = e["name"]
|
||||||
|
let etype: String = e["etype"]
|
||||||
|
let edefault: String = e["default"]
|
||||||
|
let has_default: Bool = e["has_default"]
|
||||||
|
let erequired: Bool = e["required"]
|
||||||
|
let arg_def: String = cgi_arg(edefault, has_default)
|
||||||
|
emit_line(" el_config_declare(EL_STR(" + c_str_lit(ename) + "), EL_STR(" + c_str_lit(etype) + "), " + arg_def + ", " + el_bool_arg(has_default) + ", " + el_bool_arg(erequired) + ");")
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
emit_line(" el_config_validate(EL_STR(" + c_str_lit(pname) + "));")
|
||||||
|
emit_line("}")
|
||||||
|
emit_blank()
|
||||||
|
}
|
||||||
|
|
||||||
// -- VBD role enforcement ------------------------------------------------------
|
// -- VBD role enforcement ------------------------------------------------------
|
||||||
//
|
//
|
||||||
// Scan a function body for direct calls to DHARMA-restricted builtins
|
// Scan a function body for direct calls to DHARMA-restricted builtins
|
||||||
@@ -3569,6 +3649,20 @@ fn codegen(stmts: [Map<String, Any>], source: String) -> String {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Program block: emit the cross-cutting init function before the user's
|
||||||
|
// functions so main() can call it (see emit_program_init).
|
||||||
|
let prog_have: Bool = false
|
||||||
|
let i = 0
|
||||||
|
while i < n {
|
||||||
|
let stmt = native_list_get(stmts, i)
|
||||||
|
let sk4: String = stmt["stmt"]
|
||||||
|
if str_eq(sk4, "ProgramBlock") {
|
||||||
|
emit_program_init(stmt)
|
||||||
|
let prog_have = true
|
||||||
|
}
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
|
||||||
// Function definitions
|
// Function definitions
|
||||||
let i = 0
|
let i = 0
|
||||||
while i < n {
|
while i < n {
|
||||||
@@ -3587,6 +3681,9 @@ fn codegen(stmts: [Map<String, Any>], source: String) -> String {
|
|||||||
// with the C-side parameters when fn main()'s body is folded in below.
|
// with the C-side parameters when fn main()'s body is folded in below.
|
||||||
emit_line("int main(int _argc, char** _argv) {")
|
emit_line("int main(int _argc, char** _argv) {")
|
||||||
emit_line(" el_runtime_init_args(_argc, _argv);")
|
emit_line(" el_runtime_init_args(_argc, _argv);")
|
||||||
|
if prog_have {
|
||||||
|
emit_line(" __el_program_init();")
|
||||||
|
}
|
||||||
if cgi_count >= 1 {
|
if cgi_count >= 1 {
|
||||||
let cname: String = cgi_block["name"]
|
let cname: String = cgi_block["name"]
|
||||||
let cdid: String = cgi_block["dharma_id"]
|
let cdid: String = cgi_block["dharma_id"]
|
||||||
@@ -4180,6 +4277,7 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
|
|||||||
// Fix: copy the values out BEFORE the release (strings, so no dangling reference)
|
// Fix: copy the values out BEFORE the release (strings, so no dangling reference)
|
||||||
// and emit from these. No search, so the failure mode is removed rather than moved.
|
// and emit from these. No search, so the failure mode is removed rather than moved.
|
||||||
let cgi_have: Bool = false
|
let cgi_have: Bool = false
|
||||||
|
let prog_have: Bool = false
|
||||||
let cgi_name_v: String = ""
|
let cgi_name_v: String = ""
|
||||||
let cgi_did_v: String = ""
|
let cgi_did_v: String = ""
|
||||||
let cgi_prin_v: String = ""
|
let cgi_prin_v: String = ""
|
||||||
@@ -4301,6 +4399,14 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
|
|||||||
// These are no-ops in codegen (forward decls already emitted)
|
// These are no-ops in codegen (forward decls already emitted)
|
||||||
// — except a CgiBlock, whose declared identity must survive
|
// — except a CgiBlock, whose declared identity must survive
|
||||||
// this release to be emitted as a compiled constant.
|
// this release to be emitted as a compiled constant.
|
||||||
|
// A ProgramBlock's cross-cutting declarations are
|
||||||
|
// emitted HERE, as a named init function, because the
|
||||||
|
// streaming backend cannot hold the entry list alive
|
||||||
|
// until main(). Only the bool survives.
|
||||||
|
if str_eq(sk, "ProgramBlock") {
|
||||||
|
emit_program_init(stmt)
|
||||||
|
let prog_have = true
|
||||||
|
}
|
||||||
if str_eq(sk, "CgiBlock") {
|
if str_eq(sk, "CgiBlock") {
|
||||||
let cgi_have = true
|
let cgi_have = true
|
||||||
let cgi_name_v = stmt["name"]
|
let cgi_name_v = stmt["name"]
|
||||||
@@ -4447,6 +4553,13 @@ fn codegen_streaming(tokens: [Any], sigs: [Map<String, Any>], source: String) ->
|
|||||||
let kind2: String = state_get("__program_kind")
|
let kind2: String = state_get("__program_kind")
|
||||||
emit_line("int main(int _argc, char** _argv) {")
|
emit_line("int main(int _argc, char** _argv) {")
|
||||||
emit_line(" el_runtime_init_args(_argc, _argv);")
|
emit_line(" el_runtime_init_args(_argc, _argv);")
|
||||||
|
// Cross-cutting concerns declared by a `program` block run BEFORE anything
|
||||||
|
// else — a singleton violation must refuse the start before this process
|
||||||
|
// touches a port or a data directory, and configuration must be validated
|
||||||
|
// before the first read of it rather than at each read site.
|
||||||
|
if prog_have {
|
||||||
|
emit_line(" __el_program_init();")
|
||||||
|
}
|
||||||
|
|
||||||
// cgi init if needed
|
// cgi init if needed
|
||||||
let ns2: Int = native_list_len(sigs)
|
let ns2: Int = native_list_len(sigs)
|
||||||
|
|||||||
@@ -184,6 +184,7 @@ fn keyword_kind(word: String) -> String {
|
|||||||
if word == "false" { return "Bool" }
|
if word == "false" { return "Bool" }
|
||||||
if word == "cgi" { return "Cgi" }
|
if word == "cgi" { return "Cgi" }
|
||||||
if word == "service" { return "Service" }
|
if word == "service" { return "Service" }
|
||||||
|
if word == "program" { return "Program" }
|
||||||
if word == "manager" { return "Manager" }
|
if word == "manager" { return "Manager" }
|
||||||
if word == "engine" { return "Engine" }
|
if word == "engine" { return "Engine" }
|
||||||
if word == "accessor" { return "Accessor" }
|
if word == "accessor" { return "Accessor" }
|
||||||
|
|||||||
@@ -1967,6 +1967,113 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
|
|||||||
}, p)
|
}, p)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// program block: program "name" { singleton: "id", env NAME: Type = "default", ... }
|
||||||
|
//
|
||||||
|
// The program block is El's declaration surface for CROSS-CUTTING CONCERNS —
|
||||||
|
// properties of the whole process rather than of any one function, which
|
||||||
|
// otherwise degrade into "remember to call this at every site" conventions.
|
||||||
|
//
|
||||||
|
// singleton: "id" — process identity. The runtime takes an exclusive
|
||||||
|
// lock at startup; a SECOND start is refused, loudly,
|
||||||
|
// instead of two processes sharing one data dir.
|
||||||
|
// env NAME: T = "d" — one configuration entry. Its type and its default
|
||||||
|
// are declared ONCE, here, and resolved+validated
|
||||||
|
// before main() body runs.
|
||||||
|
// env NAME: T required
|
||||||
|
// — no default; the program refuses to start unless the
|
||||||
|
// variable is set.
|
||||||
|
//
|
||||||
|
// Both compile into calls injected at the head of main() — the same boundary
|
||||||
|
// seam `cgi` already uses (codegen.el emit_program_init). No call site in the
|
||||||
|
// program body has to remember anything, which is the whole point.
|
||||||
|
if k == "Program" {
|
||||||
|
let p = pos + 1
|
||||||
|
let name = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
let p = expect(tokens, p, "LBrace")
|
||||||
|
let singleton = ""
|
||||||
|
let has_singleton = false
|
||||||
|
let entries = native_list_empty()
|
||||||
|
// Entry-scratch declared at loop-body level (not inside the branch) so
|
||||||
|
// that inner `let` forms compile to assignment rather than a C-scoped
|
||||||
|
// redeclaration — the same idiom the service block above relies on.
|
||||||
|
let ename = ""
|
||||||
|
let etype = ""
|
||||||
|
let edefault = ""
|
||||||
|
let has_default = false
|
||||||
|
let erequired = false
|
||||||
|
let fname = ""
|
||||||
|
let fval = ""
|
||||||
|
let running = true
|
||||||
|
while running {
|
||||||
|
let k2 = tok_kind(tokens, p)
|
||||||
|
if k2 == "RBrace" {
|
||||||
|
let running = false
|
||||||
|
} else {
|
||||||
|
if k2 == "Eof" {
|
||||||
|
let running = false
|
||||||
|
} else {
|
||||||
|
let fname = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
if str_eq(fname, "env") {
|
||||||
|
// env NAME: Type [= "default"] [required]
|
||||||
|
let ename = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
let p = expect(tokens, p, "Colon")
|
||||||
|
let etype = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
let edefault = ""
|
||||||
|
let has_default = false
|
||||||
|
let erequired = false
|
||||||
|
let k3 = tok_kind(tokens, p)
|
||||||
|
if str_eq(k3, "Eq") {
|
||||||
|
let p = p + 1
|
||||||
|
let edefault = tok_value(tokens, p)
|
||||||
|
let has_default = true
|
||||||
|
let p = p + 1
|
||||||
|
}
|
||||||
|
let k4 = tok_kind(tokens, p)
|
||||||
|
if str_eq(k4, "Ident") {
|
||||||
|
let w = tok_value(tokens, p)
|
||||||
|
if str_eq(w, "required") {
|
||||||
|
let erequired = true
|
||||||
|
let p = p + 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let entries = native_list_append(entries, {
|
||||||
|
"name": ename,
|
||||||
|
"etype": etype,
|
||||||
|
"default": edefault,
|
||||||
|
"has_default": has_default,
|
||||||
|
"required": erequired
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
// scalar field: `name: "value"`
|
||||||
|
let p = expect(tokens, p, "Colon")
|
||||||
|
let fval = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
if str_eq(fname, "singleton") {
|
||||||
|
let singleton = fval
|
||||||
|
let has_singleton = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let k5 = tok_kind(tokens, p)
|
||||||
|
if k5 == "Comma" {
|
||||||
|
let p = p + 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let p = expect(tokens, p, "RBrace")
|
||||||
|
return make_result({
|
||||||
|
"stmt": "ProgramBlock",
|
||||||
|
"name": name,
|
||||||
|
"singleton": singleton,
|
||||||
|
"has_singleton": has_singleton,
|
||||||
|
"entries": entries
|
||||||
|
}, p)
|
||||||
|
}
|
||||||
|
|
||||||
// assert <cond_expr> [ , <msg_expr> ]
|
// assert <cond_expr> [ , <msg_expr> ]
|
||||||
// The message is optional — if the next token after the condition is not a
|
// The message is optional — if the next token after the condition is not a
|
||||||
// Comma, emit an empty string placeholder so the test still works.
|
// Comma, emit an empty string placeholder so the test still works.
|
||||||
@@ -2419,6 +2526,7 @@ fn scan_params_c(tokens: [Any], pos: Int) -> Map<String, Any> {
|
|||||||
// toplevel_let: { "kind": "toplevel_let", "name": String, "ltype": String }
|
// toplevel_let: { "kind": "toplevel_let", "name": String, "ltype": String }
|
||||||
// cgi_block: { "kind": "cgi_block", "name": String }
|
// cgi_block: { "kind": "cgi_block", "name": String }
|
||||||
// service_block: { "kind": "service_block", "name": String }
|
// service_block: { "kind": "service_block", "name": String }
|
||||||
|
// program_block: { "kind": "program_block", "name": String }
|
||||||
//
|
//
|
||||||
// Import/TypeDef/EnumDef nodes are skipped (codegen treats them as no-ops).
|
// Import/TypeDef/EnumDef nodes are skipped (codegen treats them as no-ops).
|
||||||
//
|
//
|
||||||
@@ -2546,13 +2654,28 @@ fn scan_fn_sigs(tokens: [Any]) -> [Map<String, Any>] {
|
|||||||
"name": name
|
"name": name
|
||||||
})
|
})
|
||||||
let pos = p
|
let pos = p
|
||||||
|
} else {
|
||||||
|
// --- program block ---
|
||||||
|
if str_eq(k, "Program") {
|
||||||
|
let p: Int = pos + 1
|
||||||
|
let name: String = tok_value(tokens, p)
|
||||||
|
let p = p + 1
|
||||||
|
let k2: String = tok_kind(tokens, p)
|
||||||
|
if str_eq(k2, "LBrace") {
|
||||||
|
let p = skip_to_rbrace(tokens, p)
|
||||||
|
}
|
||||||
|
let sigs = native_list_append(sigs, {
|
||||||
|
"kind": "program_block",
|
||||||
|
"name": name
|
||||||
|
})
|
||||||
|
let pos = p
|
||||||
} else {
|
} else {
|
||||||
// Import, Type, Enum, From, or any other token.
|
// Import, Type, Enum, From, or any other token.
|
||||||
// Skip ahead to the next statement boundary.
|
// Skip ahead to the next statement boundary.
|
||||||
let p: Int = pos + 1
|
let p: Int = pos + 1
|
||||||
let p = skip_expr_to_stmt_boundary(tokens, p)
|
let p = skip_expr_to_stmt_boundary(tokens, p)
|
||||||
let pos = p
|
let pos = p
|
||||||
}}}}}
|
}}}}}}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,228 @@
|
|||||||
|
// transduce.el — transduction decomposes a signal into components and the
|
||||||
|
// relations between them. Runnable: this is the worked example for the
|
||||||
|
// transduce surface, and it exits non-zero if any claim in it stops being true.
|
||||||
|
//
|
||||||
|
// elc lang/examples/transduce.el > transduce.c
|
||||||
|
// cc -std=c11 -O2 -I lang/runtime -o transduce transduce.c \
|
||||||
|
// lang/runtime/el_runtime.c lang/runtime/el_seed.c \
|
||||||
|
// lang/runtime/engram_store.c lang/runtime/engram_vindex.c \
|
||||||
|
// lang/runtime/engram_cognition.c lang/runtime/engram_geometry.c \
|
||||||
|
// lang/runtime/engram_reason.c lang/runtime/engram_verify.c \
|
||||||
|
// -lcurl -lpthread -lm
|
||||||
|
// ./transduce # exits 0 only if every check passes
|
||||||
|
//
|
||||||
|
// It writes to an IN-MEMORY engram (leave ENGRAM_STORE unset) and contacts no
|
||||||
|
// server. The same claims are asserted by the native harness in
|
||||||
|
// lang/tests/native/test_transduce.el.
|
||||||
|
//
|
||||||
|
// WHAT CHANGED, AND WHY IT MATTERS. #144 shipped
|
||||||
|
// `transduce(signal, modality) -> Geometry`: one vector per signal. That made
|
||||||
|
// transduction a CONVERSION — take a thing, encode it, store a position — and
|
||||||
|
// what a conversion returns is a fingerprint. A fingerprint can be matched and
|
||||||
|
// ranked, and that is all it can ever do. It cannot be decomposed, cannot have
|
||||||
|
// one part grounded while another is not, and cannot be contradicted in one
|
||||||
|
// part while holding in another, because it has no parts.
|
||||||
|
//
|
||||||
|
// A song is not a point. It decomposes into pitch, interval, rhythm, harmonic
|
||||||
|
// function — components, each with its own geometry, plus the relations among
|
||||||
|
// them. THE SONG IS THE STRUCTURE OF THE RELATIONS. transduce now returns a
|
||||||
|
// Manifold, and a realizer's job is to say what its modality's components ARE.
|
||||||
|
|
||||||
|
fn check(ok: Int, label: String) -> Int {
|
||||||
|
if ok > 0 {
|
||||||
|
println(" ok " + label)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
println(" FAIL " + label)
|
||||||
|
exit(1)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
fn near(a: Float, b: Float) -> Int {
|
||||||
|
let d: Float = a - b
|
||||||
|
if d > 0.001 { return 0 }
|
||||||
|
if d < -0.001 { return 0 }
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
fn eq_int(a: Int, b: Int) -> Int {
|
||||||
|
if a == b { return 1 }
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── A DECOMPOSING realizer, written entirely in El ──────────────────────────
|
||||||
|
// "tone" signals are note letters, e.g. "CEG". This does NOT return one vector
|
||||||
|
// for the chord. It returns the PARTS — one component per note, one per
|
||||||
|
// interval between adjacent notes — and the relations that make those parts a
|
||||||
|
// chord rather than an unordered bag of pitches.
|
||||||
|
//
|
||||||
|
// The interval is deliberately a COMPONENT, not a field on a note. An interval
|
||||||
|
// is a thing with its own geometry belonging to neither endpoint; modelling it
|
||||||
|
// as an attribute of one of them is the same collapse, one level down.
|
||||||
|
fn tone_realizer(signal: String) -> Manifold {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let n: Int = str_len(signal)
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let code: Int = str_char_code(signal, i)
|
||||||
|
let g: Geometry = geometry_new(2)
|
||||||
|
let s0: Int = geometry_set(g, 0, int_to_float(code))
|
||||||
|
let s1: Int = geometry_set(g, 1, int_to_float(i))
|
||||||
|
let idx: Int = manifold_add(m, "note:" + int_to_str(i), "pitch", g)
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
let j: Int = 1
|
||||||
|
while j < n {
|
||||||
|
let a: Int = str_char_code(signal, j - 1)
|
||||||
|
let b: Int = str_char_code(signal, j)
|
||||||
|
let lo: String = "note:" + int_to_str(j - 1)
|
||||||
|
let hi: String = "note:" + int_to_str(j)
|
||||||
|
let key: String = "interval:" + int_to_str(j - 1) + "-" + int_to_str(j)
|
||||||
|
let g: Geometry = geometry_new(1)
|
||||||
|
let s: Int = geometry_set(g, 0, int_to_float(b - a))
|
||||||
|
let idx: Int = manifold_add(m, key, "interval", g)
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let e1: Int = manifold_relate(m, key, "spans", lo, 0.9)
|
||||||
|
let e2: Int = manifold_relate(m, key, "spans", hi, 0.9)
|
||||||
|
let e3: Int = manifold_relate(m, lo, "sounds_before", hi, 0.8)
|
||||||
|
j = j + 1
|
||||||
|
}
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
|
// #144's contract, kept as a control: one vector for the whole signal.
|
||||||
|
fn fingerprint_realizer(signal: String) -> Geometry {
|
||||||
|
let g: Geometry = geometry_new(4)
|
||||||
|
let n: Int = str_len(signal)
|
||||||
|
let a: Int = geometry_set(g, 0, int_to_float(n))
|
||||||
|
g
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() -> Void {
|
||||||
|
println("a realizer declared in El is a first-class realizer")
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let _c: Int = check(reg, "an El fn registers as a realizer by name")
|
||||||
|
let _c: Int = check(realizer_has("tone"), "the modality now has an organ")
|
||||||
|
|
||||||
|
println("transduction decomposes a signal into parts")
|
||||||
|
let m: Manifold = transduce("CEG", "tone")
|
||||||
|
let _c: Int = check(manifold_is(m), "transduce returns a real Manifold")
|
||||||
|
let sz: Int = manifold_size(m)
|
||||||
|
let _c: Int = check(eq_int(sz, 5), "three notes and two intervals are five parts")
|
||||||
|
let rc: Int = manifold_rel_count(m)
|
||||||
|
let _c: Int = check(eq_int(rc, 6), "and they stand in six stated relations")
|
||||||
|
|
||||||
|
println("every part is addressable BY KEY, which is what survives persistence")
|
||||||
|
let i_c: Int = manifold_index_of(m, "note:0")
|
||||||
|
let _c: Int = check(1 - eq_int(i_c, -1), "the first note is addressable on its own")
|
||||||
|
let i_iv: Int = manifold_index_of(m, "interval:0-1")
|
||||||
|
let _c: Int = check(1 - eq_int(i_iv, -1), "so is the interval between the first two")
|
||||||
|
let miss: Int = manifold_index_of(m, "never_added")
|
||||||
|
let _c: Int = check(eq_int(miss, -1), "an unknown key is -1, not component 0")
|
||||||
|
|
||||||
|
println("parts carry their own geometry, and may differ in width")
|
||||||
|
let gn: Geometry = manifold_geometry(m, i_c)
|
||||||
|
let _c: Int = check(eq_int(geometry_dim(gn), 2), "a note component is 2 wide")
|
||||||
|
let _c: Int = check(near(geometry_get(gn, 0), 67.0), "and it is C — the signal reached the realizer")
|
||||||
|
let gi: Geometry = manifold_geometry(m, i_iv)
|
||||||
|
let _c: Int = check(eq_int(geometry_dim(gi), 1), "an interval component is 1 wide")
|
||||||
|
// A single vector per signal cannot represent parts of unequal width at all.
|
||||||
|
let _c: Int = check(near(geometry_get(gi, 0), 2.0), "C to E is two semitones")
|
||||||
|
let f1: Int = geometry_free(gn)
|
||||||
|
let f2: Int = geometry_free(gi)
|
||||||
|
|
||||||
|
println("the relations are content no single part carries")
|
||||||
|
// That "2" above is not a property of C and not a property of E. It exists
|
||||||
|
// only BETWEEN them, so a representation with no relations cannot hold it.
|
||||||
|
let spans: Int = 0
|
||||||
|
let k: Int = 0
|
||||||
|
while k < rc {
|
||||||
|
if str_eq(manifold_rel_name(m, k), "spans") {
|
||||||
|
if str_eq(manifold_rel_from(m, k), "interval:0-1") { spans = spans + 1 }
|
||||||
|
}
|
||||||
|
k = k + 1
|
||||||
|
}
|
||||||
|
let _c: Int = check(eq_int(spans, 2), "the interval is wired to both notes it spans")
|
||||||
|
|
||||||
|
println("relation weight IS the grounding (correspondence-and-censorship §1)")
|
||||||
|
let wk: Int = 0
|
||||||
|
let found: Int = 0
|
||||||
|
while wk < rc {
|
||||||
|
if str_eq(manifold_rel_name(m, wk), "sounds_before") {
|
||||||
|
if near(manifold_rel_weight(m, wk), 0.8) > 0 { found = 1 }
|
||||||
|
}
|
||||||
|
wk = wk + 1
|
||||||
|
}
|
||||||
|
let _c: Int = check(found, "the ordering relation carries the weight its realizer stated")
|
||||||
|
|
||||||
|
println("the decomposition persists as real, separately addressable nodes")
|
||||||
|
let ids: [String] = el_list_empty()
|
||||||
|
let n0: Int = engram_node_count()
|
||||||
|
let e0: Int = engram_edge_count()
|
||||||
|
let pi: Int = 0
|
||||||
|
while pi < sz {
|
||||||
|
let key: String = manifold_key(m, pi)
|
||||||
|
let g: Geometry = manifold_geometry(m, pi)
|
||||||
|
let id: String = engram_node("component " + key, "Concept", 0.6)
|
||||||
|
let att: Int = node_attach_geometry(id, g)
|
||||||
|
ids = el_list_append(ids, id)
|
||||||
|
let ff: Int = geometry_free(g)
|
||||||
|
pi = pi + 1
|
||||||
|
}
|
||||||
|
let ri: Int = 0
|
||||||
|
while ri < rc {
|
||||||
|
let fi: Int = manifold_index_of(m, manifold_rel_from(m, ri))
|
||||||
|
let ti: Int = manifold_index_of(m, manifold_rel_to(m, ri))
|
||||||
|
engram_connect(el_list_get(ids, fi), el_list_get(ids, ti),
|
||||||
|
manifold_rel_weight(m, ri), manifold_rel_name(m, ri))
|
||||||
|
ri = ri + 1
|
||||||
|
}
|
||||||
|
let _c: Int = check(eq_int(engram_node_count() - n0, 5), "one signal became five nodes")
|
||||||
|
let _c: Int = check(eq_int(engram_edge_count() - e0, 6), "and six edges between them")
|
||||||
|
|
||||||
|
println("each part's geometry is independently readable back off its node")
|
||||||
|
let id_c: String = el_list_get(ids, manifold_index_of(m, "note:0"))
|
||||||
|
let id_iv: String = el_list_get(ids, manifold_index_of(m, "interval:0-1"))
|
||||||
|
let _c: Int = check(eq_int(node_geometry_dim(id_c), 2), "note:0 node carries a 2-wide geometry")
|
||||||
|
let _c: Int = check(eq_int(node_geometry_dim(id_iv), 1), "interval:0-1 node carries a 1-wide one")
|
||||||
|
|
||||||
|
println("one part can be grounded without touching its siblings")
|
||||||
|
let ear: String = engram_node("evidence: heard a C in the recording", "Memory", 0.7)
|
||||||
|
engram_connect(ear, id_c, 0.95, "corroborates")
|
||||||
|
let _c: Int = check(engram_edge_between(ear, id_c), "evidence attaches to note:0 specifically")
|
||||||
|
let id_g: String = el_list_get(ids, manifold_index_of(m, "note:2"))
|
||||||
|
let _c: Int = check(1 - engram_edge_between(ear, id_g), "and NOT to note:2 — the sibling is untouched")
|
||||||
|
// This is the whole gain, and it is impossible with a fingerprint: with one
|
||||||
|
// node per signal, "the C is corroborated" and "the G is not" have the same
|
||||||
|
// grounding target and cannot both be recorded.
|
||||||
|
let _c: Int = check(eq_int(node_geometry_dim(id_g), 2), "note:2 geometry is intact regardless")
|
||||||
|
|
||||||
|
println("a fingerprint realizer transduces NOTHING")
|
||||||
|
// #144's contract exactly: signal in, one Geometry out. It resolves, so the
|
||||||
|
// organ is present — but it does not decompose, so it does not transduce.
|
||||||
|
// "No organ" and "an organ that only fingerprints" must not look alike.
|
||||||
|
let rf: Int = realizer_register("fingerprint", "fingerprint_realizer")
|
||||||
|
let _c: Int = check(rf, "the symbol resolves, so registration succeeds")
|
||||||
|
let mf: Manifold = transduce("x", "fingerprint")
|
||||||
|
let _c: Int = check(1 - manifold_is(mf), "a single vector is not a transduction")
|
||||||
|
|
||||||
|
println("the one-part case is a size-one manifold, not a bare vector")
|
||||||
|
let g1: Geometry = geometry_new(3)
|
||||||
|
let s1: Int = geometry_set(g1, 0, 5.0)
|
||||||
|
let ms: Manifold = manifold_single("level", "scalar", g1)
|
||||||
|
let _c: Int = check(manifold_is(ms), "manifold_single yields a real Manifold")
|
||||||
|
let _c: Int = check(eq_int(manifold_size(ms), 1), "of size one — visibly degenerate, not hidden")
|
||||||
|
let fg: Int = geometry_free(g1)
|
||||||
|
let fs: Int = manifold_free(ms)
|
||||||
|
|
||||||
|
println("no organ is still reported as no organ")
|
||||||
|
let me: Manifold = transduce("anything", "echolocation")
|
||||||
|
let _c: Int = check(1 - manifold_is(me), "no realizer means no manifold, not a fake one")
|
||||||
|
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
|
||||||
|
// Reaching here means nothing called exit(1) along the way.
|
||||||
|
println("")
|
||||||
|
println("all checks passed")
|
||||||
|
}
|
||||||
+1946
-84
File diff suppressed because it is too large
Load Diff
+148
-1
@@ -586,6 +586,110 @@ void el_runtime_dharma_event_arrive(const char* event_type,
|
|||||||
const char* payload,
|
const char* payload,
|
||||||
const char* source);
|
const char* source);
|
||||||
|
|
||||||
|
/* ── Geometry: signal as a first-class El value ──────────────────────────────
|
||||||
|
*
|
||||||
|
* A Geometry is an opaque, magic-tagged heap value carried in an el_val_t —
|
||||||
|
* the same discipline as List/Map. It holds a width and a float32 payload,
|
||||||
|
* and it is the medium a non-text modality enters in. Declared HERE, above
|
||||||
|
* the engram block, because transduction is a LANGUAGE concern: every El
|
||||||
|
* program touching any modality needs it, and the engram is merely one El
|
||||||
|
* program that happens to hold a graph. See el_runtime.c ("Geometry: signal
|
||||||
|
* as a first-class el value") for the full rationale.
|
||||||
|
*
|
||||||
|
* El-side type annotation is simply `Geometry` — an opaque boxed pointer,
|
||||||
|
* exactly like Instant / Calendar / Rhythm. No codegen change is required.
|
||||||
|
*
|
||||||
|
* OWNERSHIP: a Geometry is owned by the El caller and released with
|
||||||
|
* geometry_free. node_attach_geometry COPIES, so a node and the caller's
|
||||||
|
* value have independent lifetimes. */
|
||||||
|
|
||||||
|
el_val_t geometry_new(el_val_t dim); /* zero-filled; 0 on failure */
|
||||||
|
el_val_t geometry_dim(el_val_t g); /* width, 0 if not a Geometry */
|
||||||
|
el_val_t geometry_is(el_val_t g); /* 1 if a live Geometry */
|
||||||
|
el_val_t geometry_get(el_val_t g, el_val_t i); /* Float component */
|
||||||
|
el_val_t geometry_set(el_val_t g, el_val_t i, el_val_t x); /* 1 ok / 0 out of range */
|
||||||
|
el_val_t geometry_norm(el_val_t g); /* Float L2 — lets a caller
|
||||||
|
* check a realizer emitted
|
||||||
|
* signal, not zeros */
|
||||||
|
el_val_t geometry_free(el_val_t g); /* 1 if freed, 0 if not a Geometry.
|
||||||
|
* Returns a value (not void) so it
|
||||||
|
* is safe in any El expression
|
||||||
|
* position without a codegen
|
||||||
|
* void-builtin table entry. */
|
||||||
|
|
||||||
|
/* Wire ADAPTERS — the only place an encoding appears, and only at the edge.
|
||||||
|
* `f32le hex` is little-endian float32, 8 hex chars per component: the
|
||||||
|
* encoding the perception vessel's /voice/embed already emits. The width is
|
||||||
|
* DERIVED from the input length, never supplied by a caller — which is why
|
||||||
|
* there is no max-dim constant here to validate a claimed length against. */
|
||||||
|
el_val_t geometry_from_f32le_hex(el_val_t hex); /* 0 on empty/odd-length/non-hex */
|
||||||
|
el_val_t geometry_to_f32le_hex(el_val_t g); /* "" if not a Geometry */
|
||||||
|
|
||||||
|
/* ── Manifold: the result of a transduction ──────────────────────────────────
|
||||||
|
* A transduced signal is a SUBGRAPH — named components, each with its own
|
||||||
|
* geometry, plus typed weighted relations among them — not a single vector.
|
||||||
|
* One vector is a fingerprint: matchable, rankable, and nothing else. A song
|
||||||
|
* decomposes into pitch, interval, rhythm, harmonic function; the song IS the
|
||||||
|
* structure of those relations, and collapsing it to a point discards exactly
|
||||||
|
* what made it reasonable-about. See el_runtime.c ("Manifold") for the full
|
||||||
|
* rationale, the key-addressing rule, and the ownership contract.
|
||||||
|
*
|
||||||
|
* Components are addressed BY KEY, never by index, because the key is what
|
||||||
|
* survives persistence: a component becomes a node, and it is separately
|
||||||
|
* groundable precisely because it is separately named. Relation weight IS the
|
||||||
|
* grounding (correspondence-and-censorship.md §1) — one quantity, no separate
|
||||||
|
* score, nothing computed on read.
|
||||||
|
*
|
||||||
|
* OWNERSHIP: a Manifold is owned by the El caller and released with
|
||||||
|
* manifold_free, which also releases every component's geometry. manifold_add
|
||||||
|
* COPIES the geometry it is given and manifold_geometry RETURNS a copy, so no
|
||||||
|
* component's vector is ever aliased in either direction. */
|
||||||
|
el_val_t manifold_new(void); /* empty; 0 on failure */
|
||||||
|
el_val_t manifold_is(el_val_t m); /* 1 if a live Manifold */
|
||||||
|
el_val_t manifold_add(el_val_t m, el_val_t key, el_val_t role, el_val_t g);
|
||||||
|
/* component index, or -1 on empty/duplicate
|
||||||
|
* key or a value that is not a Geometry */
|
||||||
|
el_val_t manifold_relate(el_val_t m, el_val_t from, el_val_t rel,
|
||||||
|
el_val_t to, el_val_t weight);
|
||||||
|
/* 1 ok / 0 if either endpoint is unknown —
|
||||||
|
* an unresolvable edge is REFUSED, never
|
||||||
|
* silently dropped */
|
||||||
|
el_val_t manifold_size(el_val_t m); /* component count */
|
||||||
|
el_val_t manifold_rel_count(el_val_t m); /* relation count */
|
||||||
|
el_val_t manifold_index_of(el_val_t m, el_val_t key); /* index by key, or -1 */
|
||||||
|
el_val_t manifold_key(el_val_t m, el_val_t i); /* "" if out of range */
|
||||||
|
el_val_t manifold_role(el_val_t m, el_val_t i); /* "" if out of range */
|
||||||
|
el_val_t manifold_geometry(el_val_t m, el_val_t i); /* a COPY the caller frees */
|
||||||
|
el_val_t manifold_rel_from(el_val_t m, el_val_t j); /* source component key */
|
||||||
|
el_val_t manifold_rel_name(el_val_t m, el_val_t j); /* relation name */
|
||||||
|
el_val_t manifold_rel_to(el_val_t m, el_val_t j); /* target component key */
|
||||||
|
el_val_t manifold_rel_weight(el_val_t m, el_val_t j); /* Float — the grounding */
|
||||||
|
el_val_t manifold_single(el_val_t key, el_val_t role, el_val_t g);
|
||||||
|
/* the degenerate one-part case, expressible
|
||||||
|
* but visibly a size-1 manifold rather than
|
||||||
|
* a parallel path back to a bare vector */
|
||||||
|
el_val_t manifold_free(el_val_t m); /* 1 if freed, 0 otherwise */
|
||||||
|
|
||||||
|
/* ── Realizers + transduce ───────────────────────────────────────────────────
|
||||||
|
* A REALIZER DECOMPOSES one modality into components and relations. It does
|
||||||
|
* not encode a signal to a point; that operation is one layer below and is
|
||||||
|
* called geometry. Registration is by NAME, so a new modality never requires a
|
||||||
|
* runtime patch: every El `fn name(...)` compiles to a global C symbol with
|
||||||
|
* that exact name, and the registry resolves it with dlsym against the running
|
||||||
|
* binary — the same mechanism http_set_handler already relies on.
|
||||||
|
*
|
||||||
|
* fn tone_realizer(signal: String) -> Manifold { ... }
|
||||||
|
* realizer_register("tone", "tone_realizer")
|
||||||
|
* let m: Manifold = transduce(sample, "tone")
|
||||||
|
*
|
||||||
|
* SUPERSEDES #144's `transduce -> Geometry`. A realizer that still returns a
|
||||||
|
* bare Geometry now transduces NOTHING (transduce returns 0), deliberately: an
|
||||||
|
* organ that only fingerprints must not be indistinguishable from a working
|
||||||
|
* one. A modality with genuinely one part says so with manifold_single. */
|
||||||
|
el_val_t realizer_register(el_val_t modality, el_val_t fn_name); /* 1 ok / 0 unresolved */
|
||||||
|
el_val_t realizer_has(el_val_t modality); /* 1 if a realizer is registered */
|
||||||
|
el_val_t transduce(el_val_t signal, el_val_t modality); /* Manifold, or 0 if no organ */
|
||||||
|
|
||||||
/* ── Engram local graph primitives ───────────────────────────────────────────
|
/* ── Engram local graph primitives ───────────────────────────────────────────
|
||||||
* Operate on the CGI's local Engram knowledge graph.
|
* Operate on the CGI's local Engram knowledge graph.
|
||||||
* `engram_activate` queries the local graph only; `dharma_activate` is
|
* `engram_activate` queries the local graph only; `dharma_activate` is
|
||||||
@@ -612,7 +716,28 @@ el_val_t engram_get_node(el_val_t id);
|
|||||||
void engram_strengthen(el_val_t node_id);
|
void engram_strengthen(el_val_t node_id);
|
||||||
void engram_forget(el_val_t node_id);
|
void engram_forget(el_val_t node_id);
|
||||||
el_val_t engram_prune_telemetry(el_val_t older_than_ms);
|
el_val_t engram_prune_telemetry(el_val_t older_than_ms);
|
||||||
|
/* Largest byte length <= max_bytes that does not split a UTF-8 codepoint.
|
||||||
|
* Bounded by bytes, not codepoints, so truncated strings never grow. */
|
||||||
|
size_t el_utf8_safe_len(const char* s, size_t max_bytes);
|
||||||
|
|
||||||
el_val_t engram_node_count(void);
|
el_val_t engram_node_count(void);
|
||||||
|
/* Attach a Geometry to an existing node, and read the attached width back.
|
||||||
|
* Named for the operation, not the store: a node acquires geometry. This is
|
||||||
|
* the geometry-valued ingest path — nothing about it is hex, and nothing
|
||||||
|
* about it assumes the caller's vector matches the canonical text-embedding
|
||||||
|
* width. node_geometry_dim exists so an attach is VERIFIED by reading it
|
||||||
|
* back rather than by trusting a success return. */
|
||||||
|
el_val_t node_attach_geometry(el_val_t node_id, el_val_t g); /* 1 ok / 0 otherwise */
|
||||||
|
el_val_t node_geometry_dim(el_val_t node_id); /* width, 0 if none */
|
||||||
|
|
||||||
|
/* DEPRECATED (shipped in #141, superseded 2026-08-16). Equivalent to
|
||||||
|
* geometry_from_f32le_hex + node_attach_geometry, and now implemented as
|
||||||
|
* exactly that. Kept only so anything built against the #141 runtime keeps
|
||||||
|
* linking; `dim` is accepted but treated as an assertion about the vector's
|
||||||
|
* width rather than as its source. New code should not call this — a hex
|
||||||
|
* string is a wire encoding, not a way to move geometry between two pieces
|
||||||
|
* of El. Returns 1 on success, 0 otherwise. */
|
||||||
|
el_val_t engram_node_set_emb(el_val_t id, el_val_t hex, el_val_t dim);
|
||||||
el_val_t engram_search(el_val_t query, el_val_t limit);
|
el_val_t engram_search(el_val_t query, el_val_t limit);
|
||||||
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
|
el_val_t engram_scan_nodes(el_val_t limit, el_val_t offset);
|
||||||
void engram_connect(el_val_t from_id, el_val_t to_id, el_val_t weight, el_val_t relation);
|
void engram_connect(el_val_t from_id, el_val_t to_id, el_val_t weight, el_val_t relation);
|
||||||
@@ -657,8 +782,13 @@ el_val_t engram_geo_analogy_json(el_val_t a_seeds, el_val_t b_seeds);
|
|||||||
el_val_t engram_reason_analogy_json(el_val_t a_seeds, el_val_t b_seeds, el_val_t c_seeds);
|
el_val_t engram_reason_analogy_json(el_val_t a_seeds, el_val_t b_seeds, el_val_t c_seeds);
|
||||||
/* COGNITION (2026-08-14): THE ONE OPERATION + grounding, surfaced live. */
|
/* COGNITION (2026-08-14): THE ONE OPERATION + grounding, surfaced live. */
|
||||||
el_val_t engram_think_json(el_val_t seeds, el_val_t faculty);
|
el_val_t engram_think_json(el_val_t seeds, el_val_t faculty);
|
||||||
|
/* GROUNDING (2026-08-16): grounding is an attribute of the RELATION and it IS the
|
||||||
|
* hebbian weight. ground reads; ground_record writes; trajectory reads the chain. */
|
||||||
el_val_t engram_ground_json(el_val_t claim, el_val_t evidence, el_val_t for_whom);
|
el_val_t engram_ground_json(el_val_t claim, el_val_t evidence, el_val_t for_whom);
|
||||||
el_val_t engram_assert_json(el_val_t claim_id, el_val_t for_whom, el_val_t floor);
|
el_val_t engram_ground_record_json(el_val_t claim, el_val_t evidence,
|
||||||
|
el_val_t provenance, el_val_t floor);
|
||||||
|
el_val_t engram_ground_trajectory_json(el_val_t claim, el_val_t evidence);
|
||||||
|
el_val_t engram_assert_json(el_val_t claim_id, el_val_t for_whom, el_val_t floor, el_val_t rel_floor);
|
||||||
el_val_t engram_attend_json(el_val_t node_id, el_val_t observer, el_val_t salience);
|
el_val_t engram_attend_json(el_val_t node_id, el_val_t observer, el_val_t salience);
|
||||||
el_val_t engram_correspondence_beat_json(el_val_t seeds, el_val_t faculty, el_val_t keystone);
|
el_val_t engram_correspondence_beat_json(el_val_t seeds, el_val_t faculty, el_val_t keystone);
|
||||||
el_val_t engram_consolidate_permanence(el_val_t node_id);
|
el_val_t engram_consolidate_permanence(el_val_t node_id);
|
||||||
@@ -952,6 +1082,22 @@ el_val_t __url_decode(el_val_t s);
|
|||||||
/* Environment */
|
/* Environment */
|
||||||
el_val_t __env_get(el_val_t key);
|
el_val_t __env_get(el_val_t key);
|
||||||
|
|
||||||
|
/* Cross-cutting concerns declared by a `program` block (spec §18).
|
||||||
|
* All three are COMPILER-INJECTED at the head of main() — they are not meant to
|
||||||
|
* be written by hand, which is the point: the guarantee cannot be forgotten at a
|
||||||
|
* call site because there is no call site. */
|
||||||
|
el_val_t el_singleton_acquire(el_val_t id); /* §18.1 process identity */
|
||||||
|
el_val_t el_config_declare(el_val_t name, el_val_t type,
|
||||||
|
el_val_t deflt, el_val_t has_default,
|
||||||
|
el_val_t required); /* §18.2 config schema */
|
||||||
|
el_val_t el_config_validate(el_val_t program_name); /* §18.2 startup validate */
|
||||||
|
|
||||||
|
/* config(key) — the READ side, and the only one programs write by hand. With a
|
||||||
|
* schema declared it is a validated lookup; without one it degrades to getenv.
|
||||||
|
* (Defined in el_runtime.c but previously never prototyped here, so any program
|
||||||
|
* calling it failed to compile under -Werror=implicit-function-declaration.) */
|
||||||
|
el_val_t config(el_val_t key);
|
||||||
|
|
||||||
/* Subprocess */
|
/* Subprocess */
|
||||||
el_val_t __exec(el_val_t cmd);
|
el_val_t __exec(el_val_t cmd);
|
||||||
el_val_t __exec_bg(el_val_t cmd);
|
el_val_t __exec_bg(el_val_t cmd);
|
||||||
@@ -1022,6 +1168,7 @@ el_val_t el_mem_check(void);
|
|||||||
el_val_t el_alloc_count(void);
|
el_val_t el_alloc_count(void);
|
||||||
el_val_t el_alloc_bytes(void);
|
el_val_t el_alloc_bytes(void);
|
||||||
el_val_t el_peak_rss(void);
|
el_val_t el_peak_rss(void);
|
||||||
|
el_val_t el_black_box(el_val_t v);
|
||||||
|
|
||||||
/* Semantic retrieval surface. NOT interchangeable with engram_search_json,
|
/* Semantic retrieval surface. NOT interchangeable with engram_search_json,
|
||||||
* which is lexical by design — see the note at the definition. */
|
* which is lexical by design — see the note at the definition. */
|
||||||
|
|||||||
@@ -0,0 +1,256 @@
|
|||||||
|
// runtime/elbench.el — growth-curve classifier and complexity gate.
|
||||||
|
//
|
||||||
|
// Given a geometric sweep of input sizes and the measurements taken at each,
|
||||||
|
// classify the growth curve and decide whether it violates a declared bound.
|
||||||
|
//
|
||||||
|
// ── Why this exists ──────────────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// Constant-factor regressions are annoying. Complexity regressions are outages.
|
||||||
|
// An O(n) lookup inside an O(n) loop is invisible at n=100 in a unit test and
|
||||||
|
// catastrophic at n=100000 in production. el #132 was exactly that: a strlen()
|
||||||
|
// inside a per-character accessor, quadratic, shipped for months.
|
||||||
|
//
|
||||||
|
// ── THREE signals, not one ───────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// The gate fits time AND allocation-count AND allocation-bytes, and fails if
|
||||||
|
// ANY of them exceeds its declared curve. This is not belt-and-braces; each
|
||||||
|
// signal is blind to a real defect class the others catch:
|
||||||
|
//
|
||||||
|
// * A copy-on-write accumulator rebuilding its buffer allocates ONCE per
|
||||||
|
// iteration — count is exactly linear — while bytes go quadratic.
|
||||||
|
// Count alone passes it.
|
||||||
|
// * el #132's strlen-per-character is pure CPU and allocates NOTHING.
|
||||||
|
// Both allocation signals read FLAT. Only time catches it.
|
||||||
|
//
|
||||||
|
// The deterministic signals (count, bytes) are preferable where they apply:
|
||||||
|
// no statistics, correct on the first run, machine-independent. They are
|
||||||
|
// simply not sufficient.
|
||||||
|
//
|
||||||
|
// ── SCOPE LIMIT — read this before trusting a flat curve ─────────────────────
|
||||||
|
//
|
||||||
|
// The allocation counters track EL-LEVEL allocation only: strings, ElList and
|
||||||
|
// ElMap bodies, their backing arrays, copy-on-write clones, and the realloc
|
||||||
|
// growth path. malloc inside engram_*.c and inside libcurl is NOT counted.
|
||||||
|
//
|
||||||
|
// A flat allocation curve over a workload dominated by engram or HTTP calls is
|
||||||
|
// therefore NOT evidence of anything. It means "no El-level allocation growth",
|
||||||
|
// not "no allocation growth". Gate El-level complexity with this; do not read
|
||||||
|
// third-party memory behaviour into it.
|
||||||
|
//
|
||||||
|
// ── Classification method ────────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// Sizes must form a geometric sweep (each n double the last). On such a sweep
|
||||||
|
// the ratio between consecutive measurements IS the growth exponent, directly:
|
||||||
|
//
|
||||||
|
// O(1) -> 1.0 O(log n) -> ~1.1 O(n) -> 2.0
|
||||||
|
// O(n log n) -> ~2.2 O(n^2) -> 4.0 O(n^3) -> 8.0
|
||||||
|
//
|
||||||
|
// DEVIATION FROM DESIGN.md 6.2, stated plainly: that section specified Google
|
||||||
|
// Benchmark's one-parameter least-squares fit over candidate curves. This uses
|
||||||
|
// consecutive ratios instead. The sweep is mandated geometric either way, and
|
||||||
|
// on a geometric sweep ratios are directly interpretable and need no floating
|
||||||
|
// point. The cost is weaker separation between O(n) and O(n log n), which is
|
||||||
|
// reported honestly as an ambiguous band rather than guessed at. Least-squares
|
||||||
|
// remains the better answer if that band ever needs to be resolved.
|
||||||
|
//
|
||||||
|
// All arithmetic is fixed-point, scaled by 1000 ("milli-ratio"), so a ratio of
|
||||||
|
// 2.0 is 2000. El values are int64; this avoids float-in-list handling.
|
||||||
|
|
||||||
|
// Curve identifiers. Ordered by growth — the ordering IS the comparison used
|
||||||
|
// by the gate, so an index comparison decides "worse than declared".
|
||||||
|
// 0 = O(1) 1 = O(log n) 2 = O(n) 3 = O(n log n) 4 = O(n^2) 5 = O(n^3)
|
||||||
|
|
||||||
|
fn elb_curve_name(c: Int) -> String {
|
||||||
|
if c == 0 { return "O(1)" }
|
||||||
|
if c == 1 { return "O(log n)" }
|
||||||
|
if c == 2 { return "O(n)" }
|
||||||
|
if c == 3 { return "O(n log n)" }
|
||||||
|
if c == 4 { return "O(n^2)" }
|
||||||
|
if c == 5 { return "O(n^3)" }
|
||||||
|
return "O(?)"
|
||||||
|
}
|
||||||
|
|
||||||
|
fn elb_curve_from_name(s: String) -> Int {
|
||||||
|
if str_eq(s, "O(1)") { return 0 }
|
||||||
|
if str_eq(s, "O(log n)") { return 1 }
|
||||||
|
if str_eq(s, "O(n)") { return 2 }
|
||||||
|
if str_eq(s, "O(n log n)") { return 3 }
|
||||||
|
if str_eq(s, "O(n^2)") { return 4 }
|
||||||
|
if str_eq(s, "O(n^3)") { return 5 }
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_classify_ratio — map a milli-ratio-per-doubling onto a curve.
|
||||||
|
//
|
||||||
|
// Bands are deliberately wide at the top (a quadratic measured at 3.4x is
|
||||||
|
// still a quadratic) and deliberately overlap-averse at the bottom, where a
|
||||||
|
// misclassification between O(1) and O(log n) matters least.
|
||||||
|
fn elb_classify_ratio(milli: Int) -> Int {
|
||||||
|
if milli < 1300 { return 0 }
|
||||||
|
if milli < 1700 { return 1 }
|
||||||
|
if milli < 2400 { return 2 }
|
||||||
|
if milli < 3200 { return 3 }
|
||||||
|
if milli < 6000 { return 4 }
|
||||||
|
return 5
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_ratio — milli-ratio between two consecutive measurements.
|
||||||
|
// Returns -1 when the earlier measurement is zero (ratio undefined).
|
||||||
|
fn elb_ratio(prev: Int, cur: Int) -> Int {
|
||||||
|
if prev <= 0 { return -1 }
|
||||||
|
return (cur * 1000) / prev
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── The measurement floor ────────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// A benchmark whose largest measurement is at or near zero has not been
|
||||||
|
// measured. Reporting it as O(1) would be a confident answer with nothing
|
||||||
|
// behind it — the same failure as a test that never ran reporting pass, and
|
||||||
|
// exactly what happened when clang closed a nested loop to a multiply and the
|
||||||
|
// harness read 0 microseconds at every n.
|
||||||
|
//
|
||||||
|
// So: REFUSE. Never classify below the floor.
|
||||||
|
fn elb_below_floor(vals: [Int], floor: Int) -> Bool {
|
||||||
|
let n: Int = native_list_len(vals)
|
||||||
|
let i: Int = 0
|
||||||
|
let mx: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let v: Int = native_list_get(vals, i)
|
||||||
|
if v > mx { let mx = v }
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
if mx < floor { return true }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_implausibly_flat — a measurement that does not move across a sweep whose
|
||||||
|
// input grew by 8x or more is not a flat curve, it is a broken measurement.
|
||||||
|
// Genuine O(1) work still shows noise; a hard-flat series means the work was
|
||||||
|
// optimised away, the timer has insufficient resolution, or the benchmark body
|
||||||
|
// never executed.
|
||||||
|
fn elb_implausibly_flat(vals: [Int]) -> Bool {
|
||||||
|
let n: Int = native_list_len(vals)
|
||||||
|
if n < 3 { return false }
|
||||||
|
let first: Int = native_list_get(vals, 0)
|
||||||
|
let last: Int = native_list_get(vals, n - 1)
|
||||||
|
if first == 0 {
|
||||||
|
if last == 0 { return true }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
let r: Int = (last * 1000) / first
|
||||||
|
if r < 1100 { return true }
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_spread_ok — do the consecutive ratios agree with each other?
|
||||||
|
//
|
||||||
|
// This is the ratio-method analogue of a normalised-RMS threshold. If the
|
||||||
|
// doublings disagree wildly the data is noise, a cache cliff, or a phase
|
||||||
|
// change, and the honest report is INDETERMINATE rather than a classification.
|
||||||
|
// Applies to the ASYMPTOTIC TAIL only — the last three ratios.
|
||||||
|
//
|
||||||
|
// The small-n end of any sweep is dominated by fixed overhead, cold caches and
|
||||||
|
// branch predictors that have not warmed. Measured on a genuinely linear
|
||||||
|
// character scan, the ratios ran 3.37, 2.92, 1.76, 1.65: the head looks
|
||||||
|
// quadratic, the tail is the truth. Checking spread across the whole sweep
|
||||||
|
// therefore rejects correct data. A complexity bound is an asymptotic claim, so
|
||||||
|
// it is judged on the asymptotic region — the same reason a benchmark harness
|
||||||
|
// discards warmup rather than averaging it in.
|
||||||
|
fn elb_spread_ok(ratios: [Int]) -> Bool {
|
||||||
|
let total: Int = native_list_len(ratios)
|
||||||
|
if total < 2 { return true }
|
||||||
|
let start: Int = total - 3
|
||||||
|
if start < 0 { let start = 0 }
|
||||||
|
let n: Int = total
|
||||||
|
let lo: Int = 999999
|
||||||
|
let hi: Int = 0
|
||||||
|
let i: Int = start
|
||||||
|
while i < n {
|
||||||
|
let r: Int = native_list_get(ratios, i)
|
||||||
|
if r >= 0 {
|
||||||
|
if r < lo { let lo = r }
|
||||||
|
if r > hi { let hi = r }
|
||||||
|
}
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
if lo <= 0 { return false }
|
||||||
|
// Reject when the widest ratio is more than 2.2x the narrowest. That is
|
||||||
|
// enough slack for real timing noise and tight enough to separate a clean
|
||||||
|
// 2.0 series from a clean 4.0 series.
|
||||||
|
if (hi * 1000) / lo > 2200 { return false }
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_ratios — consecutive milli-ratios across the sweep.
|
||||||
|
fn elb_ratios(vals: [Int]) -> [Int] {
|
||||||
|
let out: [Int] = native_list_empty()
|
||||||
|
let n: Int = native_list_len(vals)
|
||||||
|
let i: Int = 1
|
||||||
|
while i < n {
|
||||||
|
let out = native_list_append(out,
|
||||||
|
elb_ratio(native_list_get(vals, i - 1), native_list_get(vals, i)))
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_mean_tail_ratio — mean of the LAST TWO ratios.
|
||||||
|
//
|
||||||
|
// The tail is used deliberately: asymptotic behaviour is what a complexity
|
||||||
|
// bound claims, and the small-n end of any sweep is dominated by fixed
|
||||||
|
// overhead. This is the same reason a benchmark harness discards warmup.
|
||||||
|
fn elb_mean_tail_ratio(ratios: [Int]) -> Int {
|
||||||
|
let n: Int = native_list_len(ratios)
|
||||||
|
if n == 0 { return -1 }
|
||||||
|
if n == 1 { return native_list_get(ratios, 0) }
|
||||||
|
let a: Int = native_list_get(ratios, n - 1)
|
||||||
|
let b: Int = native_list_get(ratios, n - 2)
|
||||||
|
if a < 0 { return b }
|
||||||
|
if b < 0 { return a }
|
||||||
|
return (a + b) / 2
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Verdicts ─────────────────────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// 0 PASS measured curve is at or below the declared bound
|
||||||
|
// 1 FAIL measured curve is strictly worse than declared
|
||||||
|
// 2 INDETERMINATE ratios disagree; data is noise or a phase change
|
||||||
|
// 3 REFUSED below the measurement floor, or implausibly flat
|
||||||
|
// 4 BETTER measured strictly better than declared (warn, not fail)
|
||||||
|
|
||||||
|
fn elb_verdict_name(v: Int) -> String {
|
||||||
|
if v == 0 { return "PASS" }
|
||||||
|
if v == 1 { return "FAIL" }
|
||||||
|
if v == 2 { return "INDETERMINATE" }
|
||||||
|
if v == 3 { return "REFUSED" }
|
||||||
|
if v == 4 { return "BETTER" }
|
||||||
|
return "?"
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_gate — classify one signal against its declared bound.
|
||||||
|
//
|
||||||
|
// vals measurements, one per sweep point, in sweep order
|
||||||
|
// expect declared curve index (see elb_curve_name)
|
||||||
|
// floor minimum largest-measurement below which we refuse to classify
|
||||||
|
fn elb_gate(vals: [Int], expect: Int, floor: Int) -> Int {
|
||||||
|
if elb_below_floor(vals, floor) { return 3 }
|
||||||
|
if elb_implausibly_flat(vals) { return 3 }
|
||||||
|
let ratios: [Int] = elb_ratios(vals)
|
||||||
|
if !elb_spread_ok(ratios) { return 2 }
|
||||||
|
let m: Int = elb_mean_tail_ratio(ratios)
|
||||||
|
if m < 0 { return 2 }
|
||||||
|
let got: Int = elb_classify_ratio(m)
|
||||||
|
if got > expect { return 1 }
|
||||||
|
if got < expect { return 4 }
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// elb_measured_curve — the classified curve for a signal, or -1 if unclassifiable.
|
||||||
|
fn elb_measured_curve(vals: [Int], floor: Int) -> Int {
|
||||||
|
if elb_below_floor(vals, floor) { return -1 }
|
||||||
|
if elb_implausibly_flat(vals) { return -1 }
|
||||||
|
let ratios: [Int] = elb_ratios(vals)
|
||||||
|
let m: Int = elb_mean_tail_ratio(ratios)
|
||||||
|
if m < 0 { return -1 }
|
||||||
|
return elb_classify_ratio(m)
|
||||||
|
}
|
||||||
+372
-29
@@ -246,14 +246,6 @@ static int put_edge(EngramPagedStore* s, const char* id, const char* from, const
|
|||||||
e.metadata = (char*)meta;
|
e.metadata = (char*)meta;
|
||||||
return store_put_edge(s, &e);
|
return store_put_edge(s, &e);
|
||||||
}
|
}
|
||||||
int cog_ground_edge(EngramPagedStore* s, const char* claim_id,
|
|
||||||
const char* evidence_id, double grounding, const char* for_whom) {
|
|
||||||
if (!s || !claim_id || !evidence_id) return -1;
|
|
||||||
char id[512], meta[256];
|
|
||||||
snprintf(id, sizeof id, "gb-%s-%s-%s", claim_id, evidence_id, for_whom ? for_whom : "global");
|
|
||||||
snprintf(meta, sizeof meta, "for_whom=%s", for_whom ? for_whom : "-");
|
|
||||||
return put_edge(s, id, claim_id, evidence_id, COG_GROUNDED_BY_RELATION, grounding, meta);
|
|
||||||
}
|
|
||||||
int cog_salient_edge(EngramPagedStore* s, const char* node_id,
|
int cog_salient_edge(EngramPagedStore* s, const char* node_id,
|
||||||
const char* observer_id, double salience) {
|
const char* observer_id, double salience) {
|
||||||
if (!s || !node_id || !observer_id) return -1;
|
if (!s || !node_id || !observer_id) return -1;
|
||||||
@@ -261,35 +253,386 @@ int cog_salient_edge(EngramPagedStore* s, const char* node_id,
|
|||||||
snprintf(id, sizeof id, "st-%s-%s", node_id, observer_id);
|
snprintf(id, sizeof id, "st-%s-%s", node_id, observer_id);
|
||||||
return put_edge(s, id, node_id, observer_id, COG_SALIENT_TO_RELATION, salience, NULL);
|
return put_edge(s, id, node_id, observer_id, COG_SALIENT_TO_RELATION, salience, NULL);
|
||||||
}
|
}
|
||||||
int cog_assert_gate(EngramPagedStore* s, const char* claim_id,
|
/* ═══════════════════════════════════════════════════════════════════════════
|
||||||
const char* for_whom, double floor) {
|
* §7 GROUNDING IS THE EDGE'S WEIGHT, AND THE WEIGHT IS A VECTOR.
|
||||||
if (!s || !claim_id) return -1;
|
* See engram_cognition.h §7 for the model and for the measurements the two
|
||||||
if (!(floor > 0)) floor = 0.5;
|
* design decisions (thirteen regions, min aggregate) rest on.
|
||||||
StoreEdge* edges = NULL; size_t n = 0;
|
* ═══════════════════════════════════════════════════════════════════════════ */
|
||||||
if (store_get_edges_from(s, claim_id, &edges, &n) < 0) return -1;
|
|
||||||
double best = 0.0; int found = 0;
|
/* ── The one decay model. Moved here verbatim from el_runtime.c's
|
||||||
for (size_t i = 0; i < n; i++) {
|
* engram_temporal_decay so nodes and edges share a single implementation and a
|
||||||
if (!edges[i].relation || strcmp(edges[i].relation, COG_GROUNDED_BY_RELATION) != 0) continue;
|
* single set of constants; engram_temporal_decay now delegates. Bit-identical
|
||||||
/* grounded-for-whom: match observer if requested; global (for_whom=-) always counts */
|
* for nodes: reinforcements := activation_count, lambda_override :=
|
||||||
int match = 1;
|
* temporal_decay_rate.
|
||||||
if (for_whom && edges[i].metadata) {
|
*
|
||||||
const char* fw = strstr(edges[i].metadata, "for_whom=");
|
* This is what makes decay ANALYTIC rather than sampled: between two recorded
|
||||||
if (fw) { fw += 9; if (strcmp(fw, for_whom) != 0 && strcmp(fw, "-") != 0) match = 0; }
|
* versions the trajectory is not unknown, it is known in closed form from the
|
||||||
}
|
* last point and elapsed time. Store the point, read the curve. */
|
||||||
if (match) { found = 1; if (edges[i].weight > best) best = edges[i].weight; }
|
double cog_decay_factor(int64_t age_ms, double reinforcements, double lambda_override) {
|
||||||
}
|
if (age_ms <= 0) return 1.0;
|
||||||
store_edges_free(edges, n);
|
double lambda = (lambda_override > 0.0) ? lambda_override : COG_DECAY_LAMBDA;
|
||||||
if (!found) return 0; /* ungrounded => refuse assertion (still held) */
|
double age_hours = (double)age_ms / 3600000.0;
|
||||||
return (best >= floor) ? 1 : 0;
|
if (reinforcements < 0) reinforcements = 0;
|
||||||
|
double t_half = COG_T_HALF_HOURS * (1.0 + log(1.0 + reinforcements));
|
||||||
|
double factor = exp(-lambda * age_hours / t_half);
|
||||||
|
if (factor < COG_DECAY_FLOOR) factor = COG_DECAY_FLOOR;
|
||||||
|
return factor;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const char* cog_prov_name(CogProvClass p) {
|
||||||
|
switch (p) {
|
||||||
|
case COG_PROV_OBSERVED: return "observed";
|
||||||
|
case COG_PROV_INFERRED: return "inferred";
|
||||||
|
case COG_PROV_TOLD: return "told";
|
||||||
|
case COG_PROV_IMPRINTED: return "imprinted";
|
||||||
|
default: return "unset";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
CogProvClass cog_prov_parse(const char* s) {
|
||||||
|
if (!s) return COG_PROV_UNSET;
|
||||||
|
if (!strcmp(s, "observed")) return COG_PROV_OBSERVED;
|
||||||
|
if (!strcmp(s, "inferred")) return COG_PROV_INFERRED;
|
||||||
|
if (!strcmp(s, "told")) return COG_PROV_TOLD;
|
||||||
|
if (!strcmp(s, "imprinted")) return COG_PROV_IMPRINTED;
|
||||||
|
return COG_PROV_UNSET;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Locate the GRD1 block in an edge's metadata. It is always the tail; anything
|
||||||
|
* ahead of it is the edge's pre-existing metadata, preserved verbatim. */
|
||||||
|
static const char* cog_grd_find(const char* meta) {
|
||||||
|
if (!meta) return NULL;
|
||||||
|
size_t ml = strlen(COG_GROUNDING_META_MAGIC);
|
||||||
|
if (strncmp(meta, COG_GROUNDING_META_MAGIC, ml) == 0) return meta;
|
||||||
|
const char* p = meta;
|
||||||
|
while ((p = strstr(p, COG_GROUNDING_META_MAGIC)) != NULL) {
|
||||||
|
if (p > meta && p[-1] == '\n') return p;
|
||||||
|
p += ml;
|
||||||
|
}
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
int cog_grounding_parse(const StoreEdge* e, int64_t now_ms, CogGrounding* out) {
|
||||||
|
if (!e || !out) return -1;
|
||||||
|
memset(out, 0, sizeof *out);
|
||||||
|
|
||||||
|
/* Two dimensions exist on every edge whether or not grounding has ever been
|
||||||
|
* established, because they ARE existing substrate rather than new fields:
|
||||||
|
* associative — the accrued hebb, with its existing dynamics;
|
||||||
|
* polarity — the signed authored weight. `inhibitory` is precisely this
|
||||||
|
* distinction crushed to one bit, so it is the seed sign. */
|
||||||
|
out->associative = e->hebb;
|
||||||
|
out->polarity = e->inhibitory ? -e->weight : e->weight;
|
||||||
|
out->prov = COG_PROV_UNSET;
|
||||||
|
out->ts = e->last_fired > 0 ? e->last_fired : e->updated_at;
|
||||||
|
|
||||||
|
const char* blk = cog_grd_find(e->metadata);
|
||||||
|
if (blk) {
|
||||||
|
out->present = 1;
|
||||||
|
char* copy = dupstr(blk);
|
||||||
|
if (!copy) return -1;
|
||||||
|
for (char* line = strtok(copy, "\n"); line; line = strtok(NULL, "\n")) {
|
||||||
|
if (line[0] == '\0') continue;
|
||||||
|
char tag = line[0];
|
||||||
|
const char* rest = line + 1; while (*rest == ' ') rest++;
|
||||||
|
if (tag == 'w') { /* the four numeric dimensions */
|
||||||
|
double v[4] = {0,0,0,0}; parse_floats(rest, v, 4);
|
||||||
|
out->factual = v[0]; out->relational = v[1];
|
||||||
|
out->associative = v[2]; out->polarity = v[3];
|
||||||
|
} else if (tag == 'k') { /* provenance class */
|
||||||
|
out->prov = cog_prov_parse(rest);
|
||||||
|
} else if (tag == 't') { /* timestamp + seq + reinforcements */
|
||||||
|
double v[3] = {0,0,0}; parse_floats(rest, v, 3);
|
||||||
|
out->ts = (int64_t)v[0]; out->seq = (int64_t)v[1]; out->reinforcements = v[2];
|
||||||
|
} else if (tag == 'd') {
|
||||||
|
double v[3] = {0,0,0}; parse_floats(rest, v, 3);
|
||||||
|
out->fac_proj = v[0]; out->rel_proj = v[1]; out->cos_angle = v[2];
|
||||||
|
} else if (tag == 'v') {
|
||||||
|
snprintf(out->binding_value, sizeof out->binding_value, "%s", rest);
|
||||||
|
} else if (tag == 'c') {
|
||||||
|
double v[2] = {0,0}; parse_floats(rest, v, 2);
|
||||||
|
out->floor_at_record = v[0]; out->rel_floor_at_record = v[1];
|
||||||
|
} else if (tag == 'p') {
|
||||||
|
snprintf(out->prev_edge, sizeof out->prev_edge, "%s", rest);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
free(copy);
|
||||||
|
}
|
||||||
|
out->agreement = (out->cos_angle > 0) ? 1 : (out->cos_angle < 0 ? -1 : 0);
|
||||||
|
|
||||||
|
/* ── DERIVED. Nothing below this line is ever serialized. Recency, decay and
|
||||||
|
* staleness are read off the curve; storing them is how a number ends up
|
||||||
|
* asserting something nothing computed (§8.1 / spec §2). */
|
||||||
|
out->age_ms = (out->ts > 0 && now_ms > out->ts) ? (now_ms - out->ts) : 0;
|
||||||
|
out->decay = cog_decay_factor(out->age_ms, out->reinforcements, 0.0);
|
||||||
|
out->factual_now = out->factual * out->decay;
|
||||||
|
out->relational_now = out->relational * out->decay;
|
||||||
|
out->associative_now = out->associative * out->decay;
|
||||||
|
out->stale = (out->present && out->floor_at_record > 0 &&
|
||||||
|
out->factual_now < out->floor_at_record) ? 1 : 0;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
char* cog_grounding_metadata(const char* base_meta, const CogGrounding* g) {
|
||||||
|
if (!g) return NULL;
|
||||||
|
size_t keep = 0;
|
||||||
|
if (base_meta) {
|
||||||
|
const char* blk = cog_grd_find(base_meta);
|
||||||
|
keep = blk ? (size_t)(blk - base_meta) : strlen(base_meta);
|
||||||
|
while (keep > 0 && base_meta[keep - 1] == '\n') keep--;
|
||||||
|
}
|
||||||
|
size_t cap = keep + 1024;
|
||||||
|
char* buf = malloc(cap); if (!buf) return NULL;
|
||||||
|
size_t o = 0;
|
||||||
|
if (keep) { memcpy(buf, base_meta, keep); o = keep; buf[o++] = '\n'; }
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "%s\n", COG_GROUNDING_META_MAGIC);
|
||||||
|
/* STORED ONLY. factual / relational / associative / polarity / provenance /
|
||||||
|
* timestamp — plus the joint state a decision saw. No confidence, no
|
||||||
|
* recency, no staleness, no volatility: those are read off the curve. */
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "w %.9g %.9g %.9g %.9g\n",
|
||||||
|
g->factual, g->relational, g->associative, g->polarity);
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "k %s\n", cog_prov_name(g->prov));
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "t %lld %lld %.9g\n",
|
||||||
|
(long long)g->ts, (long long)g->seq, g->reinforcements);
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "d %.9g %.9g %.9g\n",
|
||||||
|
g->fac_proj, g->rel_proj, g->cos_angle);
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "v %s\n", g->binding_value[0] ? g->binding_value : "-");
|
||||||
|
o += (size_t)snprintf(buf + o, cap - o, "c %.9g %.9g\n", g->floor_at_record, g->rel_floor_at_record);
|
||||||
|
if (g->prev_edge[0]) o += (size_t)snprintf(buf + o, cap - o, "p %s\n", g->prev_edge);
|
||||||
|
(void)o;
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Consequence, not epsilon. Every test is a floor crossing or a sign change,
|
||||||
|
* both exact. Ordered so the two INHERENT (discrete) moves are reported in
|
||||||
|
* preference to the graded ones, because they bypass the salience gate. */
|
||||||
|
CogSignificance cog_grounding_significant(const CogGrounding* prev,
|
||||||
|
const CogGrounding* now,
|
||||||
|
double floor, double rel_floor) {
|
||||||
|
if (!now) return COG_SIG_NONE;
|
||||||
|
if (!prev || !prev->present) return COG_SIG_FIRST_RECORD;
|
||||||
|
|
||||||
|
/* INHERENT 1 — polarity sign flip. Ignorance and disagreement are different
|
||||||
|
* states, and support → contradiction is a change of state rather than a
|
||||||
|
* drift, so no threshold applies. Comparing signs, with zero its own class. */
|
||||||
|
{
|
||||||
|
int sp = prev->polarity > 0 ? 1 : (prev->polarity < 0 ? -1 : 0);
|
||||||
|
int sn = now->polarity > 0 ? 1 : (now->polarity < 0 ? -1 : 0);
|
||||||
|
if (sp != sn) return COG_SIG_POLARITY_FLIP;
|
||||||
|
}
|
||||||
|
/* INHERENT 2 — provenance class change. told → observed is a categorical
|
||||||
|
* upgrade in what the relation is entitled to, not a movement along an axis. */
|
||||||
|
if (prev->prov != now->prov) return COG_SIG_PROVENANCE_CHANGE;
|
||||||
|
|
||||||
|
/* Crossing an assert floor — the move changes whether this relation can be
|
||||||
|
* spoken. Compared on the DECAYED values, because that is what the gate reads. */
|
||||||
|
if ((prev->factual_now >= floor) != (now->factual_now >= floor)) return COG_SIG_FACTUAL_FLOOR;
|
||||||
|
if ((prev->relational_now >= rel_floor) != (now->relational_now >= rel_floor)) return COG_SIG_RELATIONAL_FLOOR;
|
||||||
|
|
||||||
|
/* Flipping factual/relational agreement — the relation stops being "true and
|
||||||
|
* meaningful" and becomes "true and misapplied", or the reverse. This is the
|
||||||
|
* 911/CPS contradiction as a measured event rather than a reviewable one. */
|
||||||
|
if (prev->agreement != now->agreement) return COG_SIG_AGREEMENT_FLIP;
|
||||||
|
|
||||||
|
/* A gradient reversing — the evidence stopped pulling the claim toward it and
|
||||||
|
* began pushing it away, or the same on the values axis. */
|
||||||
|
if ((prev->fac_proj > 0) != (now->fac_proj > 0)) return COG_SIG_DIRECTION_REVERSAL;
|
||||||
|
if ((prev->rel_proj > 0) != (now->rel_proj > 0)) return COG_SIG_DIRECTION_REVERSAL;
|
||||||
|
|
||||||
|
return COG_SIG_NONE;
|
||||||
|
}
|
||||||
|
|
||||||
|
int cog_significance_inherent(CogSignificance s) {
|
||||||
|
return (s == COG_SIG_FIRST_RECORD || s == COG_SIG_POLARITY_FLIP ||
|
||||||
|
s == COG_SIG_PROVENANCE_CHANGE) ? 1 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
const char* cog_significance_name(CogSignificance s) {
|
||||||
|
switch (s) {
|
||||||
|
case COG_SIG_FIRST_RECORD: return "first-record";
|
||||||
|
case COG_SIG_POLARITY_FLIP: return "polarity-sign-flip";
|
||||||
|
case COG_SIG_PROVENANCE_CHANGE: return "provenance-class-change";
|
||||||
|
case COG_SIG_FACTUAL_FLOOR: return "factual-floor-crossed";
|
||||||
|
case COG_SIG_RELATIONAL_FLOOR: return "relational-floor-crossed";
|
||||||
|
case COG_SIG_AGREEMENT_FLIP: return "agreement-sign-flip";
|
||||||
|
case COG_SIG_DIRECTION_REVERSAL: return "gradient-direction-reversal";
|
||||||
|
default: return "none";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Recording: a NEW edge record. The predecessor is never touched. ────────── */
|
||||||
|
int cog_grounding_record(EngramPagedStore* s, const StoreEdge* base,
|
||||||
|
const CogGrounding* g, char* out_id, size_t out_id_cap) {
|
||||||
|
if (!s || !base || !base->id || !g) return -1;
|
||||||
|
char root[192];
|
||||||
|
snprintf(root, sizeof root, "%s", base->id);
|
||||||
|
char* hash = strchr(root, '#'); if (hash) *hash = '\0';
|
||||||
|
|
||||||
|
int seq = (int)g->seq + 1;
|
||||||
|
char vid[224];
|
||||||
|
snprintf(vid, sizeof vid, "%s#%d", root, seq);
|
||||||
|
|
||||||
|
CogGrounding rec = *g;
|
||||||
|
rec.seq = seq;
|
||||||
|
snprintf(rec.prev_edge, sizeof rec.prev_edge, "%s", base->id);
|
||||||
|
|
||||||
|
char* meta = cog_grounding_metadata(base->metadata, &rec);
|
||||||
|
if (!meta) return -1;
|
||||||
|
|
||||||
|
StoreEdge e; memset(&e, 0, sizeof e);
|
||||||
|
e.id = vid; e.from_id = base->from_id; e.to_id = base->to_id;
|
||||||
|
e.relation = base->relation; e.metadata = meta;
|
||||||
|
/* The vector IS the weight, so the scalar fields carry their dimensions:
|
||||||
|
* `weight` the magnitude of polarity, `inhibitory` its sign, `hebb` the
|
||||||
|
* associative strength. Nothing here is a second copy of a derived value. */
|
||||||
|
e.weight = rec.polarity < 0 ? -rec.polarity : rec.polarity;
|
||||||
|
e.inhibitory = rec.polarity < 0 ? 1 : 0;
|
||||||
|
e.hebb = rec.associative;
|
||||||
|
e.confidence = base->confidence;
|
||||||
|
e.created_at = base->created_at;
|
||||||
|
e.updated_at = rec.ts;
|
||||||
|
e.last_fired = rec.ts;
|
||||||
|
e.layer_id = base->layer_id;
|
||||||
|
int rc = store_put_edge(s, &e);
|
||||||
|
free(meta);
|
||||||
|
if (rc != 0) return -1;
|
||||||
|
if (out_id && out_id_cap) snprintf(out_id, out_id_cap, "%s", vid);
|
||||||
|
return seq;
|
||||||
|
}
|
||||||
|
|
||||||
|
int cog_grounding_head(EngramPagedStore* s, const char* base_id,
|
||||||
|
StoreEdge* out, int max_versions) {
|
||||||
|
if (!s || !base_id || !out) return -1;
|
||||||
|
if (max_versions <= 0) max_versions = 64;
|
||||||
|
char root[192]; snprintf(root, sizeof root, "%s", base_id);
|
||||||
|
char* hash = strchr(root, '#'); if (hash) *hash = '\0';
|
||||||
|
|
||||||
|
StoreEdge cur; memset(&cur, 0, sizeof cur);
|
||||||
|
if (store_get_edge(s, root, &cur) != 1) return -1;
|
||||||
|
int found = 0;
|
||||||
|
for (int v = 1; v <= max_versions; v++) {
|
||||||
|
char vid[224]; snprintf(vid, sizeof vid, "%s#%d", root, v);
|
||||||
|
StoreEdge nx;
|
||||||
|
if (store_get_edge(s, vid, &nx) != 1) break;
|
||||||
|
store_edge_free(&cur); cur = nx; found = v;
|
||||||
|
}
|
||||||
|
*out = cur;
|
||||||
|
return found;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── VOLATILITY AND DRIFT: derived from the chain, stored nowhere. The series
|
||||||
|
* exists only because nothing was destroyed, which is the whole return on
|
||||||
|
* immutability — a derivative for free. */
|
||||||
|
int cog_grounding_trajectory(EngramPagedStore* s, const char* base_id,
|
||||||
|
int64_t now_ms, CogTrajectory* out) {
|
||||||
|
if (!s || !base_id || !out) return -1;
|
||||||
|
memset(out, 0, sizeof *out);
|
||||||
|
char root[192]; snprintf(root, sizeof root, "%s", base_id);
|
||||||
|
char* hash = strchr(root, '#'); if (hash) *hash = '\0';
|
||||||
|
|
||||||
|
double pf = 0, pr = 0, f0 = 0, r0 = 0, fN = 0, rN = 0;
|
||||||
|
double sum_df = 0, sum_dr = 0;
|
||||||
|
int n = 0;
|
||||||
|
for (int v = 0; v <= 64; v++) {
|
||||||
|
char vid[224];
|
||||||
|
if (v == 0) snprintf(vid, sizeof vid, "%s", root);
|
||||||
|
else snprintf(vid, sizeof vid, "%s#%d", root, v);
|
||||||
|
StoreEdge e;
|
||||||
|
if (store_get_edge(s, vid, &e) != 1) { if (v) break; else continue; }
|
||||||
|
CogGrounding g;
|
||||||
|
if (cog_grounding_parse(&e, now_ms, &g) == 0) {
|
||||||
|
if (n == 0) { f0 = g.factual; r0 = g.relational; }
|
||||||
|
else { sum_df += fabs(g.factual - pf); sum_dr += fabs(g.relational - pr); }
|
||||||
|
pf = g.factual; pr = g.relational; fN = pf; rN = pr;
|
||||||
|
n++;
|
||||||
|
}
|
||||||
|
store_edge_free(&e);
|
||||||
|
}
|
||||||
|
out->n_versions = n;
|
||||||
|
if (n > 1) {
|
||||||
|
out->factual_volatility = sum_df / (double)(n - 1);
|
||||||
|
out->relational_volatility = sum_dr / (double)(n - 1);
|
||||||
|
}
|
||||||
|
out->factual_drift = fN - f0;
|
||||||
|
out->relational_drift = rN - r0;
|
||||||
|
/* "STAYED TRUE, BECAME WRONG" — the event the joint record makes visible and
|
||||||
|
* that per-dimension versioning would have destroyed: the fact held while
|
||||||
|
* the meaning degraded. Expressed as signs, so there is no tolerance here
|
||||||
|
* either: factual did not fall, relational did. */
|
||||||
|
out->stayed_true_became_wrong =
|
||||||
|
(n > 1 && out->factual_drift >= 0 && out->relational_drift < 0) ? 1 : 0;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Assertion gates on BOTH floors. Traversal is untouched: activation still
|
||||||
|
* conducts on the factual/associative side, so a relation can remain thinkable
|
||||||
|
* while ceasing to be assertable. That gap is where the wide angles live. ──── */
|
||||||
|
int cog_assert_two_axis(EngramPagedStore* s, const char* claim_id,
|
||||||
|
double floor, double rel_floor, int64_t now_ms,
|
||||||
|
CogAssertion* out) {
|
||||||
|
if (!s || !claim_id || !out) return -1;
|
||||||
|
memset(out, 0, sizeof *out);
|
||||||
|
if (!(floor > 0)) floor = 0.5;
|
||||||
|
if (!(rel_floor > 0)) rel_floor = floor;
|
||||||
|
|
||||||
|
/* still_held is DERIVED, not a literal (§8.1). Holding is unconditional —
|
||||||
|
* the store gates nothing — so the question the field actually answers is
|
||||||
|
* whether the content is present and live. */
|
||||||
|
StoreNode n;
|
||||||
|
if (store_get_node(s, claim_id, &n) == 1) { out->still_held = !n.tombstoned; store_node_free(&n); }
|
||||||
|
else out->still_held = 0;
|
||||||
|
|
||||||
|
double best = -1.0;
|
||||||
|
for (int dir = 0; dir < 2; dir++) {
|
||||||
|
StoreEdge* edges = NULL; size_t ne = 0;
|
||||||
|
int rc = dir == 0 ? store_get_edges_from(s, claim_id, &edges, &ne)
|
||||||
|
: store_get_edges_to (s, claim_id, &edges, &ne);
|
||||||
|
if (rc < 0) continue;
|
||||||
|
for (size_t i = 0; i < ne; i++) {
|
||||||
|
if (edges[i].tombstoned) continue;
|
||||||
|
CogGrounding g;
|
||||||
|
if (cog_grounding_parse(&edges[i], now_ms, &g) != 0) continue;
|
||||||
|
out->n_edges++;
|
||||||
|
out->found = 1;
|
||||||
|
if (g.factual_now > best) {
|
||||||
|
best = g.factual_now;
|
||||||
|
out->factual = g.factual_now;
|
||||||
|
out->relational = g.relational_now; /* the SAME edge, not a max */
|
||||||
|
out->polarity = g.polarity;
|
||||||
|
out->cos_angle = g.cos_angle;
|
||||||
|
out->agreement = g.agreement;
|
||||||
|
out->prov = g.prov;
|
||||||
|
out->relational_established = g.present;
|
||||||
|
snprintf(out->best_edge, sizeof out->best_edge, "%s", edges[i].id ? edges[i].id : "");
|
||||||
|
snprintf(out->binding_value, sizeof out->binding_value, "%s", g.binding_value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
store_edges_free(edges, ne);
|
||||||
|
}
|
||||||
|
/* BOTH floors, and an unestablished relational axis does NOT pass by default
|
||||||
|
* — defaulting it to passing is the exemption §0 forbids. A negative polarity
|
||||||
|
* is a relation that actively contradicts and can never license assertion. */
|
||||||
|
out->may_assert = (out->found && out->relational_established &&
|
||||||
|
out->polarity > 0 &&
|
||||||
|
out->factual >= floor && out->relational >= rel_floor) ? 1 : 0;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/* ═══════════════════════════════════════════════ THE CORRESPONDENCE-LOOP ═════ */
|
/* ═══════════════════════════════════════════════ THE CORRESPONDENCE-LOOP ═════ */
|
||||||
int engram_correspondence_beat(const GeoDescriptor* region, const float* anchor,
|
int engram_correspondence_beat(const GeoDescriptor* region, const float* anchor,
|
||||||
double outcome_y, CogStance* stance,
|
double outcome_y, CogStance* stance,
|
||||||
int learn, double max_step, CogBeatResult* out) {
|
int learn, double max_step, CogBeatResult* out) {
|
||||||
if (!region || !stance || !out) return -1;
|
if (!region || !stance || !out) return -1;
|
||||||
memset(out, 0, sizeof *out);
|
memset(out, 0, sizeof *out);
|
||||||
if (stance->keystone) { learn = 0; out->wrote_keystone = 1; } /* §6: never write a keystone */
|
/* 2026-08-16: the keystone block is GONE. It refused to learn about the
|
||||||
|
* reference frame, which does not make it a good reference — it makes it
|
||||||
|
* unexaminable, trading circular calibration for an ungroundable one (spec
|
||||||
|
* §2). Measured cost of the block: on the keystone region the beat reported
|
||||||
|
* 0.00% brier reduction over n_trials 0 — it never ran, so nothing about the
|
||||||
|
* self was ever calibrated OR falsifiable. What replaces it is a provenance
|
||||||
|
* constraint, not a permission: cog_grounding_downstream refuses evidence
|
||||||
|
* that is downstream of the region being calibrated, for every region alike.
|
||||||
|
* `wrote_keystone` is retained as a reporting field only and is always 0. */
|
||||||
|
|
||||||
GeoGradient g;
|
GeoGradient g;
|
||||||
if (engram_think(region, anchor, stance, &g) != 0) return -1; /* PREDICTION */
|
if (engram_think(region, anchor, stance, &g) != 0) return -1; /* PREDICTION */
|
||||||
|
|||||||
+269
-17
@@ -23,7 +23,7 @@
|
|||||||
* and a region, and grounded-for-whom.
|
* and a region, and grounded-for-whom.
|
||||||
*
|
*
|
||||||
* PURE + (mostly) READ-ONLY, stdlib + libm only. think() and the warp are pure
|
* PURE + (mostly) READ-ONLY, stdlib + libm only. think() and the warp are pure
|
||||||
* over their inputs. Persistence (Stance <-> StoreNode, grounded-by edges) is the
|
* over their inputs. Persistence (Stance <-> StoreNode, edge grounding vectors) is the
|
||||||
* only part that touches the store, and it is additive / supersede / tombstone —
|
* only part that touches the store, and it is additive / supersede / tombstone —
|
||||||
* never mutate-in-place, never delete. It NEVER touches the live daemon: all
|
* never mutate-in-place, never delete. It NEVER touches the live daemon: all
|
||||||
* offline against a scratch store, per the design's rails.
|
* offline against a scratch store, per the design's rails.
|
||||||
@@ -152,30 +152,25 @@ int engram_express(const GeoGradient* g, const float* anchor, float* out_point);
|
|||||||
|
|
||||||
/* ═══════════════════════════════════════════════════════════════════════════
|
/* ═══════════════════════════════════════════════════════════════════════════
|
||||||
* §5 HOLD vs GROUND vs ASSERT. Holding is unconditional (the store gates nothing).
|
* §5 HOLD vs GROUND vs ASSERT. Holding is unconditional (the store gates nothing).
|
||||||
* Grounding is a RELATION — a "grounded-by" edge, probabilistic, grounded-for-whom.
|
* Grounding is an ATTRIBUTE OF a relation — carried on the edge itself, as a
|
||||||
* The honesty floor is checked only at ASSERTION.
|
* vector (§7). The honesty floor is checked only at ASSERTION, on both axes.
|
||||||
* ═══════════════════════════════════════════════════════════════════════════ */
|
* ═══════════════════════════════════════════════════════════════════════════ */
|
||||||
#define COG_GROUNDED_BY_RELATION "grounded-by"
|
/* DELETED 2026-08-16: COG_GROUNDED_BY_RELATION and cog_ground_edge.
|
||||||
|
*
|
||||||
|
* A "grounded-by" edge models grounding as a relation BETWEEN two nodes. It is a
|
||||||
|
* property OF a relation — and it is that relation's weight. Minting a new edge
|
||||||
|
* to carry a score was the error; #147 corrected which endpoints the edge landed
|
||||||
|
* on and left the wrong idea standing. There is nothing to ground a claim
|
||||||
|
* "against" that is not already an edge, and if no edge exists the honest answer
|
||||||
|
* is that the two are not related — not a freshly minted one scoring 0.98.
|
||||||
|
* See §7 for what replaced it. */
|
||||||
#define COG_SALIENT_TO_RELATION "salient-to"
|
#define COG_SALIENT_TO_RELATION "salient-to"
|
||||||
|
|
||||||
/* Write a grounded-by edge (additive). weight = grounding ∈(0,1] from the verifier;
|
|
||||||
* for_whom recorded in edge metadata (grounding is relational). Never a node flag. */
|
|
||||||
int cog_ground_edge(EngramPagedStore* s, const char* claim_id,
|
|
||||||
const char* evidence_id, double grounding, const char* for_whom);
|
|
||||||
|
|
||||||
/* Write/refresh a salient-to edge: salience is RELATIONAL (grounded-for-whom),
|
/* Write/refresh a salient-to edge: salience is RELATIONAL (grounded-for-whom),
|
||||||
* carried on the edge to the observer — not baked into the node scalar (§2.1). */
|
* carried on the edge to the observer — not baked into the node scalar (§2.1). */
|
||||||
int cog_salient_edge(EngramPagedStore* s, const char* node_id,
|
int cog_salient_edge(EngramPagedStore* s, const char* node_id,
|
||||||
const char* observer_id, double salience);
|
const char* observer_id, double salience);
|
||||||
|
|
||||||
/* The honesty floor — a QUERY at assertion time, NOT a schema constraint. Reads the
|
|
||||||
* claim's stored grounded-by edges (for the given observer) and returns:
|
|
||||||
* 1 = may assert (best grounding >= floor),
|
|
||||||
* 0 = REFUSE assertion (holds unconditionally; only asserting is gated),
|
|
||||||
* <0 = error. The content remains held either way. */
|
|
||||||
int cog_assert_gate(EngramPagedStore* s, const char* claim_id,
|
|
||||||
const char* for_whom, double floor);
|
|
||||||
|
|
||||||
/* ═══════════════════════════════════════════════════════════════════════════
|
/* ═══════════════════════════════════════════════════════════════════════════
|
||||||
* §4 THE REFLEXIVE CORRESPONDENCE-LOOP — the learning engine. think scores its
|
* §4 THE REFLEXIVE CORRESPONDENCE-LOOP — the learning engine. think scores its
|
||||||
* OWN gradient against outcome, refines the stance on the error, and (optionally)
|
* OWN gradient against outcome, refines the stance on the error, and (optionally)
|
||||||
@@ -209,8 +204,265 @@ int engram_correspondence_beat(const GeoDescriptor* region, const float* anchor,
|
|||||||
/* ═══════════════════════════════════════════════════════════════════════════
|
/* ═══════════════════════════════════════════════════════════════════════════
|
||||||
* §6 METASTABILITY. Keystones (self/values) are read-mostly: the loop reads but
|
* §6 METASTABILITY. Keystones (self/values) are read-mostly: the loop reads but
|
||||||
* never writes them. Mark by stance flag or by a keystone-id set the loop consults.
|
* never writes them. Mark by stance flag or by a keystone-id set the loop consults.
|
||||||
|
*
|
||||||
|
* SUPERSEDED BY §7's PROVENANCE CONSTRAINT (2026-08-16). The keystone flag is a
|
||||||
|
* PERMISSION: it asks who the target is, not where the evidence came from. That
|
||||||
|
* is censorship, and it costs the ability to ever ground the self (spec
|
||||||
|
* correspondence-and-censorship.md §0/§2). The constraint that actually protects
|
||||||
|
* a reference frame is cog_grounding_downstream: a region may not be calibrated
|
||||||
|
* by evidence downstream of itself. These declarations remain only so existing
|
||||||
|
* call sites keep compiling; nothing in the grounding path consults them.
|
||||||
* ═══════════════════════════════════════════════════════════════════════════ */
|
* ═══════════════════════════════════════════════════════════════════════════ */
|
||||||
typedef struct { const char** ids; int n; } CogKeystoneSet;
|
typedef struct { const char** ids; int n; } CogKeystoneSet;
|
||||||
int cog_is_keystone(const CogKeystoneSet* ks, const CogStance* s);
|
int cog_is_keystone(const CogKeystoneSet* ks, const CogStance* s);
|
||||||
|
|
||||||
|
/* ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
* §7 GROUNDING IS THE EDGE'S WEIGHT, AND THE WEIGHT IS A VECTOR
|
||||||
|
* (2026-08-16; spec correspondence-and-censorship.md §2–§6 @ 2b7e4ba.)
|
||||||
|
*
|
||||||
|
* THE MODEL. Grounding is not a subsystem, a score, or a relation BETWEEN nodes.
|
||||||
|
* It is an attribute OF a relation. The graph already IS the grounding structure:
|
||||||
|
* every edge is a grounded relation, and what that relation is worth is carried
|
||||||
|
* on the edge itself. Three things follow, and each DELETES rather than adds:
|
||||||
|
*
|
||||||
|
* 1. `grounded-by` as a relation type does not exist, and cog_ground_edge is
|
||||||
|
* gone. Minting an edge to hold a score models grounding as a relation
|
||||||
|
* between nodes when it is a property of a relation. #147 corrected which
|
||||||
|
* endpoints that edge landed on and left the wrong idea standing.
|
||||||
|
* 2. There is no observer, and no sampling rate. Change is not a consequence of
|
||||||
|
* use — it IS use, the way potentiation is the firing rather than something
|
||||||
|
* that reads the firing and writes a weight. So no supervisor compares a
|
||||||
|
* value to a threshold and decides to persist.
|
||||||
|
* 3. Between two recorded versions the trajectory is not unknown. Decay is a
|
||||||
|
* pure function of the last recorded point and elapsed time, so it is
|
||||||
|
* ANALYTIC: store the point, read the curve.
|
||||||
|
*
|
||||||
|
* WHAT IS *NOT* HERE, DELIBERATELY. An earlier draft of the spec posed "a graph
|
||||||
|
* predicate for evidence downstream of itself" as the hard problem, and this file
|
||||||
|
* briefly contained one. It is withdrawn. Non-circularity is TEMPORAL, not
|
||||||
|
* topological: you cannot recalibrate the ruler while measuring with it, so you
|
||||||
|
* do it when you are not using the frame to act. Reachability could never have
|
||||||
|
* worked — measured on the live store, reachability from the self region over
|
||||||
|
* all relations reaches 89.2% of the graph (10,580 of 11,861 nodes) and 16.0%
|
||||||
|
* over hebbian/semantic relations alone, so the predicate marks essentially all
|
||||||
|
* evidence tainted and the constraint degenerates into the total block that
|
||||||
|
* censorship started as. Nothing replaces it here; the independence is a fact
|
||||||
|
* about engagement, owned by the dreamer, not a fact about the graph.
|
||||||
|
*
|
||||||
|
* ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
* §7.1 THE VECTOR
|
||||||
|
*
|
||||||
|
* The test for a real dimension is whether it can move independently of the
|
||||||
|
* others. Five can, and each maps onto substrate that already exists:
|
||||||
|
*
|
||||||
|
* factual correspondence with evidence. [GRD1]
|
||||||
|
* relational correspondence with values — min over THIRTEEN
|
||||||
|
* value regions, carrying the binding value's NAME. [GRD1]
|
||||||
|
* associative co-activation frequency. This is the edge's `hebb`
|
||||||
|
* field with its existing dynamics — NOT a new one.
|
||||||
|
* Independent by construction: every superstition is
|
||||||
|
* a strong association with no factual grounding.
|
||||||
|
* polarity SIGNED. Near zero means "no support"; NEGATIVE means
|
||||||
|
* "this actively contradicts". The edge's `inhibitory`
|
||||||
|
* bit is exactly this distinction crushed to one bit,
|
||||||
|
* and is carried forward as the seed value. [GRD1]
|
||||||
|
* provenance observed / inferred / told / imprinted. Categorical,
|
||||||
|
* and load-bearing: it governs what the relation is
|
||||||
|
* entitled to. [GRD1]
|
||||||
|
*
|
||||||
|
* Plus a TIMESTAMP, which is what turns the supersession chain into a time
|
||||||
|
* series of vectors rather than a series of numbers.
|
||||||
|
*
|
||||||
|
* DERIVED, THEREFORE NEVER STORED. Confidence (high grounding AND low
|
||||||
|
* volatility), recency (decay read off the curve), staleness (grounding fallen
|
||||||
|
* below its floor), volatility (the derivative of a series nothing destroyed).
|
||||||
|
* Storing confidence separately is how `confidence: 0.5` ends up sitting beside
|
||||||
|
* a zero direction vector, asserting something nothing computed. Every field in
|
||||||
|
* CogGrounding below is marked STORED or DERIVED, and the serializer writes
|
||||||
|
* only the STORED ones.
|
||||||
|
*
|
||||||
|
* THE VALUES REFERENCE IS THIRTEEN REGIONS AND THE AGGREGATE IS MIN.
|
||||||
|
* Measured on the live store: the values root kn-5b606390 `contains` exactly 13
|
||||||
|
* value nodes; pairwise centroid cosine among their regions is min 0.1525,
|
||||||
|
* mean 0.5199, median 0.5282, max 0.9278 — they demonstrably do not form one
|
||||||
|
* region. Against a single union region the individual values sit at cosine
|
||||||
|
* 0.38..0.89, with constraints-as-freedom at 0.3812 and change-is-the-signal at
|
||||||
|
* 0.4677, so a union centroid under-represents precisely the values a claim is
|
||||||
|
* most likely to be measured against. MIN rather than MEAN because a mean lets
|
||||||
|
* strong agreement with twelve values mask a violation of the thirteenth, which
|
||||||
|
* is the mechanism of rationalization; min yields a binding constraint with a
|
||||||
|
* NAME attached rather than a score.
|
||||||
|
*
|
||||||
|
* TRAVERSAL CONDUCTS ON FACTUAL; ASSERTION REQUIRES BOTH. If activation
|
||||||
|
* conducted on relational weight, Neuron could not follow a chain of reasoning
|
||||||
|
* to a conclusion he then rejects — censorship arriving through the spreading
|
||||||
|
* rule. The gap between reachable and assertable is where the wide
|
||||||
|
* factual/relational angles live, and that gap is the interesting part.
|
||||||
|
* ═══════════════════════════════════════════════════════════════════════════ */
|
||||||
|
|
||||||
|
/* ── The one decay model (moved here from el_runtime.c so that node decay and
|
||||||
|
* edge-grounding decay are a single implementation with a single set of
|
||||||
|
* constants, rather than a model and a parallel copy of it). Half-life scales
|
||||||
|
* with how established the thing is: T_eff = T_HALF · (1 + ln(1 + reinforcements)).
|
||||||
|
* The floor is a preference, not a cliff — max penalty for age alone is 4x.
|
||||||
|
* `lambda_override` > 0 replaces the default rate; 0 means use the default. */
|
||||||
|
#define COG_T_HALF_HOURS 168.0
|
||||||
|
#define COG_DECAY_LAMBDA 0.693147
|
||||||
|
#define COG_DECAY_FLOOR 0.25
|
||||||
|
double cog_decay_factor(int64_t age_ms, double reinforcements, double lambda_override);
|
||||||
|
|
||||||
|
/* The compact vector block carried in the edge's own metadata. Line schema, same
|
||||||
|
* precedent as STNC1 / GEO1. Metadata the edge already carried is preserved
|
||||||
|
* verbatim ahead of the magic line. */
|
||||||
|
#define COG_GROUNDING_META_MAGIC "GRD1"
|
||||||
|
|
||||||
|
/* Provenance class — categorical, and it governs what the relation is entitled
|
||||||
|
* to. A change of class is inherently significant and needs no threshold,
|
||||||
|
* because told → observed is a categorical upgrade, not a drift. */
|
||||||
|
typedef enum {
|
||||||
|
COG_PROV_UNSET = 0,
|
||||||
|
COG_PROV_OBSERVED = 1,
|
||||||
|
COG_PROV_INFERRED = 2,
|
||||||
|
COG_PROV_TOLD = 3,
|
||||||
|
COG_PROV_IMPRINTED = 4
|
||||||
|
} CogProvClass;
|
||||||
|
const char* cog_prov_name(CogProvClass p);
|
||||||
|
CogProvClass cog_prov_parse(const char* s);
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int present; /* 1 iff the edge carries a GRD1 block */
|
||||||
|
|
||||||
|
/* ── STORED: the vector, as it stood at `ts` ─────────────────────────────── */
|
||||||
|
double factual; /* correspondence with evidence */
|
||||||
|
double relational; /* min over the thirteen value regions */
|
||||||
|
double associative; /* co-activation frequency — mirrors edge->hebb */
|
||||||
|
double polarity; /* SIGNED support; <0 = actively contradicts */
|
||||||
|
CogProvClass prov; /* observed / inferred / told / imprinted */
|
||||||
|
int64_t ts; /* when this version was recorded (ms) */
|
||||||
|
int64_t seq; /* supersession sequence number */
|
||||||
|
double reinforcements; /* uses folded into this version */
|
||||||
|
char binding_value[128]; /* the argmin value — the conflict's NAME */
|
||||||
|
/* the two gradients as frame-independent signed projections, plus the angle
|
||||||
|
* between them in full R^dim. These are part of the JOINT STATE a decision
|
||||||
|
* saw, not a convenience: near +1 evidence and values push the same way; at
|
||||||
|
* or below 0 the relation is factually supported and relationally wrong. */
|
||||||
|
double fac_proj, rel_proj, cos_angle;
|
||||||
|
int agreement; /* sign(cos_angle): +1 / 0 / −1 */
|
||||||
|
double floor_at_record, rel_floor_at_record;
|
||||||
|
char prev_edge[192]; /* the version this superseded ("" if first) */
|
||||||
|
|
||||||
|
/* ── DERIVED at read time. NEVER serialized. ─────────────────────────────── */
|
||||||
|
int64_t age_ms; /* recency: now − ts */
|
||||||
|
double decay; /* cog_decay_factor over that age */
|
||||||
|
double factual_now; /* factual · decay */
|
||||||
|
double relational_now;
|
||||||
|
double associative_now;
|
||||||
|
int stale; /* grounding fallen below its floor */
|
||||||
|
} CogGrounding;
|
||||||
|
|
||||||
|
/* Read an edge's vector as of `now_ms`. Pure — never writes. An edge with no
|
||||||
|
* GRD1 block still has an associative strength (its accrued hebb) and a polarity
|
||||||
|
* (its signed authored weight); `present` says whether the grounding dimensions
|
||||||
|
* have ever been established, and an unestablished dimension is reported as such
|
||||||
|
* rather than defaulted to a passing value. */
|
||||||
|
int cog_grounding_parse(const StoreEdge* e, int64_t now_ms, CogGrounding* out);
|
||||||
|
|
||||||
|
/* Serialize the STORED half of the vector, preserving pre-existing non-GRD1
|
||||||
|
* metadata. Returns an owned string. Derived fields are not written. */
|
||||||
|
char* cog_grounding_metadata(const char* base_meta, const CogGrounding* g);
|
||||||
|
|
||||||
|
/* ── §7.2 CONSOLIDATION-GATED SUPERSESSION ──────────────────────────────────
|
||||||
|
*
|
||||||
|
* Supersession is not recording — it is CONSOLIDATION, gated by salience, which
|
||||||
|
* is why you remember the argument and not the commute. Significance is
|
||||||
|
* evaluated PER-DIMENSION but the record is the WHOLE VECTOR: any dimension
|
||||||
|
* moving enough to matter triggers a supersession, and the new version captures
|
||||||
|
* every dimension as it stood at that instant. Versioning axes independently
|
||||||
|
* would make the joint state unreconstructable, and the joint state is the point
|
||||||
|
* — it is what makes "stayed true, became wrong" visible as an event (factual
|
||||||
|
* holding steady across versions while relational degrades).
|
||||||
|
*
|
||||||
|
* There is deliberately no epsilon in this enum or in the function that computes
|
||||||
|
* it. Every test is a floor crossing or a sign change, both exact. Two of them
|
||||||
|
* are INHERENTLY significant because they are discrete state changes rather than
|
||||||
|
* drift, and those bypass the salience gate entirely. */
|
||||||
|
typedef enum {
|
||||||
|
COG_SIG_NONE = 0, /* nothing decision-relevant moved — DO NOT RECORD */
|
||||||
|
COG_SIG_FIRST_RECORD = 1, /* no prior version exists */
|
||||||
|
COG_SIG_POLARITY_FLIP = 2, /* INHERENT: support ↔ contradiction, or ignorance
|
||||||
|
* ↔ either. A discrete change of state. */
|
||||||
|
COG_SIG_PROVENANCE_CHANGE = 3, /* INHERENT: told → observed is a categorical
|
||||||
|
* upgrade in what the relation is entitled to. */
|
||||||
|
COG_SIG_FACTUAL_FLOOR = 4, /* crossed the assert floor, factual axis */
|
||||||
|
COG_SIG_RELATIONAL_FLOOR = 5, /* crossed the assert floor, relational axis */
|
||||||
|
COG_SIG_AGREEMENT_FLIP = 6, /* factual/relational agreement changed sign */
|
||||||
|
COG_SIG_DIRECTION_REVERSAL = 7 /* a gradient reversed direction */
|
||||||
|
} CogSignificance;
|
||||||
|
|
||||||
|
CogSignificance cog_grounding_significant(const CogGrounding* prev,
|
||||||
|
const CogGrounding* now,
|
||||||
|
double floor, double rel_floor);
|
||||||
|
const char* cog_significance_name(CogSignificance s);
|
||||||
|
/* 1 iff this reason is a discrete state change that consolidates regardless of
|
||||||
|
* salience (polarity flip, provenance change, first record). */
|
||||||
|
int cog_significance_inherent(CogSignificance s);
|
||||||
|
|
||||||
|
/* ── §7.3 RECORDING: supersession of the EDGE, never an overwrite ────────────
|
||||||
|
* Writes version seq+1 as a NEW edge record with the same endpoints and relation
|
||||||
|
* and id "<root>#<seq+1>", carrying a GRD1 `p` pointer to its predecessor. The
|
||||||
|
* predecessor is never touched. The chain IS the trajectory: not only what the
|
||||||
|
* grounding is but which way it has been moving and how fast — a derivative
|
||||||
|
* obtained for free from immutability, because the points were never destroyed.
|
||||||
|
* Returns the version written (>=1), or <0 on error. */
|
||||||
|
int cog_grounding_record(EngramPagedStore* s, const StoreEdge* base,
|
||||||
|
const CogGrounding* g, char* out_id, size_t out_id_cap);
|
||||||
|
|
||||||
|
/* Walk forward from a base edge id to its newest recorded version. Point reads
|
||||||
|
* only; consolidation is gated, so the chain is short. Returns the highest
|
||||||
|
* version found (0 = the base record is the only one). */
|
||||||
|
int cog_grounding_head(EngramPagedStore* s, const char* base_id,
|
||||||
|
StoreEdge* out, int max_versions);
|
||||||
|
|
||||||
|
/* VOLATILITY — derived, never stored: the mean absolute per-version change of a
|
||||||
|
* dimension across the recorded chain. Feeds the equally-derived `confidence`
|
||||||
|
* (high grounding AND low volatility), which is likewise never stored. */
|
||||||
|
typedef struct {
|
||||||
|
int n_versions;
|
||||||
|
double factual_volatility;
|
||||||
|
double relational_volatility;
|
||||||
|
double factual_drift; /* signed: newest − oldest */
|
||||||
|
double relational_drift;
|
||||||
|
int stayed_true_became_wrong; /* factual steady while relational degraded */
|
||||||
|
} CogTrajectory;
|
||||||
|
int cog_grounding_trajectory(EngramPagedStore* s, const char* base_id,
|
||||||
|
int64_t now_ms, CogTrajectory* out);
|
||||||
|
|
||||||
|
/* ── §7.4 ASSERTION GATES ON BOTH FLOORS ────────────────────────────────────
|
||||||
|
* A well-evidenced claim must not earn the right to be asserted regardless of
|
||||||
|
* whether it means the right thing. `may_assert` requires the decayed factual
|
||||||
|
* grounding to clear `floor` AND the decayed relational grounding to clear
|
||||||
|
* `rel_floor`. A relation whose relational axis has never been established does
|
||||||
|
* not pass by default — it is reported unestablished and refused, because
|
||||||
|
* defaulting it to passing is exactly the exemption §0 forbids. Traversal is
|
||||||
|
* untouched: activation still conducts on the factual/associative side, so a
|
||||||
|
* relation can remain thinkable while ceasing to be assertable. */
|
||||||
|
typedef struct {
|
||||||
|
int may_assert;
|
||||||
|
int found; /* any relation at all on this claim */
|
||||||
|
int relational_established;
|
||||||
|
int still_held; /* DERIVED: node present and not tombstoned */
|
||||||
|
double factual; /* best decayed factual grounding */
|
||||||
|
double relational; /* the SAME edge's relational axis, not a max */
|
||||||
|
double polarity;
|
||||||
|
double cos_angle;
|
||||||
|
int agreement;
|
||||||
|
CogProvClass prov;
|
||||||
|
char best_edge[192];
|
||||||
|
char binding_value[128];
|
||||||
|
int n_edges;
|
||||||
|
} CogAssertion;
|
||||||
|
int cog_assert_two_axis(EngramPagedStore* s, const char* claim_id,
|
||||||
|
double floor, double rel_floor, int64_t now_ms,
|
||||||
|
CogAssertion* out);
|
||||||
|
|
||||||
#endif /* ENGRAM_COGNITION_H */
|
#endif /* ENGRAM_COGNITION_H */
|
||||||
|
|||||||
@@ -222,7 +222,7 @@ static double eff_w(double weight, double hebb){
|
|||||||
}
|
}
|
||||||
|
|
||||||
GeoDescriptor* engram_geometry_descriptor(
|
GeoDescriptor* engram_geometry_descriptor(
|
||||||
EngramPagedStore* store, VIndex* vindex,
|
EngramPagedStore* store, const VIndex* vindex,
|
||||||
char** vids, int n_vids,
|
char** vids, int n_vids,
|
||||||
const char* const* seed_ids, size_t n_seeds,
|
const char* const* seed_ids, size_t n_seeds,
|
||||||
const GeoParams* params,
|
const GeoParams* params,
|
||||||
@@ -1401,7 +1401,7 @@ static double geo_weighted_degree(EngramPagedStore* st, const char* id, double e
|
|||||||
return deg;
|
return deg;
|
||||||
}
|
}
|
||||||
|
|
||||||
int engram_geo_reify_store(EngramPagedStore* store, VIndex* vindex,
|
int engram_geo_reify_store(EngramPagedStore* store, const VIndex* vindex,
|
||||||
char** vids, int n_vids,
|
char** vids, int n_vids,
|
||||||
const GeoReifyParams* params){
|
const GeoReifyParams* params){
|
||||||
if(!store) return -1;
|
if(!store) return -1;
|
||||||
|
|||||||
@@ -150,7 +150,7 @@ void engram_geo_mean_free(GeoMeanCache* c);
|
|||||||
* Returns a malloc'd descriptor (free with engram_geo_free), or NULL on error
|
* Returns a malloc'd descriptor (free with engram_geo_free), or NULL on error
|
||||||
* (no seeds resolvable, OOM). */
|
* (no seeds resolvable, OOM). */
|
||||||
GeoDescriptor* engram_geometry_descriptor(
|
GeoDescriptor* engram_geometry_descriptor(
|
||||||
EngramPagedStore* store, VIndex* vindex,
|
EngramPagedStore* store, const VIndex* vindex,
|
||||||
char** vids, int n_vids,
|
char** vids, int n_vids,
|
||||||
const char* const* seed_ids, size_t n_seeds,
|
const char* const* seed_ids, size_t n_seeds,
|
||||||
const GeoParams* params,
|
const GeoParams* params,
|
||||||
@@ -375,7 +375,7 @@ void engram_geo_reify_default_params(GeoReifyParams* p);
|
|||||||
* neighborhood (+ member edges), superseding any prior same-hub record with
|
* neighborhood (+ member edges), superseding any prior same-hub record with
|
||||||
* provenance. Read-then-write over `store`. Returns #neighborhoods persisted, or <0.
|
* provenance. Read-then-write over `store`. Returns #neighborhoods persisted, or <0.
|
||||||
* Skips existing Neighborhood/GeoMeanFrame nodes when detecting (idempotent re-reify). */
|
* Skips existing Neighborhood/GeoMeanFrame nodes when detecting (idempotent re-reify). */
|
||||||
int engram_geo_reify_store(EngramPagedStore* store, VIndex* vindex,
|
int engram_geo_reify_store(EngramPagedStore* store, const VIndex* vindex,
|
||||||
char** vids, int n_vids,
|
char** vids, int n_vids,
|
||||||
const GeoReifyParams* params);
|
const GeoReifyParams* params);
|
||||||
|
|
||||||
|
|||||||
@@ -74,11 +74,6 @@ struct VIndex {
|
|||||||
|
|
||||||
int entry; /* entry-point element index, -1 if empty */
|
int entry; /* entry-point element index, -1 if empty */
|
||||||
int max_level; /* current top layer */
|
int max_level; /* current top layer */
|
||||||
|
|
||||||
/* scratch: version-stamped visited set (O(1) reset). */
|
|
||||||
uint32_t* visited;
|
|
||||||
uint32_t visit_epoch;
|
|
||||||
size_t visited_cap;
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/* ── small helpers ────────────────────────────────────────────────────────── */
|
/* ── small helpers ────────────────────────────────────────────────────────── */
|
||||||
@@ -166,37 +161,63 @@ static Pair heap_pop(Heap* h, int is_max){
|
|||||||
return top;
|
return top;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ── visited set ──────────────────────────────────────────────────────────── */
|
/* ── visited set — owned by the CALL FRAME, never by the index ──────────────
|
||||||
static int visited_ensure(VIndex* ix){
|
* This buffer is per-TRAVERSAL scratch. It used to live in struct VIndex as an
|
||||||
if (ix->visited_cap >= ix->cap && ix->visited) return 0;
|
* allocation optimisation, which made every traversal a write to shared state:
|
||||||
size_t nc = ix->cap ? ix->cap : 16;
|
* two concurrent vindex_search calls stamped each other's epoch and then walked
|
||||||
uint32_t* nv = (uint32_t*)realloc(ix->visited, nc*sizeof(uint32_t));
|
* each other's marks, so even two pure READS corrupted the traversal (measured
|
||||||
if (!nv) return -1;
|
* 2026-08-16: TSan data race at visited_reset, reached from vindex_search on one
|
||||||
if (nc > ix->visited_cap) memset(nv + ix->visited_cap, 0, (nc-ix->visited_cap)*sizeof(uint32_t));
|
* thread and vindex_insert on another; downstream SIGSEGV dereferencing a bogus
|
||||||
ix->visited = nv; ix->visited_cap = nc;
|
* element index).
|
||||||
|
*
|
||||||
|
* It is not an ownership problem and it does not want a lock or a capability —
|
||||||
|
* it was simply misfiled. A pure function's scratch belongs to the call. Moving
|
||||||
|
* it here is what lets vindex_search take a `const VIndex*`, which is in turn
|
||||||
|
* what makes "search does not mutate the index" a COMPILE-TIME property instead
|
||||||
|
* of a review comment.
|
||||||
|
*
|
||||||
|
* Cost: one calloc/free of cap*4 bytes per traversal (~55 KB at the live store's
|
||||||
|
* 13,820 elements), against thousands of dim-768 dot products in the same call.
|
||||||
|
* Deliberately NOT __thread: http_worker is a thread per connection, so a
|
||||||
|
* thread-local buffer would retain ~55 KB per connection for the process life. */
|
||||||
|
typedef struct {
|
||||||
|
uint32_t* mark; /* per-element epoch stamp */
|
||||||
|
uint32_t epoch; /* current traversal's stamp; 0 == "no traversal yet" */
|
||||||
|
size_t cap;
|
||||||
|
} VVisit;
|
||||||
|
|
||||||
|
/* calloc leaves every stamp 0 and epoch 0; the first visit_reset moves to
|
||||||
|
* epoch 1, so no element reads as visited before it is marked. */
|
||||||
|
static int visit_init(VVisit* v, size_t cap){
|
||||||
|
size_t nc = cap ? cap : 16;
|
||||||
|
v->mark = (uint32_t*)calloc(nc, sizeof(uint32_t));
|
||||||
|
if (!v->mark) return -1;
|
||||||
|
v->cap = nc; v->epoch = 0;
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
static inline void visited_reset(VIndex* ix){
|
static void visit_dispose(VVisit* v){ free(v->mark); v->mark = NULL; v->cap = 0; }
|
||||||
if (++ix->visit_epoch == 0){ /* wrapped: clear all */
|
static inline void visit_reset(VVisit* v){
|
||||||
memset(ix->visited, 0, ix->visited_cap*sizeof(uint32_t));
|
if (++v->epoch == 0){ /* wrapped: clear all */
|
||||||
ix->visit_epoch = 1;
|
memset(v->mark, 0, v->cap*sizeof(uint32_t));
|
||||||
|
v->epoch = 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
static inline int is_visited(VIndex* ix, int e){ return ix->visited[e]==ix->visit_epoch; }
|
static inline int is_visited(const VVisit* v, int e){ return v->mark[e]==v->epoch; }
|
||||||
static inline void mark_visited(VIndex* ix, int e){ ix->visited[e]=ix->visit_epoch; }
|
static inline void mark_visited(VVisit* v, int e){ v->mark[e]=v->epoch; }
|
||||||
|
|
||||||
/* ── search one layer (Algorithm 2): best-first, ef-bounded ───────────────── */
|
/* ── search one layer (Algorithm 2): best-first, ef-bounded ───────────────── */
|
||||||
/* Returns results as an unsorted Heap (max-heap on distance, size<=ef). Caller
|
/* Returns results as an unsorted Heap (max-heap on distance, size<=ef). Caller
|
||||||
* owns res->a. `q` is a normalised query. */
|
* owns res->a. `q` is a normalised query. */
|
||||||
static int search_layer(VIndex* ix, const float* q, const int* eps, int neps,
|
static int search_layer(const VIndex* ix, VVisit* vis, const float* q,
|
||||||
|
const int* eps, int neps,
|
||||||
int ef, int layer, Heap* res /*out, max-heap*/){
|
int ef, int layer, Heap* res /*out, max-heap*/){
|
||||||
Heap cand = {0,0,0}; /* min-heap: nearest to expand */
|
Heap cand = {0,0,0}; /* min-heap: nearest to expand */
|
||||||
res->a=NULL; res->n=0; res->cap=0;
|
res->a=NULL; res->n=0; res->cap=0;
|
||||||
visited_reset(ix);
|
visit_reset(vis);
|
||||||
for (int i=0;i<neps;i++){
|
for (int i=0;i<neps;i++){
|
||||||
int e = eps[i];
|
int e = eps[i];
|
||||||
if (is_visited(ix,e)) continue;
|
if (is_visited(vis,e)) continue;
|
||||||
mark_visited(ix,e);
|
mark_visited(vis,e);
|
||||||
float d = vdist(ix, q, ix->elems[e].vec);
|
float d = vdist(ix, q, ix->elems[e].vec);
|
||||||
Pair p = { d, e };
|
Pair p = { d, e };
|
||||||
if (heap_push(&cand,p,0) || heap_push(res,p,1)){ free(cand.a); return -1; }
|
if (heap_push(&cand,p,0) || heap_push(res,p,1)){ free(cand.a); return -1; }
|
||||||
@@ -212,8 +233,8 @@ static int search_layer(VIndex* ix, const float* q, const int* eps, int neps,
|
|||||||
NeighList* nl = &ce->links[layer];
|
NeighList* nl = &ce->links[layer];
|
||||||
for (int i=0;i<nl->count;i++){
|
for (int i=0;i<nl->count;i++){
|
||||||
int e = nl->ids[i];
|
int e = nl->ids[i];
|
||||||
if (is_visited(ix,e)) continue;
|
if (is_visited(vis,e)) continue;
|
||||||
mark_visited(ix,e);
|
mark_visited(vis,e);
|
||||||
float d = vdist(ix, q, ix->elems[e].vec);
|
float d = vdist(ix, q, ix->elems[e].vec);
|
||||||
if (res->n < ef || d < res->a[0].d){
|
if (res->n < ef || d < res->a[0].d){
|
||||||
Pair p = { d, e };
|
Pair p = { d, e };
|
||||||
@@ -232,7 +253,7 @@ static int search_layer(VIndex* ix, const float* q, const int* eps, int neps,
|
|||||||
* Keep c only if it is nearer to q than to every already-chosen neighbour;
|
* Keep c only if it is nearer to q than to every already-chosen neighbour;
|
||||||
* backfill from the pruned set (nearest first) to reach M for connectivity.
|
* backfill from the pruned set (nearest first) to reach M for connectivity.
|
||||||
* Writes chosen element indices into out[], returns the count. */
|
* Writes chosen element indices into out[], returns the count. */
|
||||||
static int select_neighbors(VIndex* ix, const float* q, Pair* W, int nW, int M, int* out){
|
static int select_neighbors(const VIndex* ix, const float* q, Pair* W, int nW, int M, int* out){
|
||||||
(void)q; /* q's distances are precomputed in W[].d; kept for call-site clarity */
|
(void)q; /* q's distances are precomputed in W[].d; kept for call-site clarity */
|
||||||
/* sort W ascending by (dist,elem) — deterministic. */
|
/* sort W ascending by (dist,elem) — deterministic. */
|
||||||
for (int i=1;i<nW;i++){ /* insertion sort (nW small) */
|
for (int i=1;i<nW;i++){ /* insertion sort (nW small) */
|
||||||
@@ -281,7 +302,7 @@ static int elems_reserve(VIndex* ix){
|
|||||||
Elem* ne = (Elem*)realloc(ix->elems, nc*sizeof(Elem));
|
Elem* ne = (Elem*)realloc(ix->elems, nc*sizeof(Elem));
|
||||||
if (!ne) return -1;
|
if (!ne) return -1;
|
||||||
ix->elems = ne; ix->cap = nc;
|
ix->elems = ne; ix->cap = nc;
|
||||||
return visited_ensure(ix);
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
int vindex_insert(VIndex* ix, uint64_t node_id, const float* vec){
|
int vindex_insert(VIndex* ix, uint64_t node_id, const float* vec){
|
||||||
@@ -307,13 +328,19 @@ int vindex_insert(VIndex* ix, uint64_t node_id, const float* vec){
|
|||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* This call frame owns its traversal scratch for the whole insert. ix->cap
|
||||||
|
* already covers `cur` (elems_reserve ran above), so every reachable element
|
||||||
|
* index is in range. */
|
||||||
|
VVisit vis;
|
||||||
|
if (visit_init(&vis, ix->cap)) return -1;
|
||||||
|
|
||||||
int ep = ix->entry;
|
int ep = ix->entry;
|
||||||
int L = ix->max_level;
|
int L = ix->max_level;
|
||||||
/* greedy descent through layers above `level` to refine the entry point. */
|
/* greedy descent through layers above `level` to refine the entry point. */
|
||||||
for (int lc = L; lc > level; lc--){
|
for (int lc = L; lc > level; lc--){
|
||||||
Heap r = {0,0,0};
|
Heap r = {0,0,0};
|
||||||
int eps1[1] = { ep };
|
int eps1[1] = { ep };
|
||||||
if (search_layer(ix, el->vec, eps1, 1, 1, lc, &r)){ return -1; }
|
if (search_layer(ix, &vis, el->vec, eps1, 1, 1, lc, &r)){ visit_dispose(&vis); return -1; }
|
||||||
if (r.n){ ep = r.a[0].e; float bd=r.a[0].d;
|
if (r.n){ ep = r.a[0].e; float bd=r.a[0].d;
|
||||||
for (int i=1;i<r.n;i++) if (r.a[i].d<bd){bd=r.a[i].d; ep=r.a[i].e;} }
|
for (int i=1;i<r.n;i++) if (r.a[i].d<bd){bd=r.a[i].d; ep=r.a[i].e;} }
|
||||||
free(r.a);
|
free(r.a);
|
||||||
@@ -329,7 +356,7 @@ int vindex_insert(VIndex* ix, uint64_t node_id, const float* vec){
|
|||||||
for (int lc = start; lc >= 0; lc--){
|
for (int lc = start; lc >= 0; lc--){
|
||||||
int Mmax = (lc==0) ? ix->M0 : ix->M;
|
int Mmax = (lc==0) ? ix->M0 : ix->M;
|
||||||
Heap W = {0,0,0};
|
Heap W = {0,0,0};
|
||||||
if (search_layer(ix, el->vec, eps, neps, ix->ef_construction, lc, &W)){ rc=-1; break; }
|
if (search_layer(ix, &vis, el->vec, eps, neps, ix->ef_construction, lc, &W)){ rc=-1; break; }
|
||||||
int* chosen = (int*)malloc((size_t)(W.n?W.n:1)*sizeof(int));
|
int* chosen = (int*)malloc((size_t)(W.n?W.n:1)*sizeof(int));
|
||||||
if (!chosen){ free(W.a); rc=-1; break; }
|
if (!chosen){ free(W.a); rc=-1; break; }
|
||||||
int nc = select_neighbors(ix, el->vec, W.a, W.n, Mmax, chosen);
|
int nc = select_neighbors(ix, el->vec, W.a, W.n, Mmax, chosen);
|
||||||
@@ -357,13 +384,17 @@ int vindex_insert(VIndex* ix, uint64_t node_id, const float* vec){
|
|||||||
}
|
}
|
||||||
done:
|
done:
|
||||||
free(eps_owned);
|
free(eps_owned);
|
||||||
|
visit_dispose(&vis);
|
||||||
if (rc) return -1;
|
if (rc) return -1;
|
||||||
if (level > ix->max_level){ ix->max_level = level; ix->entry = cur; }
|
if (level > ix->max_level){ ix->max_level = level; ix->entry = cur; }
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ── search ───────────────────────────────────────────────────────────────── */
|
/* ── search ───────────────────────────────────────────────────────────────── */
|
||||||
int vindex_search(VIndex* ix, const float* query, int k, int ef_search,
|
/* `ix` is const: search is pure with respect to the index. That is enforced by
|
||||||
|
* the compiler, not by convention — it is the whole point of moving the visited
|
||||||
|
* set into the frame below. */
|
||||||
|
int vindex_search(const VIndex* ix, const float* query, int k, int ef_search,
|
||||||
uint64_t* node_id_out, float* dist_out){
|
uint64_t* node_id_out, float* dist_out){
|
||||||
if (!ix || !query || k <= 0) return -1;
|
if (!ix || !query || k <= 0) return -1;
|
||||||
if (ix->entry < 0) return 0;
|
if (ix->entry < 0) return 0;
|
||||||
@@ -373,11 +404,15 @@ int vindex_search(VIndex* ix, const float* query, int k, int ef_search,
|
|||||||
float* q = vec_normalise_copy(query, ix->dim);
|
float* q = vec_normalise_copy(query, ix->dim);
|
||||||
if (!q) return -1;
|
if (!q) return -1;
|
||||||
|
|
||||||
|
/* This call frame owns its traversal scratch. */
|
||||||
|
VVisit vis;
|
||||||
|
if (visit_init(&vis, ix->cap)){ free(q); return -1; }
|
||||||
|
|
||||||
int ep = ix->entry;
|
int ep = ix->entry;
|
||||||
for (int lc = ix->max_level; lc > 0; lc--){
|
for (int lc = ix->max_level; lc > 0; lc--){
|
||||||
Heap r = {0,0,0};
|
Heap r = {0,0,0};
|
||||||
int eps[1] = { ep };
|
int eps[1] = { ep };
|
||||||
if (search_layer(ix, q, eps, 1, 1, lc, &r)){ free(q); return -1; }
|
if (search_layer(ix, &vis, q, eps, 1, 1, lc, &r)){ visit_dispose(&vis); free(q); return -1; }
|
||||||
if (r.n){ int b=r.a[0].e; float bd=r.a[0].d;
|
if (r.n){ int b=r.a[0].e; float bd=r.a[0].d;
|
||||||
for (int i=1;i<r.n;i++) if (r.a[i].d<bd){bd=r.a[i].d; b=r.a[i].e;}
|
for (int i=1;i<r.n;i++) if (r.a[i].d<bd){bd=r.a[i].d; b=r.a[i].e;}
|
||||||
ep = b; }
|
ep = b; }
|
||||||
@@ -385,7 +420,8 @@ int vindex_search(VIndex* ix, const float* query, int k, int ef_search,
|
|||||||
}
|
}
|
||||||
Heap res = {0,0,0};
|
Heap res = {0,0,0};
|
||||||
int eps[1] = { ep };
|
int eps[1] = { ep };
|
||||||
if (search_layer(ix, q, eps, 1, ef_search, 0, &res)){ free(res.a); free(q); return -1; }
|
if (search_layer(ix, &vis, q, eps, 1, ef_search, 0, &res)){ visit_dispose(&vis); free(res.a); free(q); return -1; }
|
||||||
|
visit_dispose(&vis);
|
||||||
free(q);
|
free(q);
|
||||||
|
|
||||||
/* res is a max-heap of size<=ef; pop into ascending order, keep nearest k. */
|
/* res is a max-heap of size<=ef; pop into ascending order, keep nearest k. */
|
||||||
@@ -419,7 +455,6 @@ VIndex* vindex_create(int dim, int M, int ef_construction){
|
|||||||
ix->mL = 1.0 / log((double)M > 1.0 ? (double)M : 2.0);
|
ix->mL = 1.0 / log((double)M > 1.0 ? (double)M : 2.0);
|
||||||
ix->entry = -1;
|
ix->entry = -1;
|
||||||
ix->max_level = 0;
|
ix->max_level = 0;
|
||||||
ix->visit_epoch = 0;
|
|
||||||
return ix;
|
return ix;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -432,7 +467,6 @@ void vindex_free(VIndex* ix){
|
|||||||
free(e->vec);
|
free(e->vec);
|
||||||
}
|
}
|
||||||
free(ix->elems);
|
free(ix->elems);
|
||||||
free(ix->visited);
|
|
||||||
free(ix);
|
free(ix);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -53,8 +53,15 @@ int vindex_insert(VIndex* idx, uint64_t node_id, const float* vec);
|
|||||||
* first (ascending distance). Either out array may be NULL to skip it.
|
* first (ascending distance). Either out array may be NULL to skip it.
|
||||||
* ef_search — search-time candidate width; larger == higher recall, slower.
|
* ef_search — search-time candidate width; larger == higher recall, slower.
|
||||||
* Pass <=0 for VINDEX_DEFAULT_EF_SEARCH. Internally clamped to >=k.
|
* Pass <=0 for VINDEX_DEFAULT_EF_SEARCH. Internally clamped to >=k.
|
||||||
* Returns the number of results written, or <0 on error. */
|
* Returns the number of results written, or <0 on error.
|
||||||
int vindex_search(VIndex* idx, const float* query, int k, int ef_search,
|
*
|
||||||
|
* `idx` is const BY CONTRACT AND BY TYPE: search does not mutate the index. The
|
||||||
|
* traversal's visited set is owned by the call frame, so N threads may search one
|
||||||
|
* index concurrently. Concurrent search against a vindex_insert on the same index
|
||||||
|
* is still unsafe — insert rewires existing elements' neighbour lists and reallocs
|
||||||
|
* elems[] — so the index's owner must not extend a published index under a live
|
||||||
|
* reader. See eg_vindex_view / eg_vindex_maintain in el_runtime.c. */
|
||||||
|
int vindex_search(const VIndex* idx, const float* query, int k, int ef_search,
|
||||||
uint64_t* node_id_out, float* dist_out);
|
uint64_t* node_id_out, float* dist_out);
|
||||||
|
|
||||||
/* Number of vectors currently indexed. */
|
/* Number of vectors currently indexed. */
|
||||||
|
|||||||
+169
-5
@@ -29,6 +29,8 @@ This section is the **single source of truth** for what works and what is planne
|
|||||||
- Lexer: keywords, identifiers, integer/float/string/bool literals, operators below.
|
- Lexer: keywords, identifiers, integer/float/string/bool literals, operators below.
|
||||||
- Parser: `let`, `return`, `fn`, `type`, `enum`, `import`, `from … import`, `while`, `for`, `if/else if/else`, `match`, `@decorator`, array/map literals, all listed operators, function calls, field access, index access, unary `!`/`-`, postfix `?`.
|
- Parser: `let`, `return`, `fn`, `type`, `enum`, `import`, `from … import`, `while`, `for`, `if/else if/else`, `match`, `@decorator`, array/map literals, all listed operators, function calls, field access, index access, unary `!`/`-`, postfix `?`.
|
||||||
- Codegen: function definitions, top-level `main()`, all expression forms above, control flow, decorator-as-AST-attachment.
|
- Codegen: function definitions, top-level `main()`, all expression forms above, control flow, decorator-as-AST-attachment.
|
||||||
|
- Boundary seam: decorator arguments and stacking; VBD role enforcement via `#error`; `engram_boundary_beat` auto-emit at `@manager`/`@accessor` entry; `@route` dispatch tables (Section 9).
|
||||||
|
- Program-level declarative blocks: `cgi`, `service`, and `program` — the last carrying process identity and configuration (Section 18).
|
||||||
- C runtime: I/O, string operations, integer math, lists, maps, filesystem, command-line args, basic `json_get` substring lookup.
|
- C runtime: I/O, string operations, integer math, lists, maps, filesystem, command-line args, basic `json_get` substring lookup.
|
||||||
|
|
||||||
### Planned (in flight)
|
### Planned (in flight)
|
||||||
@@ -37,7 +39,7 @@ This section is the **single source of truth** for what works and what is planne
|
|||||||
- **Match codegen.** Currently parsed; codegen does not emit. Adding `({ ... })` statement-expression emission.
|
- **Match codegen.** Currently parsed; codegen does not emit. Adding `({ ... })` statement-expression emission.
|
||||||
- **`?` propagation.** Currently no-op. Adding nil-propagation semantics.
|
- **`?` propagation.** Currently no-op. Adding nil-propagation semantics.
|
||||||
- **`cgi` block parsing.** Currently lexed (`cgi` is a keyword) but not parsed as a statement. Adding `parse_cgi_block` and codegen of `el_cgi_init` at the head of `main()`.
|
- **`cgi` block parsing.** Currently lexed (`cgi` is a keyword) but not parsed as a statement. Adding `parse_cgi_block` and codegen of `el_cgi_init` at the head of `main()`.
|
||||||
- **VBD role enforcement.** `@manager`/`@engine`/`@accessor` are accepted as decorators but not enforced. Adding compile-time check that `dharma_emit`/`dharma_field` only appear inside `@manager` functions.
|
- **Boundary epilogues.** The decorator seam injects a prologue only. Adding prologue/epilogue wrapping, the prerequisite for durability-as-an-effect (Section 19.1).
|
||||||
- **`vessel` keyword.** Replaces `package` in manifests. Adding to lexer.
|
- **`vessel` keyword.** Replaces `package` in manifests. Adding to lexer.
|
||||||
- **Real `engram_*` runtime.** Currently stub. Adding in-process graph store with spreading activation, Hebbian strengthening, and disk persistence — see Section 16.4.
|
- **Real `engram_*` runtime.** Currently stub. Adding in-process graph store with spreading activation, Hebbian strengthening, and disk persistence — see Section 16.4.
|
||||||
- **Real `dharma_*` runtime.** Currently stub. Adding network transport, channel registry, identity resolution.
|
- **Real `dharma_*` runtime.** Currently stub. Adding network transport, channel registry, identity resolution.
|
||||||
@@ -96,8 +98,10 @@ The following words are reserved and cannot be used as identifiers. Each row not
|
|||||||
| `while` | yes | Loop |
|
| `while` | yes | Loop |
|
||||||
| `import` / `from` / `as` | yes | Module import |
|
| `import` / `from` / `as` | yes | Module import |
|
||||||
| `true` / `false` | yes | Bool literals |
|
| `true` / `false` | yes | Bool literals |
|
||||||
| `cgi` | planned | Top-level CGI declaration block |
|
| `cgi` | yes | Top-level CGI declaration block |
|
||||||
| `manager` / `engine` / `accessor` | as decorators | VBD role marker on `fn` (enforcement planned) |
|
| `service` | yes | Top-level capability-bounded declaration block |
|
||||||
|
| `program` | yes | Top-level cross-cutting declaration block (Section 18) |
|
||||||
|
| `manager` / `engine` / `accessor` | as decorators | VBD role marker on `fn`; enforcement and boundary auto-emit are live (Section 9) |
|
||||||
| `vessel` | planned | Manifest declaration (replaces `package`) |
|
| `vessel` | planned | Manifest declaration (replaces `package`) |
|
||||||
| `activate` / `where` | planned | Spreading-activation construct |
|
| `activate` / `where` | planned | Spreading-activation construct |
|
||||||
| `sealed` | planned | Capability scope block |
|
| `sealed` | planned | Capability scope block |
|
||||||
@@ -446,9 +450,21 @@ Parsed. The module name is recorded; the brace-list is consumed. Both forms prod
|
|||||||
fn handle(channel: String, msg: String) -> Void { … }
|
fn handle(channel: String, msg: String) -> Void { … }
|
||||||
```
|
```
|
||||||
|
|
||||||
The `@` token followed by an identifier attaches a decorator name to the next `FnDef`. Decorators with structural meaning today: none. Planned enforcement (Section 16.2): VBD roles `@manager`, `@engine`, `@accessor`.
|
The `@` token followed by an identifier attaches a decorator to the next `FnDef`.
|
||||||
|
|
||||||
Non-VBD decorators are accepted and ignored.
|
**Decorators take arguments and they stack.** `@route("/p", "GET") @manager fn f()` attaches both to `f` as a `decorators` list of `{name, args}` records, topmost-first. Arguments are string literals only.
|
||||||
|
|
||||||
|
**Decorators have structural meaning today.** This is El's function-level boundary seam — the mechanism by which a cross-cutting concern is handled *at the boundary* rather than by a convention repeated at every call site:
|
||||||
|
|
||||||
|
| Decorator | Structural effect |
|
||||||
|
|---|---|
|
||||||
|
| `@manager` | Permits calls to `dharma_emit` / `dharma_field`. Calling either from a non-`@manager` fn emits a `#error` into the generated C — a compile-time failure, not a lint. |
|
||||||
|
| `@manager`, `@accessor` | Codegen injects one call to `engram_boundary_beat(<fn name>)` at function entry. The decorated op self-reports (chrono tick, afferent counter, self-activity strengthen, dharma bus event) with **zero** hand-written instrumentation in its body. |
|
||||||
|
| `@route(path, method, …)` | Records a route into a generated dispatch table. |
|
||||||
|
|
||||||
|
Decorators with no registered meaning are accepted and ignored.
|
||||||
|
|
||||||
|
**Limits of the seam, as it stands.** The injection is a *prologue only* — there is no epilogue, no wrapping of the call, and no way for a decorator to run code after the body returns. The injected callee is a fixed builtin chosen by the compiler, not derived from the decorator name or its arguments. Section 19 depends on lifting exactly these two limits.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1088,4 +1104,152 @@ The next minor version closes the implementation gaps named in this document. Tr
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 18. The Program Block — cross-cutting concerns [implemented]
|
||||||
|
|
||||||
|
### 18.0 Why this exists
|
||||||
|
|
||||||
|
A cross-cutting concern is one that belongs to the *process*, not to any function in it: only one of me may run; this is what my configuration is; every mutation must be durable; every request must be authorized.
|
||||||
|
|
||||||
|
El's units of encapsulation are the function and the module. Neither can hold a concern like that. So each one had been expressed the only way it could be — as a **convention**: *call this at every site.* Conventions of that shape do not hold. They are not enforced by anything, they are invisible in review, and they fail silently at the one site somebody forgot.
|
||||||
|
|
||||||
|
Measured in this codebase before this section existed:
|
||||||
|
|
||||||
|
| Concern | State | What the convention was |
|
||||||
|
|---|---|---|
|
||||||
|
| process identity | **zero** guards anywhere — no pidfile, no lock, no already-running check, at any layer | "check nothing is already running first" |
|
||||||
|
| configuration | **20** distinct environment variables in one program, each with its default written inline at the read site | "remember the right default here" |
|
||||||
|
| durability | **62** `persist_*` / `engram_save` / `wal_*` / `checkpoint` call sites | "after you mutate, remember to persist" |
|
||||||
|
| request auth | **10** per-route `_auth` checks | "check the token in this handler too" |
|
||||||
|
|
||||||
|
These are not four problems. They are one absence, four times.
|
||||||
|
|
||||||
|
That the convention form fails is observed, not predicted. Process identity failed three times in a single day: twice, two engram processes ran simultaneously against the same data directory; twice, a stale binary held a port and answered probes while a fresh build was believed to be under test, because `pkill -f` had silently failed to match its argv — which nearly produced a false "the fix does not work" conclusion. Configuration failed structurally: `ENGRAM_DATA_DIR` was read at six sites, five of them dead bindings, and the sixth defaulted to `/tmp/engram` — contradicting the canonical resolver's `$HOME/.neuron/engram` and landing a pre-destructive safety backup on ephemeral storage.
|
||||||
|
|
||||||
|
The `program` block is where a concern of this shape is declared once and enforced by the compiler at the process boundary.
|
||||||
|
|
||||||
|
### 18.1 Syntax
|
||||||
|
|
||||||
|
```
|
||||||
|
program "engram" {
|
||||||
|
singleton: "engram"
|
||||||
|
env ENGRAM_BIND: String = ":8742"
|
||||||
|
env GUIDE_PORT: Int = "8771"
|
||||||
|
env ENGRAM_API_KEY: String required
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
At most one `program` block per program. It composes with `cgi` and `service` — those declare what a program *may do*; `program` declares what a program *is*.
|
||||||
|
|
||||||
|
Grammar:
|
||||||
|
|
||||||
|
```ebnf
|
||||||
|
program_block = "program" string "{" { program_field } "}" ;
|
||||||
|
program_field = singleton_field | env_field ;
|
||||||
|
singleton_field = "singleton" ":" string [ "," ] ;
|
||||||
|
env_field = "env" ident ":" type
|
||||||
|
[ "=" string ] [ "required" ] [ "," ] ;
|
||||||
|
```
|
||||||
|
|
||||||
|
`singleton` and `env` are **not** reserved words. They are read as identifier token values by the block's own parse loop, so they remain usable as ordinary identifiers everywhere else. `program` is the only keyword this section adds.
|
||||||
|
|
||||||
|
### 18.2 Process identity — `singleton`
|
||||||
|
|
||||||
|
`singleton: "id"` compiles to an `el_singleton_acquire("id")` call injected as the **first statement of `main()`**, before any user statement runs.
|
||||||
|
|
||||||
|
The runtime takes an exclusive non-blocking `flock` on `<dir>/el-singleton-<id>.lock`, where `<dir>` is `$EL_SINGLETON_DIR`, else `$TMPDIR`, else `/tmp`. On success it writes its pid and holds the descriptor open for the life of the process. On contention it **refuses to start**: it reports the holder's pid, names the lock file, and exits 1.
|
||||||
|
|
||||||
|
Two properties are deliberate:
|
||||||
|
|
||||||
|
- **It is a lock, not a pidfile.** The kernel releases an `flock` when the owning process dies — including on `SIGKILL` and on crash. There is therefore no stale-lock state, and so no "delete the lock file to get unstuck" recovery ritual. Such a ritual would itself be a convention, which is the thing this section exists to remove.
|
||||||
|
- **It reports the holder's pid.** "Already running" is not actionable. A pid is. This is the direct answer to the observed failure where a stale process survived a `pkill` and went on answering probes.
|
||||||
|
|
||||||
|
Refusal is loud and total. It is not a warning, and the program does not continue degraded. This matters more than it looks: today a second engram whose `bind()` fails merely *returns* from `http_serve` — after it has already replayed the WAL and written boot-time backup files — and then exits **0**, indistinguishable from a clean run. `singleton` refuses before the first side effect.
|
||||||
|
|
||||||
|
### 18.3 Configuration — `env`
|
||||||
|
|
||||||
|
Each `env` entry declares one configuration variable: its name, its type (`Int` or `String`), and either a default or `required`.
|
||||||
|
|
||||||
|
Resolution happens once, at startup, in declaration order: **the environment wins; the declaration supplies the fallback.** Then `el_config_validate` checks the whole schema and reports *every* problem at once before exiting — a startup that fails one variable at a time costs one restart per variable.
|
||||||
|
|
||||||
|
Values are read with `config("NAME")`, which returns a `String`.
|
||||||
|
|
||||||
|
The enforcement that makes the declaration real: **once a program block exists, `config("X")` for an undeclared `X` is a fatal error.** Without that, the schema would be advisory, and an advisory schema is just another convention. Programs with no `program` block are unaffected — `config()` falls back to a plain environment read, so migration is incremental and per-program.
|
||||||
|
|
||||||
|
The point is not that configuration is now centralized. It is that **a default is no longer a decision made at a read site.** A read site cannot disagree with another read site about what a variable means, because a read site no longer says.
|
||||||
|
|
||||||
|
### 18.4 What is deliberately not declared here
|
||||||
|
|
||||||
|
Some values look like configuration and are not. `ENGRAM_DATA_DIR` already has a single owner — `engram_resolve_data_dir()`, which resolves it, creates the directory, and fails loud rather than silently persisting to an ephemeral path. Declaring it in the `program` block as well would give it two owners that can disagree, recreating the precise defect this section removes.
|
||||||
|
|
||||||
|
The rule: **a variable belongs in the program block when the block would be its only owner.** If a resolver already owns it, leave it there.
|
||||||
|
|
||||||
|
`HOME` is likewise not configuration. It is an environment fact, and stays a raw `env()` read.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 19. Boundary Effects — durability and request authorization [design only, not implemented]
|
||||||
|
|
||||||
|
Sections 19.1 and 19.2 specify the two remaining concerns from the table in 18.0. Both are **designed and deliberately unimplemented.** The reason is stated in 19.3 and it is not difficulty.
|
||||||
|
|
||||||
|
### 19.1 Durability as an epilogue effect
|
||||||
|
|
||||||
|
**The defect.** 62 call sites carry the convention *"after you mutate, remember to persist."* This is structurally the same defect as the index bug being fixed elsewhere in this tree — *"after you append, remember to index"* — which failed at **9 of 9** sites. A convention that failed at 100% of its sites is the strongest available evidence about what this class of convention is worth.
|
||||||
|
|
||||||
|
**Why the existing seam cannot express it.** §9's injection is a prologue. Durability is inherently an *epilogue*: persist after the mutation succeeds, and not at all if it threw. The seam has no epilogue.
|
||||||
|
|
||||||
|
**Design.** Extend the decorator seam from prologue-only to prologue/epilogue, then declare durability as an effect on the mutating function:
|
||||||
|
|
||||||
|
```
|
||||||
|
@durable("engram")
|
||||||
|
fn engram_write_node(id: String, body: String) -> Bool { … }
|
||||||
|
```
|
||||||
|
|
||||||
|
Codegen wraps rather than prefixes:
|
||||||
|
|
||||||
|
```c
|
||||||
|
el_val_t engram_write_node(el_val_t id, el_val_t body) {
|
||||||
|
el_effect_enter(EL_STR("durable"), EL_STR("engram"));
|
||||||
|
el_val_t __r = /* original body */;
|
||||||
|
el_effect_exit(EL_STR("durable"), EL_STR("engram"), __r);
|
||||||
|
return __r;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`el_effect_exit` is where the persist happens, and it is the only place it happens. Two properties follow that the 62 hand-written sites cannot have:
|
||||||
|
|
||||||
|
- **Coalescing.** The epilogue is a single choke point, so N mutations inside one request can produce one fsync instead of N. The hand-written form cannot coalesce, because no site knows about the others.
|
||||||
|
- **Failure is not silent.** A persist that fails inside `el_effect_exit` can force the mutation's return value to failure. A forgotten `persist_*` call cannot fail — it simply does not happen, which is exactly why the defect is invisible.
|
||||||
|
|
||||||
|
**Enforcement, and this is the part that actually fixes it.** Mirroring §9's `#error` for `dharma_emit`: a function that calls a mutating primitive without carrying `@durable` is a **compile error**. Otherwise this is a 63rd thing to remember rather than a replacement for 62.
|
||||||
|
|
||||||
|
### 19.2 Request authorization as a route effect
|
||||||
|
|
||||||
|
**The defect.** 10 per-route `_auth` checks. The HTTP layer has no concept of authorization, so a new route is unauthenticated by default and silently so — the failure mode is a route that forgot, and nothing anywhere reports it.
|
||||||
|
|
||||||
|
**Design.** Authorization becomes an argument to the `@route` decorator, which already takes arguments and already builds a dispatch table:
|
||||||
|
|
||||||
|
```
|
||||||
|
@route("/api/write", "POST", auth: "required")
|
||||||
|
fn route_write(body: String) -> String { … }
|
||||||
|
```
|
||||||
|
|
||||||
|
The generated dispatcher performs the check **before** dispatch, so an unauthorized request never reaches the handler and the handler contains no auth code at all.
|
||||||
|
|
||||||
|
The default must be `required`. A route that says nothing gets authorization; opening one up takes an explicit `auth: "public"`. Defaulting to public preserves the current failure mode exactly — forgetting stays silent — and a default that preserves the defect is not a fix.
|
||||||
|
|
||||||
|
Route inventory falls out for free: the dispatch table already exists, so the compiler can emit the full route/auth matrix and make "which routes are public" a fact that is read rather than audited.
|
||||||
|
|
||||||
|
### 19.3 Why these are not implemented
|
||||||
|
|
||||||
|
Not difficulty — **collision**. Both land squarely in regions two other agents hold right now:
|
||||||
|
|
||||||
|
- **Durability** requires changing the mutation and persist paths in `lang/runtime/el_runtime.c` and `engram/src/server.el` — the same files and the same read/write paths being restructured by concurrent work on VIndex read-path mutation and memory ownership, and on geometry-as-an-el-value and `transduce`.
|
||||||
|
- **Request auth** requires changing route dispatch in `engram/src/server.el`, which the geometry/`transduce` work is actively reshaping.
|
||||||
|
|
||||||
|
Implementing either now would mean editing files under concurrent modification and resolving conflicts in exactly the paths whose correctness is currently under repair. The designs are recorded here so the work is not lost, and so that whoever lands them does so against a settled tree.
|
||||||
|
|
||||||
|
The prerequisite for 19.1 is the same in both cases: **lift the §9 seam from prologue-only to prologue/epilogue.** That change is independent of both collisions and can land first.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
End of specification.
|
End of specification.
|
||||||
|
|||||||
@@ -0,0 +1,180 @@
|
|||||||
|
# El Runtime — Ownership and Capability ABI
|
||||||
|
|
||||||
|
**Status:** §0–§2 verified. §3 re-derived and **built** for the vector index (2026-08-16); not yet applied to the resident RAM graph.
|
||||||
|
**Date:** 2026-08-16
|
||||||
|
**Scope:** `lang/runtime/` — every El program (soul, engram, cgi-studio vessels) inherits this by rebuild. Nothing in this document is a change to any El *program*.
|
||||||
|
|
||||||
|
**Note on §1's line numbers:** they were read against a checkout that has since shifted by ~135 lines. Verified positions as of `a67452f` are in §2a.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. The residual
|
||||||
|
|
||||||
|
> **Builtins own memory and reach process state directly.**
|
||||||
|
|
||||||
|
That is the residual — the generator. Everything below labelled a "residue" is a deposit left by it. The distinction matters because we have spent significant effort removing deposits, and deposits regenerate.
|
||||||
|
|
||||||
|
A residue is fixed. A residual is eliminated. Fixing residues while the residual stands produces exactly the pattern observed on 2026-08-15/16: a run of individually-correct patches, each verified, followed by a new defect of the same shape in a different file.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The residues, measured
|
||||||
|
|
||||||
|
Each of these is a distinct merged or proposed fix. Each addresses one deposit. None addresses the residual.
|
||||||
|
|
||||||
|
| residue | location | fix that was applied or proposed |
|
||||||
|
|---|---|---|
|
||||||
|
| `state_get` leaked its return value per call — 15 MB over 200k calls | builtin | el #140 (merged) |
|
||||||
|
| VIndex freed under a concurrent reader | `el_runtime.c:9424` | `fb32d15` guard (merged 08:46:43) |
|
||||||
|
| `_eg_vindex_seen` realloc'd on a read path | `el_runtime.c:9412` | same guard |
|
||||||
|
| `vindex_insert` on a read path | `el_runtime.c:9434`, `9450` | same guard |
|
||||||
|
| shared `visited` / epoch scratch stomped by concurrent searches | `engram_vindex.c:79–81`, `169–186`, `195` | proposed: move to per-search frame |
|
||||||
|
| nine append sites, none indexing → lazily-embedded nodes invisible | `el_runtime.c:7806, 7988, 8148, 8224, 11526, 11731, 12050, 15295, 15312` | "embed-gap #20", patched by making the *read* path catch up (`9439` comment) |
|
||||||
|
|
||||||
|
**Measured:** all file/line references above, read 2026-08-16. Crash frames `engram_activate → eg_vindex_sync → vindex_insert → _realloc → _xzm_xzone_malloc_freelist_outlined` are accounted for by rows 2–4.
|
||||||
|
|
||||||
|
**Inferred, not yet verified:** that the nine append sites do not share a single commit point. This needs one pass before Change C is sized.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Why these are one defect
|
||||||
|
|
||||||
|
`eg_vindex_sync` (`el_runtime.c:9419`) has exactly three callers, and **all three are reads**:
|
||||||
|
|
||||||
|
- `engram_activate` — `9802`
|
||||||
|
- `eg_knn_for_node` — `13075` (its own header comment states *"No writes."*)
|
||||||
|
- `engram_geo_reify_run_json` — `13285`
|
||||||
|
|
||||||
|
It mutates five process-global statics (`9400–9404`): `_eg_vindex`, `_eg_vindex_dim`, `_eg_vindex_built_nc`, `_eg_vindex_seen`, `_eg_vindex_seen_cap`.
|
||||||
|
|
||||||
|
Reads mutate because index maintenance was never given an owner on the write side. It got bolted onto reads, because a builtin *could* reach the globals — nothing prevented it. Likewise `state_get` leaked because a builtin *owned* the value it returned; nothing prevented that either.
|
||||||
|
|
||||||
|
The store is architecturally append-only and superseding. A read path that mutates contradicts that directly. The contradiction is expressible only because the ABI permits it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2a. Verified positions and the fact §1 missed
|
||||||
|
|
||||||
|
Read directly at `a67452f`, 2026-08-16. §1's line numbers predate a ~135-line shift; these are current.
|
||||||
|
|
||||||
|
| thing | §1 said | actually |
|
||||||
|
|---|---|---|
|
||||||
|
| five process-global statics | 9400–9404 | **9535–9539** |
|
||||||
|
| `eg_vindex_seen_ensure` realloc | 9412 | **9547** |
|
||||||
|
| `eg_vindex_sync` | 9419 | **9554** |
|
||||||
|
| `vindex_free` on a read path | 9424 | **9559** |
|
||||||
|
| `vindex_insert` on a read path | 9434 / 9450 | **9569** (build) / **9585** (incremental) |
|
||||||
|
| caller: `engram_activate_inner` | 9802 | **9939** |
|
||||||
|
| caller: `eg_knn_for_node` | 13075 | **13212** |
|
||||||
|
| caller: `engram_geo_reify_run_json` | 13285 | **13422** |
|
||||||
|
| `fb32d15` guard | — | lock **1602**, depth **1631**, `eg_guard_enter` **1636**, `http_worker` acquire **1687**, `engram_activate` wrapper **14097** |
|
||||||
|
| VIndex scratch fields | 79–81 | **79–81** ✓ |
|
||||||
|
| `search_layer` race site | 195 | **195** ✓ |
|
||||||
|
|
||||||
|
**The structural fact §1 and §3 both missed:** *the index does not inherit the store's append-only property.* `vindex_insert` rewires the `NeighList` links of already-existing elements and reallocs `elems[]` — so extending the index mutates the whole structure, not just its tail. This is why "make reads pure" is necessary but **not sufficient**, and why §3 needed a publication boundary rather than only a capability split. It is reproduced as a standing test (`unsynchronized` half, §5).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The change
|
||||||
|
|
||||||
|
*(Re-derived 2026-08-16. The previous §3 — a runtime context struct carrying read/write **capability pointers** to every builtin — was written in mutable-store, C-ownership terms. It asked "who is permitted to mutate the shared thing?", which presupposes a shared mutable thing. The engram is immutable and recall is projection; what does not mutate needs no ownership discipline. So the question is not answered, it is dissolved. The implemented change is below.)*
|
||||||
|
|
||||||
|
### 3.1 Three moves, in decreasing order of how much they dissolve
|
||||||
|
|
||||||
|
**(1) Misfiled scratch is not shared state.** `visited` / `visit_epoch` were never conceptually owned by the index — they are one traversal's local, hoisted into `struct VIndex` as an allocation optimisation. Nothing about them is derived geometry. They want neither a lock nor a capability nor a checkout pool: a pure function's scratch belongs to its call frame, and the fix is to put it back there. This is not "the capability model applied by hand to one global"; it is the deletion of a false ownership claim.
|
||||||
|
|
||||||
|
**(2) `const` is the capability, and immutability hands it over for free.** Once the scratch leaves the struct, `search_layer` reads the index and nothing else — so `vindex_search` can take a `const VIndex*`. That is *precisely* the teeth old-§3 wanted from capability pointers: a read path physically cannot call `vindex_insert`, and it is a **compile error**, not a review comment. It costs one qualifier rather than a new ABI swept across hundreds of builtins. The compiler enforces it on every future caller for the same reason.
|
||||||
|
|
||||||
|
> The capability type was already in the language. It is spelled `const`.
|
||||||
|
|
||||||
|
**(3) What remains is a publication problem, not an ownership problem.** With scratch in the frame and reads const, one hazard survives, and it is real: **HNSW insert is not an append.** `vindex_insert` rewires the `NeighList` links of *already-existing* elements and reallocs `elems[]`. The store's append-only property does **not** transfer to the index derived from it. So a reader projecting against the index while its owner extends it is unsafe no matter how pure search is.
|
||||||
|
|
||||||
|
Immutability answers this too, and the answer is publication:
|
||||||
|
|
||||||
|
- **`eg_vindex_maintain`** — the sole mutator. Takes the boundary exclusively; never runs beside a reader.
|
||||||
|
- **`eg_vindex_view`** — returns a `const VIndex*` with the boundary held for read. N readers project concurrently; none can mutate.
|
||||||
|
|
||||||
|
A read path may **demand that a current snapshot exist** — that is a request to the owner, not a mutation by the reader. What it may not do is mutate the geometry it is projecting against. `view` / `maintain` is exactly that split, and it is why this replaces `eg_vindex_sync` rather than wrapping it.
|
||||||
|
|
||||||
|
**Write-side owner.** Index membership is owned by the event *"an embedding became present on this ordinal"* — not by node append, since a node without an embedding cannot be in a vector index at all. `eg_vindex_note_embedded` hooks the embedding-assignment sites: one O(log n) insert, no O(node_count) presence scan. This also retires the "STALENESS (honest tradeoff)" note in the old `eg_vindex_sync`, where a lazily-embedded *older* node stayed invisible to `route_nearest` / autoconnect until the next full rebuild.
|
||||||
|
|
||||||
|
### 3.2 What this does not claim
|
||||||
|
|
||||||
|
The **resident RAM graph** (`g->nodes` / `g->edges`) is a *separate* residue of the same residual and is untouched by this change. It is realloc'd in place (`el_runtime.c:7618`, `7629`), so an awareness-thread reader holding `EngramNode* n = &g->nodes[i]` across a concurrent append holds a dangling pointer — and `engram_activate_inner`'s embed-backfill writes `n->emb` through exactly such a pointer. It wants the same publication treatment the index just received. Until that lands, the `fb32d15` guard stays (see §5).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Why this is not a large change
|
||||||
|
|
||||||
|
The old §4 argued that El owning its compiler makes a capability-ABI sweep mechanical, since `elc` generates every builtin call site. That argument was load-bearing only for the ABI, and the ABI is gone.
|
||||||
|
|
||||||
|
The constraint now travels with the **type of the thing**, not the shape of every call site — so no sweep is needed at all. Measured extent of the implemented change: two qualifiers (`const VIndex*` on `vindex_search`, propagated to `engram_geometry_descriptor` and `engram_geo_reify_store`), one struct field group relocated to a call frame, one rwlock, and three read call sites converted from `eg_vindex_sync` to `view`/`release`.
|
||||||
|
|
||||||
|
The payoff of owning the language is unchanged and is now *cheaper*: introduced once, enforced by the compiler on every future builtin, cannot subsequently be forgotten. Contrast the current state, where the same discipline was maintained by hand across hundreds of builtins and demonstrably failed at least six times.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. What this deletes
|
||||||
|
|
||||||
|
**Deleted (done, 2026-08-16):**
|
||||||
|
|
||||||
|
- `eg_vindex_sync` — the function itself. Not renamed: split into `eg_vindex_maintain` (mutating, exclusive, sole owner) and `eg_vindex_view` (const, shared). A name that meant "read paths repair the index" had to stop existing.
|
||||||
|
- `VIndex::visited` / `visit_epoch` / `visited_cap` — the struct fields, `visited_ensure`, its call from `elems_reserve`, `ix->visit_epoch = 0` in `vindex_create`, and `free(ix->visited)` in `vindex_free`.
|
||||||
|
- The **proposed** per-search scratch *struct on the index* (a checkout pool / `VisitedListPool`) — never built. The buffer is a plain frame local; a pool is machinery for an ownership question that no longer exists.
|
||||||
|
- The **proposed** reader-view / owner-handle split for VIndex specifically — superseded. `const` already is the reader view.
|
||||||
|
- `EXPECT_RACE` in `run_vindex_concurrency_tests.sh` — a knob that let a known defect ride as "expected". Replaced by four halves with real verdicts.
|
||||||
|
|
||||||
|
**NOT deleted — the design doc was wrong about this one:**
|
||||||
|
|
||||||
|
- `fb32d15` (`eg_guard_enter` / `engram_req_lock` / `_eg_req_depth`). §5 originally called for its removal as "a lock protecting a mutation that ceases to exist." **Measured, it guards two things, and only one of them ceases to exist.** Its own comment names both: the RAM graph *and* `_eg_vindex`. The vindex justification is retired; the RAM-graph justification is independently load-bearing (§3.2), and removing the guard reintroduces the measured 11171→9579 edge-loss defect from 2026-08-14. Its comment has been narrowed to state the RAM graph only. **Precondition for deleting it:** the resident graph gets the same publication boundary the index just got.
|
||||||
|
- el #140's hand-patch. Left in place — the leak stops being *expressible* only under the abandoned capability-ABI §3, which is not what was built.
|
||||||
|
|
||||||
|
**Ordering consequence (revised):** the original ordering claim — "the residual lands first, the residues evaporate rather than get fixed" — did not survive contact. The residual here is not a single ABI that dissolves everything at once; it is a *property* (derived state is published, never edited) applied per structure. The index now has it. The RAM graph does not yet. Residues evaporate **per structure, in the order the property is applied**, and a residue whose structure has not been converted must be left standing, not deleted on the strength of the plan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Sequencing
|
||||||
|
|
||||||
|
1. **Read** how builtins are declared and dispatched, to confirm the call sites are compiler-generated in one place. *(This determines whether §4 holds. If dispatch is scattered, re-size before proceeding.)*
|
||||||
|
2. Introduce the context type and capability types.
|
||||||
|
3. Codegen emits the context at every builtin call site.
|
||||||
|
4. Mechanical sweep of builtin signatures.
|
||||||
|
5. Move index maintenance behind the write capability; the three read callers take the read capability.
|
||||||
|
6. Delete the residue-fixes listed in §5.
|
||||||
|
7. **One** build of soul from el dev — which resolves the `state_get` leak and the crash together, rather than deploying a leak fix that reintroduces the crash.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Open questions
|
||||||
|
|
||||||
|
**Answered 2026-08-16:**
|
||||||
|
|
||||||
|
- ~~Do the nine append sites share a commit point?~~ **Moot.** The question was mis-aimed: node append is not the event that owns index membership, because a node without an embedding cannot be in a vector index. The five *embedding-assignment* sites are the real owner points (`el_runtime.c:7091, 9839, 13362, 15002`, plus snapshot-restore at `7951`), and three of them carry the ordinal directly — which is all `eg_vindex_note_embedded` needs. The other two run before the node is resident, where the cold build picks it up.
|
||||||
|
- ~~Does anything outside `lang/runtime/` construct a second `VIndex`?~~ **No.** Swept: the only constructors outside the runtime are `engram/test/*` and `lang/runtime/vindex_bench.c`, all single-threaded and index-private. Inside the runtime, `engram_self_reify_beat_json` builds a **private** index deliberately and never touches the shared boundary — that was already correct and is unchanged.
|
||||||
|
- ~~Does the HTTP worker pool contend on the same globals?~~ **Yes, and it was never the whole story.** Workers serialize against each other on `engram_req_lock`, but the awareness main thread does not take it at all — that is the gap `fb32d15` closed. Now verified independent of that guard: the index boundary is its own rwlock, so worker/awareness contention on `_eg_vindex` is handled whether or not the request lock is held.
|
||||||
|
|
||||||
|
**Still open:**
|
||||||
|
|
||||||
|
- The resident RAM graph wants the same publication boundary (§3.2). Until it has one, `fb32d15` cannot be deleted.
|
||||||
|
- `eg_vindex_view` holds the boundary for read across `engram_geo_reify_store`, which is a long pass. Correct, but it stalls the owner for that duration. If reify latency becomes a problem the answer is a refcounted snapshot, not a shorter lock.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7a. Evidence (measured 2026-08-16, `engram/test/run_vindex_concurrency_tests.sh`)
|
||||||
|
|
||||||
|
| half | before | after |
|
||||||
|
|---|---|---|
|
||||||
|
| `single` — 3000 vectors, 1 thread, ASan+UBSan | clean | clean |
|
||||||
|
| `readers` — 4 readers, no writer, TSan | **race** at `engram_vindex.c:195` (`visited_reset` ← `vindex_search`) | **clean** |
|
||||||
|
| `unsynchronized` — writer+reader, bare index, TSan | race | **race, expected and permanent** — now the proof the boundary must exist |
|
||||||
|
| `published` — owner + 4 readers through the boundary, TSan | *(did not exist)* | **clean**, all 3000 inserts landed |
|
||||||
|
|
||||||
|
No recall regression: `recall@10 = 0.9365` at `ef_search=128` (gate ≥ 0.90); the determinism test still yields byte-identical results across two independent builds.
|
||||||
|
|
||||||
|
Builds locally: all seven engram runtime translation units compile `-Wall -Wextra` clean, and the full engram binary links (`engram/dist/engram.c` + runtime, arm64). The one pre-existing `-Wcomment` warning in `el_runtime.c` is present at `a67452f` too.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. What this document is not
|
||||||
|
|
||||||
|
It is not an argument for a memory model in general, a garbage collector, process isolation between soul and engram, or a client/server split of the store. Each of those was considered and each addresses mutation that this change removes. They are answers to a question that stops being asked.
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
// fitprobe.el — controlled growth-curve specimens for validating the complexity fitter.
|
||||||
|
//
|
||||||
|
// Three deliberately-shaped workloads. None depends on a real defect existing,
|
||||||
|
// which is the point: the fitter must be provable against KNOWN curves.
|
||||||
|
//
|
||||||
|
// linear — one allocation per item. count O(n), bytes O(n), time O(n)
|
||||||
|
// accum — rebuilds its accumulator. count O(n), bytes O(n^2), time O(n^2)
|
||||||
|
// compute — nested arithmetic, no alloc. count O(1), bytes O(1), time O(n^2)
|
||||||
|
//
|
||||||
|
// `compute` is the specimen that matters. It is the shape of el #132
|
||||||
|
// (strlen-per-character inside str_char_code): pure CPU, zero allocation.
|
||||||
|
// An allocation-only gate is structurally blind to it.
|
||||||
|
//
|
||||||
|
// No imports — uses runtime builtins directly so nothing collides.
|
||||||
|
|
||||||
|
fn work_linear(n: Int) -> Int {
|
||||||
|
let parts: [String] = native_list_empty()
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let parts = native_list_append(parts, int_to_str(i))
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return native_list_len(parts)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn work_accum(n: Int) -> Int {
|
||||||
|
let acc: String = ""
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let acc = acc + "x"
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return str_len(acc)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn work_compute(n: Int) -> Int {
|
||||||
|
// str_char_code is an opaque external call, so the C optimiser cannot
|
||||||
|
// reduce this nest to a closed form the way it does with `total + 1`.
|
||||||
|
// This is the exact shape of el #132: n scans over n characters, pure
|
||||||
|
// CPU, ZERO allocation.
|
||||||
|
let s: String = "abcdefghij"
|
||||||
|
let total: Int = 0
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let j: Int = 0
|
||||||
|
while j < n {
|
||||||
|
let total = total + str_char_code(s, 0)
|
||||||
|
let j = j + 1
|
||||||
|
}
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return total
|
||||||
|
}
|
||||||
|
|
||||||
|
fn run_one(mode: String, n: Int) {
|
||||||
|
let c0: Int = el_alloc_count()
|
||||||
|
let b0: Int = el_alloc_bytes()
|
||||||
|
let t0: Int = el_now_instant()
|
||||||
|
|
||||||
|
let r: Int = 0
|
||||||
|
if str_eq(mode, "linear") { let r = work_linear(n) }
|
||||||
|
if str_eq(mode, "accum") { let r = work_accum(n) }
|
||||||
|
if str_eq(mode, "compute") { let r = work_compute(n) }
|
||||||
|
|
||||||
|
let t1: Int = el_now_instant()
|
||||||
|
let c1: Int = el_alloc_count()
|
||||||
|
let b1: Int = el_alloc_bytes()
|
||||||
|
|
||||||
|
println(mode + "\t" + int_to_str(n)
|
||||||
|
+ "\t" + int_to_str(c1 - c0)
|
||||||
|
+ "\t" + int_to_str(b1 - b0)
|
||||||
|
+ "\t" + int_to_str((t1 - t0) / 1000)
|
||||||
|
+ "\t" + int_to_str(r))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sweep(mode: String) {
|
||||||
|
run_one(mode, 200)
|
||||||
|
run_one(mode, 400)
|
||||||
|
run_one(mode, 800)
|
||||||
|
run_one(mode, 1600)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() -> Int {
|
||||||
|
println("mode\tn\tallocs\tbytes\tusec\tsink")
|
||||||
|
sweep("linear")
|
||||||
|
sweep("accum")
|
||||||
|
sweep("compute")
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
import "../../runtime/eltest.el"
|
||||||
|
import "../../runtime/elbench.el"
|
||||||
|
|
||||||
|
// test_elbench.el — proves the growth-curve classifier against KNOWN curves.
|
||||||
|
//
|
||||||
|
// Every series below is real measured data from lang/tests/bench/fitprobe.el
|
||||||
|
// on a geometric sweep n = 200/400/800/1600. The classifier must be provable
|
||||||
|
// without depending on a live defect existing, which is the whole point of
|
||||||
|
// keeping controlled specimens.
|
||||||
|
|
||||||
|
fn _s4(a: Int, b: Int, c: Int, d: Int) -> [Int] {
|
||||||
|
let l: [Int] = native_list_empty()
|
||||||
|
let l = native_list_append(l, a)
|
||||||
|
let l = native_list_append(l, b)
|
||||||
|
let l = native_list_append(l, c)
|
||||||
|
let l = native_list_append(l, d)
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
|
||||||
|
test "classifies a linear allocation series as O(n)" {
|
||||||
|
// fitprobe `linear`, allocation count
|
||||||
|
let v = _s4(208, 409, 810, 1611)
|
||||||
|
assert elb_measured_curve(v, 10) == 2, "linear allocs should classify O(n)"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "classifies a linear byte series as O(n)" {
|
||||||
|
// fitprobe `linear`, allocation bytes
|
||||||
|
let v = _s4(4786, 9682, 19474, 39658)
|
||||||
|
assert elb_measured_curve(v, 10) == 2, "linear bytes should classify O(n)"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "classifies a quadratic byte series as O(n^2)" {
|
||||||
|
// fitprobe `accum`, allocation bytes -- the accumulator-rebuild shape
|
||||||
|
let v = _s4(20300, 80600, 321200, 1282400)
|
||||||
|
assert elb_measured_curve(v, 10) == 4, "accum bytes should classify O(n^2)"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "accumulator count is linear -- proves count alone misses it" {
|
||||||
|
// Same run as above. The COUNT is exactly linear while bytes are
|
||||||
|
// quadratic. A count-only gate passes this defect clean.
|
||||||
|
let v = _s4(200, 400, 800, 1600)
|
||||||
|
assert elb_measured_curve(v, 10) == 2, "accum count classifies O(n)"
|
||||||
|
assert elb_gate(v, 2, 10) == 0, "count-only gate PASSES the quadratic"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "classifies a quadratic time series as O(n^2)" {
|
||||||
|
// fitprobe `compute` -- el #132's shape: n scans over n characters
|
||||||
|
let v = _s4(67, 205, 818, 3268)
|
||||||
|
assert elb_measured_curve(v, 10) == 4, "compute time should classify O(n^2)"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "REFUSES an all-zero series instead of calling it O(1)" {
|
||||||
|
// fitprobe `compute` allocation count. Pure CPU, allocates nothing.
|
||||||
|
// Reporting O(1) here would be a confident answer with nothing behind it.
|
||||||
|
let v = _s4(0, 0, 0, 0)
|
||||||
|
assert elb_gate(v, 2, 10) == 3, "all-zero series must be REFUSED"
|
||||||
|
assert elb_measured_curve(v, 10) < 0, "unclassifiable returns -1"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "REFUSES an implausibly flat series" {
|
||||||
|
// The shape produced when clang closes a loop to a multiply: a real
|
||||||
|
// answer, no work done, no movement across an 8x input range.
|
||||||
|
let v = _s4(1000, 1001, 1002, 1003)
|
||||||
|
assert elb_gate(v, 2, 10) == 3, "hard-flat series must be REFUSED"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "gate FAILS a quadratic declared as linear" {
|
||||||
|
let v = _s4(20300, 80600, 321200, 1282400)
|
||||||
|
assert elb_gate(v, 2, 10) == 1, "O(n^2) measured vs O(n) declared must FAIL"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "gate PASSES a linear series declared as linear" {
|
||||||
|
let v = _s4(208, 409, 810, 1611)
|
||||||
|
assert elb_gate(v, 2, 10) == 0, "O(n) measured vs O(n) declared must PASS"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "gate reports BETTER when measured beats the declared bound" {
|
||||||
|
let v = _s4(208, 409, 810, 1611)
|
||||||
|
assert elb_gate(v, 4, 10) == 4, "O(n) measured vs O(n^2) declared is BETTER"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "gate reports INDETERMINATE on disagreeing ratios" {
|
||||||
|
// fitprobe `linear` WALL TIME at these sizes: 26/19/43/78 microseconds.
|
||||||
|
// Ratios 0.73, 2.26, 1.81 disagree well past the noise threshold. The
|
||||||
|
// honest answer is "cannot tell", not a classification -- this is exactly
|
||||||
|
// why benchmarks need auto-scaled iteration counts rather than one shot.
|
||||||
|
let v = _s4(26, 19, 43, 78)
|
||||||
|
assert elb_gate(v, 2, 10) == 2, "disagreeing ratios must be INDETERMINATE"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "black_box is a real barrier and returns its input" {
|
||||||
|
assert el_black_box(42) == 42, "black_box is value-preserving"
|
||||||
|
let s: Int = 0
|
||||||
|
let i: Int = 0
|
||||||
|
while i < 100 {
|
||||||
|
// Bind the call before using it in arithmetic: `x + call(...)`
|
||||||
|
// lowers to el_str_concat() on integers. Same inference defect
|
||||||
|
// as `call(...) == y` lowering to str_eq().
|
||||||
|
let bx: Int = el_black_box(1)
|
||||||
|
let s = s + bx
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
assert s == 100, "black_box does not disturb the computation"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "curve names round-trip" {
|
||||||
|
assert elb_curve_from_name("O(n)") == 2, "O(n) parses"
|
||||||
|
assert elb_curve_from_name("O(n^2)") == 4, "O(n^2) parses"
|
||||||
|
assert str_eq(elb_curve_name(4), "O(n^2)"), "O(n^2) renders"
|
||||||
|
assert elb_curve_from_name("O(nonsense)") < 0, "unknown curve is -1"
|
||||||
|
}
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
import "../../runtime/eltest.el"
|
||||||
|
import "../../runtime/elbench.el"
|
||||||
|
|
||||||
|
// test_lexer_scaling.el — THE ARMED GATE.
|
||||||
|
//
|
||||||
|
// This is the regression test that would have caught el #132.
|
||||||
|
//
|
||||||
|
// #132 was a strlen() inside str_char_code() and str_slice(). The lexer walks
|
||||||
|
// source one character at a time, so every character access rescanned the whole
|
||||||
|
// remaining input: O(n) per character over n characters = O(n^2). It shipped for
|
||||||
|
// months. It was found by a geometric sweep, not by reading code.
|
||||||
|
//
|
||||||
|
// So this test IS a geometric sweep. It scans a string of length n, character by
|
||||||
|
// character, at four doubling sizes, and asserts the cost is linear. If anyone
|
||||||
|
// reintroduces a per-character rescan — in str_char_code, in str_slice, in any
|
||||||
|
// accessor the lexer leans on — the measured curve becomes O(n^2) and this fails.
|
||||||
|
//
|
||||||
|
// The value is in it being ARMED, not in it currently failing. It passes today
|
||||||
|
// because #132 is fixed. That is the correct state for a regression gate.
|
||||||
|
//
|
||||||
|
// Note the deliberate `let c: Int = str_char_code(...)` binding in the scan loop.
|
||||||
|
// Inlining it as `total + str_char_code(s, i)` lowers to el_str_concat() on
|
||||||
|
// integers — the Plus arm of the operator-typing family, still open at the time
|
||||||
|
// of writing. Binding first is the safe form.
|
||||||
|
|
||||||
|
// _mk_string — build a string of length >= n by DOUBLING.
|
||||||
|
//
|
||||||
|
// Deliberately not `s = s + "x"` n times: that is itself quadratic in bytes and
|
||||||
|
// would contaminate the very measurement this test exists to take. Doubling
|
||||||
|
// allocates ~2n total.
|
||||||
|
fn _mk_string(n: Int) -> String {
|
||||||
|
let s: String = "abcdefgh"
|
||||||
|
while str_len(s) < n {
|
||||||
|
let s = s + s
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// _scan — walk the string one character at a time, REPS times.
|
||||||
|
//
|
||||||
|
// This is the lexer's access pattern reduced to its essential shape. The
|
||||||
|
// repetitions lift the measurement clear of timer resolution; without them the
|
||||||
|
// smaller sizes land in noise and the classifier correctly reports
|
||||||
|
// INDETERMINATE rather than guessing.
|
||||||
|
fn _scan(s: String, n: Int, reps: Int) -> Int {
|
||||||
|
let total: Int = 0
|
||||||
|
let r: Int = 0
|
||||||
|
while r < reps {
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let c: Int = str_char_code(s, i)
|
||||||
|
let total = total + c
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
let r = r + 1
|
||||||
|
}
|
||||||
|
return total
|
||||||
|
}
|
||||||
|
|
||||||
|
// _measure_scan — microseconds for a full scan sweep point.
|
||||||
|
fn _measure_scan(n: Int, reps: Int) -> Int {
|
||||||
|
let s: String = _mk_string(n)
|
||||||
|
// WARMUP, discarded. Without it the small-n end of the sweep is dominated
|
||||||
|
// by cold caches and reads as superlinear on genuinely linear work --
|
||||||
|
// measured ratios 3.37 2.92 1.76 1.65 on exactly this workload.
|
||||||
|
let w: Int = _scan(s, n, 2)
|
||||||
|
let wj: Int = el_black_box(w)
|
||||||
|
let t0: Int = el_now_instant()
|
||||||
|
let got: Int = _scan(s, n, reps)
|
||||||
|
let t1: Int = el_now_instant()
|
||||||
|
// Feed the result through the barrier so the scan cannot be elided.
|
||||||
|
let sink: Int = el_black_box(got)
|
||||||
|
if sink == 0 { println("") }
|
||||||
|
return (t1 - t0) / 1000
|
||||||
|
}
|
||||||
|
|
||||||
|
fn _series4(a: Int, b: Int, c: Int, d: Int) -> [Int] {
|
||||||
|
let l: [Int] = native_list_empty()
|
||||||
|
let l = native_list_append(l, a)
|
||||||
|
let l = native_list_append(l, b)
|
||||||
|
let l = native_list_append(l, c)
|
||||||
|
let l = native_list_append(l, d)
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
|
||||||
|
test "character scan is LINEAR in time -- regression gate for el #132" {
|
||||||
|
let reps: Int = 40
|
||||||
|
let t1: Int = _measure_scan(16384, reps)
|
||||||
|
let t2: Int = _measure_scan(32768, reps)
|
||||||
|
let t3: Int = _measure_scan(65536, reps)
|
||||||
|
let t4: Int = _measure_scan(131072, reps)
|
||||||
|
let series: [Int] = _series4(t1, t2, t3, t4)
|
||||||
|
|
||||||
|
let verdict: Int = elb_gate(series, 2, 50)
|
||||||
|
let measured: Int = elb_measured_curve(series, 50)
|
||||||
|
|
||||||
|
// Report the actual numbers regardless of outcome. A gate that fires
|
||||||
|
// without showing its evidence is just an assertion.
|
||||||
|
println(" scan us: " + int_to_str(t1) + " " + int_to_str(t2) + " "
|
||||||
|
+ int_to_str(t3) + " " + int_to_str(t4)
|
||||||
|
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
|
||||||
|
|
||||||
|
// PASS (0) or BETTER (4) are both acceptable. FAIL (1) means someone
|
||||||
|
// reintroduced superlinear per-character cost. REFUSED (3) or
|
||||||
|
// INDETERMINATE (2) mean the measurement is untrustworthy -- which is
|
||||||
|
// also a failure of this test, deliberately: a gate that cannot measure
|
||||||
|
// must not report success.
|
||||||
|
assert verdict == 0 || verdict == 4, "character scan must measure O(n) or better"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "string building by doubling stays linear in allocated bytes" {
|
||||||
|
let b1: Int = el_alloc_bytes()
|
||||||
|
let s1: String = _mk_string(8192)
|
||||||
|
let b2: Int = el_alloc_bytes()
|
||||||
|
let s2: String = _mk_string(16384)
|
||||||
|
let b3: Int = el_alloc_bytes()
|
||||||
|
let s3: String = _mk_string(32768)
|
||||||
|
let b4: Int = el_alloc_bytes()
|
||||||
|
let s4: String = _mk_string(65536)
|
||||||
|
let b5: Int = el_alloc_bytes()
|
||||||
|
|
||||||
|
let series: [Int] = _series4(b2 - b1, b3 - b2, b4 - b3, b5 - b4)
|
||||||
|
let verdict: Int = elb_gate(series, 2, 1000)
|
||||||
|
let measured: Int = elb_measured_curve(series, 1000)
|
||||||
|
println(" bytes: " + int_to_str(b2 - b1) + " " + int_to_str(b3 - b2) + " "
|
||||||
|
+ int_to_str(b4 - b3) + " " + int_to_str(b5 - b4)
|
||||||
|
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
|
||||||
|
|
||||||
|
assert verdict == 0 || verdict == 4, "doubling build must be O(n) in bytes"
|
||||||
|
assert str_len(s4) >= 65536, "final string reached the requested size"
|
||||||
|
}
|
||||||
|
|
||||||
|
// _scan_quadratic — a DELIBERATELY quadratic scan: for each position, rescan
|
||||||
|
// from the start. This is precisely what el #132 did — strlen() from offset 0
|
||||||
|
// on every character access — reproduced here so the gate can be proven to
|
||||||
|
// FIRE, not merely to pass on healthy code. An unproven gate is decoration.
|
||||||
|
fn _scan_quadratic(s: String, n: Int) -> Int {
|
||||||
|
let total: Int = 0
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let j: Int = 0
|
||||||
|
while j < i {
|
||||||
|
let c: Int = str_char_code(s, j)
|
||||||
|
let total = total + c
|
||||||
|
let j = j + 1
|
||||||
|
}
|
||||||
|
let i = i + 1
|
||||||
|
}
|
||||||
|
return total
|
||||||
|
}
|
||||||
|
|
||||||
|
fn _measure_quadratic(n: Int) -> Int {
|
||||||
|
let s: String = _mk_string(n)
|
||||||
|
let w: Int = _scan_quadratic(s, 64)
|
||||||
|
let wj: Int = el_black_box(w)
|
||||||
|
let t0: Int = el_now_instant()
|
||||||
|
let got: Int = _scan_quadratic(s, n)
|
||||||
|
let t1: Int = el_now_instant()
|
||||||
|
let sink: Int = el_black_box(got)
|
||||||
|
return (t1 - t0) / 1000
|
||||||
|
}
|
||||||
|
|
||||||
|
test "the gate FIRES on a live quadratic scan -- proves it is armed" {
|
||||||
|
let q1: Int = _measure_quadratic(1024)
|
||||||
|
let q2: Int = _measure_quadratic(2048)
|
||||||
|
let q3: Int = _measure_quadratic(4096)
|
||||||
|
let q4: Int = _measure_quadratic(8192)
|
||||||
|
let series: [Int] = _series4(q1, q2, q3, q4)
|
||||||
|
|
||||||
|
let verdict: Int = elb_gate(series, 2, 50)
|
||||||
|
let measured: Int = elb_measured_curve(series, 50)
|
||||||
|
println(" quad us: " + int_to_str(q1) + " " + int_to_str(q2) + " "
|
||||||
|
+ int_to_str(q3) + " " + int_to_str(q4)
|
||||||
|
+ " -> " + elb_curve_name(measured) + " [" + elb_verdict_name(verdict) + "]")
|
||||||
|
|
||||||
|
assert measured == 4, "a rescan-from-zero workload must classify O(n^2)"
|
||||||
|
assert verdict == 1, "declared O(n) against measured O(n^2) must FAIL the gate"
|
||||||
|
}
|
||||||
@@ -0,0 +1,534 @@
|
|||||||
|
import "../../runtime/eltest.el"
|
||||||
|
// test_transduce.el — transduction produces a SUBGRAPH, not a point.
|
||||||
|
//
|
||||||
|
// WHAT IS ACTUALLY UNDER TEST. #144 moved transduction into the language and
|
||||||
|
// got the dispatch right: realizers declared in El, resolved by name, no
|
||||||
|
// runtime patch per modality. It got the RESULT TYPE wrong —
|
||||||
|
// `transduce(signal, modality) -> Geometry`, one vector per signal.
|
||||||
|
//
|
||||||
|
// One vector is a FINGERPRINT. It can be matched and it can be ranked, and
|
||||||
|
// that is the whole of what it can ever do. It cannot be decomposed, cannot
|
||||||
|
// have one part grounded while another is not, and cannot be contradicted in
|
||||||
|
// one part while holding in another — because it has no parts. Treating
|
||||||
|
// transduction as a CONVERSION (signal in, position out) is the premise this
|
||||||
|
// file exists to falsify.
|
||||||
|
//
|
||||||
|
// A song is not a point. It decomposes into pitch, interval, rhythm, harmonic
|
||||||
|
// function — components, each with its own geometry, plus the relations among
|
||||||
|
// them. THE SONG IS THE STRUCTURE OF THE RELATIONS. So transduction yields a
|
||||||
|
// Manifold: named components carrying geometry, and typed weighted relations
|
||||||
|
// between them.
|
||||||
|
//
|
||||||
|
// The geometry tests below are UNCHANGED from #144 and still pass, which is
|
||||||
|
// the point: Geometry was never wrong, it was misplaced. A vector is the right
|
||||||
|
// representation for a COMPONENT. It was only ever wrong as the representation
|
||||||
|
// of a whole transduced signal.
|
||||||
|
//
|
||||||
|
// COMPARISON DISCIPLINE IN THIS FILE (measured 2026-08-16, not stylistic):
|
||||||
|
// elc lowers `a == b` to a NUMERIC comparison only when both operand names are
|
||||||
|
// in the per-function int-name set, which `let x: Int` populates. A bare call
|
||||||
|
// like `manifold_size(m) == 5` is not a registered name, so it lowers to
|
||||||
|
// `str_eq(...)` — strcmp on two integers reinterpreted as pointers. `<` and `>`
|
||||||
|
// lower directly via binop_to_c with no type inference at all, so truthiness is
|
||||||
|
// written `> 0` / `< 1` here, and any exact `==` is done on a value first bound
|
||||||
|
// through `let x: Int`.
|
||||||
|
//
|
||||||
|
// ONE FURTHER RULE, measured while writing this file: that int-name set LEAKS
|
||||||
|
// ACROSS `test` BLOCKS. Binding `dn` as a Float in one test and as an Int in
|
||||||
|
// another silently demoted the Int comparison to str_eq and failed an
|
||||||
|
// assertion that was arithmetically true. Every Int-bound name compared with
|
||||||
|
// `==` here is therefore spelled UNIQUELY across the whole file (note_dim,
|
||||||
|
// iv_dim, ...), rather than reusing a short name per test.
|
||||||
|
|
||||||
|
// ── A DECOMPOSING realizer, written entirely in El ──────────────────────────
|
||||||
|
// "tone" signals are note letters, e.g. "CEG". This realizer does NOT return
|
||||||
|
// one vector for the chord. It returns the PARTS — one component per note, one
|
||||||
|
// per interval between adjacent notes — and the relations that make those
|
||||||
|
// parts a chord rather than an unordered bag of pitches.
|
||||||
|
//
|
||||||
|
// The interval is deliberately a COMPONENT, not an attribute of a note. An
|
||||||
|
// interval is a thing with its own geometry that belongs to neither endpoint;
|
||||||
|
// modelling it as a field on a note is exactly the collapse this change
|
||||||
|
// rejects, one level down.
|
||||||
|
fn tone_realizer(signal: String) -> Manifold {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let n: Int = str_len(signal)
|
||||||
|
|
||||||
|
let i: Int = 0
|
||||||
|
while i < n {
|
||||||
|
let code: Int = str_char_code(signal, i)
|
||||||
|
let g: Geometry = geometry_new(2)
|
||||||
|
let s0: Int = geometry_set(g, 0, int_to_float(code))
|
||||||
|
let s1: Int = geometry_set(g, 1, int_to_float(i))
|
||||||
|
let idx: Int = manifold_add(m, "note:" + int_to_str(i), "pitch", g)
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
i = i + 1
|
||||||
|
}
|
||||||
|
|
||||||
|
let j: Int = 1
|
||||||
|
while j < n {
|
||||||
|
let a: Int = str_char_code(signal, j - 1)
|
||||||
|
let b: Int = str_char_code(signal, j)
|
||||||
|
let lo: String = "note:" + int_to_str(j - 1)
|
||||||
|
let hi: String = "note:" + int_to_str(j)
|
||||||
|
let key: String = "interval:" + int_to_str(j - 1) + "-" + int_to_str(j)
|
||||||
|
let g: Geometry = geometry_new(1)
|
||||||
|
let s: Int = geometry_set(g, 0, int_to_float(b - a))
|
||||||
|
let idx: Int = manifold_add(m, key, "interval", g)
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let e1: Int = manifold_relate(m, key, "spans", lo, 0.9)
|
||||||
|
let e2: Int = manifold_relate(m, key, "spans", hi, 0.9)
|
||||||
|
let e3: Int = manifold_relate(m, lo, "sounds_before", hi, 0.8)
|
||||||
|
j = j + 1
|
||||||
|
}
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
|
// A second realizer for a different modality, to prove the registry keys on
|
||||||
|
// modality and does not just hand back "the last thing registered". Its
|
||||||
|
// decomposition has a DIFFERENT shape — two components, one relation — so a
|
||||||
|
// test can tell the two organs apart by structure alone.
|
||||||
|
fn pulse_realizer(signal: String) -> Manifold {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let ga: Geometry = geometry_new(1)
|
||||||
|
let sa: Int = geometry_set(ga, 0, 1.0)
|
||||||
|
let ia: Int = manifold_add(m, "onset", "event", ga)
|
||||||
|
let fa: Int = geometry_free(ga)
|
||||||
|
let gb: Geometry = geometry_new(1)
|
||||||
|
let sb: Int = geometry_set(gb, 0, 0.0)
|
||||||
|
let ib: Int = manifold_add(m, "decay", "envelope", gb)
|
||||||
|
let fb: Int = geometry_free(gb)
|
||||||
|
let e: Int = manifold_relate(m, "onset", "decays_into", "decay", 0.7)
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
|
// #144's ACTUAL CONTRACT, preserved verbatim as a control: a realizer that
|
||||||
|
// returns one vector for the whole signal. This is not a strawman — it is what
|
||||||
|
// the merged primitive asked realizers to be. It must now transduce NOTHING.
|
||||||
|
fn fingerprint_realizer(signal: String) -> Geometry {
|
||||||
|
let g: Geometry = geometry_new(4)
|
||||||
|
let n: Int = str_len(signal)
|
||||||
|
let a: Int = geometry_set(g, 0, int_to_float(n))
|
||||||
|
let b: Int = geometry_set(g, 1, int_to_float(n * 2))
|
||||||
|
g
|
||||||
|
}
|
||||||
|
|
||||||
|
// A realizer returning something that is not a value at all.
|
||||||
|
fn bogus_realizer(signal: String) -> Manifold {
|
||||||
|
return 12345
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
// Geometry — unchanged from #144. A vector is the right representation for a
|
||||||
|
// COMPONENT; it was only ever wrong as the representation of a whole signal.
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
test "geometry-is-a-value-with-its-own-width" {
|
||||||
|
let g: Geometry = geometry_new(8)
|
||||||
|
let live: Int = geometry_is(g)
|
||||||
|
assert live > 0, "geometry_new returns a live Geometry"
|
||||||
|
let d: Int = geometry_dim(g)
|
||||||
|
assert d == 8, "a Geometry carries its own width"
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
assert freed > 0, "geometry_free reports what it did"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "geometry-rejects-nonsense-without-an-arbitrary-bound" {
|
||||||
|
let zero: Geometry = geometry_new(0)
|
||||||
|
let z: Int = geometry_is(zero)
|
||||||
|
assert z < 1, "dim 0 is not a geometry"
|
||||||
|
let neg: Geometry = geometry_new(-4)
|
||||||
|
let n: Int = geometry_is(neg)
|
||||||
|
assert n < 1, "negative dim is not a geometry"
|
||||||
|
let nd: Int = geometry_dim(0)
|
||||||
|
assert nd < 1, "geometry_dim of a non-geometry is 0"
|
||||||
|
let ni: Int = geometry_is(0)
|
||||||
|
assert ni < 1, "geometry_is of a non-geometry is 0"
|
||||||
|
let nf: Int = geometry_free(0)
|
||||||
|
assert nf < 1, "geometry_free of a non-geometry is a no-op"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "geometry-components-round-trip" {
|
||||||
|
let g: Geometry = geometry_new(3)
|
||||||
|
let s0: Int = geometry_set(g, 0, 1.5)
|
||||||
|
let s1: Int = geometry_set(g, 1, -2.5)
|
||||||
|
assert s0 > 0, "set in range succeeds"
|
||||||
|
let oob: Int = geometry_set(g, 3, 9.0)
|
||||||
|
assert oob < 1, "set out of range is refused, not silently dropped"
|
||||||
|
let v0: Float = geometry_get(g, 0)
|
||||||
|
let d0: Float = v0 - 1.5
|
||||||
|
assert d0 < 0.001, "component 0 round-trips"
|
||||||
|
assert d0 > -0.001, "component 0 round-trips"
|
||||||
|
let v1: Float = geometry_get(g, 1)
|
||||||
|
let d1: Float = v1 + 2.5
|
||||||
|
assert d1 < 0.001, "component 1 round-trips (negative)"
|
||||||
|
assert d1 > -0.001, "component 1 round-trips (negative)"
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "hex-is-an-edge-adapter-and-derives-its-own-width" {
|
||||||
|
let g: Geometry = geometry_from_f32le_hex("0000803f00000040")
|
||||||
|
let live: Int = geometry_is(g)
|
||||||
|
assert live > 0, "valid hex decodes to a Geometry"
|
||||||
|
let hex_dim: Int = geometry_dim(g)
|
||||||
|
assert hex_dim == 2, "width is DERIVED from the input, never supplied"
|
||||||
|
let back: String = geometry_to_f32le_hex(g)
|
||||||
|
assert str_eq(back, "0000803f00000040"), "hex round-trips exactly"
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "hex-rejects-malformed-input" {
|
||||||
|
let empty: Geometry = geometry_from_f32le_hex("")
|
||||||
|
let e: Int = geometry_is(empty)
|
||||||
|
assert e < 1, "empty hex is not a geometry"
|
||||||
|
let ragged: Geometry = geometry_from_f32le_hex("0000803f0000")
|
||||||
|
let r: Int = geometry_is(ragged)
|
||||||
|
assert r < 1, "length not a multiple of 8 is refused"
|
||||||
|
let nonhex: Geometry = geometry_from_f32le_hex("zzzzzzzz")
|
||||||
|
let nh: Int = geometry_is(nonhex)
|
||||||
|
assert nh < 1, "non-hex characters are refused"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "norm-lets-a-caller-check-a-realizer-emitted-signal" {
|
||||||
|
let g: Geometry = geometry_new(2)
|
||||||
|
let z: Float = geometry_norm(g)
|
||||||
|
assert z < 0.001, "a fresh geometry is zero — norm says so"
|
||||||
|
let s0: Int = geometry_set(g, 0, 3.0)
|
||||||
|
let s1: Int = geometry_set(g, 1, 4.0)
|
||||||
|
let nrm: Float = geometry_norm(g)
|
||||||
|
let dnorm: Float = nrm - 5.0
|
||||||
|
assert dnorm < 0.001, "3-4-5: norm is 5"
|
||||||
|
assert dnorm > -0.001, "3-4-5: norm is 5"
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
// Manifold — the corrected result of a transduction
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
test "a-manifold-is-a-value-that-holds-parts-and-relations" {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live > 0, "manifold_new returns a live Manifold"
|
||||||
|
let fresh_sz: Int = manifold_size(m)
|
||||||
|
assert fresh_sz == 0, "a fresh manifold has no components"
|
||||||
|
let fresh_rc: Int = manifold_rel_count(m)
|
||||||
|
assert fresh_rc == 0, "a fresh manifold has no relations"
|
||||||
|
let freed: Int = manifold_free(m)
|
||||||
|
assert freed > 0, "manifold_free reports what it did"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "manifold-accessors-are-total" {
|
||||||
|
let ni2: Int = manifold_is(0)
|
||||||
|
assert ni2 < 1, "manifold_is of a non-manifold is 0"
|
||||||
|
let ns: Int = manifold_size(0)
|
||||||
|
assert ns < 1, "manifold_size of a non-manifold is 0"
|
||||||
|
let nf2: Int = manifold_free(0)
|
||||||
|
assert nf2 < 1, "manifold_free of a non-manifold is a no-op"
|
||||||
|
let k: String = manifold_key(0, 0)
|
||||||
|
assert str_eq(k, ""), "manifold_key of a non-manifold is empty, never a crash"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "components-are-addressed-by-key-not-by-index" {
|
||||||
|
// The key is what survives persistence: a component becomes a node, and it
|
||||||
|
// is separately groundable precisely because it is separately NAMED.
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let g: Geometry = geometry_new(1)
|
||||||
|
let s: Int = geometry_set(g, 0, 7.0)
|
||||||
|
let first_idx: Int = manifold_add(m, "rhythm", "temporal", g)
|
||||||
|
assert first_idx == 0, "the first component is index 0"
|
||||||
|
let found_idx: Int = manifold_index_of(m, "rhythm")
|
||||||
|
assert found_idx == 0, "a component is found by its key"
|
||||||
|
let missing: Int = manifold_index_of(m, "never_added")
|
||||||
|
assert missing < 0, "an unknown key resolves to -1, not to component 0"
|
||||||
|
let role: String = manifold_role(m, 0)
|
||||||
|
assert str_eq(role, "temporal"), "a component carries what KIND of part it is"
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "a-duplicate-key-is-refused-because-addressing-must-be-unambiguous" {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let g: Geometry = geometry_new(1)
|
||||||
|
let ok_idx: Int = manifold_add(m, "pitch", "spectral", g)
|
||||||
|
assert ok_idx == 0, "first add succeeds"
|
||||||
|
let dup: Int = manifold_add(m, "pitch", "spectral", g)
|
||||||
|
assert dup < 0, "two components answering to one name is not an addressing scheme"
|
||||||
|
let dup_sz: Int = manifold_size(m)
|
||||||
|
assert dup_sz == 1, "and the duplicate did not land"
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "a-part-with-no-geometry-is-not-a-part" {
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let bad: Int = manifold_add(m, "ghost", "none", 0)
|
||||||
|
assert bad < 0, "a non-Geometry is refused as a component"
|
||||||
|
let empty_key: Int = manifold_add(m, "", "none", geometry_new(1))
|
||||||
|
assert empty_key < 0, "an unaddressable component is refused"
|
||||||
|
let none_sz: Int = manifold_size(m)
|
||||||
|
assert none_sz < 1, "nothing landed"
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "an-edge-to-a-nonexistent-endpoint-is-refused-not-dropped" {
|
||||||
|
// A decomposition that silently loses edges is indistinguishable from one
|
||||||
|
// that never had them.
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let g: Geometry = geometry_new(1)
|
||||||
|
let a: Int = manifold_add(m, "here", "part", g)
|
||||||
|
let dangling: Int = manifold_relate(m, "here", "points_at", "nowhere", 0.5)
|
||||||
|
assert dangling < 1, "an edge to an unknown target is refused"
|
||||||
|
let backwards: Int = manifold_relate(m, "nowhere", "points_at", "here", 0.5)
|
||||||
|
assert backwards < 1, "an edge from an unknown source is refused"
|
||||||
|
let dang_rc: Int = manifold_rel_count(m)
|
||||||
|
assert dang_rc < 1, "and no relation was recorded"
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "a-component-owns-its-geometry-independently-of-the-caller" {
|
||||||
|
// manifold_add COPIES. Freeing the caller's vector must not disturb the
|
||||||
|
// component, or a decomposition would be unusable the moment it was built.
|
||||||
|
let m: Manifold = manifold_new()
|
||||||
|
let g: Geometry = geometry_new(2)
|
||||||
|
let s0: Int = geometry_set(g, 0, 42.0)
|
||||||
|
let idx: Int = manifold_add(m, "part", "kind", g)
|
||||||
|
let freed: Int = geometry_free(g)
|
||||||
|
assert freed > 0, "the caller freed its own vector"
|
||||||
|
let back: Geometry = manifold_geometry(m, 0)
|
||||||
|
let live: Int = geometry_is(back)
|
||||||
|
assert live > 0, "the component still has geometry"
|
||||||
|
let v: Float = geometry_get(back, 0)
|
||||||
|
let dv: Float = v - 42.0
|
||||||
|
assert dv < 0.001, "and it is the right geometry"
|
||||||
|
assert dv > -0.001, "and it is the right geometry"
|
||||||
|
let fb: Int = geometry_free(back)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
// transduce — signal in, SUBGRAPH out
|
||||||
|
// ═══════════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
test "a-realizer-declared-in-el-is-a-first-class-realizer" {
|
||||||
|
// THE CLAIM, unchanged from #144: tone_realizer is an ordinary El function.
|
||||||
|
// It is not in the runtime and the compiler knows nothing about it.
|
||||||
|
// Registering it by name is enough to make it the organ for a modality.
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
assert reg > 0, "an El fn registers as a realizer by name"
|
||||||
|
let has: Int = realizer_has("tone")
|
||||||
|
assert has > 0, "the modality now has an organ"
|
||||||
|
|
||||||
|
let m: Manifold = transduce("CEG", "tone")
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live > 0, "transduce returns a real Manifold"
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "transduction-decomposes-a-signal-into-parts" {
|
||||||
|
// THE CENTRAL CLAIM. "CEG" is three notes. What comes back is not one
|
||||||
|
// vector standing for a chord — it is five addressable parts (three notes,
|
||||||
|
// two intervals) and six relations. A fingerprint has one part by
|
||||||
|
// construction and could not express this at any width.
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let m: Manifold = transduce("CEG", "tone")
|
||||||
|
|
||||||
|
let ceg_sz: Int = manifold_size(m)
|
||||||
|
assert ceg_sz == 5, "three notes and two intervals are five distinct parts"
|
||||||
|
let ceg_rc: Int = manifold_rel_count(m)
|
||||||
|
assert ceg_rc == 6, "and the parts stand in six stated relations"
|
||||||
|
|
||||||
|
// Every part is independently addressable BY NAME.
|
||||||
|
let n0: Int = manifold_index_of(m, "note:0")
|
||||||
|
assert n0 > -1, "the first note is addressable on its own"
|
||||||
|
let n2: Int = manifold_index_of(m, "note:2")
|
||||||
|
assert n2 > -1, "so is the third"
|
||||||
|
let iv: Int = manifold_index_of(m, "interval:0-1")
|
||||||
|
assert iv > -1, "so is the interval between the first two"
|
||||||
|
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "each-part-carries-its-own-geometry" {
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let m: Manifold = transduce("CEG", "tone")
|
||||||
|
|
||||||
|
// 'C' is 67. The note component's geometry is the note's, not the chord's.
|
||||||
|
let note_i: Int = manifold_index_of(m, "note:0")
|
||||||
|
let gn: Geometry = manifold_geometry(m, note_i)
|
||||||
|
let note_dim: Int = geometry_dim(gn)
|
||||||
|
assert note_dim == 2, "a note component has the width its realizer gave it"
|
||||||
|
let pitch: Float = geometry_get(gn, 0)
|
||||||
|
let dpitch: Float = pitch - 67.0
|
||||||
|
assert dpitch < 0.001, "and it is C, so the signal reached the El realizer"
|
||||||
|
assert dpitch > -0.001, "and it is C, so the signal reached the El realizer"
|
||||||
|
|
||||||
|
// Parts may have DIFFERENT widths. A single vector per signal cannot
|
||||||
|
// represent parts of unequal dimensionality at all.
|
||||||
|
let iv_i: Int = manifold_index_of(m, "interval:0-1")
|
||||||
|
let gi: Geometry = manifold_geometry(m, iv_i)
|
||||||
|
let iv_dim: Int = geometry_dim(gi)
|
||||||
|
assert iv_dim == 1, "an interval component has its own, different width"
|
||||||
|
|
||||||
|
let f1: Int = geometry_free(gn)
|
||||||
|
let f2: Int = geometry_free(gi)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "the-relations-are-content-no-single-part-carries" {
|
||||||
|
// THE POINT OF THE WHOLE CHANGE. C->E is two semitones. That "2" is not a
|
||||||
|
// property of C and not a property of E; it exists only BETWEEN them. A
|
||||||
|
// representation with no relations cannot hold it, which is why collapsing
|
||||||
|
// a signal to one vector does not merely lose resolution — it loses a
|
||||||
|
// category of content.
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let m: Manifold = transduce("CEG", "tone")
|
||||||
|
|
||||||
|
let step_i: Int = manifold_index_of(m, "interval:0-1")
|
||||||
|
let gi: Geometry = manifold_geometry(m, step_i)
|
||||||
|
let step: Float = geometry_get(gi, 0)
|
||||||
|
let dstep: Float = step - 2.0
|
||||||
|
assert dstep < 0.001, "C to E is two semitones"
|
||||||
|
assert dstep > -0.001, "C to E is two semitones"
|
||||||
|
|
||||||
|
// And the interval is WIRED to both endpoints, so the structure says which
|
||||||
|
// two things it is the interval between.
|
||||||
|
let spans: Int = 0
|
||||||
|
let span_rc: Int = manifold_rel_count(m)
|
||||||
|
let k: Int = 0
|
||||||
|
while k < span_rc {
|
||||||
|
let rn: String = manifold_rel_name(m, k)
|
||||||
|
let rf: String = manifold_rel_from(m, k)
|
||||||
|
if str_eq(rn, "spans") {
|
||||||
|
if str_eq(rf, "interval:0-1") { spans = spans + 1 }
|
||||||
|
}
|
||||||
|
k = k + 1
|
||||||
|
}
|
||||||
|
assert spans == 2, "the interval is related to both notes it spans"
|
||||||
|
|
||||||
|
let fg: Int = geometry_free(gi)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "relation-weight-is-the-grounding-carried-on-the-edge" {
|
||||||
|
// correspondence-and-censorship.md §1: grounding is an attribute of the
|
||||||
|
// edge and it IS the weight — one quantity, not a score computed beside
|
||||||
|
// it. A realizer states a relation and its weight is the claim.
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let m: Manifold = transduce("CE", "tone")
|
||||||
|
|
||||||
|
let ce_rc: Int = manifold_rel_count(m)
|
||||||
|
assert ce_rc == 3, "one interval yields two spans and one ordering"
|
||||||
|
|
||||||
|
let found_w: Int = 0
|
||||||
|
let k: Int = 0
|
||||||
|
while k < ce_rc {
|
||||||
|
let rn: String = manifold_rel_name(m, k)
|
||||||
|
if str_eq(rn, "sounds_before") {
|
||||||
|
let w: Float = manifold_rel_weight(m, k)
|
||||||
|
let dw: Float = w - 0.8
|
||||||
|
if dw < 0.001 { if dw > -0.001 { found_w = found_w + 1 } }
|
||||||
|
}
|
||||||
|
k = k + 1
|
||||||
|
}
|
||||||
|
assert found_w == 1, "the ordering relation carries the weight its realizer stated"
|
||||||
|
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "distinct-signals-decompose-differently" {
|
||||||
|
let reg: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let m2: Manifold = transduce("CE", "tone")
|
||||||
|
let m3: Manifold = transduce("CEG", "tone")
|
||||||
|
let two_sz: Int = manifold_size(m2)
|
||||||
|
let three_sz: Int = manifold_size(m3)
|
||||||
|
assert two_sz == 3, "two notes decompose into two notes and one interval"
|
||||||
|
assert three_sz == 5, "three notes decompose into three notes and two intervals"
|
||||||
|
// Structure differs, not just position: fingerprints of a two-note and a
|
||||||
|
// three-note signal have identical shape and differ only numerically.
|
||||||
|
let two_rc: Int = manifold_rel_count(m2)
|
||||||
|
let three_rc: Int = manifold_rel_count(m3)
|
||||||
|
assert two_rc < three_rc, "and the relational structure itself differs"
|
||||||
|
let f2: Int = manifold_free(m2)
|
||||||
|
let f3: Int = manifold_free(m3)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "the-registry-keys-on-modality" {
|
||||||
|
let r1: Int = realizer_register("tone", "tone_realizer")
|
||||||
|
let rp: Int = realizer_register("pulse", "pulse_realizer")
|
||||||
|
assert rp > 0, "a second modality registers independently"
|
||||||
|
let mt: Manifold = transduce("CEG", "tone")
|
||||||
|
let mp: Manifold = transduce("CEG", "pulse")
|
||||||
|
let tone_sz: Int = manifold_size(mt)
|
||||||
|
let pulse_sz: Int = manifold_size(mp)
|
||||||
|
assert tone_sz == 5, "tone still routes to its own realizer"
|
||||||
|
assert pulse_sz == 2, "pulse routes to a different realizer, with its own decomposition"
|
||||||
|
let onset: Int = manifold_index_of(mp, "onset")
|
||||||
|
assert onset > -1, "and to that realizer's own component vocabulary"
|
||||||
|
let f1: Int = manifold_free(mt)
|
||||||
|
let f2: Int = manifold_free(mp)
|
||||||
|
}
|
||||||
|
|
||||||
|
test "no-organ-is-reported-as-no-organ" {
|
||||||
|
// A modality with no realizer must transduce to NOTHING. It must never
|
||||||
|
// fall back to embedding a description of the signal and calling that
|
||||||
|
// perception — that silent substitution is the original defect.
|
||||||
|
let has: Int = realizer_has("echolocation")
|
||||||
|
assert has < 1, "unregistered modality has no organ"
|
||||||
|
let m: Manifold = transduce("anything", "echolocation")
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live < 1, "no realizer means no manifold, not a fake one"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "registration-of-an-unresolvable-name-fails-loudly" {
|
||||||
|
let bad: Int = realizer_register("ghost", "no_such_function_anywhere")
|
||||||
|
assert bad < 1, "an unresolvable realizer name is a registration failure"
|
||||||
|
let has: Int = realizer_has("ghost")
|
||||||
|
assert has < 1, "and nothing gets registered"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "a-fingerprint-realizer-transduces-nothing" {
|
||||||
|
// THE SUPERSESSION OF #144, asserted directly. fingerprint_realizer is
|
||||||
|
// exactly what the merged primitive asked a realizer to be: signal in, one
|
||||||
|
// Geometry out. It resolves, so registration succeeds — the organ is
|
||||||
|
// present. But it does not decompose, so it does not transduce.
|
||||||
|
//
|
||||||
|
// This is a deliberate hard failure. "No organ" and "an organ that only
|
||||||
|
// fingerprints" must not be indistinguishable, which is the same
|
||||||
|
// distinction realizer_register already draws between an absent and a
|
||||||
|
// broken organ. A modality with genuinely one part says so with
|
||||||
|
// manifold_single, and is then visibly a size-1 manifold.
|
||||||
|
let reg: Int = realizer_register("fingerprint", "fingerprint_realizer")
|
||||||
|
assert reg > 0, "the symbol resolves, so registration succeeds"
|
||||||
|
let m: Manifold = transduce("x", "fingerprint")
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live < 1, "a single vector is not a transduction"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "a-realizer-returning-nonsense-transduces-nothing" {
|
||||||
|
let reg: Int = realizer_register("bogus", "bogus_realizer")
|
||||||
|
assert reg > 0, "the symbol resolves, so registration succeeds"
|
||||||
|
let m: Manifold = transduce("x", "bogus")
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live < 1, "a non-Manifold return transduced nothing"
|
||||||
|
}
|
||||||
|
|
||||||
|
test "the-one-part-case-is-a-size-one-manifold-not-a-bare-vector" {
|
||||||
|
// Some modalities really do have one part. That is a manifold of size 1 —
|
||||||
|
// a special case of decomposition, not a parallel path back to a
|
||||||
|
// fingerprint. Anything reading it still asks manifold_size and still gets
|
||||||
|
// a real answer, and a second part can be added later without changing the
|
||||||
|
// type of the thing.
|
||||||
|
let g: Geometry = geometry_new(3)
|
||||||
|
let s: Int = geometry_set(g, 0, 5.0)
|
||||||
|
let m: Manifold = manifold_single("level", "scalar", g)
|
||||||
|
let live: Int = manifold_is(m)
|
||||||
|
assert live > 0, "manifold_single yields a real Manifold"
|
||||||
|
let one_sz: Int = manifold_size(m)
|
||||||
|
assert one_sz == 1, "of size one — visibly degenerate, not hidden"
|
||||||
|
let idx: Int = manifold_index_of(m, "level")
|
||||||
|
assert idx == 0, "and its one part is still addressable by name"
|
||||||
|
let f: Int = geometry_free(g)
|
||||||
|
let fm: Int = manifold_free(m)
|
||||||
|
}
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
fn getstr(x: String) -> String { return x }
|
||||||
|
fn getint(x: Int) -> Int { return x }
|
||||||
|
fn ok(label: String) -> Void { println("ok " + label) }
|
||||||
|
fn bad(label: String) -> Void { println("FAIL " + label) }
|
||||||
|
|
||||||
|
let s1: String = "hello"
|
||||||
|
let s2: String = "hello"
|
||||||
|
let s3: String = "world"
|
||||||
|
let i1: Int = 5
|
||||||
|
let i2: Int = 5
|
||||||
|
let i3: Int = 9
|
||||||
|
|
||||||
|
if "abc" == "abc" { ok("str literal eq") } else { bad("str literal eq") }
|
||||||
|
if "abc" == "xyz" { bad("str literal ne") } else { ok("str literal ne") }
|
||||||
|
if s1 == s2 { ok("str var eq") } else { bad("str var eq") }
|
||||||
|
if s1 == s3 { bad("str var ne") } else { ok("str var ne") }
|
||||||
|
if getstr("hi") == "hi" { ok("str call vs literal") } else { bad("str call vs literal") }
|
||||||
|
if s1 == getstr("hello") { ok("str var vs call") } else { bad("str var vs call") }
|
||||||
|
if s1 == getstr("nope") { bad("str var vs call ne") } else { ok("str var vs call ne") }
|
||||||
|
if i1 == i2 { ok("int var eq") } else { bad("int var eq") }
|
||||||
|
if i1 == i3 { bad("int var ne") } else { ok("int var ne") }
|
||||||
|
if getint(5) == i1 { ok("int call vs var") } else { bad("int call vs var") }
|
||||||
|
if getint(9) == i1 { bad("int call vs var ne") } else { ok("int call vs var ne") }
|
||||||
|
if s1 != s3 { ok("str NOTEQ") } else { bad("str NOTEQ") }
|
||||||
|
if s1 != s2 { bad("str NOTEQ same") } else { ok("str NOTEQ same") }
|
||||||
|
if i1 != i3 { ok("int NOTEQ") } else { bad("int NOTEQ") }
|
||||||
|
if getint(9) != i1 { ok("int call NOTEQ") } else { bad("int call NOTEQ") }
|
||||||
|
println("done")
|
||||||
Reference in New Issue
Block a user