Prediction 3 was FALSE. I expected refusal to be impossible through the seam
because the entry indirection discarded its return. One line:
{ el_val_t __s = el_seam_run(EL_STR(f), 0, 0); if (__s) return __s; }
work() returns 7; bound to a refusing construct AFTER the build it returns 42.
So three of the five compile-time kinds are runtime-bindable: entry injection,
exit injection, and refusal. wraps_body needs invocation control and
prohibits_outside is compile-time by nature.
104/104 native compiler tests pass.
The other half of a boundary: not what runs when something crosses, but what
may not cross at all. It was two string literals in vbd_is_restricted_name and
one #error in cg_fn — one prohibition, uneditable without a compiler release.
@decorator("prohibits_outside", "raw_sql")
fn repository() {}
fn sneaky() -> Int { raw_sql("DROP") }
// #error "boundary violation: raw_sql may only be called from an
// @repository fn, but 'sneaky' is not one"
The recursive matcher is parameterised through a state key rather than by
threading an argument through every branch of the walk — the mechanism codegen
already uses for __match_counter and __if_expr_counter. Each prohibition is
checked in its own turn, so the owning construct is known by construction and
the diagnostic names it instead of hardcoding one rule's wording.
PREDICTIONS AND RESULTS
1 the 3 duplicated uniqueness rules are textually identical TRUE
2 a declared prohibition reproduces @manager's #error TRUE
3 existing output byte-identical TRUE
4 a program can declare its own prohibition TRUE
5 fixpoint holds TRUE
I misread result 2 on first pass: a @manager fn calling dharma_emit still
emitted one #error, which looked like a failure. It is the CAPABILITY-tier rule
at codegen.el:2578, a separate prohibition system, and it fires identically on
the pre-change compiler.
MEASURED DEFECTS STILL OPEN
- two independent prohibition systems (VBD constructs, capability tiers);
only the first is declarable
- 3 uniqueness rules written 6 times, once per codegen path, kept in sync by
hand and identical today
102/102 native compiler tests pass, compiler self-hosts byte-identically.
§6 records 62 persist-after-mutate sites, 10 auth-per-route, and
index-after-append that failed at 9 of 9 — every one an obligation at a
crossing that decayed into "remember to do this afterwards." An obligation a
human must remember is not an obligation, and the 9-of-9 figure is what that
costs.
@decorator("injects_at_exit", "persist_now")
fn durable() {}
The body moves into a static helper and the visible fn becomes a wrapper, so
EARLY RETURNS pass through the exit injection. Emitting it only before the
fall-through return would have silently missed every early return — the exact
failure class this seam exists to remove. Fns with no exit construct emit
byte-identically to before.
Three independent constructs now compose on one fn, none known to the compiler:
el_val_t mutate(el_val_t k) {
{ el_val_t __g = my_auth(EL_STR("mutate"), EL_STR("authenticate")); if (__g) return __g; }
engram_boundary_beat(EL_STR("mutate"), EL_STR("manager"));
el_val_t __r = __el_body_mutate(k);
persist_now(EL_STR("mutate"), EL_STR("durable"), __r);
return __r;
}
Guard, then entry, then body, then exit. §5.2 asked whether `hold` is one
construct or two; the implementation answers one construct with two faces,
selected by declared kind rather than by two mechanisms.
Verified: existing output byte-identical, compiler self-hosts byte-identically,
early returns pass through the exit, ordering holds under composition. 98/98
native compiler tests pass.
@authenticate (6 uses), @authorize (3), @rate_limit (3) and @validate (2)
parsed, attached, and compiled to nothing. Fourteen applications that read as
protection and emitted no instruction — a function decorated @authenticate
compiled byte-identically to an undecorated one.
The missing capability was not authentication. It was that a construct could
observe a boundary but never refuse one. injects_at_entry discards the target's
result; there was no form in which a construct could say no.
@decorator("guards_at_entry", "my_auth")
fn authenticate() {}
@authenticate
@authorize
fn handler() -> String { ... }
emits, at entry:
{ el_val_t __g = my_auth(EL_STR("handler"), EL_STR("authenticate")); if (__g) return __g; }
{ el_val_t __g = my_roles(EL_STR("handler"), EL_STR("authorize")); if (__g) return __g; }
Guards precede injections because a refused call must not report a crossing,
and every guard runs where the topmost injecting construct wins — refusal is
not a role, so it does not follow the role convention.
The compiler still knows nothing about auth. The program points the construct
at its own function, which is where that decision belongs.
Verified: existing @manager/@accessor output byte-identical, compiler
self-hosts byte-identically, guards stack in declaration order and emit before
the beat. 94/94 native compiler tests pass.
codegen called fn_has_decorator for exactly three names — manager, accessor,
route. Twelve others parsed, attached as {name,args}, and compiled to nothing,
including four that look like protection: @authenticate (6 uses), @authorize
(3), @rate_limit (3), @validate (2). The cause was not that the branches were
untidy. A construct had nothing to BE, so its meaning had nowhere to live
except the emitter, and every construct was therefore a compiler edit.
A name -> injection table would have moved the enumeration twenty lines up
without removing it. So the construct now carries its own meaning:
@decorator("injects_at_entry", "engram_boundary_beat")
fn audited() {}
@audited
fn risky_op() -> Int { ... } // gets the beat, attributed to "audited"
scan_declared_decorators is a token-level pre-pass beside scan_routes, forced
by streaming codegen having no whole-program AST. manager and accessor are
seeded as the compiled-in core — the fixedSelf shape from substrate.go: a
complete fallback exists, declaration is enrichment.
This is the injection half of the seam only. The prohibition half (@manager's
#error on dharma_emit) stays hardcoded, because "which calls may appear inside
this boundary" is a query over program structure and there is nothing yet to
ask.
Verified three ways: emitted C for existing @manager/@accessor code is
byte-identical to the hardcoded path; a construct with a name the compiler has
never heard of injects correctly; the compiler self-hosts byte-identically.
90/90 native compiler tests pass.
The beat reported which function crossed a boundary, never which decorator
put the beat there. So the graph accumulated boundary events with no
attribution, and no construct could be measured — "is this decorator
earning its keep" stayed an argument instead of a traversal.
engram_boundary_beat now takes the construct and carries it on the bus as
{"construct":"..."}. The injection point, the beat, and the accumulation
already existed; only the attribution was missing.
Also pins a known defect as a test: codegen calls fn_has_decorator for
exactly three names (manager, accessor, route). Twelve others parse, attach,
and compile to nothing — including @authenticate (6 uses), @authorize (3),
@rate_limit (3) and @validate (2), which look like protection and are not.
decorator-authenticate-compiles-to-nothing asserts that @authenticate emits
byte-identical C to no decorator at all, so fixing it will be a visible flip.
Verified: compiler self-hosts byte-identically, 86/86 native compiler tests
pass, emitted C carries the construct for both @manager and @accessor.
#144 moved transduction into the language and got the dispatch right. It got
the result type wrong: transduce(signal, modality) -> Geometry yields one
vector per signal, and one vector is a fingerprint. A fingerprint can be
matched and ranked; that is all. It cannot be decomposed, cannot have one part
grounded while another is not, and cannot be contradicted in one part while
holding in another, because it has no parts.
A song is not a point. It decomposes into pitch, interval, rhythm, harmonic
function -- components, each with its own geometry, plus the relations among
them. The song IS the structure of the relations.
transduce now returns a Manifold: named components carrying geometry, and
typed weighted relations between them. Signal in, subgraph out.
Components are addressed by key, never by index, because the key is what
survives persistence -- a component becomes a node and is separately groundable
precisely because it is separately named. Relation weight IS the grounding
(correspondence-and-censorship.md 1), so a realizer's relations arrive already
grounded and there is no score computed beside them.
#141 let signal enter as geometry and it worked, but it was placed at the
CONSUMER and said so in its own commit message. This is the correction.
Three defects, all of them placement:
1. It sat in the engram. Ingest is a LANGUAGE concern — every el program
touching any modality needs it, and the engram is merely one el program
that happens to hold a graph. The geometry surface is now defined in
el_runtime.c immediately ABOVE the engram section and depends on nothing
inside it. Delete the entire engram and geometry still enters el.
2. It marshalled the vector as a hex STRING, because el had no first-class
geometry value — which reintroduced text as the TRANSPORT medium one layer
below the problem being fixed. Geometry is now an el value: a magic-tagged
heap object carried in el_val_t, same discipline as List/Map. Hex survives
only as an adapter at the edge, which is all an encoding should ever be.
3. It needed an arbitrary `dim <= 8192` bound purely to size an allocation
from a caller's CLAIM about a string's length. A value carries its own
width, so the width is derived and never asserted. The bound is gone, not
raised — there is nothing left to validate.
Language surface, none of it engram-prefixed: geometry_new / _dim / _is /
_get / _set / _norm / _free, geometry_from_f32le_hex + geometry_to_f32le_hex
as the wire adapters, realizer_register(modality, fn_name), realizer_has, and
transduce(signal, modality) -> Geometry.
REALIZERS ARE DECLARABLE IN EL. This is the part that makes the move real
rather than nominal: registration resolves a name with dlsym against the
running binary, the identical mechanism http_set_handler already relies on,
because every el `fn name(...)` compiles to a global C symbol with that exact
name. So an ordinary el function IS a realizer and a new modality needs no
runtime patch. Verified end to end in lang/examples/transduce.el: an el-defined
tone_realizer is registered by name, transduce dispatches to it, and the
signal demonstrably reaches it (distinct signals produce distinct geometry).
A modality with no realizer transduces to NOTHING. There is deliberately no
built-in realizer, not even for text — silently embedding a description of a
signal and calling that perception is the exact defect this ends.
engram/src/server.el is migrated: POST /api/nodes decodes "emb" hex exactly
once, at the edge, into a Geometry, and everything below that line moves
geometry. The wire is unchanged because production clients speak it. "dim" is
now an ASSERTION about the vector, not the source of its width; disagreement
is a rejected ingest, not a silent reinterpretation.
#141's engram_node_set_emb becomes a DEPRECATED WRAPPER over
geometry_from_f32le_hex + node_attach_geometry — kept only because the runtime
ships as an SDK asset and a downstream binary may link the symbol. Its exact
contract, negative cases included, is preserved and re-verified.
ingest.el's `fn transduce` is renamed transduce_manifold. Mechanically it had
to yield the name (duplicate C symbol, a hard compile error, measured). But it
was never signal->geometry: it chunks already-extracted content into a node+edge
manifold, one layer up, and had taken the name belonging to the primitive
underneath it. Behaviour unchanged.
PROPERTIES FROM #141 PRESERVED, each re-measured on a scratch engram (:8971,
never prod :8742):
- off-dimension vectors stored but NOT indexed — the HNSW build loop still
filters on n->emb_dim == dim at four sites, so a 64-dim voice vector is
durable and addressable without perturbing the 768-dim canonical index
- geometry makes a node ineligible for embed_backfill: after backfill the
64-dim voice node was still 64-dim while the text control acquired 768
- the create response reports whether geometry landed, and the node document
always emits emb_dim and embedded
Read-back with control and negatives, all verified against a PID-confirmed
fresh binary: geometry node emb_dim=64 embedded=true / emb_set=1; text-only
control emb_dim=0 embedded=false / emb_set=0; malformed hex, ragged length,
and dim-disagreement each emb_set=0.
Two compiler landmines found by reading the generated C rather than trusting a
successful build, both documented at their sites: elc lowers `a == b` to
str_eq unless both operand NAMES are in the per-function int-name set (which
does NOT propagate into nested if-expression blocks — the first cut would have
strcmp'd two integers as pointers on the first geometry-bearing request), and
`+` lowers to string concat when either operand is a user-defined call.
Adds tests/native/test_lexer_scaling.el, the regression gate for el #132.
Both directions are proven on LIVE workloads, not synthetic series:
healthy per-character scan 1821 3251 6007 10422 us -> O(n) PASS
rescan-from-zero (the #132 shape) 922 3667 13524 44792 -> O(n^2) FAIL
A gate only proven to pass is decoration. The quadratic specimen exists so
the gate is proven to FIRE.
Also fixes elb_spread_ok to judge the ASYMPTOTIC TAIL (last three ratios)
rather than the whole sweep. Measured on a genuinely linear scan the ratios
ran 3.37 2.92 1.76 1.65 -- the head looks quadratic because it is cold
cache, the tail is the truth. Whole-sweep spread rejected correct data. A
complexity bound is an asymptotic claim and must be judged asymptotically.
That fix came from the classifier refusing to rubber-stamp my own bad
measurement: it reported INDETERMINATE on an unwarmed sweep rather than
passing it. Warmup is now taken and discarded at every sweep point.
Reverts the == workarounds in test_elbench.el now that el #137 has landed;
the natural form generates no str_eq and all 13 fitter tests stay green.
The workaround remains -- the Plus arm is still open.
fn check(label: String, cond: Bool, want: Bool) -> Void {
if cond == want { ... } -> if (str_eq(cond, want)) SIGSEGV
}
Bool has always been an integer in the value model — type_to_c maps Bool to
"int", and el_runtime.h states "Bool -> el_val_t (0 = false, nonzero = true)".
But Bool names were registered NOWHERE: build_int_names_for_params tracked Int
and Float params, and the `let` path tracked Int and Float bindings. Neither
knew about Bool.
So comparing two Bools fell through to str_eq, which dereferenced 0 or 1 as a
char* and segfaulted immediately.
This is the third instance of one family found tonight, after el #137 (a call
on either side of == poisoned the operator) and el #136 (a missing import
compiled clean). All three are the same shape: something the compiler could not
type, silently handled as a string.
Found while writing #137's own test harness — the first version of that harness
crashed on exactly this, on both the old and new compiler, which is how it
surfaced. A test harness that cannot compare two Bools is a good way to notice.
VERIFIED:
- the harness that segfaulted on every prior compiler (exit 139, no output)
now runs clean: 14 passed, 0 failed
- self-hosting fixpoint byte-identical
- the compiler's own generated C differs by 8 lines — only the intended
registration
- neuron's full soul amalgam regenerates in 424ms, exit 0, BYTE-IDENTICAL
- test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12
Adds tests/runtime/operator_typing_test.el, the 15-case suite from #137, so
this family is covered going forward rather than rediscovered.
Adds el_black_box (inline asm, +r constraint, memory clobber) and
runtime/elbench.el: a growth-curve classifier that gates time AND
allocation-count AND allocation-bytes, failing if any exceeds its
declared curve.
Refusal is a first-class verdict. The classifier REFUSES rather than
classifying when the largest measurement is below the floor, or when a
series is hard-flat across an 8x input range -- the shape produced when
the optimiser deletes the work. Reporting O(1) there would be a
confident answer with nothing behind it. Disagreeing ratios report
INDETERMINATE rather than a guess.
Deviation from DESIGN.md 6.2, stated in the source: uses consecutive
ratios on a mandated geometric sweep rather than least-squares over
candidate curves. Ratios are directly interpretable on a doubling sweep
and need no floating point; the cost is weaker O(n) vs O(n log n)
separation, reported as an ambiguous band rather than guessed.
Documents the counter scope limit: engram_*.c and libcurl malloc are
NOT tracked, so a flat curve over engram/HTTP-dominated work is not
evidence of anything.
13 tests prove the classifier against real measured series from
fitprobe.el -- including that an accumulator's allocation COUNT is
linear while its bytes are quadratic, and that el #132's pure-CPU shape
reads FLAT on both allocation signals and is caught only by time.
el #132's quadratic (strlen per character in str_char_code/str_slice) is
pure CPU and allocates NOTHING. Measured on three controlled specimens:
specimen allocs bytes time
linear 2.00 -> O(n) 2.16 -> O(n) 2.05 -> O(n)
accum 2.00 -> O(n) 3.99 -> O(n2) noisy
compute FLAT FLAT 3.96 -> O(n2)
'compute' is #132's shape. A gate fitting only allocation count and bytes
classifies it FLAT and passes -- it would not have caught the defect it
was created for. The gate now fits time AND count AND bytes, failing if
any exceeds its declared curve.
Also: black_box is mandatory and consuming the result is NOT sufficient.
The first 'compute' reported 0us at every n while returning a correct n2 --
clang closed the loop to a multiply. Only an opaque call restored the curve.
Adds lang/tests/bench/fitprobe.el as the fitter's known-good/known-bad set,
so the classifier is provable without depending on a real bug existing.
Marks DESIGN.md 1.3 stale: test_compiler 3.58s -> 0.03s (119x).
Completes the Phase 1 runner and migrates the 11 test files onto it.
- forward-declare the registry accessors in the test preamble; they are
defined at the end of the unit but the El runner is compiled in between
- eltest.el: explicit trailing return in the void emit_* helpers, which
otherwise lower to 'return println(...)' and fail to compile
- test files import runtime/eltest.el explicitly, using the language's own
textual import mechanism rather than compiler-side auto-injection
- DESIGN.md 6.5: gate on allocation COUNT AND BYTES, not count alone
Verified: self-hosting fixpoint byte-identical (gen2 == gen3). 6 of 11
suites run and report per-test timing. The other 5 fail to COMPILE, and
fail identically under the committed compiler -- pre-existing breakage
this framework makes visible for the first time.
THE BUG. str_char_code() and str_slice() each called strlen() on every
invocation. The lexer walks source one character at a time, so every character
access rescanned the whole remaining input: O(n) per character over n
characters = O(n^2).
el_val_t str_char_code(el_val_t s, el_val_t i) {
...
int64_t n = (int64_t)strlen(str); // <- O(n), every call
if (idx < 0 || idx >= n) return 0;
return str[idx];
}
HOW IT WAS FOUND. Not by reading code — by sampling the running process, which
is the same method that resolved tonight's engram outage after four wrong
theories. A geometric sweep of synthetic sources showed wall-clock rising 3.0x,
3.0x, 4.0x, 4.14x per doubling (converging on 4x = quadratic), and a stack
sample put 779 of 779 samples inside lex(), every one bottoming out in
_platform_strlen via str_char_code and str_slice.
THE FIX. Remember the length instead of recomputing it. The subtlety is
INVALIDATION: El strings are arena-allocated, so a freed pointer can be reused
for a different string at the same address, and a naive pointer-keyed cache
would hand back a stale length and read past the end of the new string —
trading a performance bug for a memory-safety one. So entries carry a
generation, a hit requires pointer AND generation to match, and every path that
frees or mutates a runtime string bumps the generation: el_arena_pop,
seed_request_end, __str_set_char. Stale entries cannot be believed; they miss
and recompute.
MEASURED, same host, same inputs:
n(fns) before after
512 0.10s 0.01s
1024 0.37s 0.02s
2048 1.51s 0.03s 50x
the compiler's own 422 KB source concatenated (DESIGN.md's 3.58s case):
3.55s -> 0.03s 118x
The speedup GROWS with input size, which is the signature of removing a
complexity class rather than a constant factor. After the fix each doubling
adds ~0.01s: linear.
CORRECTNESS, verified rather than assumed:
- byte-identical output on every sweep input (n = 128..2048)
- byte-identical output on the 422 KB compiler concatenation
- byte-identical output on tests/runtime/string_test.el
- self-hosting fixpoint byte-identical
- new tests/runtime/str_cache_test.el: 17 assertions covering bounds, empty
strings, negative indices, slice clamping, distinct strings not sharing a
cached length, 1000 interleaved strings forcing cache-slot collisions, and
a grown string not reporting its old length. All pass.
This is the defect that made dist/soul.c a committed artifact: elc could not run
in CI because it needed 24 GB+ and minutes. It needs neither now.
Cancellation-token control channel checked at every step boundary lets a
coordinator PAUSE/RESUME/REDIRECT/KILL a running worker mid-task instead of
waiting for the whole (possibly wrong) plan to finish. Bounded purviews
mean no half-committed state to unwind on interrupt. Includes a proof
harness (proof.el, run.sh) comparing a broken non-interruptible worker
against the new one under identical kill/redirect/pause timing.
Distinct from the already-preserved swarm-ccr orchestrator (fan-out/
converge dispatch): this is single-worker interruptibility, a
complementary mechanism, not a duplicate.
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.
Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.
WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
bigmerge
will.anderson