ISHIKAWA: el_val_t carries integers AND tagged heap pointers, so "is this a
pointer" is undecidable without checking first. That check was a CONVENTION
every author had to know rather than a GATE they had to pass through, and
looks_like_heap_obj was static -- so every sibling translation unit re-derived
it.
MEASURED, across the five existing tags
geom_of looks_like_heap_obj full guard correct
mfld_of looks_like_heap_obj full guard correct
el_bin_lookup (uintptr_t)p < 4096 floor only reads 8 bytes BACKWARD
el_input_len s ? ... : 0 NULL only strlen's an integer
sha256_hex(50000) -> exit 139, SIGSEGV, compiled clean
PREDICTIONS AND RESULTS
P1 looks_like_heap_obj is static, not exported TRUE
P2 each tagged type re-derives the check TRUE
P3 at least one is missing guard components TRUE (two are)
P6 sha256_hex(<int>) reads out of bounds TRUE
P8 routing el_bin_lookup through the gate fixes it FALSE
P9 the legitimate hash is unchanged TRUE
P11 fixpoint and suites hold TRUE
P8 IS THE USEFUL FAILURE. Guarding the tagged lookup changed nothing --
looks_like_heap_obj(49992) correctly returns 0, el_bin_lookup bails, and then
el_input_len falls through to strlen() on address 50000. The FALLBACK was the
hazard, not the tagged path. A NULL check does not establish that a slot is a
pointer. I would have shipped the wrong fix and called it verified.
A MEASUREMENT DEFECT, fourth today: my first run of the crash reported exit=0,
because $? read head's exit through a pipe rather than the program's. I nearly
recorded a segfault as a clean run. Same shape as grepping only parser.el and
searching by variable name instead of by operation.
AND I PROVED THE HAZARD FROM THE INSIDE. Sixty seconds after diagnosing
`let s: String = 42` as an arbitrary-read primitive, I wrote the identical
defect into el_await -- dereferencing ->magic off an unvalidated slot -- and
only then found the runtime had already made it twice.
el_tagged() is now exported in el_runtime.h. Anything that dereferences a slot
without passing through it is the defect.
105/105 native, 42/42 integration across eight harnesses, fixpoint ok.
The module question ended with a limit: textual inlining destroys file
provenance, so a duplicate-definition message could name the symbol but not the
files. Threading it exposed a bigger absence first.
TOKENS HAD NO POSITION AT ALL. A token was a flat (kind, value) pair, so NO
diagnostic in El could name a place -- every error named a symbol and never a
line. That is the prerequisite the module question was resting on.
THE CHAIN, end to end
lexer counts newlines; tok_append mints (kind, value, line)
parser stride 2 -> 3; tok_line added; FnDef carries its line
codegen records <fn> defines_at:<line>
resolve_imports publishes <file> spans <start> <end> for the combined source
checker maps a combined line back to file:line-within-that-file
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
/tmp/modtest/a.el:1
/tmp/modtest/b.el:1
PREDICTIONS AND RESULTS
P1 15 stride sites, encapsulated in tok_kind/tok_value TRUE, but see below
P2 adding a line field is mechanical TRUE
P3 the lexer must count newlines TRUE
P4 resolve_imports can record per-file line ranges TRUE
P5 the message can then name both files TRUE
P6 token memory grows TRUE, 25.0 -> 33.9 MB (+36%)
FOUR DEFECTS, EACH FOUND BY RUNNING AND NOT BY READING
1. interp_tokens_append_all walks the token list DIRECTLY with its own copy of
the stride. Gen1 built fine and gen2 emitted corrupt C, because the
compiler's own source uses string interpolation. My search missed it because
I grepped for the variable name `tokens`; it is called `dst`/`result`.
Searching by name instead of by shape -- third time today.
2. tok_count in test_compiler.el carried the stride too. I had scoped the search
to compiler sources and it had escaped into the tests.
3. Nested resolve_imports calls accumulated spans into shared state, so each
republished meaningless line ranges under the parent's name. Making the
buffer local fixed it; guarding the WRITE did not, which is what I tried
first.
4. The first working version reported b.el:3 -- the COMBINED line against a
filename that has no line 3. A file:line that does not match the file is
worse than no line at all.
105/105 native, 37/37 integration, fixpoint ok, compiler self-checks clean.
The question is premature, and measuring says why. El's partition is a
FILESYSTEM PATH, not a neighbourhood, and there is no namespacing at all.
MEASURED
import is textual inlining (resolve_imports), guarded against double
inclusion by a __elc_imp__:<path> state key
when a .elh header exists the header is inlined instead and the .el is marked
seen, so symbols resolve at C link time -- so linking IS real, delegated to C
two modules defining `helper` emit two C functions into one translation unit
So linking barely survives the PATH partition. Whether it survives a
neighbourhood partition cannot be asked yet.
A DIAGNOSTIC REGRESSION I CAUSED, found by asking this question. cc does catch
the collision, but reports:
error: redefinition of '__el_body_helper'
error: redefinition of '__env_helper'
error: redefinition of '__thunk_helper'
error: redefinition of 'helper'
The user's own function is FOURTH. The first three are generated symbols
introduced by the unconditional-wrapper pass earlier today -- before it, there
was one clear message. Repaired by catching the collision at El level instead:
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
LIMIT, stated rather than hidden: textual inlining destroys file provenance. By
the time codegen runs there is one source string, so the message can say WHICH
name collides but not which files. Naming a.el and b.el needs provenance
threaded through resolve_imports.
104/104 native, 4/4 definitions_query.sh, the compiler itself reports clean,
fixpoint ok.
Both, at different layers, and the split is the same as everywhere else. The
NUMERAL is convention -- int_to_str was already form 1, because no position
determines that twelve is written 1 then 2 in base ten. The NUMBER is a
position: three things are three things regardless of notation.
But the sharper answer follows from `love = 0`. A bare `3` is a MAGNITUDE WITH
NO AXIS. It is not a position until something gives it a direction, which is
exactly why 3.days needs a calendar and why time_add(t, n, "min") had to carry
its axis as a string.
PREDICTIONS AND RESULTS
P1 numeral = convention, number = position TRUE
P2 a bare literal is dimensionless until context types it TRUE
P3 there is a measurable place where El guesses TRUE
P4 Instant + Int is not caught though Duration + Int is TRUE
P5 the rule catches it TRUE
P6 nothing legitimate in the tree relies on it TRUE
P3/P4 IS THE DEFECT, and it was found by reasoning from the philosophy and then
measured. Duration + Int was refused -- "an Int carries no unit" -- while
let t: Instant = now()
let u: Instant = t + 3
compiled to raw (t + 3) and reported CLEAN. Adding a dimensionless number to a
point is worse than adding it to a displacement: it silently moves the instant
by an unspecified amount. 3 of what? Whatever the representation happens to be,
which is the leak itself. The asymmetry had no justification; the rule was
simply never written.
P6 MATTERED. Two calendar tests looked like Instant + Int:
let later: Instant = i + 1.hour
let later: Instant = base + 15.hours
They are not. `1.hour` lexes to a Duration -- el_duration_from_nanos(1LL *
3600000000000LL) -- and both stay clean. That is the whole answer demonstrated
in one line: t + 3 is refused because 3 has no axis; t + 1.hour is accepted
because .hour supplies one.
104/104 native + 2 new, integration green, fixpoint ok.
Both, at different layers -- and it is the same split as serialization: the
convention is the BASIS, never the ACT.
lexeme -> token `fn` means function-start because someone said so CONVENTION
shape recognition given tokens, which construct is this REGION
source -> structure parsing is transduction onto that basis GEOMETRY
byte traversal something must read them in order IRREDUCIBLE
Three things push the ACT toward region rather than convention: ambiguity
(a * b needs context; a grammar resolves it with the lexer hack, a region by
neighbourhood), error recovery (nearest-region is free), and precedence, which
is ordering along an axis with a conventional parameter.
AND THE SHOULD GATE SAYS NO TO THE OBVIOUS MOVE
Every other table this session moved to data. This one stays code. The keyword
set is CLOSED by the language definition -- it does not leak the way an
allowlist does -- and the lexer runs before the program is understood, so a
program can never declare its own keywords. Externalising it costs file I/O on
every compile and buys nothing. Same verdict as is_digit in ASCII.
WHAT WAS ACTUALLY WRONG: five of 46 keywords were consumed by no parser or
codegen path. sealed, activate, seed, protocol, impl. Each stole an identifier
from users for nothing.
SECOND SILENT MISCOMPILATION OF THE DAY. Using one did not fail to parse:
let seed = 42
let impl = seed + 1
compiled CLEAN -- zero cc errors -- and printed 0 instead of 44. No diagnostic
at any layer. Fixed by removing the five.
A DEFECT IN MY OWN MEASUREMENT, caught before it did damage: my first pass
checked only parser.el and reported `test` as inert too. codegen consumes it at
4135 for --test mode, and the tree has 408 uses. Removing it would have broken
every test in the suite. The measurement was re-run across all four consumers.
100/100 native + 2 new, 31/31 integration, fixpoint ok.
PREDICTIONS AND RESULTS
P1 is_int_call's 35 hardcoded names move to data TRUE
P2 is_int_name stays -- it is annotation propagation TRUE
P3 the dispatch stays -- it is emission TRUE
P4 codegen shrinks ~40 lines TRUE 4507 -> 4469
P5 the design doc's characterisation is WRONG TRUE
P6 the moved data also fixes the bug it exposed TRUE
P5 CORRECTS THE RECORD. el-language-design.md and geometry-vs-code.md both cite
"== lowering to str_eq unless both operand names are in a hardcoded int-name
set -- a literal list of variable names treated as integers" as the paradigm
defect. It is not one. __int_names is populated from TYPE ANNOTATIONS
(param["type"] == "Int"), which is primitive but legitimate type propagation.
The actual defect was is_int_call: 35 hardcoded builtin return types, the same
shape as the temporal 19.
P6 IS A LIVE CORRECTNESS BUG, PRE-EXISTING, NOW FIXED
let a = str_len("hello") // no annotation
let b = str_len("hi")
let c = a + b // -> el_str_concat(a, b) on two integers
Verified identical on the pre-change compiler, so not a regression. It compiled
clean, ran, and printed NOTHING where it should print 7. No error at any layer.
The repair is three lines: an unannotated let takes its type from what the
initialiser returns. The return types were already required for dispatch and
were simply never consulted at the binding site. Moving them into data is what
made the gap visible -- reading the code for eight hours did not.
98/98 native + 2 new, 31/31 integration, fixpoint ok.
The previous pass moved the type DATA and left the judgment inline, which I
stated rather than hid. This finishes it.
PREDICTIONS AND RESULTS
P1 codegen can emit operand-type relations TRUE
"main calls temporal:instant_plus_instant"
P2 the affine rules are a small closed set as data TRUE 6 rules
P3 violations still caught at build time TRUE exit=1
P4 the reporter leaves codegen TRUE 4538 -> 4507
P5 the TIME_TYPE_ERROR placeholder must STAY TRUE
P5 is the boundary of this whole approach. The emitter has to emit SOMETHING
for an illegal expression -- it cannot emit nothing and it cannot decide what
the program meant. So the placeholder is irreducible in the same way the AST
traversal was: what moved is the judgment and the wording, not the fact that
something must be written.
The rules are affine algebra and the set is closed because there are only two
kinds of thing. An Instant is a POINT, a Duration is a DISPLACEMENT: add a
displacement to a point, subtract two points for a displacement, combine
displacements. Nothing else is meaningful, which is why the enumeration in
temporal.rel cannot grow the way an allowlist does.
A defect in my own checker, found by running it: the .rel file uses aligned
columns and my awk assumed a single space, so the message came out with the
rule key still prefixed. Same class as the multi-line header parse in the arity
pass -- formatting assumptions that only fail when you look at the output.
98/98 native, 6/6 temporal_query.sh, fixpoint ok.
This block is structurally unlike the previous four. It does not only
adjudicate, it DISPATCHES: Instant + Duration must become el_instant_add_dur,
LocalDate + Duration must become el_local_date_add_dur. The emitted C depends on
the type answer, so it cannot move to a post-hoc query. Selecting which call to
emit is an emitter's actual job.
PREDICTIONS AND RESULTS
P1 the block conflates dispatch with adjudication TRUE
P2 adjudication can move, dispatch cannot TRUE
P3 this pass shrinks codegen far less than the last TRUE, and worse:
4513 -> 4537, it GREW
by 24 lines
P4 the rules are affine algebra, closed by construction TRUE
P5 no type propagation -- name tracking plus a
hardcoded list of which builtins return which type TRUE, 19 names
P3 is the honest result and it is not spun: moving 19 names into a data file
cost more lines than it saved, because a generic loader is larger than the
enumeration it replaces. The win is not line count. It is that adding a 20th
temporal builtin is now a one-line edit to signatures.rel instead of a compiler
change, and that the data is inspectable.
WHY THE HEADER CANNOT SUPPLY THIS, unlike arity: el_runtime.h declares every
builtin as returning el_val_t, because El has ONE type. That single type is why
the whole seam is cheap and it is exactly why the C boundary cannot say that
now() returns an Instant while unix_seconds() returns an Int. The El-level type
is real and the boundary erases it.
INCOMPLETE, and stated rather than hidden: P2 said adjudication could move to a
query. It has NOT. Violations still emit TIME_TYPE_ERROR inline from the
emitter. Only the type DATA moved. Moving the adjudication needs the operand
types recorded as relations, which is a further pass.
98/98 native, 4/4 temporal_signatures.sh, fixpoint ok.
codegen.el carried builtin_arity(): 344 lines, 300 entries, a hand-maintained
second copy of el_runtime.h.
PREDICTIONS AND RESULTS
P1 the table duplicates the header TRUE 243 shared names
P2 they have already drifted FALSE ZERO drift. The
duplicate had been
maintained correctly.
P3 codegen can emit call-arity relations TRUE
P4 the check becomes a query against the header TRUE
P5 codegen drops to roughly baseline TRUE 4903 -> 4512,
149 BELOW the 4661
it started at
P2 being false is the better result: the table was not WRONG, it was
INCOMPLETE. 110 functions the runtime declares had no entry, so calling them
with the wrong argument count produced no El-level diagnostic at all. Measured:
the old compiler reports 0 arity errors for __http_do_map_to_file(1); the query
reports "takes 5 arguments, called with 1".
Deriving from the header fixes coverage AND makes drift impossible by
construction. 503 signatures, versus 300 entries maintained by hand.
THREE DEFECTS IN MY OWN CHECKER, each found by running it rather than reading it
1. El names and C names differ -- `println` is `__println`. 60 of 500 decls
carry the prefix and codegen owns the mapping; the old table carried both
keys. One rule covers all 60.
2. Multi-line declarations parsed as zero params, so the checker reported
"takes 0" for a function taking 5. A diagnostic with the wrong number in it
is worse than none -- the same shape as the stale caller attribution in the
previous pass.
3. Fixing (2) by joining lines dropped 500 signatures to 334, because a
declaration preceded by a comment no longer started its record. Comments
are stripped first now.
98/98 native, 5/5 arity_query.sh, fixpoint ok.
Capability differs from prohibits_outside in one way that matters: a utility
program cannot be trusted to declare its own restrictions, because it would
declare none. So the policy comes from OUTSIDE the program -- it ships with the
language as data, editable without a compiler release.
tools/check/capabilities.rel 18 names that were string literals in codegen
tools/check/capabilities.sh the query that decides
PREDICTIONS AND RESULTS
P1 codegen emits kind + call graph, drops the 4 name tests TRUE zero #errors
P2 the 18 literals become a data file TRUE
P3 the checker catches capability violations TRUE exit=1
P4 codegen drops ~76 lines TRUE 4963 -> 4881
P5 below the 4661 baseline FALSE ~+230
TWO DEFECTS THE HARNESS FOUND THAT READING WOULD NOT HAVE
1. Calls inside main became invisible. cg_fn returns early for main -- C
provides its own -- so hooking the recording there left every call in main
unrecorded: a blind spot exactly where a program does its work. The old
cap_check_call ran from cg_expr and did see main. Moved the recording to
cg_expr.
2. Caller attribution was stale. __cg_current_fn kept whatever cg_fn set last,
so a violation in main was reported against the previously emitted function.
The test still PASSED, because the violation was detected -- only the name
was wrong, and a diagnostic naming the wrong fn is worse than none. Fixed at
all three main-emission sites; the first patch missed two because the live
path is codegen_streaming.
98/98 native, 7/7 + 4/4 + 5/5 integration, fixpoint ok.
I said prohibition could not move because "a #error has no runtime". That
conflated two separable things: WHEN a violation is detected (build time --
correct, and unchanged) and WHERE the rule and the checker live (the compiler
-- assumed).
A prohibition is a containment relation over the call graph. So codegen now
records what it saw:
sneaky calls raw_sql
allowed calls raw_sql
allowed calls @repository
repository calls prohibits:raw_sql
and tools/check/prohibitions.sh decides, at build time, outside the compiler.
PREDICTIONS AND RESULTS
P1 codegen can emit the call graph it already walks TRUE
P2 the check becomes a query outside the compiler TRUE
P3 all prohibition decisions leave codegen TRUE zero #errors now
P4 violations still caught at build time TRUE exit=1
P5 codegen drops below the 4661 baseline FALSE 4962, +301
P5 is the finding. The TRAVERSAL is irreducible -- you must walk the AST to
find calls, and those ~120 lines do not move no matter who decides. What is not
irreducible is the rule (which names) or the decision (#error). Those left. I
predicted the whole 223 lines would go because I had not separated walking from
adjudicating.
Still compiled, and measured rather than assumed: the capability-tier system
(cap_check_call, is_self_formation_call, is_dharma_call, is_llm_call,
cap_record_violation, emit_cap_violations) is 76 lines of the same shape --
prohibits_WITHIN rather than prohibits_outside, so the checker needs the
opposite polarity to absorb it.
98/98 native, 4/4 prohibition_query.sh, 7/7 seam_binding.sh, fixpoint ok.
ISHIKAWA: why did wraps_body need compile-time knowledge? Because the wrapper
called the target directly. If the wrapper calls through the seam instead, the
seam can call the body itself, and a construct bound after the build decides
how and whether to invoke it.
PREDICTIONS AND RESULTS
P1 wrap becomes runtime-bindable TRUE body x3 -> 21,
never invoked -> 111
P2 codegen shrinks TRUE 5042 -> 4977
P3 cost 5-10% from an indirect call on every fn TRUE 0.36s -> 0.39s, ~8%
P4 zero-param fns break on the empty struct TRUE empty struct is a GNU
extension, empty init
is C23. Fixed with a
char field.
P5 fixpoint holds TRUE
PROCESS FAILURE worth recording: my first patch silently did not apply because
I dropped the assert on the string replacement. The build then failed with
"undeclared identifier __thunk_noargs", which I nearly attributed to the
empty-struct prediction. The guard that would have caught it existed and I
removed it -- the same shape as every other defect found tonight.
Removed: declare_wrap, decorator_wrap, cg_wrap_target, cg_wrap_construct,
params_to_call_args, and the wraps_body scanner branch.
prohibits_outside is now the ONLY construct kind left at compile time, and it
cannot move: a #error has no runtime.
ISHIKAWA: why did exit injection still need compile-time knowledge? Because the
body-helper wrapper was only emitted when codegen already knew an exit
construct existed. The wrapper being conditional was the cause, not the wrapper
being necessary.
PREDICTIONS AND RESULTS
P1 exit becomes runtime-bindable TRUE returns 14, bound
after the build
P2 codegen shrinks TRUE 5094 -> 5044
P3 cost 5-15% from a call frame on every fn FALSE 0.37s -> 0.38s, ~3%
P4 fixpoint holds TRUE
Every fn now gets a body helper and a wrapper. It has to be unconditional:
early returns must route through something for an exit construct to observe
them, and codegen cannot know which fns will be bound after the binary exists.
Removed with the machinery: declare_exit, decorator_exit, cg_exit_target,
cg_exit_construct, and the injects_at_exit scanner branch.
Two controls failed and were rewritten rather than repaired --
no-exit-construct-emits-no-wrapper asserted the optimisation this removes, so
it is now inverted. The integration harness gained a seventh assertion: an exit
construct declared after the build replaces the result.
99/99 native, 7/7 integration, fixpoint gen2==gen3.
The seam's whole claim is that a construct declared AFTER a binary exists
applies to that already-built program. compile_capture only sees emitted text,
so it structurally cannot check this: it needs a built binary, a linked target,
and an environment. Verified by hand until now, which is the standing problem
this session has been about.
tests/integration/seam_binding.sh builds a probe from El source containing no
construct at all, links a target that El never references, and asserts:
ok unbound program is unaffected
ok a construct declared AFTER the build applies
ok a construct declared after the build can REFUSE
ok an unlinked target is skipped, not fatal
ok a binding for a different fn does not fire
ok two constructs compose on one crossing
6 assertions, 6 passed, 0 failed
The eight controls that failed after the strip were replaced, not repaired.
They asserted compile-time emission of capability that moved to runtime;
contorting them would have kept an assertion whose subject no longer exists.
Three took their place, asserting the emitted shape, and the behaviour they
used to cover is now the integration harness's job -- which is the honest
division, since the shape and the behaviour are no longer the same fact.
99/99 native compiler tests pass. Fixpoint holds.
Prediction 3 was FALSE. I expected refusal to be impossible through the seam
because the entry indirection discarded its return. One line:
{ el_val_t __s = el_seam_run(EL_STR(f), 0, 0); if (__s) return __s; }
work() returns 7; bound to a refusing construct AFTER the build it returns 42.
So three of the five compile-time kinds are runtime-bindable: entry injection,
exit injection, and refusal. wraps_body needs invocation control and
prohibits_outside is compile-time by nature.
104/104 native compiler tests pass.
The other half of a boundary: not what runs when something crosses, but what
may not cross at all. It was two string literals in vbd_is_restricted_name and
one #error in cg_fn — one prohibition, uneditable without a compiler release.
@decorator("prohibits_outside", "raw_sql")
fn repository() {}
fn sneaky() -> Int { raw_sql("DROP") }
// #error "boundary violation: raw_sql may only be called from an
// @repository fn, but 'sneaky' is not one"
The recursive matcher is parameterised through a state key rather than by
threading an argument through every branch of the walk — the mechanism codegen
already uses for __match_counter and __if_expr_counter. Each prohibition is
checked in its own turn, so the owning construct is known by construction and
the diagnostic names it instead of hardcoding one rule's wording.
PREDICTIONS AND RESULTS
1 the 3 duplicated uniqueness rules are textually identical TRUE
2 a declared prohibition reproduces @manager's #error TRUE
3 existing output byte-identical TRUE
4 a program can declare its own prohibition TRUE
5 fixpoint holds TRUE
I misread result 2 on first pass: a @manager fn calling dharma_emit still
emitted one #error, which looked like a failure. It is the CAPABILITY-tier rule
at codegen.el:2578, a separate prohibition system, and it fires identically on
the pre-change compiler.
MEASURED DEFECTS STILL OPEN
- two independent prohibition systems (VBD constructs, capability tiers);
only the first is declarable
- 3 uniqueness rules written 6 times, once per codegen path, kept in sync by
hand and identical today
102/102 native compiler tests pass, compiler self-hosts byte-identically.
§6 records 62 persist-after-mutate sites, 10 auth-per-route, and
index-after-append that failed at 9 of 9 — every one an obligation at a
crossing that decayed into "remember to do this afterwards." An obligation a
human must remember is not an obligation, and the 9-of-9 figure is what that
costs.
@decorator("injects_at_exit", "persist_now")
fn durable() {}
The body moves into a static helper and the visible fn becomes a wrapper, so
EARLY RETURNS pass through the exit injection. Emitting it only before the
fall-through return would have silently missed every early return — the exact
failure class this seam exists to remove. Fns with no exit construct emit
byte-identically to before.
Three independent constructs now compose on one fn, none known to the compiler:
el_val_t mutate(el_val_t k) {
{ el_val_t __g = my_auth(EL_STR("mutate"), EL_STR("authenticate")); if (__g) return __g; }
engram_boundary_beat(EL_STR("mutate"), EL_STR("manager"));
el_val_t __r = __el_body_mutate(k);
persist_now(EL_STR("mutate"), EL_STR("durable"), __r);
return __r;
}
Guard, then entry, then body, then exit. §5.2 asked whether `hold` is one
construct or two; the implementation answers one construct with two faces,
selected by declared kind rather than by two mechanisms.
Verified: existing output byte-identical, compiler self-hosts byte-identically,
early returns pass through the exit, ordering holds under composition. 98/98
native compiler tests pass.
@authenticate (6 uses), @authorize (3), @rate_limit (3) and @validate (2)
parsed, attached, and compiled to nothing. Fourteen applications that read as
protection and emitted no instruction — a function decorated @authenticate
compiled byte-identically to an undecorated one.
The missing capability was not authentication. It was that a construct could
observe a boundary but never refuse one. injects_at_entry discards the target's
result; there was no form in which a construct could say no.
@decorator("guards_at_entry", "my_auth")
fn authenticate() {}
@authenticate
@authorize
fn handler() -> String { ... }
emits, at entry:
{ el_val_t __g = my_auth(EL_STR("handler"), EL_STR("authenticate")); if (__g) return __g; }
{ el_val_t __g = my_roles(EL_STR("handler"), EL_STR("authorize")); if (__g) return __g; }
Guards precede injections because a refused call must not report a crossing,
and every guard runs where the topmost injecting construct wins — refusal is
not a role, so it does not follow the role convention.
The compiler still knows nothing about auth. The program points the construct
at its own function, which is where that decision belongs.
Verified: existing @manager/@accessor output byte-identical, compiler
self-hosts byte-identically, guards stack in declaration order and emit before
the beat. 94/94 native compiler tests pass.
codegen called fn_has_decorator for exactly three names — manager, accessor,
route. Twelve others parsed, attached as {name,args}, and compiled to nothing,
including four that look like protection: @authenticate (6 uses), @authorize
(3), @rate_limit (3), @validate (2). The cause was not that the branches were
untidy. A construct had nothing to BE, so its meaning had nowhere to live
except the emitter, and every construct was therefore a compiler edit.
A name -> injection table would have moved the enumeration twenty lines up
without removing it. So the construct now carries its own meaning:
@decorator("injects_at_entry", "engram_boundary_beat")
fn audited() {}
@audited
fn risky_op() -> Int { ... } // gets the beat, attributed to "audited"
scan_declared_decorators is a token-level pre-pass beside scan_routes, forced
by streaming codegen having no whole-program AST. manager and accessor are
seeded as the compiled-in core — the fixedSelf shape from substrate.go: a
complete fallback exists, declaration is enrichment.
This is the injection half of the seam only. The prohibition half (@manager's
#error on dharma_emit) stays hardcoded, because "which calls may appear inside
this boundary" is a query over program structure and there is nothing yet to
ask.
Verified three ways: emitted C for existing @manager/@accessor code is
byte-identical to the hardcoded path; a construct with a name the compiler has
never heard of injects correctly; the compiler self-hosts byte-identically.
90/90 native compiler tests pass.
The beat reported which function crossed a boundary, never which decorator
put the beat there. So the graph accumulated boundary events with no
attribution, and no construct could be measured — "is this decorator
earning its keep" stayed an argument instead of a traversal.
engram_boundary_beat now takes the construct and carries it on the bus as
{"construct":"..."}. The injection point, the beat, and the accumulation
already existed; only the attribution was missing.
Also pins a known defect as a test: codegen calls fn_has_decorator for
exactly three names (manager, accessor, route). Twelve others parse, attach,
and compile to nothing — including @authenticate (6 uses), @authorize (3),
@rate_limit (3) and @validate (2), which look like protection and are not.
decorator-authenticate-compiles-to-nothing asserts that @authenticate emits
byte-identical C to no decorator at all, so fixing it will be a visible flip.
Verified: compiler self-hosts byte-identically, 86/86 native compiler tests
pass, emitted C carries the construct for both @manager and @accessor.
#144 moved transduction into the language and got the dispatch right. It got
the result type wrong: transduce(signal, modality) -> Geometry yields one
vector per signal, and one vector is a fingerprint. A fingerprint can be
matched and ranked; that is all. It cannot be decomposed, cannot have one part
grounded while another is not, and cannot be contradicted in one part while
holding in another, because it has no parts.
A song is not a point. It decomposes into pitch, interval, rhythm, harmonic
function -- components, each with its own geometry, plus the relations among
them. The song IS the structure of the relations.
transduce now returns a Manifold: named components carrying geometry, and
typed weighted relations between them. Signal in, subgraph out.
Components are addressed by key, never by index, because the key is what
survives persistence -- a component becomes a node and is separately groundable
precisely because it is separately named. Relation weight IS the grounding
(correspondence-and-censorship.md 1), so a realizer's relations arrive already
grounded and there is no score computed beside them.
#141 let signal enter as geometry and it worked, but it was placed at the
CONSUMER and said so in its own commit message. This is the correction.
Three defects, all of them placement:
1. It sat in the engram. Ingest is a LANGUAGE concern — every el program
touching any modality needs it, and the engram is merely one el program
that happens to hold a graph. The geometry surface is now defined in
el_runtime.c immediately ABOVE the engram section and depends on nothing
inside it. Delete the entire engram and geometry still enters el.
2. It marshalled the vector as a hex STRING, because el had no first-class
geometry value — which reintroduced text as the TRANSPORT medium one layer
below the problem being fixed. Geometry is now an el value: a magic-tagged
heap object carried in el_val_t, same discipline as List/Map. Hex survives
only as an adapter at the edge, which is all an encoding should ever be.
3. It needed an arbitrary `dim <= 8192` bound purely to size an allocation
from a caller's CLAIM about a string's length. A value carries its own
width, so the width is derived and never asserted. The bound is gone, not
raised — there is nothing left to validate.
Language surface, none of it engram-prefixed: geometry_new / _dim / _is /
_get / _set / _norm / _free, geometry_from_f32le_hex + geometry_to_f32le_hex
as the wire adapters, realizer_register(modality, fn_name), realizer_has, and
transduce(signal, modality) -> Geometry.
REALIZERS ARE DECLARABLE IN EL. This is the part that makes the move real
rather than nominal: registration resolves a name with dlsym against the
running binary, the identical mechanism http_set_handler already relies on,
because every el `fn name(...)` compiles to a global C symbol with that exact
name. So an ordinary el function IS a realizer and a new modality needs no
runtime patch. Verified end to end in lang/examples/transduce.el: an el-defined
tone_realizer is registered by name, transduce dispatches to it, and the
signal demonstrably reaches it (distinct signals produce distinct geometry).
A modality with no realizer transduces to NOTHING. There is deliberately no
built-in realizer, not even for text — silently embedding a description of a
signal and calling that perception is the exact defect this ends.
engram/src/server.el is migrated: POST /api/nodes decodes "emb" hex exactly
once, at the edge, into a Geometry, and everything below that line moves
geometry. The wire is unchanged because production clients speak it. "dim" is
now an ASSERTION about the vector, not the source of its width; disagreement
is a rejected ingest, not a silent reinterpretation.
#141's engram_node_set_emb becomes a DEPRECATED WRAPPER over
geometry_from_f32le_hex + node_attach_geometry — kept only because the runtime
ships as an SDK asset and a downstream binary may link the symbol. Its exact
contract, negative cases included, is preserved and re-verified.
ingest.el's `fn transduce` is renamed transduce_manifold. Mechanically it had
to yield the name (duplicate C symbol, a hard compile error, measured). But it
was never signal->geometry: it chunks already-extracted content into a node+edge
manifold, one layer up, and had taken the name belonging to the primitive
underneath it. Behaviour unchanged.
PROPERTIES FROM #141 PRESERVED, each re-measured on a scratch engram (:8971,
never prod :8742):
- off-dimension vectors stored but NOT indexed — the HNSW build loop still
filters on n->emb_dim == dim at four sites, so a 64-dim voice vector is
durable and addressable without perturbing the 768-dim canonical index
- geometry makes a node ineligible for embed_backfill: after backfill the
64-dim voice node was still 64-dim while the text control acquired 768
- the create response reports whether geometry landed, and the node document
always emits emb_dim and embedded
Read-back with control and negatives, all verified against a PID-confirmed
fresh binary: geometry node emb_dim=64 embedded=true / emb_set=1; text-only
control emb_dim=0 embedded=false / emb_set=0; malformed hex, ragged length,
and dim-disagreement each emb_set=0.
Two compiler landmines found by reading the generated C rather than trusting a
successful build, both documented at their sites: elc lowers `a == b` to
str_eq unless both operand NAMES are in the per-function int-name set (which
does NOT propagate into nested if-expression blocks — the first cut would have
strcmp'd two integers as pointers on the first geometry-bearing request), and
`+` lowers to string concat when either operand is a user-defined call.
Adds tests/native/test_lexer_scaling.el, the regression gate for el #132.
Both directions are proven on LIVE workloads, not synthetic series:
healthy per-character scan 1821 3251 6007 10422 us -> O(n) PASS
rescan-from-zero (the #132 shape) 922 3667 13524 44792 -> O(n^2) FAIL
A gate only proven to pass is decoration. The quadratic specimen exists so
the gate is proven to FIRE.
Also fixes elb_spread_ok to judge the ASYMPTOTIC TAIL (last three ratios)
rather than the whole sweep. Measured on a genuinely linear scan the ratios
ran 3.37 2.92 1.76 1.65 -- the head looks quadratic because it is cold
cache, the tail is the truth. Whole-sweep spread rejected correct data. A
complexity bound is an asymptotic claim and must be judged asymptotically.
That fix came from the classifier refusing to rubber-stamp my own bad
measurement: it reported INDETERMINATE on an unwarmed sweep rather than
passing it. Warmup is now taken and discarded at every sweep point.
Reverts the == workarounds in test_elbench.el now that el #137 has landed;
the natural form generates no str_eq and all 13 fitter tests stay green.
The workaround remains -- the Plus arm is still open.
fn check(label: String, cond: Bool, want: Bool) -> Void {
if cond == want { ... } -> if (str_eq(cond, want)) SIGSEGV
}
Bool has always been an integer in the value model — type_to_c maps Bool to
"int", and el_runtime.h states "Bool -> el_val_t (0 = false, nonzero = true)".
But Bool names were registered NOWHERE: build_int_names_for_params tracked Int
and Float params, and the `let` path tracked Int and Float bindings. Neither
knew about Bool.
So comparing two Bools fell through to str_eq, which dereferenced 0 or 1 as a
char* and segfaulted immediately.
This is the third instance of one family found tonight, after el #137 (a call
on either side of == poisoned the operator) and el #136 (a missing import
compiled clean). All three are the same shape: something the compiler could not
type, silently handled as a string.
Found while writing #137's own test harness — the first version of that harness
crashed on exactly this, on both the old and new compiler, which is how it
surfaced. A test harness that cannot compare two Bools is a good way to notice.
VERIFIED:
- the harness that segfaulted on every prior compiler (exit 139, no output)
now runs clean: 14 passed, 0 failed
- self-hosting fixpoint byte-identical
- the compiler's own generated C differs by 8 lines — only the intended
registration
- neuron's full soul amalgam regenerates in 424ms, exit 0, BYTE-IDENTICAL
- test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12
Adds tests/runtime/operator_typing_test.el, the 15-case suite from #137, so
this family is covered going forward rather than rediscovered.
Adds el_black_box (inline asm, +r constraint, memory clobber) and
runtime/elbench.el: a growth-curve classifier that gates time AND
allocation-count AND allocation-bytes, failing if any exceeds its
declared curve.
Refusal is a first-class verdict. The classifier REFUSES rather than
classifying when the largest measurement is below the floor, or when a
series is hard-flat across an 8x input range -- the shape produced when
the optimiser deletes the work. Reporting O(1) there would be a
confident answer with nothing behind it. Disagreeing ratios report
INDETERMINATE rather than a guess.
Deviation from DESIGN.md 6.2, stated in the source: uses consecutive
ratios on a mandated geometric sweep rather than least-squares over
candidate curves. Ratios are directly interpretable on a doubling sweep
and need no floating point; the cost is weaker O(n) vs O(n log n)
separation, reported as an ambiguous band rather than guessed.
Documents the counter scope limit: engram_*.c and libcurl malloc are
NOT tracked, so a flat curve over engram/HTTP-dominated work is not
evidence of anything.
13 tests prove the classifier against real measured series from
fitprobe.el -- including that an accumulator's allocation COUNT is
linear while its bytes are quadratic, and that el #132's pure-CPU shape
reads FLAT on both allocation signals and is caught only by time.
el #132's quadratic (strlen per character in str_char_code/str_slice) is
pure CPU and allocates NOTHING. Measured on three controlled specimens:
specimen allocs bytes time
linear 2.00 -> O(n) 2.16 -> O(n) 2.05 -> O(n)
accum 2.00 -> O(n) 3.99 -> O(n2) noisy
compute FLAT FLAT 3.96 -> O(n2)
'compute' is #132's shape. A gate fitting only allocation count and bytes
classifies it FLAT and passes -- it would not have caught the defect it
was created for. The gate now fits time AND count AND bytes, failing if
any exceeds its declared curve.
Also: black_box is mandatory and consuming the result is NOT sufficient.
The first 'compute' reported 0us at every n while returning a correct n2 --
clang closed the loop to a multiply. Only an opaque call restored the curve.
Adds lang/tests/bench/fitprobe.el as the fitter's known-good/known-bad set,
so the classifier is provable without depending on a real bug existing.
Marks DESIGN.md 1.3 stale: test_compiler 3.58s -> 0.03s (119x).
Completes the Phase 1 runner and migrates the 11 test files onto it.
- forward-declare the registry accessors in the test preamble; they are
defined at the end of the unit but the El runner is compiled in between
- eltest.el: explicit trailing return in the void emit_* helpers, which
otherwise lower to 'return println(...)' and fail to compile
- test files import runtime/eltest.el explicitly, using the language's own
textual import mechanism rather than compiler-side auto-injection
- DESIGN.md 6.5: gate on allocation COUNT AND BYTES, not count alone
Verified: self-hosting fixpoint byte-identical (gen2 == gen3). 6 of 11
suites run and report per-test timing. The other 5 fail to COMPILE, and
fail identically under the committed compiler -- pre-existing breakage
this framework makes visible for the first time.
THE BUG. str_char_code() and str_slice() each called strlen() on every
invocation. The lexer walks source one character at a time, so every character
access rescanned the whole remaining input: O(n) per character over n
characters = O(n^2).
el_val_t str_char_code(el_val_t s, el_val_t i) {
...
int64_t n = (int64_t)strlen(str); // <- O(n), every call
if (idx < 0 || idx >= n) return 0;
return str[idx];
}
HOW IT WAS FOUND. Not by reading code — by sampling the running process, which
is the same method that resolved tonight's engram outage after four wrong
theories. A geometric sweep of synthetic sources showed wall-clock rising 3.0x,
3.0x, 4.0x, 4.14x per doubling (converging on 4x = quadratic), and a stack
sample put 779 of 779 samples inside lex(), every one bottoming out in
_platform_strlen via str_char_code and str_slice.
THE FIX. Remember the length instead of recomputing it. The subtlety is
INVALIDATION: El strings are arena-allocated, so a freed pointer can be reused
for a different string at the same address, and a naive pointer-keyed cache
would hand back a stale length and read past the end of the new string —
trading a performance bug for a memory-safety one. So entries carry a
generation, a hit requires pointer AND generation to match, and every path that
frees or mutates a runtime string bumps the generation: el_arena_pop,
seed_request_end, __str_set_char. Stale entries cannot be believed; they miss
and recompute.
MEASURED, same host, same inputs:
n(fns) before after
512 0.10s 0.01s
1024 0.37s 0.02s
2048 1.51s 0.03s 50x
the compiler's own 422 KB source concatenated (DESIGN.md's 3.58s case):
3.55s -> 0.03s 118x
The speedup GROWS with input size, which is the signature of removing a
complexity class rather than a constant factor. After the fix each doubling
adds ~0.01s: linear.
CORRECTNESS, verified rather than assumed:
- byte-identical output on every sweep input (n = 128..2048)
- byte-identical output on the 422 KB compiler concatenation
- byte-identical output on tests/runtime/string_test.el
- self-hosting fixpoint byte-identical
- new tests/runtime/str_cache_test.el: 17 assertions covering bounds, empty
strings, negative indices, slice clamping, distinct strings not sharing a
cached length, 1000 interleaved strings forcing cache-slot collisions, and
a grown string not reporting its old length. All pass.
This is the defect that made dist/soul.c a committed artifact: elc could not run
in CI because it needed 24 GB+ and minutes. It needs neither now.
Cancellation-token control channel checked at every step boundary lets a
coordinator PAUSE/RESUME/REDIRECT/KILL a running worker mid-task instead of
waiting for the whole (possibly wrong) plan to finish. Bounded purviews
mean no half-committed state to unwind on interrupt. Includes a proof
harness (proof.el, run.sh) comparing a broken non-interruptible worker
against the new one under identical kill/redirect/pause timing.
Distinct from the already-preserved swarm-ccr orchestrator (fan-out/
converge dispatch): this is single-worker interruptibility, a
complementary mechanism, not a duplicate.
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.
Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.
WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
bigmerge
will.anderson