6df362f84d2953945945e2d0ca90feeb6feaa4e8
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bigmerge
|
f1a7e224a7 |
verify the annotation against what it annotates
ISHIKAWA: three silent miscompilations found the same day shared one shape.
method type tracked by per-function name sets, fed from annotations
machine el_val_t erases everything at the C boundary
material no propagation through expressions
measurement nothing verifies an annotation against what it annotates
root cause El has type ANNOTATIONS and no type CHECKING. The annotation
feeds dispatch and is never itself verified.
MEASURED, and it is not merely a wrong answer
let x: Int = "hello" ; x + 1 -> printed 4343631981, a string POINTER
interpreted as an integer
let s: String = 42 ; println -> dereferenced address 42
The first leaks a raw memory address into program output. The second is an
arbitrary-read primitive if the integer is ever attacker-influenced.
PREDICTIONS AND RESULTS
P1 let x: Int = "hello" compiles clean TRUE
P2 let s: String = 42 compiles clean TRUE
P3 the annotation drives dispatch, unverified TRUE
P4 same root cause as all three bugs found today TRUE
P5 checking literal-vs-annotation catches both TRUE
P6 zero false positives across the compiler's source TRUE
The emitter only RECORDS the mismatch; tools/check/annotations.sh decides,
consistent with every other check landed today.
INCOMPLETE, stated rather than hidden: only literals are checked.
let x: Int = some_string_fn() still passes, because signatures.rel carries
Int/Instant/Duration and no String entries. That is a DATA gap, not a capability
limit -- every El function declares its return type in source and codegen
already holds ret_type on every FnDef.
105/105 native, 5/5 annotation_query.sh, fixpoint ok.
|
||
|
bigmerge
|
6c975b1d50 |
thread provenance through resolve_imports
The module question ended with a limit: textual inlining destroys file
provenance, so a duplicate-definition message could name the symbol but not the
files. Threading it exposed a bigger absence first.
TOKENS HAD NO POSITION AT ALL. A token was a flat (kind, value) pair, so NO
diagnostic in El could name a place -- every error named a symbol and never a
line. That is the prerequisite the module question was resting on.
THE CHAIN, end to end
lexer counts newlines; tok_append mints (kind, value, line)
parser stride 2 -> 3; tok_line added; FnDef carries its line
codegen records <fn> defines_at:<line>
resolve_imports publishes <file> spans <start> <end> for the combined source
checker maps a combined line back to file:line-within-that-file
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
/tmp/modtest/a.el:1
/tmp/modtest/b.el:1
PREDICTIONS AND RESULTS
P1 15 stride sites, encapsulated in tok_kind/tok_value TRUE, but see below
P2 adding a line field is mechanical TRUE
P3 the lexer must count newlines TRUE
P4 resolve_imports can record per-file line ranges TRUE
P5 the message can then name both files TRUE
P6 token memory grows TRUE, 25.0 -> 33.9 MB (+36%)
FOUR DEFECTS, EACH FOUND BY RUNNING AND NOT BY READING
1. interp_tokens_append_all walks the token list DIRECTLY with its own copy of
the stride. Gen1 built fine and gen2 emitted corrupt C, because the
compiler's own source uses string interpolation. My search missed it because
I grepped for the variable name `tokens`; it is called `dst`/`result`.
Searching by name instead of by shape -- third time today.
2. tok_count in test_compiler.el carried the stride too. I had scoped the search
to compiler sources and it had escaped into the tests.
3. Nested resolve_imports calls accumulated spans into shared state, so each
republished meaningless line ranges under the parent's name. Making the
buffer local fixed it; guarding the WRITE did not, which is what I tried
first.
4. The first working version reported b.el:3 -- the COMBINED line against a
filename that has no line 3. A file:line that does not match the file is
worse than no line at all.
105/105 native, 37/37 integration, fixpoint ok, compiler self-checks clean.
|
||
|
bigmerge
|
79f6cb7985 |
ANSWER: if the partition is a neighbourhood, does linking survive?
The question is premature, and measuring says why. El's partition is a
FILESYSTEM PATH, not a neighbourhood, and there is no namespacing at all.
MEASURED
import is textual inlining (resolve_imports), guarded against double
inclusion by a __elc_imp__:<path> state key
when a .elh header exists the header is inlined instead and the .el is marked
seen, so symbols resolve at C link time -- so linking IS real, delegated to C
two modules defining `helper` emit two C functions into one translation unit
So linking barely survives the PATH partition. Whether it survives a
neighbourhood partition cannot be asked yet.
A DIAGNOSTIC REGRESSION I CAUSED, found by asking this question. cc does catch
the collision, but reports:
error: redefinition of '__el_body_helper'
error: redefinition of '__env_helper'
error: redefinition of '__thunk_helper'
error: redefinition of 'helper'
The user's own function is FOURTH. The first three are generated symbols
introduced by the unconditional-wrapper pass earlier today -- before it, there
was one clear message. Repaired by catching the collision at El level instead:
duplicate definition: 'helper' is defined 2 times — El has no namespacing,
so imported modules share one global scope
LIMIT, stated rather than hidden: textual inlining destroys file provenance. By
the time codegen runs there is one source string, so the message can say WHICH
name collides but not which files. Naming a.el and b.el needs provenance
threaded through resolve_imports.
104/104 native, 4/4 definitions_query.sh, the compiler itself reports clean,
fixpoint ok.
|
||
|
bigmerge
|
c48db6c2a8 |
ANSWER: is 3 a position, or a convention we agreed on?
Both, at different layers, and the split is the same as everywhere else. The
NUMERAL is convention -- int_to_str was already form 1, because no position
determines that twelve is written 1 then 2 in base ten. The NUMBER is a
position: three things are three things regardless of notation.
But the sharper answer follows from `love = 0`. A bare `3` is a MAGNITUDE WITH
NO AXIS. It is not a position until something gives it a direction, which is
exactly why 3.days needs a calendar and why time_add(t, n, "min") had to carry
its axis as a string.
PREDICTIONS AND RESULTS
P1 numeral = convention, number = position TRUE
P2 a bare literal is dimensionless until context types it TRUE
P3 there is a measurable place where El guesses TRUE
P4 Instant + Int is not caught though Duration + Int is TRUE
P5 the rule catches it TRUE
P6 nothing legitimate in the tree relies on it TRUE
P3/P4 IS THE DEFECT, and it was found by reasoning from the philosophy and then
measured. Duration + Int was refused -- "an Int carries no unit" -- while
let t: Instant = now()
let u: Instant = t + 3
compiled to raw (t + 3) and reported CLEAN. Adding a dimensionless number to a
point is worse than adding it to a displacement: it silently moves the instant
by an unspecified amount. 3 of what? Whatever the representation happens to be,
which is the leak itself. The asymmetry had no justification; the rule was
simply never written.
P6 MATTERED. Two calendar tests looked like Instant + Int:
let later: Instant = i + 1.hour
let later: Instant = base + 15.hours
They are not. `1.hour` lexes to a Duration -- el_duration_from_nanos(1LL *
3600000000000LL) -- and both stay clean. That is the whole answer demonstrated
in one line: t + 3 is refused because 3 has no axis; t + 1.hour is accepted
because .hour supplies one.
104/104 native + 2 new, integration green, fixpoint ok.
|
||
|
bigmerge
|
cbef1c1ebb |
EXPERIMENT: Int return types as data — and the bug that fell out
PREDICTIONS AND RESULTS
P1 is_int_call's 35 hardcoded names move to data TRUE
P2 is_int_name stays -- it is annotation propagation TRUE
P3 the dispatch stays -- it is emission TRUE
P4 codegen shrinks ~40 lines TRUE 4507 -> 4469
P5 the design doc's characterisation is WRONG TRUE
P6 the moved data also fixes the bug it exposed TRUE
P5 CORRECTS THE RECORD. el-language-design.md and geometry-vs-code.md both cite
"== lowering to str_eq unless both operand names are in a hardcoded int-name
set -- a literal list of variable names treated as integers" as the paradigm
defect. It is not one. __int_names is populated from TYPE ANNOTATIONS
(param["type"] == "Int"), which is primitive but legitimate type propagation.
The actual defect was is_int_call: 35 hardcoded builtin return types, the same
shape as the temporal 19.
P6 IS A LIVE CORRECTNESS BUG, PRE-EXISTING, NOW FIXED
let a = str_len("hello") // no annotation
let b = str_len("hi")
let c = a + b // -> el_str_concat(a, b) on two integers
Verified identical on the pre-change compiler, so not a regression. It compiled
clean, ran, and printed NOTHING where it should print 7. No error at any layer.
The repair is three lines: an unannotated let takes its type from what the
initialiser returns. The return types were already required for dispatch and
were simply never consulted at the binding site. Moving them into data is what
made the gap visible -- reading the code for eight hours did not.
98/98 native + 2 new, 31/31 integration, fixpoint ok.
|
||
|
bigmerge
|
e8e25a07b4 |
EXPERIMENT: temporal adjudication moves out; the placeholder stays
The previous pass moved the type DATA and left the judgment inline, which I
stated rather than hid. This finishes it.
PREDICTIONS AND RESULTS
P1 codegen can emit operand-type relations TRUE
"main calls temporal:instant_plus_instant"
P2 the affine rules are a small closed set as data TRUE 6 rules
P3 violations still caught at build time TRUE exit=1
P4 the reporter leaves codegen TRUE 4538 -> 4507
P5 the TIME_TYPE_ERROR placeholder must STAY TRUE
P5 is the boundary of this whole approach. The emitter has to emit SOMETHING
for an illegal expression -- it cannot emit nothing and it cannot decide what
the program meant. So the placeholder is irreducible in the same way the AST
traversal was: what moved is the judgment and the wording, not the fact that
something must be written.
The rules are affine algebra and the set is closed because there are only two
kinds of thing. An Instant is a POINT, a Duration is a DISPLACEMENT: add a
displacement to a point, subtract two points for a displacement, combine
displacements. Nothing else is meaningful, which is why the enumeration in
temporal.rel cannot grow the way an allowlist does.
A defect in my own checker, found by running it: the .rel file uses aligned
columns and my awk assumed a single space, so the message came out with the
rule key still prefixed. Same class as the multi-line header parse in the arity
pass -- formatting assumptions that only fail when you look at the output.
98/98 native, 6/6 temporal_query.sh, fixpoint ok.
|
||
|
bigmerge
|
d2d89fcb60 |
EXPERIMENT: temporal types as data — and the pass that GREW the compiler
This block is structurally unlike the previous four. It does not only
adjudicate, it DISPATCHES: Instant + Duration must become el_instant_add_dur,
LocalDate + Duration must become el_local_date_add_dur. The emitted C depends on
the type answer, so it cannot move to a post-hoc query. Selecting which call to
emit is an emitter's actual job.
PREDICTIONS AND RESULTS
P1 the block conflates dispatch with adjudication TRUE
P2 adjudication can move, dispatch cannot TRUE
P3 this pass shrinks codegen far less than the last TRUE, and worse:
4513 -> 4537, it GREW
by 24 lines
P4 the rules are affine algebra, closed by construction TRUE
P5 no type propagation -- name tracking plus a
hardcoded list of which builtins return which type TRUE, 19 names
P3 is the honest result and it is not spun: moving 19 names into a data file
cost more lines than it saved, because a generic loader is larger than the
enumeration it replaces. The win is not line count. It is that adding a 20th
temporal builtin is now a one-line edit to signatures.rel instead of a compiler
change, and that the data is inspectable.
WHY THE HEADER CANNOT SUPPLY THIS, unlike arity: el_runtime.h declares every
builtin as returning el_val_t, because El has ONE type. That single type is why
the whole seam is cheap and it is exactly why the C boundary cannot say that
now() returns an Instant while unix_seconds() returns an Int. The El-level type
is real and the boundary erases it.
INCOMPLETE, and stated rather than hidden: P2 said adjudication could move to a
query. It has NOT. Violations still emit TIME_TYPE_ERROR inline from the
emitter. Only the type DATA moved. Moving the adjudication needs the operand
types recorded as relations, which is a further pass.
98/98 native, 4/4 temporal_signatures.sh, fixpoint ok.
|
||
|
bigmerge
|
9cc6040df2 |
EXPERIMENT: derive arity from the runtime's own declarations
codegen.el carried builtin_arity(): 344 lines, 300 entries, a hand-maintained
second copy of el_runtime.h.
PREDICTIONS AND RESULTS
P1 the table duplicates the header TRUE 243 shared names
P2 they have already drifted FALSE ZERO drift. The
duplicate had been
maintained correctly.
P3 codegen can emit call-arity relations TRUE
P4 the check becomes a query against the header TRUE
P5 codegen drops to roughly baseline TRUE 4903 -> 4512,
149 BELOW the 4661
it started at
P2 being false is the better result: the table was not WRONG, it was
INCOMPLETE. 110 functions the runtime declares had no entry, so calling them
with the wrong argument count produced no El-level diagnostic at all. Measured:
the old compiler reports 0 arity errors for __http_do_map_to_file(1); the query
reports "takes 5 arguments, called with 1".
Deriving from the header fixes coverage AND makes drift impossible by
construction. 503 signatures, versus 300 entries maintained by hand.
THREE DEFECTS IN MY OWN CHECKER, each found by running it rather than reading it
1. El names and C names differ -- `println` is `__println`. 60 of 500 decls
carry the prefix and codegen owns the mapping; the old table carried both
keys. One rule covers all 60.
2. Multi-line declarations parsed as zero params, so the checker reported
"takes 0" for a function taking 5. A diagnostic with the wrong number in it
is worse than none -- the same shape as the stale caller attribution in the
previous pass.
3. Fixing (2) by joining lines dropped 500 signatures to 334, because a
declaration preceded by a comment no longer started its record. Comments
are stripped first now.
98/98 native, 5/5 arity_query.sh, fixpoint ok.
|
||
|
bigmerge
|
c2d9596e76 |
EXPERIMENT: the capability tier becomes shipped policy plus a query
Capability differs from prohibits_outside in one way that matters: a utility program cannot be trusted to declare its own restrictions, because it would declare none. So the policy comes from OUTSIDE the program -- it ships with the language as data, editable without a compiler release. tools/check/capabilities.rel 18 names that were string literals in codegen tools/check/capabilities.sh the query that decides PREDICTIONS AND RESULTS P1 codegen emits kind + call graph, drops the 4 name tests TRUE zero #errors P2 the 18 literals become a data file TRUE P3 the checker catches capability violations TRUE exit=1 P4 codegen drops ~76 lines TRUE 4963 -> 4881 P5 below the 4661 baseline FALSE ~+230 TWO DEFECTS THE HARNESS FOUND THAT READING WOULD NOT HAVE 1. Calls inside main became invisible. cg_fn returns early for main -- C provides its own -- so hooking the recording there left every call in main unrecorded: a blind spot exactly where a program does its work. The old cap_check_call ran from cg_expr and did see main. Moved the recording to cg_expr. 2. Caller attribution was stale. __cg_current_fn kept whatever cg_fn set last, so a violation in main was reported against the previously emitted function. The test still PASSED, because the violation was detected -- only the name was wrong, and a diagnostic naming the wrong fn is worse than none. Fixed at all three main-emission sites; the first patch missed two because the live path is codegen_streaming. 98/98 native, 7/7 + 4/4 + 5/5 integration, fixpoint ok. |
||
|
bigmerge
|
c741cfe928 |
EXPERIMENT: prohibition becomes a query over emitted relations
I said prohibition could not move because "a #error has no runtime". That
conflated two separable things: WHEN a violation is detected (build time --
correct, and unchanged) and WHERE the rule and the checker live (the compiler
-- assumed).
A prohibition is a containment relation over the call graph. So codegen now
records what it saw:
sneaky calls raw_sql
allowed calls raw_sql
allowed calls @repository
repository calls prohibits:raw_sql
and tools/check/prohibitions.sh decides, at build time, outside the compiler.
PREDICTIONS AND RESULTS
P1 codegen can emit the call graph it already walks TRUE
P2 the check becomes a query outside the compiler TRUE
P3 all prohibition decisions leave codegen TRUE zero #errors now
P4 violations still caught at build time TRUE exit=1
P5 codegen drops below the 4661 baseline FALSE 4962, +301
P5 is the finding. The TRAVERSAL is irreducible -- you must walk the AST to
find calls, and those ~120 lines do not move no matter who decides. What is not
irreducible is the rule (which names) or the decision (#error). Those left. I
predicted the whole 223 lines would go because I had not separated walking from
adjudicating.
Still compiled, and measured rather than assumed: the capability-tier system
(cap_check_call, is_self_formation_call, is_dharma_call, is_llm_call,
cap_record_violation, emit_cap_violations) is 76 lines of the same shape --
prohibits_WITHIN rather than prohibits_outside, so the checker needs the
opposite polarity to absorb it.
98/98 native, 4/4 prohibition_query.sh, 7/7 seam_binding.sh, fixpoint ok.
|
||
 will.anderson
|
0a72fced28 |
engram: WAL persistence + integrity hardening + single canonical runtime
El SDK CI - dev / build-and-test (pull_request) Failing after 13m17s
Establish lang/runtime/ as the ONE canonical el runtime (from the active runtime that carries hebb/emb persistence + the new WAL); repoint the el CI publish, engram build, elb default, and in-repo build scripts to it; delete the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh drift guard. Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian edge weights — learned co-activation was wiped on every restart. Publishing from canonical ships the stranded 'learning that cannot outlive the process' fix. WAL storage engine + integrity fixes (DELETE->tombstone + store-layer protection, safe data-dir default) ride in behind ENGRAM_WAL (default off = byte-identical to today). Verified: engram elb per-module build clean, WAL gate 66/66, native smoke ok, drift-guard green. |
||
|
will.anderson
|
027ad82db2 |
fix elb linker: remove runtime imports from el-install, add --clean, catch in dev/stage CI
El SDK CI - dev / build-and-test (pull_request) Successful in 3m35s
el-install.el explicitly imported runtime/*.el modules (string, env, fs, exec, json, http), which elb compiled to .c files in the shared dist/bin out_dir. Linking those alongside el_runtime.c caused multiple definition errors for every runtime function (http_get, http_patch, etc.). The runtime .el files are thin wrappers over seed primitives already compiled into el_runtime.c — no import needed. Fixes: - Remove all explicit runtime imports from el-install.el (root cause) - Add --clean to every elb invocation in sdk-release.yaml so each build starts with a clean out_dir (defense-in-depth against stale .c files) - Add elb build + epm/el-install build steps to ci-dev.yaml and ci-stage.yaml so linker errors are caught on every PR, not just stage->main |
||
|
Will Anderson
|
592f8f482a |
add el-install binary and SDK bundle to release pipeline
- lang/tools/install/el-install.el: El program that fetches the latest release from the Gitea API, downloads el-sdk-latest.tar.gz, and extracts it into ~/.el (or a custom prefix passed as argv[1]) - lang/tools/install/manifest.el: build manifest for the el-install package - .gitea/workflows/sdk-release.yaml: build elb, epm, and el-install binaries; bundle elc + elb + epm + runtime files into el-sdk-latest.tar.gz; attach both the tarball and el-install binary to the Gitea release alongside the existing per-file GCP uploads |
||
|
Will Anderson
|
1ae68962cf | restructure: move el compiler content into lang/ |