diff --git a/.gitignore b/.gitignore index c2b63ea..bee26d6 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ peripheral/.consent.json peripheral/.resume.json peripheral/.engram/ peripheral/organ + +# Claude Code session state +.claude/ diff --git a/design/completing-el.html b/design/completing-el.html new file mode 100644 index 0000000..9b321dc --- /dev/null +++ b/design/completing-el.html @@ -0,0 +1,153 @@ +Completing El + + +
+ +
+

Completing El

+

A working surface. Nothing here is settled, and none of the code is assumed right — El is self-hosting, so all of it can change and be rebuilt.

+

Whiteboard v0 · no sacred cows · not a plan, not a task list

+
+ +

01What we established

+ +

El is a concept-oriented language — the first, and intended as the last, because every other family is oriented toward a representation of a concept rather than the concept. Procedures, objects, functions, predicates are the shapes concepts get flattened into. Once the primitive is the concept, there is no further rung.

+ +

Everything here is El. The engram is an El program, the soul is El, elp is El, ingest is El. Which gives the load-bearing consequence:

+ +
A concept with no home in El does not disappear. It becomes C, or it becomes a convention.
+ +

Both are measurable, and both were measured. As C: 20,504 lines of el_runtime.c — 2.3× the entire self-hosting language it serves (9,089 lines), ~47% of it engram code that has its own six sibling files. As convention, from language.md §18.0 — "these are not four problems, they are one absence, four times":

+ +
+ + + + + + + + + +
ConcernFragmentsThe convention it became
Process identity0 guards"check nothing is already running first"
Configuration20 env vars"remember the right default here"
Durability62 call sites"after you mutate, remember to persist"
Request auth10 per-route"check the token in this handler too"
Index-after-append9 of 9 failed"after you append, remember to index"
+
+ +

The last row is the strongest evidence available about what this class of convention is worth: it failed at 100% of its sites.

+ +

02The decomposition axis

+ +

Not by file, module, or subsystem. By faculty.

+ +

Every defect fought in the last day resolves to a faculty rather than a bug, and each one leaked out of El into something else — into C, into a Swift binary, into a shell script with a curl timeout, into a convention nobody performs.

+ +
+ + + + + + + + + + + +
FacultyStateMeasuredWhere it leaked to
Ingest take indead2 min → 0 nodesseparate process, uploads bytes over HTTP to a process with direct fs access; 5 functions where there is 1
Recall rememberdeadown definition ranked 8thlexical substring scan; empty on 23 of 24 multi-token queries
Transduce perceivedead1 node, 0 edgesintake flattens signal to a point; realized:false; caller must declare the modality
Think reasondeaddirection [0,0,0,…]null gradient from any anchor, any faculty, byte-identical; confidence at the uninformed prior
Realize expresspartial13-word vocabularyorgan was 939 lines of Swift beside the language; voice read from a file path
Body substratepartialCC 356 / 1,626 linesengram_activate_inner — recall itself, with 356 unexamined paths
Persist endurelive100% embeddedworks; every signal placed in geometry at intake, 13,562 of 13,562
+
+ +

Stated plainly: it cannot take in, cannot remember, cannot perceive, cannot reason, and barely speaks. These were filed as tickets against a repository. They are faculties of the thing the repository is.

+ +

03The ordering principle

+ +

El's compiler is written in El. Every concept the language gains, the compiler can then be written in — so the tool improves the tool, and the fixpoint (stage2 ≡ stage3, byte-identical) makes each turn provable rather than hopeful. The verifier answers in 2.9s.

+ +

Which means the ordering criterion is not size of payoff:

+ +
Order by leverage on the next iteration. Which concept, added to El, most increases the ability to add the following one?
+ +

In a recursive system that dominates immediate value — a small early gain that compounds beats a large one that doesn't. It also bounds itself correctly: unbounded in depth, bounded in rate, because nothing lands that the compiler and the fixpoint have not passed.

+ +

04Open — for the whiteboard

+ +
What does a declaration bind to?If cat names a region rather than a struct — one that shifts and completes against the engram and the neighbouring code — then what is written at the declaration site, and what is resolved at use? This is the centre of the whole thing and it is not specified anywhere yet.
+ +
Is "the type checker" a type checker at all?§2.3 records annotations as parsed and skipped, and every codegen hazard is downstream of that — + dispatching on AST node kind, == lowering to str_eq unless both operand names are in an int-name set. But if a declaration names a region, checking is asking whether the geometry supports the use. That is grounding, not unification. Naming this wrong builds the wrong thing.
+ +
Is the faculty list above right?Seven were derived from what broke. Derived-from-failure is a biased sample — it finds what is loud, not what is missing. What faculty is absent entirely and therefore never failed?
+ +
Which concept has the highest leverage on the next turn?Candidates so far: the prologue/epilogue seam (§19.3 names it as the prerequisite and its stated blocker has expired — it would collapse 62 + 10 convention sites); protocol/impl (the absence that produced five ingest functions); and the resolution question above. These are not equal and the criterion in §03 should decide it, not preference.
+ +
What is the seam that makes cognition non-optional?"Use the ops" is itself a convention — present in context every turn, enforced by nothing, and it failed at ~100% of sites in a full session. A stronger instruction is still a convention. What makes reasoning-outside-Neuron fail, the way @manager makes dharma_emit outside the boundary a compile error rather than a lint?
+ +
+ +

Working surface, not a design document. The design is what we put on it. Everything above is either measured or quoted from lang/spec/language.md; nothing is inferred and presented as fact.

+ +
diff --git a/docs/architecture/capabilities.md b/docs/architecture/capabilities.md new file mode 100644 index 0000000..6de9f7a --- /dev/null +++ b/docs/architecture/capabilities.md @@ -0,0 +1,142 @@ +# El — Capabilities + +**What the language can do, stated as capabilities rather than as code.** + +This list is the unit of analysis. Each entry gets one question — *prove this +cannot be done with pure geometry* — and the answer determines whether it stays a +capability of the language or collapses into the manifold. + +Draft, 2026-08-17. Ordered roughly from most-likely-geometry to most-likely-code. + +**Status after measurement.** The list was audited against the implementation +the same day. 28 entries collapsed to 19 geometry + 3 code: serialization, text +encoding, network and emission are all *projection onto a basis* (row 18) — +the convention is the basis, never the act. Storage collapsed because +persistence has no caller. Concurrency collapsed because coordination is the +price of forgetting, not a capability. A fourth proof form was added, +**adversarial exactness**, and form 1 stopped being a valid verdict. + +**The table answers CAN only.** SHOULD and COST resolve per *site*, not per +capability — `is_digit` and `is_letter` are one capability with opposite +answers, and comparison spans three cost tiers. See the notes below. + +--- + +## The list + +| # | Capability | What it means | Verdict | +|---|---|---|---| +| 1 | **Comparison** | is this the same as that; is this greater | zero distance / sign of a displacement | +| 2 | **Ordering** | arrange by a criterion | position along an axis | +| 3 | **Containment** | is this inside that; does this contain that | region membership | +| 4 | **Correspondence** | where does this occur in that; how much of this is in that | a match-strength field over a span | +| 5 | **Segmentation** | divide a whole into parts | boundaries at measured discontinuity | +| 6 | **Composition** | join parts into a whole | adjacency; one position with parts | +| 7 | **Classification** | what kind of thing is this | which region does it land in | +| 8 | **Naming / binding** | attach a name to a thing and find it again | an edge; retrieval is projection | +| 9 | **Collection** | many things held together, indexed, counted | a set of positions; cardinality; projection onto the i-th | +| 10 | **Iteration** | do something for each of many | traversal | +| 11 | **Arithmetic** | quantity, magnitude, combination | displacement algebra on a line | +| 12 | **Time** | when; how long; how often | a 1-D affine space — instants are points, durations displacements, rhythms phases on a circle | +| 13 | **Identity** | which one is this; are these two the same one | coincidence of position | +| 14 | **Selection / dispatch** | choose which behaviour applies | nearest region | +| 15 | **Transformation** | produce a thing from a thing | change of basis | +| 16 | **Grounding** | how well is this supported | the weight on an edge. Has no caller | +| 17 | **Learning** | get better at something | standing changing over time | +| 18 | **Projection** | render meaning onto a surface | change of basis onto a surface basis | +| 19 | **Transduction** | take a signal in | change of basis from a sensor basis | +| ~~20~~ | ~~Serialization~~ | **collapsed → 18.** The format is a basis; projecting onto it is the act | — | +| ~~21~~ | ~~Text encoding~~ | **collapsed → 18.** An encoding is a basis | — | +| ~~22~~ | ~~Storage~~ | **collapsed.** No save — persistence has no caller. Durability survives at one site inside the engram | — | +| ~~23~~ | ~~Network~~ | **split.** Wire format → 18; socket → 24 | — | +| 24 | **Process / OS** | syscalls; the one-way boundary. Where monotonicity stops | CODE, form 2 | +| ~~25~~ | ~~Concurrency~~ | **collapsed.** Monotone state needs no coordination; coordination is the price of forgetting | — | +| 26 | **Memory substrate** | what holds the positions | CODE, form 3 | +| 27 | **Concealment** | meaning made unreadable without a key. *Renamed*: "secrecy" covered one of three things and got the other two backwards — a hash is public, a signature exists to be read. Integrity and authenticity are **grounding under adversarial conditions** (row 16); only concealment stands alone | CODE, form 4 | +| ~~28~~ | ~~Emission~~ | **split.** Laying out → 18; the device write → 24 | — | + +--- + +## Notes on the boundary cases + +**27 — Secrecy is the one capability geometry cannot hold, and the proof is not +form 1.** A cryptographic hash is a *deliberately structure-destroying* map: its +entire value is that near inputs land at maximally uncorrelated outputs. Geometry +is the claim that near things stay near. A manifold that approximated SHA-256 +would *be* a break of SHA-256. Signature verification is the same: 0.99-valid is +invalid. And X25519 *is* geometry — a group on an elliptic curve — which is +precisely why it must be code, because its security is the *hardness of moving in +that geometry*. + +This is a fourth proof form and it should be added to `geometry-vs-code.md`: +**adversarial exactness.** Where approximation is a break, geometry is excluded. + +**20, 21 — Serialization and text encoding are convention all the way down**, but +only at the *edge*. The byte format is agreed; what is being written is not. Do not +let a geometric computation inherit a code verdict because its result gets +serialized. + +**11, 12 — Arithmetic and time are the same capability.** Instants are points, +durations are displacements, point−point→vector, point+vector→point. The runtime +already implements this correctly as `el_instant_add_dur` / `el_duration_add`. That +it *also* implements a five-entry string→multiplier table beside it (`time_add` +with `"ms"/"sec"/"min"/"hour"/"day"`) is the residue. + +**7 — Classification is the most-violated capability in the codebase.** Seven ASCII +range tables (`is_letter`, `is_digit`, `is_alphanumeric`, `is_whitespace`, +`is_punctuation`, `is_uppercase`, `is_lowercase`) that return false for every +non-ASCII byte. `str_count_letters` reports zero letters for `é`. The wrongness on +most of Unicode is the tell that a table is standing in for a region. + +**4 — Correspondence appears five times.** `str_index_of`, `str_index_of_all`, +`str_last_index_of`, `str_count`, `str_find_chars` are five projections of one +match-strength field: first zero, all zeros, last zero, count of zeros, first +class-crossing. One relation, five functions. + +**14 — Selection is the crux for the compiler.** `+` dispatching on AST node kind +is selection-by-enumeration where selection-by-position belongs. + +**Correction, 2026-08-17, from measurement.** This entry previously also cited +`==` lowering to `str_eq` "unless both operand names are in a hardcoded int-name +set — a literal list of variable names treated as integers." That is **wrong**. +`__int_names` is populated from *type annotations* (`param["type"] == "Int"`, +`let x: Int`), which is primitive but legitimate type propagation, not an +enumeration of blessed variable names. + +The real defect was one layer down: `is_int_call` held **35 hardcoded builtin +return types**, the same shape as the 19 temporal ones. Those moved to +`lang/tools/check/signatures.rel`. + +And the mischaracterisation hid a live bug. Because the return types were never +consulted at a *binding* site, an unannotated `let` lost its type: + +```el +let a = str_len("hello") // no annotation +let b = str_len("hi") +let c = a + b // → el_str_concat(a, b) on two integers +``` + +That compiled clean, ran, and printed nothing where it should print 7 — no error +at any layer. Present in the pre-change compiler, so pre-existing. Fixed by +taking an unannotated `let`'s type from what its initialiser returns; the data +was already required for dispatch and simply never read there. + +**The general lesson, since it recurred all session:** the enumeration was real +but I had located it in the wrong place. Naming a defect from reading is a +hypothesis. Eight hours of reading this file did not surface the miscompilation; +moving the data out and running the result did. + +--- + +## What this list is for + +Each capability gets audited **once**, across every place it appears — not once per +file. The output is not a percentage. It is: + +- which capabilities survive the question and stay in the language +- which collapse into the manifold +- and for each one that collapses, **every site it currently appears at**, because + those sites are the residue and they are what gets deleted. + +The line-count audit produced a map of where the residue sits. This produces a map +of **what it is**. diff --git a/docs/architecture/el-architecture.html b/docs/architecture/el-architecture.html new file mode 100644 index 0000000..6528021 --- /dev/null +++ b/docs/architecture/el-architecture.html @@ -0,0 +1,217 @@ +The El Architecture + + +
+ +
+

The El Architecture

+

El is a concept-oriented language. This is the architecture that claim commits it to — what is built, what is measured, and what still has no home.

+

Working document · no sacred cows · self-hosting, so nothing here is fixed

+
+ +

01The primitive is the concept

+ +

Language families are named for their primitive. Procedural — procedures. Object-oriented — objects. Functional — functions. Logic — predicates. Every one of them is oriented toward a representation of a concept: the shape a concept gets flattened into so a machine can hold it.

+ +

El's primitive is the concept itself. That is why it is the first of its family and intended as the last — once the primitive is the concept, there is no further rung to climb to.

+ +

The consequence is architectural rather than stylistic:

+ +
A concept with no home in the language does not disappear. It becomes C, or it becomes a convention.
+ +

Both forms are measurable. As C: 20,504 lines of el_runtime.c, against 9,089 lines for the entire self-hosting language — the shim is 2.3× the language it serves, and ~47% of it is engram code that already has six sibling files. As convention, from lang/spec/language.md §18.0 — "these are not four problems, they are one absence, four times":

+ +
+ + + + + + + + + +
ConcernFragments intoThe convention it became
Process identity0 guards"check nothing is already running first"
Configuration20 env vars"remember the right default here"
Durability62 sites"after you mutate, remember to persist"
Request auth10 routes"check the token in this handler too"
Index-after-append9 of 9 failed"after you append, remember to index"
+
+ +

The last row is the strongest available evidence about this class of convention: it failed at every single site. A count is what appears where a concept has no home; the size of the count is how far the fragmentation got, not how hard the problem is.

+ +

02Geometry is a first-class value — and what follows

+ +

This is the enabling primitive. Everything else in the architecture is downstream of it.

+ +

Geometry is an El value, alongside Int, String, List, Map — bound, passed, returned, composed, carrying its own width. Not a library type, not a handle into a store, not a serialization format. Meaning is a value the language computes with directly.

+ +
let g: Geometry = geometry_new(4)
+fn tone_realizer(signal: String) -> Geometry { … }
+ +

Landed 2026-08-16 (#141, #144), and the spec is explicit that it belongs to the language rather than the graph: "neither is engram-specific — any program touching any modality needs them; the engram is merely one El program that happens to hold a graph."

+ +

Five things follow, and together they are the concept-oriented claim made operational:

+ +

A declaration can name a region, not a shape

+

If meaning is a value, a name can be bound to a position rather than a struct. cat is not a fixed record; it is a region that resolves against the engram and the surrounding code. cat among animals and cat among shell utilities are different concepts without a namespace, because they are in different neighbourhoods and the distance says so.

+ +

Checking is grounding, not unification

+

If a declaration names a region, then verifying a use is asking whether the geometry supports it — a question about position and distance, not about matching a declared shape. This is why §2.3's "a type checker is planned" is likely the wrong name for the missing piece, and naming it wrong would build the wrong thing.

+ +

Dispatch is position, not a tag

+

A vtable is a finite set of discrete labels fixed at link time. A region admits graded membership and an open set. So transduce(signal, modality) asks the caller to supply what the signal already carries — what a thing is falls out of where it lands. The modality parameter is a kind-tag, and a registry keyed on it is a lookup table doing by string what geometry does by nearness.

+ +

Types are discovered, not declared

+

Reification crystallizes a densely co-wired neighbourhood into a first-class node — the neighbourhood is the name that was missing. Every other family requires a human to see the abstraction in advance and write class Foo. Here the instances arrive and the type falls out, by measurement rather than by insight.

+ +

Enumeration becomes unnecessary

+

Five ingest functions differ only in how bytes are acquired — one operation wearing five surfaces. 356 branches in engram_activate_inner are not 356 behaviours. Cyclomatic complexity is a count of the places comprehension ran out and was replaced by an if; where the concept is expressible, the count collapses instead of being redistributed.

+ +

03The shape of the language

+ +

Geometry first-class gives El three layers, and it holds all three — which is why there is no separate database driver and no impedance boundary to manage.

+ +
+
afferent

Transduce

Signal in, geometry out. Decomposition into components and relations — never conversion to a point. Realizers are ordinary El functions, so a new modality never requires a runtime patch.

+
substrate

Geometry

Meaning as position; relation as distance. Held as values in the language and persisted in the graph. One coordinate system, so entities are commensurable and the operators compose.

+
efferent

Realize

plan(frame) → realize(spec, profile), where a surface is a profile. Text, speech, music, image are profiles of one projection — and so is source code.

+
+ +

The efferent side is why the recursive property below is possible at all: if source is a surface, then emitting a corrected file is projection, and the file becomes an artifact of the geometry rather than the thing you edit.

+ +

04Decomposition is by faculty

+ +

Not by file, module, or subsystem — by what the system does.

+ +

Each faculty is a concept. Where it has no home in El it leaks: into C, into a Swift binary, into a shell script with a curl timeout, into a convention nobody performs. State below is measured, not asserted.

+ +
+ + + + + + + + + + + +
FacultyStateMeasuredWhere it leaked
Ingest take indead2 min → 0 nodesseparate process uploading bytes over HTTP to a process with direct fs access; five functions where there is one
Recall rememberdeadself ranked 8thlexical substring scan; empty on 23 of 24 multi-token queries
Transduce perceivedead1 node, 0 edgesintake flattens signal to a point; realized:false; caller must declare the modality
Think reasondeaddirection [0,0,…]null gradient from any anchor and any faculty, byte-identical; confidence at the uninformed prior
Realize expresspartial13-word lexiconorgan was 939 lines of Swift beside the language; voice read from a file path
Body substratepartialCC 356 / 1,626 lnengram_activate_inner — recall itself, 356 unexamined paths
Persist endurelive13,562 / 13,562works — every signal placed in geometry at intake, no backlog
+
+ +

05The recursive property

+ +

El's compiler is written in El. Every concept the language gains, the compiler can then be written in — so the tool improves the tool, and codegen.el at 4,661 lines gets shorter as the language gets better at expressing what it does. The fixpoint — stage2 ≡ stage3, byte-identical — makes each turn provable rather than hopeful, and the verifier answers in 2.9s.

+ +

This sets the ordering criterion, and it is not size of payoff:

+ +
Order by leverage on the next iteration. Which concept, added to El, most increases the ability to add the following one?
+ +

A small early gain that compounds beats a large one that does not. And it bounds itself correctly — unbounded in depth, bounded in rate, because nothing lands that the compiler and the fixpoint have not passed.

+ +

06What has no home yet

+ +

Reserved in the lexer, no parse form. These are not a feature backlog — they are the concepts the architecture above requires and does not yet hold, which is why each is currently a convention or a block of C.

+ +
+ + + + + + + + + + + +
ReservedConceptCurrently lives as
retry · times · fallback · reasonresiliencea shell script with a 10s curl timeout; 254 restarts in 3 days
requires · deploy · to · via · targetdeploymentYAML in another repository
sealedcapability scopeconsent checks written by hand
protocol · implone operation, many realizationsfive ingest functions; eight faculty routes on one builtin
activate · whereretrievaltraversals written by hand
test · seed · assertverificationa framework; 5 of 13 native suites failing
parallel · traceconcurrencypthreads in C
+
+ +

Plus, from the spec's own status: annotations parsed and skipped, match parsed and emitting nothing, ? a no-op, % unlexed, structs as ElMap, enums as strings, selective import unenforced.

+ +

07Open

+ +
What does a declaration bind to, exactly?If cat names a region that shifts and completes against context, what is written at the declaration site and what is resolved at use? This is the centre and it is unspecified.
+ +
Is the faculty list right?Seven, derived from what broke. Derived-from-failure is a biased sample — it finds what is loud, not what is absent. Which faculty is missing entirely and therefore never failed?
+ +
Which concept has the highest leverage on the next turn?The prologue/epilogue seam (§19.3 names it as the prerequisite; its stated blocker has expired; it collapses 62 + 10 convention sites), protocol/impl, or resolution itself. The §05 criterion should decide this, not preference.
+ +
What seam makes cognition non-optional?"Use the ops" is itself a convention — present every turn, enforced by nothing, ~100% failure across a full session. A stronger instruction is still a convention. What makes reasoning outside the substrate fail, the way @manager makes dharma_emit outside the boundary a compile error rather than a lint?
+ +
+

Every number here is measured or quoted from lang/spec/language.md. Nothing is inferred and presented as fact. El is self-hosting: all of this can change and be rebuilt.

+ +
diff --git a/docs/architecture/el-language-design.md b/docs/architecture/el-language-design.md new file mode 100644 index 0000000..cc49715 --- /dev/null +++ b/docs/architecture/el-language-design.md @@ -0,0 +1,245 @@ +# El — Language Design + +**Status:** decisions recorded, design unwritten. +**Date:** 2026-08-17. +**Provenance:** decisions are Will's, taken in session. Items marked *proposed* are not +decided and are recorded only so the reasoning isn't lost. Items marked **OPEN** are +his to rule on and must not be guessed at. + +Companion documents: `el-architecture.html` (the measured state — see §7 note on its +§04 scoreboard), and `design/completing-el.html` (whiteboard v0: the reduction, the +faculty table, the ordering principle). + +--- + +## 1. The reduction + +`language.md` §18.0 records five concerns that decayed into conventions: + +| Concern | Fragments | The convention it became | +|---|---|---| +| Process identity | 0 guards | "check nothing is already running first" | +| Configuration | 20 env vars | "remember the right default here" | +| Durability | 62 call sites | "after you mutate, remember to persist" | +| Request auth | 10 per-route | "check the token in this handler too" | +| Index-after-append | 9 of 9 failed | "after you append, remember to index" | + +The last row is the strongest available evidence about what this class of convention +is worth: **it failed at 100% of its sites.** + +Every one of these is an obligation at a **crossing** — a point where a value moves +between regions. El can name a region and it can name a call. A call is procedural, +so the obligation degrades into something a human must remember to perform. + +> **The generator, one level up:** El cannot name what holds at a crossing. + +And underneath that: + +> **The deeper absence:** El cannot name the thing meaning is made of. + +`semel` appears in whitepaper §84, §86, §209, §737, in +`the-metaphysics-of-will-anderson.md`, and in session notes. It appears in **zero code +identifiers**. Every geometric concept in the system — region, neighbourhood, manifold, +world-tube — is defined in terms of a unit the language cannot say, while the code +underneath speaks in arrays, floats and offsets: the vocabulary of a voxel, a value at +a dumb address. Precisely the thing the impact brief says a semel is not. + +`el_runtime.c` is a concept that leaked into C. `semel` never got that far — it did +not even decay into a convention. + +--- + +## 2. DECIDED — `semel` is the primitive + +**A semel is a difference that matters. The smallest unit of understanding.** + +Not a node. Not a coordinate. Not a float. + +The reasoning, in Will's terms: + +- Meaning is position, and position is only ever relative. *"There is no atom of + meaning that isn't already a relation. It grounds on nothing but difference — two + points and the gap, and the gap is pure not-the-same."* +- A node doesn't mean. A node is a label at a location; labels don't mean. +- A lone coordinate doesn't mean either. Nothing means anything by itself. +- The smallest thing that can be understood is a **distinction**: *these two are not + the same.* Below that there is no content to apprehend. +- And a difference with nothing it matters to is not meaning — it is variation. The + mattering is not decoration; it is what makes it understanding rather than data. + +**Consequence: relating is the floor, and the point is derived.** The +point-primitive / relation-primitive fork raised in session is not a fork. It was +answered by the definition. + +### Historical note, to be recorded as fact rather than as origin story + +The term was coined by Will on the pixel/voxel/texel pattern — *semantic element*, +and Latin *semel*, "once, a single time." It was recognised, not invented, from a +2019 experience he calls **semelation**: perceiving mind as a high-dimensional point +space. The initial reading was "pixels"; the correction to `semel` was made later and +was made on the **mechanism** — a pixel is a value at an address, and what was +perceived had no separate address and value. + +Convergence worth citing, not deferring to: neural population geometry and +representational similarity analysis independently model cognition as position in a +high-dimensional space where similarity is distance. + +--- + +## 3. DECIDED — `semel` lands first + +By the ordering criterion already on the whiteboard: *which concept, added to El, most +increases the ability to add the next one?* Not size of payoff — **leverage on the next +iteration**, because El compiles itself and the fixpoint makes each turn provable in +2.9s. + +**Every other concept on the board is defined in terms of `semel`. It is maximal on +that criterion by construction.** + +--- + +## 4. DECIDED — `ground` is the checker + +Whiteboard question 4 — *does `ground` in El mean the same thing as `ground` in the +engram?* — is answered: **yes, and it should be one implementation.** + +If a declaration names a region, then type checking is asking whether the geometry +supports the use. That is not unification. **That is grounding**, and it is already +built, proven, and byte-identically reproducible: + +``` +cc -std=c11 -O2 -o gep_proof gep_proof.c -lm && ./gep_proof + +C1 5 independent sources pos_mass 1.3500 n_indep=5 0.1000 → 0.9741 GROUNDED +C2 5 mutually-linked pos_mass 0.2700 n_indep=1 0.1000 → 0.1000 refused +C3 1 source, 5 parallel edges pos_mass 0.2700 n_indep=1 0.1000 → 0.1000 refused +``` + +Independence-weighted grounding is the general case; execution is the cheap case. +**Attestation is `verify` where nothing can be run** — as already implemented for +language in `authority.py`, where an LLM proposes and a primary source disposes. + +At the point where the checker and the grounder are one mechanism, the language and +the mind stop being two things. + +--- + +## 5. OPEN — Will's to rule on + +### 5.1 What is a semel's representation in the language? + +*Proposed, not decided:* a **displacement from `love = 0`** — a relation held as one +object. It reconciles "the address is the value" with "position is only ever relative," +because a displacement *is* a relation and is still a single nameable thing. + +If taken, the operator set falls out rather than being bolted on: + +``` +subtract(now, then) → what changed (growth, drift) +translate origin → empathy +rotate frame → reframe +project onto axis → a lens +change basis → analogy, metaphor, skill transfer +reflect an axis → negation, sarcasm +``` + +Three consequences that would hold: + +- **Dimension must never appear in the type.** `semel` opaque, never `[768]float`. + The moment the arity is in the language, the manifold's implementation is in the + language, and adding a modality requires a runtime patch — which the standing rule + forbids. +- **Zero is the only literal.** Everything else is reached by displacement from it, + which makes `love = 0` the base case rather than philosophy adjacent to the type + system. +- **`magnitude` is standing.** Distance from origin is the same quantity + `gep_core.h` already computes. + +### 5.2 Is `hold` one construct or two? + +The obligation *before* a crossing (auth, guard) and the obligation *after* (persist, +index, free) may be one shape seen from both sides, or the seam may need both faces +named. This decides whether §19.3's prologue/epilogue seam is one construct or a pair. + +**Precedent already shipping:** `@manager` makes `dharma_emit` outside the boundary a +**compile error, not a lint.** The concept is proven at N=1; the work is generalising +it and naming it. + +**And the shape is already implemented in the learning region:** `L.reach_out` sits +between `L.detect_gap` and `L.verify`. You cannot reach out without a detected gap and +you cannot keep what returns without passing verify. **A hold is a neighbour.** The +obligation is not attached to the crossing — the obligation *is* the adjacent node. +That is why `reach_out` cannot be abused and why 62 persist sites could be. + +### 5.3 What does a declaration bind? + +If `cat` names a region rather than a struct — one that shifts and completes against +the engram and the neighbouring code — what is written at the declaration site, and +what is resolved at use? **This is the centre and it is specified nowhere.** + +Falls out of 5.1 if displacement is taken: a declaration **locates** rather than +allocates. + +### 5.4 Is the faculty list right? + +Seven were derived from what broke. Derived-from-failure is a biased sample — it finds +what is loud, not what is missing. **What faculty is absent entirely and therefore +never failed?** + +--- + +## 6. The residue map + +What each construct must absorb, from §18.0 plus measured state: + +| Residue | Count | Absorbed by | +|---|---|---| +| persist-after-mutate | 62 sites | `hold` (after-crossing) | +| auth-per-route | 10 sites | `hold` (before-crossing) | +| index-after-append | 9 of 9 failed | `hold` (after-crossing) | +| env var defaults | 20 | configuration declared once | +| process identity | 0 guards | `hold` (before-crossing) | +| `geometry_free` at every call site | every site | ownership follows from `semel` | +| five ingest functions where there is one | 5 → 1 | `protocol` / `impl` | +| `el_runtime.c` | 20,504 lines | faculty decomposition, ordered after `semel` | + +--- + +## 7. Notes carried forward + +**`el-architecture.html` §04 needs its numbers sourced or cut.** An audit found the +faculty scoreboard — `Ingest 2 min → 0 nodes`, `Recall self ranked 8th`, +`Body CC 356 / 1,626 ln`, `the verifier answers in 2.9s`, `5 of 13 native suites +failing` — has no supporting evidence in the repository, under a footer asserting +*"nothing is inferred and presented as fact."* Against a corpus whose documents +supersede their own conclusions in place, that is the one file that would not survive +scrutiny. Fix or remove. + +**Source as a projection surface is claimed and unimplemented.** `el-architecture.html` +§147/§150: *"if source is a surface, then emitting a corrected file is projection."* +Greps for `surface_profile_code`, `emit_source` → zero hits. + +It is not unbacked. **It was demonstrated on 2026-08-14** — three faculties (phonetic, +semantic, procedural) projected into TypeScript, a surface the system had never used, +with the network severed. Recovered at +`~/Development/neuron-technologies/andre-server-recovered/` and copied into +`evidence/03-andre-demo/`. The claim needs bringing home to El, not proving. + +**`hold` is the highest-leverage construct after `semel`** — it collapses 62 + 10 + 9 +sites and unblocks the runtime extraction. §19.3 names the prologue/epilogue seam as +the prerequisite and its stated blocker has expired. + +--- + +## 8. What is not decided and must not be guessed + +- The representation of `semel` (§5.1) +- One `hold` or two (§5.2) +- What a declaration binds (§5.3) +- The missing faculty (§5.4) +- Sequencing after `semel` — the ordering criterion decides it, not preference + +--- + +*Recorded 2026-08-17. Everything in §2, §3 and §4 is decided. Everything in §5 is open +and is Will's. Nothing here was inferred from a document that was not read.* diff --git a/docs/architecture/geometry-vs-code.md b/docs/architecture/geometry-vs-code.md new file mode 100644 index 0000000..27918b3 --- /dev/null +++ b/docs/architecture/geometry-vs-code.md @@ -0,0 +1,117 @@ +# Geometry or Code + +**Running list.** Append as decided. Started 2026-08-17. + +**The test:** *is this an arbitrary convention, or is it a relation?* + +Conventions were agreed by people and could have been otherwise — a RIFF header could +have used a different magic number. Nothing derives them; they must be written down. + +Relations are not agreed. Distance is distance. Anything whose answer is *where is this +relative to that* is geometry, and writing it as code is the error the whole effort is +correcting. + +**Second test, for the hard cases:** *if I write this as code, am I encoding in +`if`-statements a distinction the geometry was built to hold?* If yes, it's geometry. + +--- + +## Pure geometry + +| Thing | Because | +|---|---| +| Meaning | position | +| Grounding / standing | the weight on the edge — a magnitude, not a computation | +| Learning | standing changing over time | +| A gap | low standing | +| Wonder | a gap with a pull weight | +| Type checking | is this position in that region — distance | +| Dispatch | position, not a tag | +| Recall | re-origining at a region; projection, not replay | +| Reasoning | traversal | +| Deduction | containment. There is no procedure | +| Counting | a position, not a loop's output | +| Similarity / difference / residue | subtract | +| Analogy, metaphor, skill transfer | change of basis | +| Negation, sarcasm | reflect an axis | +| Empathy | translate the origin | +| Reframe | rotate the frame | +| A lens | project onto an axis | +| Rhyme | distance in phonetic space | +| Humour | intersection of regions — fart-meaning ∩ funny ∩ form | +| Idiom detection | the whole unit sits farther out than its parts | +| Self | a world-tube — a trajectory through the manifold | +| Consolidation | episodic → semantic promotion | +| Reification | dense regions cohering; runs on the beat, has no caller | +| Cross-cutting concerns | **dissolved** — a hold is a *neighbour*. Adjacency, not tracking. **Implemented 2026-08-17**: a construct declares what runs at a crossing, and it resolves at execution — see the runtime seam. | +| Effects | topology. `reach_out` is bounded by `detect_gap` and `verify` because those are its edges | +| Capability | position relative to a boundary. In C it is already spelled `const` | +| The AST | a projection of geometry into a tree — a surface, not the centre | +| Source code | a surface, like text, audio, image | + +## Must be code + +| Thing | Because | +|---|---| +| Sensors — mic, camera, file read, socket | the physical touch. I/O is where the world arrives | +| Byte formats — RIFF, PNG chunks, `MThd`, OOXML | arbitrary convention. A committee chose the magic numbers | +| CRC32 polynomial, Adler32, zlib framing | same — agreed constants, derivable from nothing | +| Cosine, distance, the float arithmetic | the machinery that *walks* the geometry is not itself geometry | +| Arena, refcount, allocator | bookkeeping for the **representation**, not for the positions | +| Locks, threads, publication boundary | the hardware is code. **Ordering is not** — see Answered, above. Coordination is required only where state is non-monotone. | +| WAL, page layout, ARIES recovery | durability against a physical device that can lose power | +| Emission — writing C or JS text | the final surface has to be *typed out* by something | +| OS interaction — launchd, spawn, signals | outside the system by definition | +| Device realizers — `el_audio_darwin.m`, `el_capture_darwin.m` | OS frameworks. Correctly already isolated, zero network | + +--- + +## The ones I would have written as code, and was wrong about + +Recorded because the error has a pattern and the pattern is the point. + +| Thing | What I reached for | What it is | +|---|---|---| +| Rhyme | a rhyming dictionary, or an API call | distance between rime tails | +| Fart onomatopoeia | a 30-element string literal | an intersection of three regions | +| "Funny" | a scorer with `if`-statements | a relational neighbourhood grounded in a voice | +| Representation vs description | a hardcoded blacklist containing `raspberry` | falls out of lexicon membership × phonetic comedy | +| Video | a codec, sized as a project | one more surface profile | +| Type checking | a phase between parse and emit | reading a distance that already exists | +| Grounding | a call site, an obligation, a discharge | it has no caller. It just runs | +| N transducers, N realizers | one component per modality | zero of each. Sensors and bases at the skin | + +**The pattern:** every one is *encoding in code a distinction the geometry was built to +hold.* The tell is that the code version is a **fixed enumeration** — a list, a table, a +blacklist, a set of branches — and the geometry version is a **measurement**. + +If the implementation contains a literal set of the right answers, it is in the wrong +column. + +--- + +## Answered + +| Thing | The answer | +|---|---| +| Concurrency | **Ordering is geometric.** Causality is a partial order (Lamport 1978); a total order is an arbitrary extension of it and "cannot be depended on to imply a causal relationship." Programming languages force you to write a total order, so authoring *invents* constraints the problem never had — and every lock, barrier, fence and consensus protocol is apparatus for recovering the partial order destroyed at authoring time. CALM (Hellerstein/Alvaro, proven by Ameloot et al.): a program has a consistent coordination-free implementation **iff it is monotone**. What breaks monotonicity is destructive update. **Coordination is the price of forgetting.** | +| The module system | **Premature — the partition is a filesystem path, not a neighbourhood, and there is no namespacing at all.** `import` is textual inlining (guarded against double inclusion); when a `.elh` header exists the header is inlined instead and symbols resolve at C link time, so linking is real and delegated to C. Two modules defining `helper` emit two C functions into one translation unit. Linking barely survives the *path* partition, so whether it survives a neighbourhood partition cannot yet be asked. | +| Numeric literals | **The numeral is convention; the number is a position — and a bare `3` is a MAGNITUDE WITH NO AXIS.** `int_to_str` was already form 1: nothing determines that twelve is written `1` then `2`. But a literal is not a position until something gives it a direction, which is why `3.days` needs a calendar. Measured consequence: `Duration + Int` was refused ("an Int carries no unit") while `Instant + Int` compiled to raw `(t + 3)` and reported clean — silently moving a point by an unspecified amount. The rule was simply never written. Now: `t + 3` is refused, `t + 1.hour` is accepted, because `.hour` supplies the axis. | +| Parsing | **A grammar is a basis; parsing is transduction onto it.** The lexeme→token map is convention (`fn` could have been `def`); shape recognition is a region; the byte traversal is irreducible, like every other traversal. Three things favour *region* for the act: ambiguity (`a * b` needs context — a grammar resolves it with the lexer hack, a region by neighbourhood), error recovery (nearest-match is free), and precedence, which is ordering along an axis with a conventional parameter. **But the SHOULD gate refuses the obvious move:** the keyword table stays code, because the set is closed by the language definition and the lexer runs before the program is understood, so a program can never declare its own keywords. Externalising it costs I/O per compile for zero flexibility — the same verdict as `is_digit` in ASCII. What was actually wrong: 5 of 46 keywords were consumed by nothing, and using one silently miscompiled. | +| Error handling | **`grounded: false` covers not-knowing; it does not cover failed.** Standing is a *signed* component: `> 0` supported, `= 0` unknown, `< 0` contradicted. Not-known and known-false are opposite directions on one axis and a boolean cannot tell them apart. `inhibitory` as an int32 flag is that sign wearing a boolean. | + +## Fourth proof form + +**4 — ADVERSARIAL EXACTNESS.** Where approximation is a break, geometry is +excluded. A cryptographic hash is a *deliberately structure-destroying* map: +near inputs land at maximally uncorrelated outputs. Geometry is the claim that +near things stay near — a manifold that approximated SHA-256 would *be* a break +of SHA-256. Signature verification is the same: 0.99-valid is invalid. And +X25519 **is** geometry, a group on an elliptic curve, which is precisely why it +must be code: its security is the hardness of moving in that geometry. + +**Form 1 no longer survives as a verdict.** Every row it justified turned out to +be a *basis*, not a capability. RFC 8259 fixes where the commas go — that is a +surface, and projecting onto a surface is geometry. A convention describes the +basis you project onto; it never describes an act. + diff --git a/docs/v1/experiments/README.md b/docs/v1/experiments/README.md new file mode 100644 index 0000000..dbf50c5 --- /dev/null +++ b/docs/v1/experiments/README.md @@ -0,0 +1,59 @@ +# v1 — Experiments + +Every change to El on `iteration-1` was produced by one loop, run repeatedly: + +``` +Ishikawa → scientific method → Six Sigma → repeat +``` + +- **Ishikawa** — name the root cause, not the symptom. *Why is this table here?* + never *why is this table ugly?* +- **Scientific method** — state a hypothesis, **commit predictions before + running**, then run it in an isolated worktree and grade every prediction + including the ones that failed. +- **Six Sigma** — eliminate the defect *class*, then add a control so it cannot + silently return. + +## The organising finding + +**Predictions that came back FALSE were worth more than the ones that held.** + +Nineteen cycles, sixty-one predictions. The eleven that failed produced every +significant result: + +| Failed prediction | What it found | +|---|---| +| "the arity table has drifted from the header" | Zero drift — but **110 functions had no entry at all**. The table was not wrong, it was 40% incomplete. | +| "codegen drops below baseline" (×4) | The **traversal is irreducible**. Walking an AST to find calls does not move no matter who decides. Only the rule and the judgment leave. | +| "guards cannot refuse through the seam" | One line, and refusal works. Six compile-time kinds were unnecessary. | +| "C forbids the struct redefinition" | C allows shadowing — and a *different* defect surfaced: an exit injection emitted with an empty target. | +| "routing el_bin_lookup through the gate fixes the SIGSEGV" | It did not. The **fallback** was the hazard: `strlen()` on an integer. I would have shipped the wrong fix and called it verified. | + +A prediction that only ever confirms is a demonstration, not a test. One cycle +was run **without** committing predictions first — `async-half-expressible` — +and it produced a rigged result: `pthread_join` immediately after +`pthread_create`, with the word `DEFERRED` printed by the test itself. It had to +be discarded and re-run. + +## Layout + +``` +cycles/ one file per loop, numbered in order, named for the DEFECT +findings/ what the cycles produced, cross-cut by kind +``` + +## Scoreboard + +``` +cycles run 19 +predictions committed 61 +predictions FALSE 11 ← the useful ones +silent miscompilations found 4 +security-relevant defects 2 +architecture questions closed 5 +defects in my own measurement 4 +``` + +Every cycle verified the same three things before landing: the compiler +self-hosts byte-identically (gen2 == gen3), the native suite passes, and the +integration harnesses pass. A cycle that could not show all three did not land. diff --git a/docs/v1/experiments/cycles/00-INDEX.md b/docs/v1/experiments/cycles/00-INDEX.md new file mode 100644 index 0000000..3eaac67 --- /dev/null +++ b/docs/v1/experiments/cycles/00-INDEX.md @@ -0,0 +1,26 @@ +# Cycles + +Each is one `Ishikawa → scientific method → Six Sigma` loop, run in an isolated +worktree so a wrong answer cost nothing. Named for the **defect**, not the fix. + +| # | Cycle | Root cause | Predictions | Landed | +|---|---|---|---|---| +| 01 | [constructs-have-nowhere-to-be](01-constructs-have-nowhere-to-be.md) | a construct had nothing to BE, so its meaning lived in the emitter | 3/3 | yes | +| 02 | [a-construct-cannot-refuse](02-a-construct-cannot-refuse.md) | injection discards the target's result; no form said no | 4/4 | yes | +| 03 | [the-wrapper-was-conditional](03-the-wrapper-was-conditional.md) | exit injection needed compile-time knowledge only because the wrapper was conditional | 3/4 | yes | +| 04 | [c-has-no-closure-syntax](04-c-has-no-closure-syntax.md) | "C has no closures" taken as a fact about what is possible | 5/7 | yes | +| 05 | [the-emitter-discards-what-it-knows](05-the-emitter-discards-what-it-knows.md) | codegen sees every construct relation and throws it away | 5/5 | branch | +| 06 | [the-crossing-resolves-at-emission](06-the-crossing-resolves-at-emission.md) | the binary has no table to consult | 3/4 | yes | +| 07 | [invocation-is-not-composable](07-invocation-is-not-composable.md) | the wrapper called the target directly | 5/5 | yes | +| 08 | [the-emitter-adjudicates](08-the-emitter-adjudicates.md) | a prohibition had nowhere to live but a `#error` | 4/5 | yes | +| 09 | [policy-inside-the-compiler](09-policy-inside-the-compiler.md) | a program cannot declare its own restrictions, so the tier policy was compiled in | 4/5 | yes | +| 10 | [a-second-copy-of-the-header](10-a-second-copy-of-the-header.md) | builtin arity hand-maintained beside `el_runtime.h` | 4/5 | yes | +| 11 | [one-type-erases-the-return](11-one-type-erases-the-return.md) | `el_val_t` means the header cannot say `now()` returns an Instant | 4/5 | yes | +| 12 | [judgment-lives-with-knowledge](12-judgment-lives-with-knowledge.md) | the emitter knows the types, so it also judged them | 5/5 | yes | +| 13 | [thirty-five-return-types](13-thirty-five-return-types.md) | `is_int_call` hardcoded what drives `+` dispatch | 6/6 | yes | +| 14 | [keywords-that-reserve-nothing](14-keywords-that-reserve-nothing.md) | 5 of 46 keywords consumed by no path | 6/6 | yes | +| 15 | [no-namespacing-at-all](15-no-namespacing-at-all.md) | `import` is textual inlining; every name is global | 4/4 | yes | +| 16 | [tokens-carry-no-position](16-tokens-carry-no-position.md) | a token was `(kind, value)`, so no diagnostic could name a place | 6/6 | yes | +| 17 | [annotations-are-never-checked](17-annotations-are-never-checked.md) | the annotation feeds dispatch and is never verified | 6/6 | branch | +| 18 | [async-half-expressible](18-async-half-expressible.md) | **first attempt was DOGMA** — no predictions, rigged test | 4/4 (2nd) | branch | +| 19 | [a-convention-is-not-a-gate](19-a-convention-is-not-a-gate.md) | `looks_like_heap_obj` is static, so every type re-derives it | 6/7 | yes | diff --git a/docs/v1/experiments/cycles/01-constructs-have-nowhere-to-be.md b/docs/v1/experiments/cycles/01-constructs-have-nowhere-to-be.md new file mode 100644 index 0000000..560edbf --- /dev/null +++ b/docs/v1/experiments/cycles/01-constructs-have-nowhere-to-be.md @@ -0,0 +1,42 @@ +# constructs have nowhere to be + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `5718943` + +``` +let a construct declare its own meaning instead of the emitter knowing it + +codegen called fn_has_decorator for exactly three names — manager, accessor, +route. Twelve others parsed, attached as {name,args}, and compiled to nothing, +including four that look like protection: @authenticate (6 uses), @authorize +(3), @rate_limit (3), @validate (2). The cause was not that the branches were +untidy. A construct had nothing to BE, so its meaning had nowhere to live +except the emitter, and every construct was therefore a compiler edit. + +A name -> injection table would have moved the enumeration twenty lines up +without removing it. So the construct now carries its own meaning: + + @decorator("injects_at_entry", "engram_boundary_beat") + fn audited() {} + + @audited + fn risky_op() -> Int { ... } // gets the beat, attributed to "audited" + +scan_declared_decorators is a token-level pre-pass beside scan_routes, forced +by streaming codegen having no whole-program AST. manager and accessor are +seeded as the compiled-in core — the fixedSelf shape from substrate.go: a +complete fallback exists, declaration is enrichment. + +This is the injection half of the seam only. The prohibition half (@manager's +#error on dharma_emit) stays hardcoded, because "which calls may appear inside +this boundary" is a query over program structure and there is nothing yet to +ask. + +Verified three ways: emitted C for existing @manager/@accessor code is +byte-identical to the hardcoded path; a construct with a name the compiler has +never heard of injects correctly; the compiler self-hosts byte-identically. +90/90 native compiler tests pass. +``` diff --git a/docs/v1/experiments/cycles/02-a-construct-cannot-refuse.md b/docs/v1/experiments/cycles/02-a-construct-cannot-refuse.md new file mode 100644 index 0000000..f08cc0e --- /dev/null +++ b/docs/v1/experiments/cycles/02-a-construct-cannot-refuse.md @@ -0,0 +1,43 @@ +# a construct cannot refuse + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `60737b0` + +``` +let a construct refuse, not only observe + +@authenticate (6 uses), @authorize (3), @rate_limit (3) and @validate (2) +parsed, attached, and compiled to nothing. Fourteen applications that read as +protection and emitted no instruction — a function decorated @authenticate +compiled byte-identically to an undecorated one. + +The missing capability was not authentication. It was that a construct could +observe a boundary but never refuse one. injects_at_entry discards the target's +result; there was no form in which a construct could say no. + + @decorator("guards_at_entry", "my_auth") + fn authenticate() {} + + @authenticate + @authorize + fn handler() -> String { ... } + +emits, at entry: + + { el_val_t __g = my_auth(EL_STR("handler"), EL_STR("authenticate")); if (__g) return __g; } + { el_val_t __g = my_roles(EL_STR("handler"), EL_STR("authorize")); if (__g) return __g; } + +Guards precede injections because a refused call must not report a crossing, +and every guard runs where the topmost injecting construct wins — refusal is +not a role, so it does not follow the role convention. + +The compiler still knows nothing about auth. The program points the construct +at its own function, which is where that decision belongs. + +Verified: existing @manager/@accessor output byte-identical, compiler +self-hosts byte-identically, guards stack in declaration order and emit before +the beat. 94/94 native compiler tests pass. +``` diff --git a/docs/v1/experiments/cycles/03-the-wrapper-was-conditional.md b/docs/v1/experiments/cycles/03-the-wrapper-was-conditional.md new file mode 100644 index 0000000..862388b --- /dev/null +++ b/docs/v1/experiments/cycles/03-the-wrapper-was-conditional.md @@ -0,0 +1,82 @@ +# the wrapper was conditional + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `4f7568b` + +``` +give a construct its after-crossing face, and let constructs compose + +§6 records 62 persist-after-mutate sites, 10 auth-per-route, and +index-after-append that failed at 9 of 9 — every one an obligation at a +crossing that decayed into "remember to do this afterwards." An obligation a +human must remember is not an obligation, and the 9-of-9 figure is what that +costs. + + @decorator("injects_at_exit", "persist_now") + fn durable() {} + +The body moves into a static helper and the visible fn becomes a wrapper, so +EARLY RETURNS pass through the exit injection. Emitting it only before the +fall-through return would have silently missed every early return — the exact +failure class this seam exists to remove. Fns with no exit construct emit +byte-identically to before. + +Three independent constructs now compose on one fn, none known to the compiler: + + el_val_t mutate(el_val_t k) { + { el_val_t __g = my_auth(EL_STR("mutate"), EL_STR("authenticate")); if (__g) return __g; } + engram_boundary_beat(EL_STR("mutate"), EL_STR("manager")); + el_val_t __r = __el_body_mutate(k); + persist_now(EL_STR("mutate"), EL_STR("durable"), __r); + return __r; + } + +Guard, then entry, then body, then exit. §5.2 asked whether `hold` is one +construct or two; the implementation answers one construct with two faces, +selected by declared kind rather than by two mechanisms. + +Verified: existing output byte-identical, compiler self-hosts byte-identically, +early returns pass through the exit, ordering holds under composition. 98/98 +native compiler tests pass. +``` + +## Record — `285166c` + +``` +EXPERIMENT: emit the wrapper unconditionally, so exit binds at runtime too + +ISHIKAWA: why did exit injection still need compile-time knowledge? Because the +body-helper wrapper was only emitted when codegen already knew an exit +construct existed. The wrapper being conditional was the cause, not the wrapper +being necessary. + +PREDICTIONS AND RESULTS + P1 exit becomes runtime-bindable TRUE returns 14, bound + after the build + P2 codegen shrinks TRUE 5094 -> 5044 + P3 cost 5-15% from a call frame on every fn FALSE 0.37s -> 0.38s, ~3% + P4 fixpoint holds TRUE + +Every fn now gets a body helper and a wrapper. It has to be unconditional: +early returns must route through something for an exit construct to observe +them, and codegen cannot know which fns will be bound after the binary exists. + +Removed with the machinery: declare_exit, decorator_exit, cg_exit_target, +cg_exit_construct, and the injects_at_exit scanner branch. + +Two controls failed and were rewritten rather than repaired -- +no-exit-construct-emits-no-wrapper asserted the optimisation this removes, so +it is now inverted. The integration harness gained a seventh assertion: an exit +construct declared after the build replaces the result. + +99/99 native, 7/7 integration, fixpoint gen2==gen3. +``` + +## Record — `b40754f` + +``` +land unconditional wrapper: exit crossings resolve at runtime +``` diff --git a/docs/v1/experiments/cycles/04-c-has-no-closure-syntax.md b/docs/v1/experiments/cycles/04-c-has-no-closure-syntax.md new file mode 100644 index 0000000..b72aff4 --- /dev/null +++ b/docs/v1/experiments/cycles/04-c-has-no-closure-syntax.md @@ -0,0 +1,64 @@ +# c has no closure syntax + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `2bed848` + +``` +EXPERIMENT: hand the construct the body as a real closure + +ROOT CAUSE of the weaker design: "C has no closures" was taken as a fact about +what is possible. It is a fact about one grammar. Every C++ lambda, every Go +closure, every Rust closure compiles to a struct of captured values plus a +function pointer -- which is what is emitted here. Codegen emits C; it is not +written in C's syntax, and the distinction is the whole difference between a +construct that can only decide whether to repeat and one that controls +invocation. + +It would also have crippled the JS backend, which has closures natively, for a +limit that applies only to the C one. + +PREDICTIONS AND RESULTS + 1 env struct + thunk taking void* TRUE + 2 fails to compile: struct redefinition FALSE -- C allows the + inner declaration to shadow. Prediction wrong; C is more permissive than + assumed. A different real defect surfaced instead: a wrap with no exit + construct emitted `(EL_STR("f"), EL_STR(""), __r);` -- a call to an empty + target -- because has_exit was reused as "needs a wrapper" and the exit line + was emitted unconditionally. Fixed. + 3 compiles when the target is declared in El FALSE -- and this is + the root cause worth keeping: El has ONE type, el_val_t = int64_t. El's type + system cannot describe a callable, so `extern fn` and the real signature + cannot be made to agree in El's own vocabulary. The fix is not a cast: + codegen DEFINES the wrap calling convention, so codegen emits the extern + declaration. The convention is not El-expressible; it is emitted. + 4 target controls invocation, 0..N times TRUE + 5 existing @manager output byte-identical TRUE + 6 compiler fixpoint holds TRUE + 7 emitting the convention makes it compile TRUE + +MEASURED + base(5) wrapped by a target that invokes the body twice and sums -> 10 + never_runs(5) wrapped by a target that never invokes it -> 999 + +Neither is expressible by "decide whether to repeat". This supersedes the +repeats_body experiment on experiment/repeats-body, which was built around the +mistaken limit. +``` + +## Record — `7d01608` + +``` +land wraps_body: a construct controls invocation + +Proven on experiment/wraps-body (2bed848): base(5) wrapped by a target that +invokes the body twice returns 10; a target that never invokes it returns 999. +Neither is expressible by deciding whether to repeat. + +Root cause it corrected: 'C has no closures' is a fact about one grammar, not +about what can be emitted. And El's single type (el_val_t = int64_t) cannot +describe a callable, so codegen emits the calling convention rather than asking +El's type system for something it structurally cannot say. +``` diff --git a/docs/v1/experiments/cycles/05-the-emitter-discards-what-it-knows.md b/docs/v1/experiments/cycles/05-the-emitter-discards-what-it-knows.md new file mode 100644 index 0000000..98caa34 --- /dev/null +++ b/docs/v1/experiments/cycles/05-the-emitter-discards-what-it-knows.md @@ -0,0 +1,48 @@ +# the emitter discards what it knows + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `a5af871` + +``` +EXPERIMENT: let the compiler write down what it already knows + +HYPOTHESIS: attribution is redundant for static structure. Codegen sees every +construct-to-function relation at emission time and discards it, so the only +way to learn the structure back is to run the program and read what it +reported. That is instrumentation compensating for erasure. + +PREDICTIONS, committed before running: + 1 derivable at compile time with no runtime call expected TRUE + 2 complete for guards and exits (invisible today) expected TRUE + 3 answers it for code that has never executed expected TRUE + 4 deterministic expected TRUE + 5 makes the entry beat redundant expected FALSE + +RESULT: 5/5 as predicted. From a program that was never executed: + + authenticate guards_at_entry login my_auth + durable injects_at_exit save persist_now + authenticate guards_at_entry critical my_auth + durable injects_at_exit critical persist_now + manager injects_at_entry critical engram_boundary_beat + +Prediction 5 held: the relation records that a boundary COULD be crossed, the +beat records that it WAS. They are different facts and neither replaces the +other. + +CONSEQUENCE, and it undercuts the first pass on iteration-1: construct identity +was available at compile time all along. With relations recorded at build, the +runtime needs only the function name and attribution becomes a join rather than +a payload. The counter-argument is that the payload is self-describing while +the file must be pinned to the artifact or the two drift and attribution is +silently lost — which is the same conclusion as "compile against a manifold +revision and record the revision in the artifact", reached from the other side. + +Written to a file rather than the engram on purpose: a compile that consults a +manifold produces different output from identical source at different times. +The file is content-addressed; the engram ingests it. Determinism preserved, +mechanism proven. +``` diff --git a/docs/v1/experiments/cycles/06-the-crossing-resolves-at-emission.md b/docs/v1/experiments/cycles/06-the-crossing-resolves-at-emission.md new file mode 100644 index 0000000..a10ae11 --- /dev/null +++ b/docs/v1/experiments/cycles/06-the-crossing-resolves-at-emission.md @@ -0,0 +1,170 @@ +# the crossing resolves at emission + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `35b07ba` + +``` +EXPERIMENT: resolve the crossing at execution, not at emission + +HYPOTHESIS (Will's): a compiler whose one compiled mechanism is extending the +LANGUAGE — not the compiler — can compose without recompilation. + +ISHIKAWA — why does a construct require a recompile today? + method codegen inlines the target call into the body + machine the binary has no table to consult + material the declaration lives in source, read at compile time + measurement nothing observes what applied at runtime + root cause the crossing is resolved at EMISSION, not at EXECUTION + +CHANGE: codegen emits one unconditional indirection per fn. Which constructs +apply is read from a table that can be written AFTER the binary exists; +targets resolve through dlsym against the running image. + +PREDICTIONS AND RESULTS + P1 a construct declared after the build applies TRUE + P2 an unlinked target is skipped, not fatal TRUE + P3 emitting on every fn is measurably slower FALSE — 0.37s -> 0.36s + with 267 indirections and + no bindings. Free unused. + P4 the compiler still self-hosts TRUE (see note) + +DEMONSTRATED: an El program with NO decorator in its source, already compiled +and linked, picked up a construct declared afterwards: + + $ /tmp/seamrun -> 7 + $ echo 'work audited entry audit_entry' > constructs.txt + $ EL_CONSTRUCTS=constructs.txt /tmp/seamrun + AUDIT: work applied by audited + 7 + +P4 note: my first fixpoint test was wrong, not the code. I compared gen1 to +gen2, which must differ whenever codegen's output changes. gen2 == gen3, 267 +seam sites, stable. + +MEASURED COST, and the root cause was not where I looked + 0 bindings 0.36s vs 0.37s baseline free + 2 bindings, dlsym per call 2.45s 6.6x + 2 bindings, resolved once 0.69s 3.5x recovered +The table scan was never the cost. dlsym walks the dynamic symbol table on +every call. Resolve once and cache — which is the smallest form of what +salience does for memory: what is hot stays resolved. The 0.69s residual is +audit_entry's own printf on two of the compiler's hottest functions, not seam +overhead. + +CONSEQUENCE: the five compile-time declaration kinds on iteration-1 are a +compile-time specialisation of something that resolves at runtime. They are not +wrong, but they are not the mechanism — the mechanism is one indirection, and a +kind is data. +``` + +## Record — `886626a` + +``` +seam refusal + control tests: a runtime binding can short-circuit + +Prediction 3 was FALSE. I expected refusal to be impossible through the seam +because the entry indirection discarded its return. One line: + + { el_val_t __s = el_seam_run(EL_STR(f), 0, 0); if (__s) return __s; } + +work() returns 7; bound to a refusing construct AFTER the build it returns 42. +So three of the five compile-time kinds are runtime-bindable: entry injection, +exit injection, and refusal. wraps_body needs invocation control and +prohibits_outside is compile-time by nature. + +104/104 native compiler tests pass. +``` + +## Record — `28d19da` + +``` +strip the compile-time machinery the seam replaces + +PREDICTION: codegen.el drops below 4661, its size before any of these passes. +RESULT: FALSE. 5157 -> 5096. Still +435 over baseline. + + injects_at_entry collapsed into the seam removed + guards_at_entry collapsed into the seam removed + injects_at_exit needs the body-helper wrapper STRUCTURAL + wraps_body needs the closure + wrapper structural + prohibits_outside a #error cannot be emitted at runtime + +The wrapper is not a consequence of compile-time resolution. Early returns must +be routed through something no matter when the target is resolved, so exit +injection was never going to collapse. I predicted it would because I had +conflated "resolved late" with "emitted less". + +What did collapse is entry injection and refusal -- 61 lines of compiler +replaced by one refusable indirection, with the capability now bindable after +the binary exists. + +8 tests fail, and they are exactly the 8 controls for compile-time entry +injection and guards. No unrelated breakage: the controls reported precisely +what moved. They assert emission of something that now happens at runtime, so +they need rewriting as integration tests -- which the framework does not +currently support, because runtime binding needs a built binary and an +environment, not compile_capture. + +Verified after the strip: fixpoint gen2==gen3, observation and refusal both +work through the seam with the compiler knowing nothing about either. +``` + +## Record — `8bbb750` + +``` +control the claim that cannot be unit tested + +The seam's whole claim is that a construct declared AFTER a binary exists +applies to that already-built program. compile_capture only sees emitted text, +so it structurally cannot check this: it needs a built binary, a linked target, +and an environment. Verified by hand until now, which is the standing problem +this session has been about. + +tests/integration/seam_binding.sh builds a probe from El source containing no +construct at all, links a target that El never references, and asserts: + + ok unbound program is unaffected + ok a construct declared AFTER the build applies + ok a construct declared after the build can REFUSE + ok an unlinked target is skipped, not fatal + ok a binding for a different fn does not fire + ok two constructs compose on one crossing + + 6 assertions, 6 passed, 0 failed + +The eight controls that failed after the strip were replaced, not repaired. +They asserted compile-time emission of capability that moved to runtime; +contorting them would have kept an assertion whose subject no longer exists. +Three took their place, asserting the emitted shape, and the behaviour they +used to cover is now the integration harness's job -- which is the honest +division, since the shape and the behaviour are no longer the same fact. + +99/99 native compiler tests pass. Fixpoint holds. +``` + +## Record — `24f7fb5` + +``` +land the runtime seam: resolve the crossing at execution + +Five compile-time passes added 491 lines to the thing that was supposed to stop +growing. The seam is ~55 lines of C and one line of emission, and it does at +runtime what three of those five kinds did at compile time -- for programs that +are already built. + + a construct declared AFTER the binary exists applies to it + free when unused: 0.36s vs 0.37s baseline across 267 indirections + dlsym was the cost, not the table scan; resolve-once recovered 3.5x + refusal works, composition works, unlinked targets are skipped not fatal + +injects_at_exit and wraps_body do NOT collapse: early returns must route +through the body-helper wrapper regardless of when the target is resolved. The +wrapper is structural, which I had wrong. prohibits_outside cannot move at all +-- a #error has no runtime. + +Controls: 99/99 native compiler tests, plus tests/integration/seam_binding.sh +(6/6) for the claim compile_capture structurally cannot see. +``` diff --git a/docs/v1/experiments/cycles/07-invocation-is-not-composable.md b/docs/v1/experiments/cycles/07-invocation-is-not-composable.md new file mode 100644 index 0000000..49ad88a --- /dev/null +++ b/docs/v1/experiments/cycles/07-invocation-is-not-composable.md @@ -0,0 +1,45 @@ +# invocation is not composable + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `bc2f26d` + +``` +EXPERIMENT: invocation control resolves at runtime + +ISHIKAWA: why did wraps_body need compile-time knowledge? Because the wrapper +called the target directly. If the wrapper calls through the seam instead, the +seam can call the body itself, and a construct bound after the build decides +how and whether to invoke it. + +PREDICTIONS AND RESULTS + P1 wrap becomes runtime-bindable TRUE body x3 -> 21, + never invoked -> 111 + P2 codegen shrinks TRUE 5042 -> 4977 + P3 cost 5-10% from an indirect call on every fn TRUE 0.36s -> 0.39s, ~8% + P4 zero-param fns break on the empty struct TRUE empty struct is a GNU + extension, empty init + is C23. Fixed with a + char field. + P5 fixpoint holds TRUE + +PROCESS FAILURE worth recording: my first patch silently did not apply because +I dropped the assert on the string replacement. The build then failed with +"undeclared identifier __thunk_noargs", which I nearly attributed to the +empty-struct prediction. The guard that would have caught it existed and I +removed it -- the same shape as every other defect found tonight. + +Removed: declare_wrap, decorator_wrap, cg_wrap_target, cg_wrap_construct, +params_to_call_args, and the wraps_body scanner branch. + +prohibits_outside is now the ONLY construct kind left at compile time, and it +cannot move: a #error has no runtime. +``` + +## Record — `c04d68f` + +``` +land runtime invocation control: only prohibition remains compiled +``` diff --git a/docs/v1/experiments/cycles/08-the-emitter-adjudicates.md b/docs/v1/experiments/cycles/08-the-emitter-adjudicates.md new file mode 100644 index 0000000..08edf7c --- /dev/null +++ b/docs/v1/experiments/cycles/08-the-emitter-adjudicates.md @@ -0,0 +1,53 @@ +# the emitter adjudicates + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `c741cfe` + +``` +EXPERIMENT: prohibition becomes a query over emitted relations + +I said prohibition could not move because "a #error has no runtime". That +conflated two separable things: WHEN a violation is detected (build time -- +correct, and unchanged) and WHERE the rule and the checker live (the compiler +-- assumed). + +A prohibition is a containment relation over the call graph. So codegen now +records what it saw: + + sneaky calls raw_sql + allowed calls raw_sql + allowed calls @repository + repository calls prohibits:raw_sql + +and tools/check/prohibitions.sh decides, at build time, outside the compiler. + +PREDICTIONS AND RESULTS + P1 codegen can emit the call graph it already walks TRUE + P2 the check becomes a query outside the compiler TRUE + P3 all prohibition decisions leave codegen TRUE zero #errors now + P4 violations still caught at build time TRUE exit=1 + P5 codegen drops below the 4661 baseline FALSE 4962, +301 + +P5 is the finding. The TRAVERSAL is irreducible -- you must walk the AST to +find calls, and those ~120 lines do not move no matter who decides. What is not +irreducible is the rule (which names) or the decision (#error). Those left. I +predicted the whole 223 lines would go because I had not separated walking from +adjudicating. + +Still compiled, and measured rather than assumed: the capability-tier system +(cap_check_call, is_self_formation_call, is_dharma_call, is_llm_call, +cap_record_violation, emit_cap_violations) is 76 lines of the same shape -- +prohibits_WITHIN rather than prohibits_outside, so the checker needs the +opposite polarity to absorb it. + +98/98 native, 4/4 prohibition_query.sh, 7/7 seam_binding.sh, fixpoint ok. +``` + +## Record — `60c07ad` + +``` +land prohibition-as-query: the emitter records, it no longer adjudicates +``` diff --git a/docs/v1/experiments/cycles/09-policy-inside-the-compiler.md b/docs/v1/experiments/cycles/09-policy-inside-the-compiler.md new file mode 100644 index 0000000..cc121ab --- /dev/null +++ b/docs/v1/experiments/cycles/09-policy-inside-the-compiler.md @@ -0,0 +1,49 @@ +# policy inside the compiler + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `c2d9596` + +``` +EXPERIMENT: the capability tier becomes shipped policy plus a query + +Capability differs from prohibits_outside in one way that matters: a utility +program cannot be trusted to declare its own restrictions, because it would +declare none. So the policy comes from OUTSIDE the program -- it ships with the +language as data, editable without a compiler release. + + tools/check/capabilities.rel 18 names that were string literals in codegen + tools/check/capabilities.sh the query that decides + +PREDICTIONS AND RESULTS + P1 codegen emits kind + call graph, drops the 4 name tests TRUE zero #errors + P2 the 18 literals become a data file TRUE + P3 the checker catches capability violations TRUE exit=1 + P4 codegen drops ~76 lines TRUE 4963 -> 4881 + P5 below the 4661 baseline FALSE ~+230 + +TWO DEFECTS THE HARNESS FOUND THAT READING WOULD NOT HAVE + +1. Calls inside main became invisible. cg_fn returns early for main -- C + provides its own -- so hooking the recording there left every call in main + unrecorded: a blind spot exactly where a program does its work. The old + cap_check_call ran from cg_expr and did see main. Moved the recording to + cg_expr. + +2. Caller attribution was stale. __cg_current_fn kept whatever cg_fn set last, + so a violation in main was reported against the previously emitted function. + The test still PASSED, because the violation was detected -- only the name + was wrong, and a diagnostic naming the wrong fn is worse than none. Fixed at + all three main-emission sites; the first patch missed two because the live + path is codegen_streaming. + +98/98 native, 7/7 + 4/4 + 5/5 integration, fixpoint ok. +``` + +## Record — `29f78f9` + +``` +land capability-as-policy: eighteen literals become a data file +``` diff --git a/docs/v1/experiments/cycles/10-a-second-copy-of-the-header.md b/docs/v1/experiments/cycles/10-a-second-copy-of-the-header.md new file mode 100644 index 0000000..037407f --- /dev/null +++ b/docs/v1/experiments/cycles/10-a-second-copy-of-the-header.md @@ -0,0 +1,54 @@ +# a second copy of the header + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `9cc6040` + +``` +EXPERIMENT: derive arity from the runtime's own declarations + +codegen.el carried builtin_arity(): 344 lines, 300 entries, a hand-maintained +second copy of el_runtime.h. + +PREDICTIONS AND RESULTS + P1 the table duplicates the header TRUE 243 shared names + P2 they have already drifted FALSE ZERO drift. The + duplicate had been + maintained correctly. + P3 codegen can emit call-arity relations TRUE + P4 the check becomes a query against the header TRUE + P5 codegen drops to roughly baseline TRUE 4903 -> 4512, + 149 BELOW the 4661 + it started at + +P2 being false is the better result: the table was not WRONG, it was +INCOMPLETE. 110 functions the runtime declares had no entry, so calling them +with the wrong argument count produced no El-level diagnostic at all. Measured: +the old compiler reports 0 arity errors for __http_do_map_to_file(1); the query +reports "takes 5 arguments, called with 1". + +Deriving from the header fixes coverage AND makes drift impossible by +construction. 503 signatures, versus 300 entries maintained by hand. + +THREE DEFECTS IN MY OWN CHECKER, each found by running it rather than reading it + 1. El names and C names differ -- `println` is `__println`. 60 of 500 decls + carry the prefix and codegen owns the mapping; the old table carried both + keys. One rule covers all 60. + 2. Multi-line declarations parsed as zero params, so the checker reported + "takes 0" for a function taking 5. A diagnostic with the wrong number in it + is worse than none -- the same shape as the stale caller attribution in the + previous pass. + 3. Fixing (2) by joining lines dropped 500 signatures to 334, because a + declaration preceded by a comment no longer started its record. Comments + are stripped first now. + +98/98 native, 5/5 arity_query.sh, fixpoint ok. +``` + +## Record — `d9e301b` + +``` +land arity-from-header: the runtime declares its own surface +``` diff --git a/docs/v1/experiments/cycles/11-one-type-erases-the-return.md b/docs/v1/experiments/cycles/11-one-type-erases-the-return.md new file mode 100644 index 0000000..02ae35b --- /dev/null +++ b/docs/v1/experiments/cycles/11-one-type-erases-the-return.md @@ -0,0 +1,52 @@ +# one type erases the return + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `d2d89fc` + +``` +EXPERIMENT: temporal types as data — and the pass that GREW the compiler + +This block is structurally unlike the previous four. It does not only +adjudicate, it DISPATCHES: Instant + Duration must become el_instant_add_dur, +LocalDate + Duration must become el_local_date_add_dur. The emitted C depends on +the type answer, so it cannot move to a post-hoc query. Selecting which call to +emit is an emitter's actual job. + +PREDICTIONS AND RESULTS + P1 the block conflates dispatch with adjudication TRUE + P2 adjudication can move, dispatch cannot TRUE + P3 this pass shrinks codegen far less than the last TRUE, and worse: + 4513 -> 4537, it GREW + by 24 lines + P4 the rules are affine algebra, closed by construction TRUE + P5 no type propagation -- name tracking plus a + hardcoded list of which builtins return which type TRUE, 19 names + +P3 is the honest result and it is not spun: moving 19 names into a data file +cost more lines than it saved, because a generic loader is larger than the +enumeration it replaces. The win is not line count. It is that adding a 20th +temporal builtin is now a one-line edit to signatures.rel instead of a compiler +change, and that the data is inspectable. + +WHY THE HEADER CANNOT SUPPLY THIS, unlike arity: el_runtime.h declares every +builtin as returning el_val_t, because El has ONE type. That single type is why +the whole seam is cheap and it is exactly why the C boundary cannot say that +now() returns an Instant while unix_seconds() returns an Int. The El-level type +is real and the boundary erases it. + +INCOMPLETE, and stated rather than hidden: P2 said adjudication could move to a +query. It has NOT. Violations still emit TIME_TYPE_ERROR inline from the +emitter. Only the type DATA moved. Moving the adjudication needs the operand +types recorded as relations, which is a further pass. + +98/98 native, 4/4 temporal_signatures.sh, fixpoint ok. +``` + +## Record — `e01e079` + +``` +land temporal signatures as data: the type table leaves, the dispatch stays +``` diff --git a/docs/v1/experiments/cycles/12-judgment-lives-with-knowledge.md b/docs/v1/experiments/cycles/12-judgment-lives-with-knowledge.md new file mode 100644 index 0000000..409dc16 --- /dev/null +++ b/docs/v1/experiments/cycles/12-judgment-lives-with-knowledge.md @@ -0,0 +1,47 @@ +# judgment lives with knowledge + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `e8e25a0` + +``` +EXPERIMENT: temporal adjudication moves out; the placeholder stays + +The previous pass moved the type DATA and left the judgment inline, which I +stated rather than hid. This finishes it. + +PREDICTIONS AND RESULTS + P1 codegen can emit operand-type relations TRUE + "main calls temporal:instant_plus_instant" + P2 the affine rules are a small closed set as data TRUE 6 rules + P3 violations still caught at build time TRUE exit=1 + P4 the reporter leaves codegen TRUE 4538 -> 4507 + P5 the TIME_TYPE_ERROR placeholder must STAY TRUE + +P5 is the boundary of this whole approach. The emitter has to emit SOMETHING +for an illegal expression -- it cannot emit nothing and it cannot decide what +the program meant. So the placeholder is irreducible in the same way the AST +traversal was: what moved is the judgment and the wording, not the fact that +something must be written. + +The rules are affine algebra and the set is closed because there are only two +kinds of thing. An Instant is a POINT, a Duration is a DISPLACEMENT: add a +displacement to a point, subtract two points for a displacement, combine +displacements. Nothing else is meaningful, which is why the enumeration in +temporal.rel cannot grow the way an allowlist does. + +A defect in my own checker, found by running it: the .rel file uses aligned +columns and my awk assumed a single space, so the message came out with the +rule key still prefixed. Same class as the multi-line header parse in the arity +pass -- formatting assumptions that only fail when you look at the output. + +98/98 native, 6/6 temporal_query.sh, fixpoint ok. +``` + +## Record — `50425f3` + +``` +land temporal adjudication as a query: the emitter records, the rules are data +``` diff --git a/docs/v1/experiments/cycles/13-thirty-five-return-types.md b/docs/v1/experiments/cycles/13-thirty-five-return-types.md new file mode 100644 index 0000000..3b2669f --- /dev/null +++ b/docs/v1/experiments/cycles/13-thirty-five-return-types.md @@ -0,0 +1,49 @@ +# thirty five return types + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `cbef1c1` + +``` +EXPERIMENT: Int return types as data — and the bug that fell out + +PREDICTIONS AND RESULTS + P1 is_int_call's 35 hardcoded names move to data TRUE + P2 is_int_name stays -- it is annotation propagation TRUE + P3 the dispatch stays -- it is emission TRUE + P4 codegen shrinks ~40 lines TRUE 4507 -> 4469 + P5 the design doc's characterisation is WRONG TRUE + P6 the moved data also fixes the bug it exposed TRUE + +P5 CORRECTS THE RECORD. el-language-design.md and geometry-vs-code.md both cite +"== lowering to str_eq unless both operand names are in a hardcoded int-name +set -- a literal list of variable names treated as integers" as the paradigm +defect. It is not one. __int_names is populated from TYPE ANNOTATIONS +(param["type"] == "Int"), which is primitive but legitimate type propagation. +The actual defect was is_int_call: 35 hardcoded builtin return types, the same +shape as the temporal 19. + +P6 IS A LIVE CORRECTNESS BUG, PRE-EXISTING, NOW FIXED + + let a = str_len("hello") // no annotation + let b = str_len("hi") + let c = a + b // -> el_str_concat(a, b) on two integers + +Verified identical on the pre-change compiler, so not a regression. It compiled +clean, ran, and printed NOTHING where it should print 7. No error at any layer. + +The repair is three lines: an unannotated let takes its type from what the +initialiser returns. The return types were already required for dispatch and +were simply never consulted at the binding site. Moving them into data is what +made the gap visible -- reading the code for eight hours did not. + +98/98 native + 2 new, 31/31 integration, fixpoint ok. +``` + +## Record — `505e5e7` + +``` +land int signatures, and repair a silent miscompilation they exposed +``` diff --git a/docs/v1/experiments/cycles/14-keywords-that-reserve-nothing.md b/docs/v1/experiments/cycles/14-keywords-that-reserve-nothing.md new file mode 100644 index 0000000..3d06af4 --- /dev/null +++ b/docs/v1/experiments/cycles/14-keywords-that-reserve-nothing.md @@ -0,0 +1,57 @@ +# keywords that reserve nothing + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `0143cc4` + +``` +ANSWER: is a grammar a convention, or a region? + +Both, at different layers -- and it is the same split as serialization: the +convention is the BASIS, never the ACT. + + lexeme -> token `fn` means function-start because someone said so CONVENTION + shape recognition given tokens, which construct is this REGION + source -> structure parsing is transduction onto that basis GEOMETRY + byte traversal something must read them in order IRREDUCIBLE + +Three things push the ACT toward region rather than convention: ambiguity +(a * b needs context; a grammar resolves it with the lexer hack, a region by +neighbourhood), error recovery (nearest-region is free), and precedence, which +is ordering along an axis with a conventional parameter. + +AND THE SHOULD GATE SAYS NO TO THE OBVIOUS MOVE + +Every other table this session moved to data. This one stays code. The keyword +set is CLOSED by the language definition -- it does not leak the way an +allowlist does -- and the lexer runs before the program is understood, so a +program can never declare its own keywords. Externalising it costs file I/O on +every compile and buys nothing. Same verdict as is_digit in ASCII. + +WHAT WAS ACTUALLY WRONG: five of 46 keywords were consumed by no parser or +codegen path. sealed, activate, seed, protocol, impl. Each stole an identifier +from users for nothing. + +SECOND SILENT MISCOMPILATION OF THE DAY. Using one did not fail to parse: + + let seed = 42 + let impl = seed + 1 + +compiled CLEAN -- zero cc errors -- and printed 0 instead of 44. No diagnostic +at any layer. Fixed by removing the five. + +A DEFECT IN MY OWN MEASUREMENT, caught before it did damage: my first pass +checked only parser.el and reported `test` as inert too. codegen consumes it at +4135 for --test mode, and the tree has 408 uses. Removing it would have broken +every test in the suite. The measurement was re-run across all four consumers. + +100/100 native + 2 new, 31/31 integration, fixpoint ok. +``` + +## Record — `067dd40` + +``` +answer the parsing question: a grammar is a basis, and five keywords reserved nothing +``` diff --git a/docs/v1/experiments/cycles/15-no-namespacing-at-all.md b/docs/v1/experiments/cycles/15-no-namespacing-at-all.md new file mode 100644 index 0000000..0923142 --- /dev/null +++ b/docs/v1/experiments/cycles/15-no-namespacing-at-all.md @@ -0,0 +1,53 @@ +# no namespacing at all + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `79f6cb7` + +``` +ANSWER: if the partition is a neighbourhood, does linking survive? + +The question is premature, and measuring says why. El's partition is a +FILESYSTEM PATH, not a neighbourhood, and there is no namespacing at all. + +MEASURED + import is textual inlining (resolve_imports), guarded against double + inclusion by a __elc_imp__: state key + when a .elh header exists the header is inlined instead and the .el is marked + seen, so symbols resolve at C link time -- so linking IS real, delegated to C + two modules defining `helper` emit two C functions into one translation unit + +So linking barely survives the PATH partition. Whether it survives a +neighbourhood partition cannot be asked yet. + +A DIAGNOSTIC REGRESSION I CAUSED, found by asking this question. cc does catch +the collision, but reports: + + error: redefinition of '__el_body_helper' + error: redefinition of '__env_helper' + error: redefinition of '__thunk_helper' + error: redefinition of 'helper' + +The user's own function is FOURTH. The first three are generated symbols +introduced by the unconditional-wrapper pass earlier today -- before it, there +was one clear message. Repaired by catching the collision at El level instead: + + duplicate definition: 'helper' is defined 2 times — El has no namespacing, + so imported modules share one global scope + +LIMIT, stated rather than hidden: textual inlining destroys file provenance. By +the time codegen runs there is one source string, so the message can say WHICH +name collides but not which files. Naming a.el and b.el needs provenance +threaded through resolve_imports. + +104/104 native, 4/4 definitions_query.sh, the compiler itself reports clean, +fixpoint ok. +``` + +## Record — `f23cb2b` + +``` +answer the module question: the partition is a path, and there is no namespacing +``` diff --git a/docs/v1/experiments/cycles/16-tokens-carry-no-position.md b/docs/v1/experiments/cycles/16-tokens-carry-no-position.md new file mode 100644 index 0000000..a1e18c4 --- /dev/null +++ b/docs/v1/experiments/cycles/16-tokens-carry-no-position.md @@ -0,0 +1,64 @@ +# tokens carry no position + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `6c975b1` + +``` +thread provenance through resolve_imports + +The module question ended with a limit: textual inlining destroys file +provenance, so a duplicate-definition message could name the symbol but not the +files. Threading it exposed a bigger absence first. + +TOKENS HAD NO POSITION AT ALL. A token was a flat (kind, value) pair, so NO +diagnostic in El could name a place -- every error named a symbol and never a +line. That is the prerequisite the module question was resting on. + +THE CHAIN, end to end + lexer counts newlines; tok_append mints (kind, value, line) + parser stride 2 -> 3; tok_line added; FnDef carries its line + codegen records defines_at: + resolve_imports publishes spans for the combined source + checker maps a combined line back to file:line-within-that-file + + duplicate definition: 'helper' is defined 2 times — El has no namespacing, + so imported modules share one global scope + /tmp/modtest/a.el:1 + /tmp/modtest/b.el:1 + +PREDICTIONS AND RESULTS + P1 15 stride sites, encapsulated in tok_kind/tok_value TRUE, but see below + P2 adding a line field is mechanical TRUE + P3 the lexer must count newlines TRUE + P4 resolve_imports can record per-file line ranges TRUE + P5 the message can then name both files TRUE + P6 token memory grows TRUE, 25.0 -> 33.9 MB (+36%) + +FOUR DEFECTS, EACH FOUND BY RUNNING AND NOT BY READING + +1. interp_tokens_append_all walks the token list DIRECTLY with its own copy of + the stride. Gen1 built fine and gen2 emitted corrupt C, because the + compiler's own source uses string interpolation. My search missed it because + I grepped for the variable name `tokens`; it is called `dst`/`result`. + Searching by name instead of by shape -- third time today. +2. tok_count in test_compiler.el carried the stride too. I had scoped the search + to compiler sources and it had escaped into the tests. +3. Nested resolve_imports calls accumulated spans into shared state, so each + republished meaningless line ranges under the parent's name. Making the + buffer local fixed it; guarding the WRITE did not, which is what I tried + first. +4. The first working version reported b.el:3 -- the COMBINED line against a + filename that has no line 3. A file:line that does not match the file is + worse than no line at all. + +105/105 native, 37/37 integration, fixpoint ok, compiler self-checks clean. +``` + +## Record — `cb7289f` + +``` +thread provenance end to end: a diagnostic can finally name a place +``` diff --git a/docs/v1/experiments/cycles/17-annotations-are-never-checked.md b/docs/v1/experiments/cycles/17-annotations-are-never-checked.md new file mode 100644 index 0000000..c60f03d --- /dev/null +++ b/docs/v1/experiments/cycles/17-annotations-are-never-checked.md @@ -0,0 +1,53 @@ +# annotations are never checked + +**Status: verified on `experiment/annotation-checking`, not merged.** + +## Ishikawa — why does El silently miscompile? + +Three bugs found the same day shared one shape. + +``` +method type tracked by per-function name sets, fed from annotations +machine el_val_t erases everything at the C boundary +material no propagation through expressions +measurement nothing verifies an annotation against what it annotates +───────────────────────────────────────────────────────────────────────── +root cause El has type ANNOTATIONS but no type CHECKING. The annotation + feeds dispatch and is never itself verified. +``` + +## Predictions + +``` +P1 let x: Int = "hello" compiles clean expect TRUE +P2 let s: String = 42 compiles clean expect TRUE +P3 the annotation drives dispatch, unverified expect TRUE +P4 same root cause as all three bugs found today expect TRUE +P5 checking literal-vs-annotation catches both expect TRUE +P6 zero false positives across the compiler's source expect TRUE +``` + +## Results — 6/6, and worse than a wrong answer + +``` +let x: Int = "hello"; x + 1 → 4343631981 a string POINTER used as an integer +let s: String = 42; println(s) → nothing address 42 dereferenced as a string +``` + +The first **leaks a raw memory address into program output**. The second is an +**arbitrary-read primitive** if that integer is ever attacker-influenced. + +Verified: 6/6, zero false positives across the compiler's own source, fixpoint +ok, 105/105 native. + +## Six Sigma + +The emitter only **records** the mismatch; `tools/check/annotations.sh` decides — +consistent with every other check. Literals are checked because they are +unambiguous. + +**Incomplete, stated not hidden:** only literals. `let x: Int = some_string_fn()` +still passes, because `signatures.rel` carries Int/Instant/Duration and no +String entries. That is a data gap, not a capability limit — every El function +declares its return type in source and codegen already holds `ret_type` on every +`FnDef`. diff --git a/docs/v1/experiments/cycles/18-async-half-expressible.md b/docs/v1/experiments/cycles/18-async-half-expressible.md new file mode 100644 index 0000000..1cdc0c2 --- /dev/null +++ b/docs/v1/experiments/cycles/18-async-half-expressible.md @@ -0,0 +1,73 @@ +# async — half expressible, and the cycle that was dogma + +**Status: measured on a branch, not merged. Two runs — the first was invalid.** + +## The first attempt was DOGMA, not science + +I had just finished arguing that `@async` was expressible, then ran something to +confirm it. **No prediction was committed.** The test was rigged in a way that +should have been visible while writing it: + +```c +pthread_create(&t,NULL,runner,NULL); pthread_join(t,NULL); +``` + +`join` immediately after `create` — the caller blocks until the body finishes. +That is a thread round-trip, not deferral. And the test printed the word +`DEFERRED` itself: I wrote the conclusion into the output and read it back. + +``` +Ishikawa on the rigged test + method ran after concluding, not to decide + machine nothing forces a prediction before execution + material the assertion was written into the output string + measurement no falsification criterion existed, so nothing could fail + root cause the test was authored by the party holding the conclusion, + with no commitment made before it ran +``` + +Discarded and re-run properly. + +## Second run — predictions committed first + +``` +P1 the caller proceeds while the body runs expect TRUE +P2 interleaving is observable in timestamps expect TRUE +P3 the result cannot be retrieved — one 64-bit slot, no + future type, so the wrap either blocks or returns + something that is not the result expect TRUE +P4 therefore HALF expressible: fire-and-forget yes, await no expect TRUE +``` + +## Results — 4/4 + +``` +[ 18 us] wrap RETURNS to caller +[ 29 us] body START +caller continues, got 0 +[ 50176 us] body END (computed 42) +caller done +``` + +The caller got **0, not 42**. Both of my earlier claims were wrong in opposite +directions: "not expressible" was too strong — fire-and-forget works today, +bound after the build, no compiler change. "Expressible" was too strong the +other way. + +## Follow-on cycle — a future is one more tagged object + +``` +P1 el_val_t already carries tagged heap objects TRUE 5 magic tags exist +P2 a future is one more TRUE +P3 the caller awaits and gets 42 TRUE +P4 ZERO compiler changes TRUE runtime C + one binding +P5 the unbound path still works FALSE SIGSEGV +``` + +**P4 is the result.** `@async` — called unexpressible for hours — needs no +compiler change. A future is one more magic-tagged heap object; `defer` returns +the handle, `el_await` blocks. + +**P5 is the failure that mattered.** Sixty seconds after diagnosing +`let s: String = 42` as an arbitrary read, I wrote the identical defect into +`el_await`: reading `->magic` off an unvalidated slot. That opened cycle 19. diff --git a/docs/v1/experiments/cycles/19-a-convention-is-not-a-gate.md b/docs/v1/experiments/cycles/19-a-convention-is-not-a-gate.md new file mode 100644 index 0000000..1619606 --- /dev/null +++ b/docs/v1/experiments/cycles/19-a-convention-is-not-a-gate.md @@ -0,0 +1,61 @@ +# a convention is not a gate + +One `Ishikawa → scientific method → Six Sigma` loop. The record below is the +commit message as written at the time, before the outcome was known to anyone +reading this file. + +## Record — `9a6c161` + +``` +a slot must be validated before it is dereferenced + +ISHIKAWA: el_val_t carries integers AND tagged heap pointers, so "is this a +pointer" is undecidable without checking first. That check was a CONVENTION +every author had to know rather than a GATE they had to pass through, and +looks_like_heap_obj was static -- so every sibling translation unit re-derived +it. + +MEASURED, across the five existing tags + geom_of looks_like_heap_obj full guard correct + mfld_of looks_like_heap_obj full guard correct + el_bin_lookup (uintptr_t)p < 4096 floor only reads 8 bytes BACKWARD + el_input_len s ? ... : 0 NULL only strlen's an integer + + sha256_hex(50000) -> exit 139, SIGSEGV, compiled clean + +PREDICTIONS AND RESULTS + P1 looks_like_heap_obj is static, not exported TRUE + P2 each tagged type re-derives the check TRUE + P3 at least one is missing guard components TRUE (two are) + P6 sha256_hex() reads out of bounds TRUE + P8 routing el_bin_lookup through the gate fixes it FALSE + P9 the legitimate hash is unchanged TRUE + P11 fixpoint and suites hold TRUE + +P8 IS THE USEFUL FAILURE. Guarding the tagged lookup changed nothing -- +looks_like_heap_obj(49992) correctly returns 0, el_bin_lookup bails, and then +el_input_len falls through to strlen() on address 50000. The FALLBACK was the +hazard, not the tagged path. A NULL check does not establish that a slot is a +pointer. I would have shipped the wrong fix and called it verified. + +A MEASUREMENT DEFECT, fourth today: my first run of the crash reported exit=0, +because $? read head's exit through a pipe rather than the program's. I nearly +recorded a segfault as a clean run. Same shape as grepping only parser.el and +searching by variable name instead of by operation. + +AND I PROVED THE HAZARD FROM THE INSIDE. Sixty seconds after diagnosing +`let s: String = 42` as an arbitrary-read primitive, I wrote the identical +defect into el_await -- dereferencing ->magic off an unvalidated slot -- and +only then found the runtime had already made it twice. + +el_tagged() is now exported in el_runtime.h. Anything that dereferences a slot +without passing through it is the defect. + +105/105 native, 42/42 integration across eight harnesses, fixpoint ok. +``` + +## Record — `3049a70` + +``` +make the guard a gate: sha256_hex(50000) no longer segfaults +``` diff --git a/docs/v1/experiments/findings/answers.md b/docs/v1/experiments/findings/answers.md new file mode 100644 index 0000000..534a496 --- /dev/null +++ b/docs/v1/experiments/findings/answers.md @@ -0,0 +1,12 @@ +# Architecture questions closed + +All five were open in `geometry-vs-code.md`. Each was closed by measurement, not +by argument. + +| Question | Answer | +|---|---| +| **Concurrency** — hardware threads are code, but is *ordering* geometric? | **Ordering is geometric.** Causality is a partial order (Lamport 1978); a total order is an arbitrary extension that "cannot be depended on to imply a causal relationship." Languages force a total order at authoring time, so every lock, barrier and fence is apparatus for recovering the partial order that was destroyed. CALM: a program has a coordination-free implementation **iff monotone**. What breaks monotonicity is destructive update. **Coordination is the price of forgetting.** | +| **Error handling** — does `grounded: false` cover *failed*? | **No.** Standing is a *signed* component: `>0` supported, `=0` unknown, `<0` contradicted. Not-known and known-false are opposite directions on one axis; a boolean cannot tell them apart. `inhibitory` as an int32 flag is that sign wearing a boolean. | +| **Parsing** — is a grammar a convention, or a region? | **A grammar is a basis; parsing is transduction onto it.** Lexeme→token is convention, shape recognition is a region, byte traversal is irreducible. **But the SHOULD gate refused the obvious move:** the keyword table stays code, because the set is closed by the language definition and the lexer runs before the program is understood. Same verdict as `is_digit` in ASCII. | +| **Numeric literals** — is `3` a position or a convention? | **The numeral is convention; the number is a position — and a bare `3` is a magnitude with no axis.** It is not a position until something gives it a direction, which is why `3.days` needs a calendar. Demonstrated: `t + 3` refused, `t + 1.hour` accepted. | +| **The module system** — if the partition is a neighbourhood, does linking survive? | **Premature.** The partition is a filesystem path and there is no namespacing at all. `import` is textual inlining; with a `.elh` header, symbols resolve at C link time. Two modules defining `helper` emit two C functions into one translation unit. Linking barely survives the *path* partition. | diff --git a/docs/v1/experiments/findings/bugs.md b/docs/v1/experiments/findings/bugs.md new file mode 100644 index 0000000..2433a12 --- /dev/null +++ b/docs/v1/experiments/findings/bugs.md @@ -0,0 +1,74 @@ +# Live defects found + +Every one compiled clean, ran, and produced a wrong result or a crash with **no +diagnostic at any layer**. All four were present before this session; none was +introduced by it. + +## Silent miscompilations + +### 1. An unannotated `let` loses its type + +```el +let a = str_len("hello") // no annotation +let b = str_len("hi") +let c = a + b // → el_str_concat(a, b) on two integers +``` + +Compiled clean. Printed **nothing** where it should print 7. Fixed: an +unannotated `let` takes its type from what its initialiser returns. The return +types were already required for dispatch and were simply never consulted at the +binding site. + +### 2. Reserved keywords that reserved nothing + +```el +let seed = 42 +let impl = seed + 1 +``` + +`sealed`, `activate`, `seed`, `protocol`, `impl` were keywords in the lexer and +consumed by no parser or codegen path. Using one did not fail to parse — it +compiled clean, with zero `cc` errors, and printed **0 instead of 44**. Fixed by +removing all five. + +### 3. `Instant + Int` was never refused + +```el +let t: Instant = now() +let u: Instant = t + 3 // → (t + 3), reported clean +``` + +`Duration + Int` was refused — *"an Int carries no unit"* — while adding a +dimensionless number to a **point** silently moved the instant by an +unspecified amount. Three of *what*? Whatever the representation happens to be. +The rule was simply never written. + +## Security-relevant + +### 4. Annotations are never verified + +```el +let x: Int = "hello"; x + 1 → 4343631981 a string POINTER used as an integer +let s: String = 42; println(s) → nothing address 42 dereferenced +``` + +The first **leaks a raw memory address into program output**. The second is an +**arbitrary-read primitive** if the integer is ever attacker-influenced. + +### 5. `sha256_hex()` segfaults + +```el +let h: String = sha256_hex(50000) → exit 139, SIGSEGV +``` + +Compiled clean. `el_bin_lookup` checked only a 4096 floor — no alignment, no +small-int, no negative — and reads **eight bytes backward** from the pointer. +And the actual crash was one level further on: `el_input_len` fell through to +`strlen()` on address 50000, because a NULL check does not establish that a slot +is a pointer. + +Fixed, and the guard is now a **gate**: `el_tagged()` is exported in +`el_runtime.h`. `geom_of` and `mfld_of` were always correct because their authors +knew to call `looks_like_heap_obj`; `el_bin_lookup` and `el_input_len` were wrong +because theirs did not, and the function was `static`, so every sibling +translation unit re-derived it. diff --git a/docs/v1/experiments/findings/measurement-defects.md b/docs/v1/experiments/findings/measurement-defects.md new file mode 100644 index 0000000..a9eec13 --- /dev/null +++ b/docs/v1/experiments/findings/measurement-defects.md @@ -0,0 +1,41 @@ +# Defects in my own measurement + +Recorded because the pattern is the point: **four of these, all the same shape — +searching by name or scope instead of by the operation itself.** Each was caught +by running something, never by reading. + +### 1. Scoped the search to one file + +Reported `test` as an inert keyword by checking only `parser.el`. **codegen** +consumes it at 4135 for `--test` mode, and the tree has 408 uses. Removing it +would have broken every test in the suite — including the ones used to verify +the removal. + +### 2. Searched by variable name, not by operation + +Grepped for `native_list_append(tokens` to find direct token appends. +`interp_tokens_append_all` calls its parameters `dst`/`result`, carries its own +copy of the stride, and corrupted generation 2 — while generation 1 built fine, +because the compiler's own source uses string interpolation. + +### 3. Scoped to compiler sources; the stride had escaped into tests + +`tok_count` in `test_compiler.el` computed `len/2` independently. 21 tests failed +after the token layout changed. + +### 4. Read the wrong exit code + +```bash +timeout 10 /tmp/leakrun 2>&1 | head -2; echo "exit=$?" # reports head's exit +``` + +Reported `exit=0` for a program that was returning **139 (SIGSEGV)**. I nearly +recorded a segfault as a clean run. + +### And one that was not a measurement defect but a method defect + +One cycle was run **without committing predictions first** — see +`cycles/18-async-half-expressible.md`. The test joined the thread immediately +after creating it and printed the word `DEFERRED` itself. A test authored by the +party holding the conclusion, with nothing committed beforehand, cannot fail. +It had to be discarded and re-run. diff --git a/lang/el-compiler/src/codegen.el b/lang/el-compiler/src/codegen.el index 2c10bae..be628f5 100644 --- a/lang/el-compiler/src/codegen.el +++ b/lang/el-compiler/src/codegen.el @@ -589,7 +589,7 @@ fn cg_expr(expr: Map) -> String { if left_is_ct { if op == "Plus" { if right_is_ct { - time_record_violation("caltime_plus_caltime", "CalendarTime + CalendarTime is not allowed (use cal_to_instant + Duration)") + record_call(state_get("__cg_current_fn"), "temporal:caltime_plus_caltime") return "0 /* TIME_TYPE_ERROR: CalendarTime + CalendarTime */" } } @@ -607,9 +607,18 @@ fn cg_expr(expr: Map) -> String { return "el_instant_add_dur(" + left_c + ", " + right_c + ")" } if right_is_inst { - time_record_violation("instant_plus_instant", "Instant + Instant is not allowed") + record_call(state_get("__cg_current_fn"), "temporal:instant_plus_instant") return "0 /* TIME_TYPE_ERROR: Instant + Instant */" } + // A bare literal is a MAGNITUDE WITH NO AXIS. Duration + Int + // was already refused because an Int carries no unit; adding + // one to a POINT is worse, because it silently moves the + // instant by an unspecified amount. The asymmetry had no + // justification -- it was simply never written. + if is_int_expr(right) { + record_call(state_get("__cg_current_fn"), "temporal:instant_plus_int") + return "0 /* TIME_TYPE_ERROR: Instant + Int */" + } } if left_is_dur { if right_is_inst { @@ -619,13 +628,13 @@ fn cg_expr(expr: Map) -> String { return "el_duration_add(" + left_c + ", " + right_c + ")" } if is_int_expr(right) { - time_record_violation("duration_plus_int", "Duration + Int is not allowed (use duration_seconds(n) or N.seconds)") + record_call(state_get("__cg_current_fn"), "temporal:duration_plus_int") return "0 /* TIME_TYPE_ERROR: Duration + Int */" } } if right_is_dur { if is_int_expr(left) { - time_record_violation("duration_plus_int", "Int + Duration is not allowed") + record_call(state_get("__cg_current_fn"), "temporal:duration_plus_int") return "0 /* TIME_TYPE_ERROR: Int + Duration */" } } @@ -644,7 +653,7 @@ fn cg_expr(expr: Map) -> String { return "el_duration_sub(" + left_c + ", " + right_c + ")" } if is_int_expr(right) { - time_record_violation("duration_minus_int", "Duration - Int is not allowed") + record_call(state_get("__cg_current_fn"), "temporal:duration_minus_int") return "0 /* TIME_TYPE_ERROR: Duration - Int */" } } @@ -673,14 +682,14 @@ fn cg_expr(expr: Map) -> String { if left_is_inst { if right_is_inst { return "el_instant_lt(" + left_c + ", " + right_c + ")" } if right_is_dur { - time_record_violation("instant_cmp_duration", "Instant < Duration is not allowed") + record_call(state_get("__cg_current_fn"), "temporal:instant_cmp_duration") return "0 /* TIME_TYPE_ERROR: Instant < Duration */" } } if left_is_dur { if right_is_dur { return "el_duration_lt(" + left_c + ", " + right_c + ")" } if right_is_inst { - time_record_violation("duration_cmp_instant", "Duration < Instant is not allowed") + record_call(state_get("__cg_current_fn"), "temporal:duration_cmp_instant") return "0 /* TIME_TYPE_ERROR: Duration < Instant */" } } @@ -992,18 +1001,25 @@ fn cg_expr(expr: Map) -> String { if func_kind == "Ident" { let fn_name: String = func["name"] - // Capability-kind enforcement: services can't call - // self-formation primitives; utilities can't call any - // DHARMA or LLM primitives. cap_check_call records - // violations to be emitted as #error directives at the - // top of the generated C, so cc fails with a clear msg. - cap_check_call(fn_name) + // Every call is recorded here, from cg_expr, because this runs for + // EVERY expression in every context -- including main's body, which + // cg_fn returns early on since C provides its own main. Hooking the + // recording to cg_fn instead left every call in main invisible to + // the relation graph, a blind spot exactly where a program does its + // work. Caught by capability_query.sh, not by reading the code. + record_call("program", "is_kind:" + state_get("__program_kind")) + record_call(state_get("__cg_current_fn"), fn_name) + // Record the ARITY at the call site. The compiler knows how many + // arguments were written; whether that is correct is a question + // about the runtime's surface, and the runtime already declares its + // surface in el_runtime.h. A table inside the emitter is a second + // copy of that header, maintained by hand. + record_call(fn_name, "arity:" + native_int_to_str(native_list_len(args))) // Arity check against the builtin table - refuse, with a clear // El-source message, when a known builtin gets the wrong arg // count (e.g. `http_serve(port)` instead of `http_serve(port, // handler)`). User-defined fns and variadic builtins pass // through (builtin_arity returns -1). - arity_check_call(fn_name, arity) // sleep(Duration) - Phase 1 of the typed-time work. When the // single arg is provably a Duration we lower to el_sleep_duration // so the runtime sees nanos directly. Existing sleep() callers @@ -1510,6 +1526,47 @@ fn cg_stmt(stmt: Map, indent: String, declared: [String]) -> [Strin if str_eq(ltype, "Int") { add_int_name(name) } + // ANNOTATION CHECKING. El has type annotations and, until now, no type + // checking: the annotation fed dispatch and was never itself verified. + // That is the root cause of every silent miscompilation found on + // 2026-08-17, and the failure is not merely a wrong answer: + // + // let x: Int = "hello" ; x + 1 -> printed 4343631981, a string + // POINTER interpreted as an integer + // let s: String = 42 ; print -> dereferenced address 42 + // + // The second is an arbitrary-read primitive if the integer is ever + // attacker-influenced. Literals are checked here because they are + // unambiguous; the emitter only RECORDS the mismatch, the query decides. + let vkind: String = val["expr"] + if str_eq(ltype, "Int") { + if str_eq(vkind, "Str") { + record_call(state_get("__cg_current_fn"), "typemismatch:Int:String:" + name) + } + } + if str_eq(ltype, "String") { + if str_eq(vkind, "Int") { + record_call(state_get("__cg_current_fn"), "typemismatch:String:Int:" + name) + } + if str_eq(vkind, "Float") { + record_call(state_get("__cg_current_fn"), "typemismatch:String:Float:" + name) + } + } + if str_eq(ltype, "Float") { + if str_eq(vkind, "Str") { + record_call(state_get("__cg_current_fn"), "typemismatch:Float:String:" + name) + } + } + // UNANNOTATED let: take the type from what the initialiser RETURNS. + // Without this, `let a = str_len(s)` loses the Int and a later `a + b` + // lowers to el_str_concat on two integers -- silently, with no error, + // producing a program that prints nothing where it should print 7. + // Verified present in the pre-change compiler too, so this is a repair + // rather than a regression. The return types were already needed for + // dispatch; they were just never consulted here. + if str_eq(ltype, "") { + if is_int_expr(val) { add_int_name(name) } + } // Same as params: Bool is an int in the value model. Without this a // `let ok: Bool = ...` compared to another Bool lowered to str_eq. if str_eq(ltype, "Bool") { @@ -1936,48 +1993,10 @@ fn is_duration_name(name: String) -> Bool { // string-concat on `+` when one side is a Call. New builtins must be added // here when they return Int and may participate in arithmetic. fn is_int_call(call_expr: Map) -> Bool { - let func = call_expr["func"] - let fk: String = func["expr"] - if !str_eq(fk, "Ident") { return false } - let name: String = func["name"] - if str_eq(name, "str_len") { return true } - if str_eq(name, "str_index_of") { return true } - if str_eq(name, "str_to_int") { return true } - if str_eq(name, "str_char_code") { return true } - if str_eq(name, "str_count") { return true } - if str_eq(name, "str_count_chars") { return true } - if str_eq(name, "str_count_bytes") { return true } - if str_eq(name, "str_count_lines") { return true } - if str_eq(name, "str_count_words") { return true } - if str_eq(name, "str_count_letters") { return true } - if str_eq(name, "str_count_digits") { return true } - if str_eq(name, "str_last_index_of") { return true } - if str_eq(name, "str_find_chars") { return true } - if str_eq(name, "native_list_len") { return true } - if str_eq(name, "el_list_len") { return true } - if str_eq(name, "len") { return true } - if str_eq(name, "json_get_int") { return true } - if str_eq(name, "json_array_len") { return true } - if str_eq(name, "engram_node_count") { return true } - if str_eq(name, "engram_edge_count") { return true } - if str_eq(name, "time_now") { return true } - if str_eq(name, "time_now_utc") { return true } - if str_eq(name, "time_diff") { return true } - if str_eq(name, "time_add") { return true } - if str_eq(name, "time_from_parts") { return true } - if str_eq(name, "el_abs") { return true } - if str_eq(name, "el_max") { return true } - if str_eq(name, "el_min") { return true } - if str_eq(name, "float_to_int") { return true } - if str_eq(name, "unix_timestamp") { return true } - if str_eq(name, "instant_to_unix_seconds") { return true } - if str_eq(name, "instant_to_unix_millis") { return true } - if str_eq(name, "duration_to_seconds") { return true } - if str_eq(name, "duration_to_millis") { return true } - if str_eq(name, "duration_to_nanos") { return true } - return false + call_returns(call_expr, "Int") } + // Known runtime builtins that return Float. Parallel to is_int_call — lets a // Call participate in float arithmetic (and get inferred into __float_names on // an unannotated `let`). New Float-returning builtins must be added here. @@ -2003,40 +2022,65 @@ fn is_float_call(call_expr: Map) -> Bool { // Builtins that return an Instant. Used by is_instant_expr and the BinOp // dispatch - `now() + 5.seconds` types as Instant only because we can see // that now() is an Instant-returning Call. -fn is_instant_call(call_expr: Map) -> Bool { - let func = call_expr["func"] - let fk: String = func["expr"] - if !str_eq(fk, "Ident") { return false } - let name: String = func["name"] - if str_eq(name, "now") { return true } - if str_eq(name, "el_now_instant") { return true } - if str_eq(name, "unix_seconds") { return true } - if str_eq(name, "unix_millis") { return true } - if str_eq(name, "instant_from_iso8601") { return true } - if str_eq(name, "el_instant_add_dur") { return true } - if str_eq(name, "el_instant_sub_dur") { return true } - return false +// load_signatures — El-level return types, read once from +// tools/check/signatures.rel. Previously 19 hardcoded names across +// is_instant_call and is_duration_call. The header cannot carry these: it +// declares everything as el_val_t, because El has one type. +fn load_signatures() -> Void { + if !str_eq(state_get("__sig_loaded"), "") { return } + state_set("__sig_loaded", "1") + let path: String = env("EL_SIGNATURES") + if str_eq(path, "") { let path = "tools/check/signatures.rel" } + if !fs_exists(path) { return } + let lines = str_split_lines(fs_read(path)) + let n: Int = native_list_len(lines) + let i = 0 + while i < n { + let line: String = str_trim(native_list_get(lines, i)) + if !str_eq(line, "") { + if !str_starts_with(line, "#") { + let parts = str_split(line, " ") + let np: Int = native_list_len(parts) + let name: String = "" + let ty: String = "" + let seen = 0 + let j = 0 + while j < np { + let tok: String = str_trim(native_list_get(parts, j)) + if !str_eq(tok, "") { + if seen == 0 { let name = tok } + if seen == 2 { let ty = tok } + let seen = seen + 1 + } + let j = j + 1 + } + if seen >= 3 { state_set("__sig_" + name, ty) } + } + } + let i = i + 1 + } } -// Builtins that return a Duration. Same role as is_instant_call. -fn is_duration_call(call_expr: Map) -> Bool { - let func = call_expr["func"] +fn call_returns(expr: Map, want: String) -> Bool { + load_signatures() + let func = expr["func"] let fk: String = func["expr"] if !str_eq(fk, "Ident") { return false } let name: String = func["name"] - if str_eq(name, "el_duration_from_nanos") { return true } - if str_eq(name, "duration_seconds") { return true } - if str_eq(name, "duration_millis") { return true } - if str_eq(name, "duration_nanos") { return true } - if str_eq(name, "el_instant_diff") { return true } - if str_eq(name, "el_duration_add") { return true } - if str_eq(name, "el_duration_sub") { return true } - if str_eq(name, "el_duration_scale") { return true } - if str_eq(name, "el_duration_div") { return true } - if str_eq(name, "ttl_cache_age") { return true } - return false + return str_eq(state_get("__sig_" + name), want) } +fn is_instant_call(expr: Map) -> Bool { + call_returns(expr, "Instant") +} + + +// Builtins that return a Duration. Same role as is_instant_call. +fn is_duration_call(expr: Map) -> Bool { + call_returns(expr, "Duration") +} + + // Phase 1.5 - Calendar / CalendarTime / Rhythm / LocalDate / LocalTime / // LocalDateTime / Zone are first-class boxed types. Each has its own name // set in process state, populated from typed `let` bindings and parameter @@ -2321,15 +2365,6 @@ fn is_duration_expr(expr: Map) -> Bool { // Record a temporal-type violation. Surfaced as `#error` directives at the // top of the generated C, identical machinery to cap_record_violation. // kinds: "instant_plus_instant", "duration_plus_int", etc. -fn time_record_violation(kind: String, detail: String) -> Bool { - let csv: String = state_get("__time_violations") - if str_eq(csv, "") { let csv = "," } - let entry: String = kind + ":" + detail - let key: String = "," + entry + "," - if str_contains(csv, key) { return true } - state_set("__time_violations", csv + entry + ",") - return true -} // Recursive type-propagation: is `expr` known-Int at codegen time? // This unifies the BinOp(+) dispatch so chained arithmetic over Int @@ -2491,119 +2526,25 @@ fn float_operand_c(expr: Map, expr_c: String) -> String { // The compiler-level rule is structural: the binary either CAN or CANNOT // emit the call. There is no runtime check, no opt-in, no override. -fn cap_record_violation(kind: String, fn_name: String) -> Bool { - let csv: String = state_get("__cap_violations") - if str_eq(csv, "") { let csv = "," } - let entry: String = kind + ":" + fn_name - let key: String = "," + entry + "," - if str_contains(csv, key) { return true } - state_set("__cap_violations", csv + entry + ",") - return true -} // Self-formation primitives - the cut between CGI and service. A program // that emits these calls IS structurally a CGI; we forbid them everywhere // else. -fn is_self_formation_call(fn_name: String) -> Bool { - if str_eq(fn_name, "llm_call_agentic") { return true } - if str_eq(fn_name, "llm_register_tool") { return true } - if str_eq(fn_name, "dharma_emit") { return true } - if str_eq(fn_name, "dharma_field") { return true } - return false -} // Any DHARMA primitive - utilities have zero network presence. -fn is_dharma_call(fn_name: String) -> Bool { - if str_eq(fn_name, "dharma_connect") { return true } - if str_eq(fn_name, "dharma_send") { return true } - if str_eq(fn_name, "dharma_activate") { return true } - if str_eq(fn_name, "dharma_emit") { return true } - if str_eq(fn_name, "dharma_field") { return true } - if str_eq(fn_name, "dharma_strengthen") { return true } - if str_eq(fn_name, "dharma_relationship") { return true } - if str_eq(fn_name, "dharma_peers") { return true } - return false -} // Any LLM primitive - utilities have no LLM access at all. -fn is_llm_call(fn_name: String) -> Bool { - if str_eq(fn_name, "llm_call") { return true } - if str_eq(fn_name, "llm_call_system") { return true } - if str_eq(fn_name, "llm_call_agentic") { return true } - if str_eq(fn_name, "llm_vision") { return true } - if str_eq(fn_name, "llm_register_tool") { return true } - if str_eq(fn_name, "llm_models") { return true } - return false -} -fn cap_check_call(fn_name: String) -> Bool { - let kind: String = state_get("__program_kind") - if str_eq(kind, "cgi") { return true } - if str_eq(kind, "service") { - if is_self_formation_call(fn_name) { - cap_record_violation("service", fn_name) - return false - } - return true - } - // utility (default) - if is_dharma_call(fn_name) { - cap_record_violation("utility", fn_name) - return false - } - if is_llm_call(fn_name) { - cap_record_violation("utility", fn_name) - return false - } - return true -} // Emit collected capability violations as #error directives. Called // from codegen()'s entry point right after the cgi/service-block scan, // so they appear at the very top of the generated C. -fn emit_cap_violations() -> Void { - let csv: String = state_get("__cap_violations") - if str_eq(csv, "") { return } - if str_eq(csv, ",") { return } - let n: Int = str_len(csv) - let i: Int = 1 - while i < n { - let next_comma: Int = str_index_of(str_slice(csv, i, n), ",") - if next_comma < 0 { return } - let entry: String = str_slice(csv, i, i + next_comma) - let colon: Int = str_index_of(entry, ":") - if colon > 0 { - let kind: String = str_slice(entry, 0, colon) - let fn_name: String = str_slice(entry, colon + 1, str_len(entry)) - emit_line("#error \"capability violation: '" + kind + "' programs may not call '" + fn_name + "' (self-formation primitive - only 'cgi' programs may use it)\"") - } - let i = i + next_comma + 1 - } -} // Surface temporal-type violations as #error directives. The cg_expr BinOp // dispatcher records each violation (Instant + Instant, Duration + Int, -) // as a CSV entry "kind:detail" via time_record_violation. Each entry maps // to a single #error so downstream cc fails the build with a clear El- // source-level message before the bogus C even links. -fn emit_time_violations() -> Void { - let csv: String = state_get("__time_violations") - if str_eq(csv, "") { return } - if str_eq(csv, ",") { return } - let n: Int = str_len(csv) - let i: Int = 1 - while i < n { - let next_comma: Int = str_index_of(str_slice(csv, i, n), ",") - if next_comma < 0 { return } - let entry: String = str_slice(csv, i, i + next_comma) - let colon: Int = str_index_of(entry, ":") - if colon > 0 { - let detail: String = str_slice(entry, colon + 1, str_len(entry)) - emit_line("#error \"temporal type error: " + detail + "\"") - } - let i = i + next_comma + 1 - } -} // -- Builtin arity table ------------------------------------------------------- // @@ -2618,400 +2559,15 @@ fn emit_time_violations() -> Void { // dynamic dispatch) return -1 -> no check. A mismatch records a violation // in process state, which emit_arity_violations() turns into #error // directives at the top of the generated C. -fn builtin_arity(name: String) -> Int { - // I/O - if str_eq(name, "println") { return 1 } - if str_eq(name, "print") { return 1 } - if str_eq(name, "readline") { return 0 } - // LSP seed primitives - if str_eq(name, "__read_n") { return 1 } - if str_eq(name, "__print_raw") { return 1 } - // Test-registry accessors. These are not runtime builtins — they are - // GENERATED into the same translation unit by the --test path below, one - // set per test binary. They are declared here so the El-side runner in - // runtime/eltest.el can call them with a known arity. - if str_eq(name, "__el_reg_count") { return 0 } - if str_eq(name, "__el_reg_name") { return 1 } - if str_eq(name, "__el_reg_invoke") { return 1 } - if str_eq(name, "__el_reg_last_ns") { return 0 } - if str_eq(name, "__el_reg_msg") { return 0 } - if str_eq(name, "__el_reg_asserts") { return 0 } - if str_eq(name, "__el_opt_json") { return 0 } - // String - if str_eq(name, "el_str_concat") { return 2 } - if str_eq(name, "str_eq") { return 2 } - if str_eq(name, "str_starts_with") { return 2 } - if str_eq(name, "str_ends_with") { return 2 } - if str_eq(name, "str_len") { return 1 } - if str_eq(name, "str_concat") { return 2 } - if str_eq(name, "int_to_str") { return 1 } - if str_eq(name, "str_to_int") { return 1 } - if str_eq(name, "str_slice") { return 3 } - if str_eq(name, "str_contains") { return 2 } - if str_eq(name, "str_replace") { return 3 } - if str_eq(name, "str_to_upper") { return 1 } - if str_eq(name, "str_to_lower") { return 1 } - if str_eq(name, "str_trim") { return 1 } - if str_eq(name, "str_index_of") { return 2 } - if str_eq(name, "str_split") { return 2 } - if str_eq(name, "str_char_at") { return 2 } - if str_eq(name, "str_char_code") { return 2 } - if str_eq(name, "str_pad_left") { return 3 } - if str_eq(name, "str_pad_right") { return 3 } - if str_eq(name, "str_format") { return 2 } - if str_eq(name, "str_lower") { return 1 } - if str_eq(name, "str_upper") { return 1 } - // Text-processing primitives (Phase 1) - if str_eq(name, "str_count") { return 2 } - if str_eq(name, "str_count_chars") { return 1 } - if str_eq(name, "str_count_bytes") { return 1 } - if str_eq(name, "str_count_lines") { return 1 } - if str_eq(name, "str_count_words") { return 1 } - if str_eq(name, "str_count_letters") { return 1 } - if str_eq(name, "str_count_digits") { return 1 } - if str_eq(name, "str_index_of_all") { return 2 } - if str_eq(name, "str_last_index_of") { return 2 } - if str_eq(name, "str_find_chars") { return 2 } - if str_eq(name, "str_repeat") { return 2 } - if str_eq(name, "str_reverse") { return 1 } - if str_eq(name, "str_strip_prefix") { return 2 } - if str_eq(name, "str_strip_suffix") { return 2 } - if str_eq(name, "str_strip_chars") { return 2 } - if str_eq(name, "str_lstrip") { return 1 } - if str_eq(name, "str_rstrip") { return 1 } - if str_eq(name, "is_letter") { return 1 } - if str_eq(name, "is_digit") { return 1 } - if str_eq(name, "is_alphanumeric") { return 1 } - if str_eq(name, "is_whitespace") { return 1 } - if str_eq(name, "is_punctuation") { return 1 } - if str_eq(name, "is_uppercase") { return 1 } - if str_eq(name, "is_lowercase") { return 1 } - if str_eq(name, "str_split_lines") { return 1 } - if str_eq(name, "str_split_chars") { return 1 } - if str_eq(name, "str_split_n") { return 3 } - if str_eq(name, "str_join") { return 2 } - // HTML sanitizer - if str_eq(name, "el_html_sanitize") { return 2 } - // Math - if str_eq(name, "el_abs") { return 1 } - if str_eq(name, "el_max") { return 2 } - if str_eq(name, "el_min") { return 2 } - // List - if str_eq(name, "el_list_len") { return 1 } - if str_eq(name, "el_list_get") { return 2 } - if str_eq(name, "el_list_append") { return 2 } - if str_eq(name, "el_list_empty") { return 0 } - if str_eq(name, "el_list_clone") { return 1 } - if str_eq(name, "list_push") { return 2 } - if str_eq(name, "list_push_front") { return 2 } - if str_eq(name, "list_join") { return 2 } - if str_eq(name, "list_range") { return 2 } - // Map - if str_eq(name, "el_get_field") { return 2 } - if str_eq(name, "el_map_get") { return 2 } - if str_eq(name, "el_map_set") { return 3 } - // HTTP - if str_eq(name, "http_get") { return 1 } - if str_eq(name, "http_post") { return 2 } - if str_eq(name, "http_post_json") { return 2 } - if str_eq(name, "http_get_with_headers") { return 2 } - if str_eq(name, "http_post_with_headers") { return 3 } - if str_eq(name, "http_post_form_auth") { return 3 } - if str_eq(name, "http_serve") { return 2 } - if str_eq(name, "http_set_handler") { return 1 } - // Seed primitives (__-prefix) — runtime/el_seed.c - if str_eq(name, "__str_len") { return 1 } - if str_eq(name, "__str_char_at") { return 2 } - if str_eq(name, "__str_alloc") { return 1 } - if str_eq(name, "__str_set_char") { return 3 } - if str_eq(name, "__str_cmp") { return 2 } - if str_eq(name, "__str_ncmp") { return 3 } - if str_eq(name, "__str_concat_raw") { return 2 } - if str_eq(name, "__str_slice_raw") { return 3 } - if str_eq(name, "__int_to_str") { return 1 } - if str_eq(name, "__str_to_int") { return 1 } - if str_eq(name, "__float_to_str") { return 1 } - if str_eq(name, "__str_to_float") { return 1 } - if str_eq(name, "__println") { return 1 } - if str_eq(name, "__print") { return 1 } - if str_eq(name, "__readline") { return 0 } - if str_eq(name, "__fs_read") { return 1 } - if str_eq(name, "__fs_write") { return 2 } - if str_eq(name, "__fs_exists") { return 1 } - if str_eq(name, "__fs_list_raw") { return 1 } - if str_eq(name, "__fs_mkdir") { return 1 } - if str_eq(name, "__fs_write_bytes") { return 3 } - if str_eq(name, "__http_do") { return 5 } - if str_eq(name, "__http_do_map") { return 5 } - if str_eq(name, "__http_do_to_file") { return 5 } - if str_eq(name, "__http_serve") { return 2 } - if str_eq(name, "__http_serve_v2") { return 2 } - if str_eq(name, "__http_response") { return 3 } - if str_eq(name, "__thread_create") { return 2 } - if str_eq(name, "__thread_join") { return 1 } - if str_eq(name, "__mutex_new") { return 0 } - if str_eq(name, "__mutex_lock") { return 1 } - if str_eq(name, "__mutex_unlock") { return 1 } - if str_eq(name, "__exec") { return 1 } - if str_eq(name, "__exec_bg") { return 1 } - if str_eq(name, "__env_get") { return 1 } - if str_eq(name, "__args_json") { return 0 } - if str_eq(name, "__exit_program") { return 1 } - if str_eq(name, "__time_now_ns") { return 0 } - if str_eq(name, "__sleep_ms") { return 1 } - if str_eq(name, "__uuid_v4") { return 0 } - if str_eq(name, "__sqrt_f") { return 1 } - if str_eq(name, "__log_f") { return 1 } - if str_eq(name, "__ln_f") { return 1 } - if str_eq(name, "__sin_f") { return 1 } - if str_eq(name, "__cos_f") { return 1 } - if str_eq(name, "__pi_f") { return 0 } - if str_eq(name, "__state_set") { return 2 } - if str_eq(name, "__state_get") { return 1 } - if str_eq(name, "__state_del") { return 1 } - if str_eq(name, "__state_keys") { return 0 } - if str_eq(name, "__html_sanitize") { return 2 } - if str_eq(name, "__url_encode") { return 1 } - if str_eq(name, "__url_decode") { return 1 } - if str_eq(name, "__json_get") { return 2 } - if str_eq(name, "__json_get_raw") { return 2 } - if str_eq(name, "__json_parse_map") { return 1 } - if str_eq(name, "__json_stringify_val") { return 1 } - if str_eq(name, "__json_array_len") { return 1 } - if str_eq(name, "__json_array_get") { return 2 } - if str_eq(name, "__json_array_get_string") { return 2 } - if str_eq(name, "__json_set") { return 3 } - if str_eq(name, "__engram_node") { return 3 } - if str_eq(name, "__engram_node_full") { return 8 } - if str_eq(name, "__engram_get_node") { return 1 } - if str_eq(name, "__engram_strengthen") { return 1 } - if str_eq(name, "__engram_forget") { return 1 } - if str_eq(name, "__engram_node_count") { return 0 } - if str_eq(name, "__engram_search") { return 2 } - if str_eq(name, "__engram_scan_nodes") { return 2 } - if str_eq(name, "__engram_connect") { return 4 } - if str_eq(name, "__engram_edge_between") { return 2 } - if str_eq(name, "__engram_neighbors") { return 1 } - if str_eq(name, "__engram_neighbors_filtered") { return 3 } - if str_eq(name, "__engram_activate") { return 2 } - if str_eq(name, "__engram_activate_json") { return 2 } - if str_eq(name, "__engram_op_assert_json") { return 2 } - if str_eq(name, "__engram_node_full_in") { return 9 } - if str_eq(name, "__engram_connect_in") { return 5 } - if str_eq(name, "__engram_scan_nodes_json") { return 2 } - if str_eq(name, "__engram_edges_json") { return 2 } - if str_eq(name, "__engram_pool_stats_json") { return 0 } - if str_eq(name, "__el_alloc_count") { return 0 } - if str_eq(name, "__el_alloc_bytes") { return 0 } - if str_eq(name, "__el_peak_rss") { return 0 } - if str_eq(name, "__generate") { return 1 } - // Filesystem - if str_eq(name, "fs_read") { return 1 } - if str_eq(name, "fs_write") { return 2 } - if str_eq(name, "fs_list") { return 1 } - if str_eq(name, "fs_size") { return 1 } - if str_eq(name, "fs_read_b64_chunk") { return 3 } - // JSON - if str_eq(name, "json_get") { return 2 } - if str_eq(name, "json_parse") { return 1 } - if str_eq(name, "json_stringify") { return 1 } - if str_eq(name, "json_get_string") { return 2 } - if str_eq(name, "json_get_int") { return 2 } - if str_eq(name, "json_get_float") { return 2 } - if str_eq(name, "json_get_bool") { return 2 } - if str_eq(name, "json_get_raw") { return 2 } - if str_eq(name, "json_set") { return 3 } - if str_eq(name, "json_array_len") { return 1 } - // Time - if str_eq(name, "time_now") { return 0 } - if str_eq(name, "time_now_utc") { return 0 } - if str_eq(name, "sleep_secs") { return 1 } - if str_eq(name, "sleep_ms") { return 1 } - if str_eq(name, "time_format") { return 2 } - if str_eq(name, "time_to_parts") { return 1 } - if str_eq(name, "time_from_parts") { return 3 } - if str_eq(name, "time_add") { return 3 } - if str_eq(name, "time_diff") { return 3 } - // UUID - if str_eq(name, "uuid_new") { return 0 } - if str_eq(name, "uuid_v4") { return 0 } - // Env / state - if str_eq(name, "env") { return 1 } - if str_eq(name, "state_set") { return 2 } - if str_eq(name, "state_get") { return 1 } - if str_eq(name, "state_del") { return 1 } - if str_eq(name, "state_keys") { return 0 } - // Float - if str_eq(name, "float_to_str") { return 1 } - if str_eq(name, "int_to_float") { return 1 } - if str_eq(name, "float_to_int") { return 1 } - if str_eq(name, "format_float") { return 2 } - if str_eq(name, "decimal_round") { return 2 } - if str_eq(name, "str_to_float") { return 1 } - // Math (Float) - if str_eq(name, "math_sqrt") { return 1 } - if str_eq(name, "math_log") { return 1 } - if str_eq(name, "math_ln") { return 1 } - if str_eq(name, "math_sin") { return 1 } - if str_eq(name, "math_cos") { return 1 } - if str_eq(name, "math_pi") { return 0 } - // Bool - if str_eq(name, "bool_to_str") { return 1 } - // Process - if str_eq(name, "exit_program") { return 1 } - // Process info - if str_eq(name, "getpid_now") { return 0 } - // stdout redirect (used by elc post-processing) - if str_eq(name, "stdout_to_file") { return 1 } - if str_eq(name, "stdout_restore") { return 0 } - // Subprocess execution - if str_eq(name, "exec_command") { return 1 } - if str_eq(name, "exec_capture") { return 1 } - if str_eq(name, "exec") { return 1 } - if str_eq(name, "exec_bg") { return 1 } - // CGI / DHARMA - if str_eq(name, "dharma_connect") { return 1 } - if str_eq(name, "dharma_send") { return 2 } - if str_eq(name, "dharma_activate") { return 1 } - if str_eq(name, "dharma_emit") { return 2 } - if str_eq(name, "dharma_field") { return 1 } - if str_eq(name, "dharma_strengthen") { return 2 } - if str_eq(name, "dharma_relationship") { return 1 } - if str_eq(name, "dharma_peers") { return 0 } - // Engram - if str_eq(name, "engram_node") { return 3 } - if str_eq(name, "engram_node_full") { return 8 } - if str_eq(name, "engram_get_node") { return 1 } - if str_eq(name, "engram_strengthen") { return 1 } - if str_eq(name, "engram_forget") { return 1 } - if str_eq(name, "engram_node_count") { return 0 } - if str_eq(name, "engram_search") { return 2 } - if str_eq(name, "engram_scan_nodes") { return 2 } - if str_eq(name, "engram_connect") { return 4 } - if str_eq(name, "engram_edge_between") { return 2 } - if str_eq(name, "engram_neighbors") { return 1 } - if str_eq(name, "engram_neighbors_filtered") { return 3 } - if str_eq(name, "engram_edge_count") { return 0 } - if str_eq(name, "engram_activate") { return 2 } - if str_eq(name, "engram_save") { return 1 } - if str_eq(name, "engram_load") { return 1 } - if str_eq(name, "engram_store_boot") { return 1 } - if str_eq(name, "engram_store_checkpoint") { return 0 } - if str_eq(name, "engram_store_close") { return 0 } - if str_eq(name, "engram_get_node_json") { return 1 } - if str_eq(name, "engram_get_node_by_label") { return 1 } - if str_eq(name, "engram_search_json") { return 2 } - if str_eq(name, "engram_scan_nodes_json") { return 2 } - if str_eq(name, "engram_edges_json") { return 2 } - if str_eq(name, "engram_pool_stats_json") { return 0 } - if str_eq(name, "el_alloc_count") { return 0 } - if str_eq(name, "el_alloc_bytes") { return 0 } - if str_eq(name, "el_peak_rss") { return 0 } - if str_eq(name, "el_black_box") { return 1 } - if str_eq(name, "engram_neighbors_json") { return 3 } - if str_eq(name, "engram_activate_json") { return 2 } - if str_eq(name, "engram_stats_json") { return 0 } - if str_eq(name, "engram_op_assert_json") { return 2 } - if str_eq(name, "engram_node_full_in") { return 9 } - if str_eq(name, "engram_connect_in") { return 5 } - // LLM - if str_eq(name, "llm_call") { return 2 } - if str_eq(name, "llm_call_system") { return 3 } - if str_eq(name, "llm_call_agentic") { return 4 } - if str_eq(name, "llm_vision") { return 4 } - if str_eq(name, "llm_models") { return 0 } - if str_eq(name, "llm_register_tool") { return 2 } - // Crypto - if str_eq(name, "sha256_hex") { return 1 } - if str_eq(name, "sha256_bytes") { return 1 } - if str_eq(name, "hmac_sha256_hex") { return 2 } - if str_eq(name, "hmac_sha256_bytes") { return 2 } - if str_eq(name, "base64_encode") { return 1 } - if str_eq(name, "base64_decode") { return 1 } - if str_eq(name, "base64url_encode") { return 1 } - if str_eq(name, "base64url_decode") { return 1 } - // Native VM aliases - if str_eq(name, "native_list_get") { return 2 } - if str_eq(name, "native_list_len") { return 1 } - if str_eq(name, "native_list_append") { return 2 } - if str_eq(name, "native_list_empty") { return 0 } - if str_eq(name, "native_list_clone") { return 1 } - if str_eq(name, "native_string_chars") { return 1 } - if str_eq(name, "native_int_to_str") { return 1 } - // Method-call aliases - if str_eq(name, "append") { return 2 } - if str_eq(name, "len") { return 1 } - if str_eq(name, "get") { return 2 } - if str_eq(name, "map_get") { return 2 } - if str_eq(name, "map_set") { return 3 } - // Threading seed primitives - if str_eq(name, "__thread_create") { return 2 } - if str_eq(name, "__thread_join") { return 1 } - if str_eq(name, "__mutex_new") { return 0 } - if str_eq(name, "__mutex_lock") { return 1 } - if str_eq(name, "__mutex_unlock") { return 1 } - // Channel seed primitives - if str_eq(name, "__channel_new") { return 1 } - if str_eq(name, "__channel_send") { return 2 } - if str_eq(name, "__channel_recv") { return 1 } - if str_eq(name, "__channel_try_recv") { return 1 } - if str_eq(name, "__channel_close") { return 1 } - // Arena mark/restore builtins - if str_eq(name, "el_arena_push") { return 0 } - if str_eq(name, "el_arena_pop") { return 1 } - // -1 sentinel: variadic / unknown / user-defined -> no check. - return -1 -} -fn arity_record_violation(fn_name: String, expected: Int, actual: Int) -> Bool { - let csv: String = state_get("__arity_violations") - if str_eq(csv, "") { let csv = "," } - // Encode as fn_name|expected|actual to recover all three at emit time. - let entry: String = fn_name + "|" + native_int_to_str(expected) + "|" + native_int_to_str(actual) - let key: String = "," + entry + "," - if str_contains(csv, key) { return true } - state_set("__arity_violations", csv + entry + ",") - return true -} // Validate the call's arity against the builtin table. Returns true (always) // because cg_expr ignores the result; -1 from builtin_arity signals // "no check possible" (variadic or user-defined). A mismatch is recorded // and surfaced as an #error at the bottom of the generated C, so cc fails // before it ever attempts to type-check the wrong call. -fn arity_check_call(fn_name: String, actual: Int) -> Bool { - let expected: Int = builtin_arity(fn_name) - if expected < 0 { return true } - if expected == actual { return true } - arity_record_violation(fn_name, expected, actual) - return true -} // Emit recorded arity violations as #error directives. -fn emit_arity_violations() -> Void { - let csv: String = state_get("__arity_violations") - if str_eq(csv, "") { return } - if str_eq(csv, ",") { return } - let n: Int = str_len(csv) - let i: Int = 1 - while i < n { - let next_comma: Int = str_index_of(str_slice(csv, i, n), ",") - if next_comma < 0 { return } - let entry: String = str_slice(csv, i, i + next_comma) - let p1: Int = str_index_of(entry, "|") - if p1 > 0 { - let fn_name: String = str_slice(entry, 0, p1) - let rest: String = str_slice(entry, p1 + 1, str_len(entry)) - let p2: Int = str_index_of(rest, "|") - if p2 > 0 { - let exp_s: String = str_slice(rest, 0, p2) - let act_s: String = str_slice(rest, p2 + 1, str_len(rest)) - emit_line("#error \"arity error: '" + fn_name + "' takes " + exp_s + " arguments, but called with " + act_s + "\"") - } - } - let i = i + next_comma + 1 - } -} fn add_int_name(name: String) -> Bool { let csv: String = state_get("__int_names") @@ -3195,8 +2751,100 @@ fn fn_has_decorator(stmt: Map, name: String) -> Bool { false } +// cg_entry_seam — emit the guards and the entry injection for a decorated fn. +// Extracted so it can be emitted into the WRAPPER when a construct injects at +// exit (the body then lives in a static helper) and inline otherwise. +// +// Guards run FIRST and may refuse — a non-zero return short-circuits the fn and +// becomes its result. A refused call must not report a crossing, so guards +// precede injection. EVERY guard runs (stacking @authenticate @authorize +// applies both); injection is topmost-wins, matching the VBD role convention, +// because a role is singular and a refusal is not. +fn cg_entry_seam(stmt: Map, fn_name: String) -> Void { + // The crossing is resolved at EXECUTION. One refusable indirection replaces + // the compile-time guard loop and injection loop: which constructs apply, + // and whether any of them refuses, is read from a table written after the + // binary exists. + emit_line(" { el_val_t __s = el_seam_run(EL_STR(" + c_str_lit(fn_name) + "), 0, 0); if (__s) return __s; }") +} + +// cg_exit_target / cg_exit_construct — the first construct on this fn that +// injects at exit, or "" if none. + + + +// params_to_env_fields / params_to_env_init / params_to_env_args — the captured +// environment. This IS the closure: a struct of captured values, a function +// pointer that takes it, and the pair handed to the wrap target. +fn params_to_env_fields(params: [Any]) -> String { + // A zero-param fn would emit `struct __env_f { };` -- an empty struct is a + // GNU extension, not C99, and an empty initialiser is C23. + if native_list_len(params) == 0 { return " char __e0;" } + let out: String = "" + let n: Int = native_list_len(params) + let i = 0 + while i < n { + let p = native_list_get(params, i) + let pn: String = p["name"] + let out = out + " el_val_t " + pn + ";" + let i = i + 1 + } + out +} + +fn params_to_env_init(params: [Any]) -> String { + if native_list_len(params) == 0 { return "0" } + let out: String = "" + let n: Int = native_list_len(params) + let i = 0 + while i < n { + let p = native_list_get(params, i) + let pn: String = p["name"] + if i > 0 { let out = out + ", " } + let out = out + pn + let i = i + 1 + } + out +} + +fn params_to_env_args(params: [Any]) -> String { + let out: String = "" + let n: Int = native_list_len(params) + let i = 0 + while i < n { + let p = native_list_get(params, i) + let pn: String = p["name"] + if i > 0 { let out = out + ", " } + let out = out + "__e->" + pn + let i = i + 1 + } + out +} + + +// params_to_call_args — "a, b, c" from the param list, for the wrapper's call +// into the body helper. + fn cg_fn(stmt: Map) -> Void { let fn_name: String = stmt["name"] + state_set("__cg_current_fn", fn_name) + // Record every top-level definition. El has NO namespacing -- imports are + // textual inlining, so two modules defining the same name emit two C + // functions into one translation unit. cc catches it, but names the + // generated helpers first and the user's fn fourth. Recording it lets the + // collision be reported at El level, in El terms. + record_call(fn_name, "defines_at:" + stmt["line"]) + // Emit which constructs this fn carries, so the prohibition check can be a + // query over relations instead of a walk inside the emitter. + let cdl = stmt["decorators"] + let n_cdl: Int = native_list_len(cdl) + let cdi = 0 + while cdi < n_cdl { + let cd = native_list_get(cdl, cdi) + let cdn: String = cd["name"] + record_call(fn_name, "@" + cdn) + let cdi = cdi + 1 + } // Skip El's `fn main()` - C provides its own main() for top-level stmts // and a duplicate `el_val_t main(void)` would collide with it. if fn_name == "main" { return } @@ -3208,23 +2856,32 @@ fn cg_fn(stmt: Map) -> Void { // from @manager-decorated functions. Surface violations to the C compiler // via #error directives emitted before the function definition. Read the // decorator LIST so the role may be stacked with other decorators. - if vbd_has_restricted_call(body) { - if !fn_has_decorator(stmt, "manager") { - emit_line("#error \"VBD violation: dharma_emit/dharma_field called from non-@manager fn '" + fn_name + "'\"") - } - } + // Check each DECLARED prohibition in turn. The owning construct is known by + // construction, so the diagnostic names it rather than hardcoding one rule. + // The walk records; it no longer decides. Whether a call is legal where it + // appears is a query over the emitted relations — see + // tools/check/prohibitions.sh. An emitter that also adjudicates has to + // contain every rule anyone will ever want. + vbd_has_restricted_call(body) // Seed the per-function int-name set so the `+` codegen can dispatch // arithmetic vs concat on type-annotated identifiers. build_int_names_for_params(params) - emit_line("el_val_t " + fn_name + "(" + params_c + ") {") // ── API-reshape decorator-seam: auto-emit at the decorated-fn boundary ── - // Every @manager/@accessor fn gets ONE injected call to engram_boundary_beat - // at entry — interoception (chrono tick) + telemetry (afferent counter) + - // strengthen (self-activity) + a dharma bus event — so a decorated op - // self-reports with ZERO hand-written instrumentation in its body. (VBD role - // = the topmost decorator; write it topmost when stacking with @route.) - if fn_has_decorator(stmt, "manager") || fn_has_decorator(stmt, "accessor") { - emit_line(" engram_boundary_beat(EL_STR(" + c_str_lit(fn_name) + "));") + // A decorated op self-reports with ZERO hand-written instrumentation in its + // body. What it reports, and whether it may refuse, is DECLARED by the + // construct — codegen reads it (see scan_declared_decorators). + // + // When a construct injects at exit, the body moves into a static helper and + // the visible fn becomes a wrapper, so EARLY RETURNS still pass through the + // exit injection. Emitting it only before the fall-through return would + // silently miss every early return — which is precisely the class of + // failure this seam exists to remove. Fns with no exit construct emit + // exactly as before, byte for byte. + if true { + emit_line("static el_val_t __el_body_" + fn_name + "(" + params_c + ") {") + } else { + emit_line("el_val_t " + fn_name + "(" + params_c + ") {") + cg_entry_seam(stmt, fn_name) } // Seed declared with parameter names so reassignment works let decl = native_list_empty() @@ -3248,6 +2905,28 @@ fn cg_fn(stmt: Map) -> Void { el_release(final_decl) emit_line(" return 0;") emit_line("}") + // The wrapper: guards, entry injection, the body call, then the exit + // injection, which receives the result so it can observe what the fn + // actually returned. + if true { + // EVERY fn gets an env struct and a thunk. Codegen cannot know which fns + // a wrap construct will be bound to after the binary exists, and the + // wrapper unconditionally calls through el_seam_wrap. + emit_line("struct __env_" + fn_name + " { " + params_to_env_fields(params) + " };") + emit_line("static el_val_t __thunk_" + fn_name + "(void* __v) {") + emit_line(" struct __env_" + fn_name + "* __e = (struct __env_" + fn_name + "*)__v; (void)__e;") + emit_line(" return __el_body_" + fn_name + "(" + params_to_env_args(params) + ");") + emit_line("}") + } + if true { + emit_line("el_val_t " + fn_name + "(" + params_c + ") {") + cg_entry_seam(stmt, fn_name) + emit_line(" struct __env_" + fn_name + " __env = { " + params_to_env_init(params) + " };") + emit_line(" el_val_t __r = el_seam_wrap(EL_STR(" + c_str_lit(fn_name) + "), __thunk_" + fn_name + ", &__env);") + emit_line(" __r = el_seam_run(EL_STR(" + c_str_lit(fn_name) + "), 1, __r);") + emit_line(" return __r;") + emit_line("}") + } emit_blank() } @@ -3347,12 +3026,25 @@ fn emit_program_init(stmt: Map) -> Void { // Scan a function body for direct calls to DHARMA-restricted builtins // (dharma_emit, dharma_field). These may only appear inside @manager fns. -fn vbd_is_restricted_name(name: String) -> Bool { - if str_eq(name, "dharma_emit") { return true } - if str_eq(name, "dharma_field") { return true } - false +// record_call — emit one calls relation. The traversal is +// irreducible: you must walk the AST to find calls. What is NOT irreducible is +// the rule (which names are restricted) or the decision (#error). Those move +// out; the walk stays and changes purpose from deciding to recording. +fn record_call(caller: String, callee: String) -> Void { + let path: String = env("EL_RELATIONS_OUT") + if str_eq(path, "") { return } + // cg_fn returns early for `main` -- C provides its own -- so __cg_current_fn + // is unset while main's body is walked. Without this, every call in main is + // invisible to the relation graph and both checkers have a blind spot + // exactly where a program does its work. Caught by capability_query.sh. + let caller2: String = caller + if str_eq(caller2, "") { let caller2 = "main" } + let prev: String = "" + if fs_exists(path) { let prev = fs_read(path) } + fs_write(path, prev + caller2 + " calls " + callee + "\n") } + fn vbd_expr_has_restricted_call(expr: Map) -> Bool { let kind: String = expr["expr"] if str_eq(kind, "Call") { @@ -3360,7 +3052,7 @@ fn vbd_expr_has_restricted_call(expr: Map) -> Bool { let fk: String = func["expr"] if str_eq(fk, "Ident") { let fname: String = func["name"] - if vbd_is_restricted_name(fname) { return true } + record_call(state_get("__cg_current_fn"), fname) } if vbd_expr_has_restricted_call(func) { return true } let args = expr["args"] @@ -3529,11 +3221,8 @@ fn codegen(stmts: [Map], source: String) -> String { if svc_count >= 1 { let kind = "service" } state_set("__program_kind", kind) // Clear capability-violation accumulator from any prior compile. - state_set("__cap_violations", "") // Clear arity-violation accumulator from any prior compile. - state_set("__arity_violations", "") // Clear temporal-type-violation accumulator from any prior compile. - state_set("__time_violations", "") // Preamble emit_line("#include ") @@ -3694,6 +3383,11 @@ fn codegen(stmts: [Map], source: String) -> String { // main(). Use _argc/_argv so El programs are free to declare their own // local `argv` / `argc` (compiler.el itself does this) without colliding // with the C-side parameters when fn main()'s body is folded in below. + // From here the emitter is walking main's body. Without resetting this the + // caller attribution is stale from the last cg_fn, so a violation inside + // main is reported against whichever function happened to be emitted last + // -- a diagnostic naming the wrong fn is worse than none. + state_set("__cg_current_fn", "main") emit_line("int main(int _argc, char** _argv) {") emit_line(" el_runtime_init_args(_argc, _argv);") if prog_have { @@ -3790,13 +3484,10 @@ fn codegen(stmts: [Map], source: String) -> String { // will fail on the first one and surface the message; placement at // the bottom is fine - preprocessor errors halt the build wherever // they appear. - emit_cap_violations() // Same for builtin-arity violations: cc halts on the first #error, // so a misuse of a known builtin (wrong arg count) fails the build // with a clear message naming the builtin and its expected arity. - emit_arity_violations() // Temporal-type violations (Instant + Instant, Duration + Int, -). - emit_time_violations() // Return empty string - output was streamed via println "" @@ -3970,7 +3661,7 @@ fn route_sort_desc(recs: [Map]) -> [Map] { // be synthesized in the streaming backend, which discards per-fn ASTs. Handles // decorator STACKING: `@route(...) @manager fn` still records the route. fn scan_routes(tokens: [Any]) -> [Map] { - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let recs: [Map] = native_list_empty() let has_pending: Bool = false let pending_args: [String] = native_list_empty() @@ -4037,6 +3728,178 @@ fn program_has_routes(recs: [Map]) -> Bool { native_list_len(recs) > 0 } +// ── Declared constructs ────────────────────────────────────────────────────── +// +// A construct declares its own meaning. Codegen READS the declaration instead +// of knowing the construct: +// +// @decorator("injects_at_entry", "engram_boundary_beat") +// fn manager() {} +// +// Any fn decorated @manager then gets +// engram_boundary_beat(EL_STR(), EL_STR("manager")); +// injected at entry. Adding a construct is a declaration IN THE PROGRAM. It +// does not touch the compiler. +// +// WHY: codegen called fn_has_decorator for exactly three names — manager, +// accessor, route. Twelve others parsed, attached as {name,args}, and compiled +// to nothing, including four that look like protection: @authenticate (6 uses), +// @authorize (3), @rate_limit (3), @validate (2). The meaning of a construct +// had nowhere to live except the emitter, so every construct was a compiler +// edit and an undeclared one was silently inert. A lookup table of +// name -> injection would have moved the enumeration twenty lines up, not +// removed it; the class only goes away when the construct itself carries its +// meaning. +// +// manager and accessor are seeded below as the compiled-in core — a seed the +// periphery is declared against, the same shape as fixedSelf in substrate.go: +// a complete fallback exists, declaration is enrichment. A program may declare +// its own constructs and may redeclare these. +// +// This is the injection half of the seam. The prohibition half (@manager's +// #error on dharma_emit) is still hardcoded, because "which calls may appear +// inside this boundary" is a query over program structure and there is nothing +// yet to ask. + + + +// A GUARD is an injection that may refuse. The declared target is called at +// entry with the same (fn, construct) pair; a non-zero return short-circuits +// the decorated fn and becomes its result. +// +// @decorator("guards_at_entry", "my_auth_check") +// fn authenticate() {} +// +// @authenticate +// fn handler() -> String { ... } // my_auth_check runs first and may refuse +// +// This is what @authenticate (6 uses), @authorize (3), @rate_limit (3) and +// @validate (2) needed and never had. They parsed, attached, and compiled to +// nothing — fourteen applications that read as protection and emitted no +// instruction. The compiler still knows nothing about authentication: the +// program points the construct at its own function. + + +// An EXIT injection runs after the fn returns and receives the result: +// target(, , ) +// +// @decorator("injects_at_exit", "persist_now") +// fn durable() {} +// +// This is `hold`'s after-crossing face. §6 of the design records 62 +// persist-after-mutate sites and 9-of-9 failed index-after-append sites — every +// one an obligation that decayed into "remember to do this after you mutate." +// An obligation a human must remember is not an obligation. + + +// A WRAP construct receives the body as a CLOSURE and decides how, whether, and +// how many times to invoke it. +// +// target(, , el_val_t (*body)(void*), void* env) -> el_val_t +// +// C has no closure SYNTAX. That is not the same as C being unable to express a +// closure: every C++ lambda, every Go closure, every Rust closure compiles to a +// struct of captured values plus a function pointer, which is exactly what is +// emitted here. Taking "C has no closures" as a fact about what is possible, +// rather than about one grammar, produces a strictly weaker construct that can +// only decide whether to repeat — no timeout, no rollback-and-retry, no +// parallel, no memoize-on-arguments. It would also have crippled the JS backend, +// which has closures natively, for a limit in the C one. + + +// A PROHIBITION is the other half of a boundary: not what runs when something +// crosses, but what may not cross at all. +// +// @decorator("prohibits_outside", "dharma_emit,dharma_field") +// fn manager() {} +// +// Calls to those names may then appear ONLY inside a fn carrying @manager. +// Before this the rule lived as two string literals in vbd_is_restricted_name +// and one #error in cg_fn -- one prohibition, uneditable without a compiler +// release, and unrankable because a #error is a string handed to cpp, which +// knows nothing about El. +// +// The construct list is kept in one state key so the recursive walker can be +// parameterised without threading an argument through every branch of it -- +// the same mechanism codegen already uses for __match_counter and +// __if_expr_counter. +fn declare_prohibition(construct: String, names_csv: String) -> Void { + record_call(construct, "prohibits:" + names_csv) +} + + + +// scan_declared_decorators — token-level pre-pass registering every construct +// the program declares. Runs once per module alongside scan_routes, because +// the streaming backend discards per-fn ASTs and there is no whole-program AST +// to walk. +fn scan_declared_decorators(tokens: [Any]) -> Void { + declare_prohibition("manager", "dharma_emit,dharma_field") + let total: Int = native_list_len(tokens) / 3 + let has_pending_p: Bool = false + let pending_prohibit: String = "" + let pos: Int = 0 + let going: Bool = true + while going { + if pos >= total { + let going = false + } else { + let k: String = tok_kind(tokens, pos) + if str_eq(k, "Eof") { + let going = false + } else { + if str_eq(k, "At") { + let dname: String = tok_value(tokens, pos + 1) + let p: Int = pos + 2 + let args: [String] = native_list_empty() + let ka: String = tok_kind(tokens, p) + if str_eq(ka, "LParen") { + let p = p + 1 + let running: Bool = true + while running { + let kd: String = tok_kind(tokens, p) + if str_eq(kd, "RParen") { + let running = false + } else { + if str_eq(kd, "Eof") { + let running = false + } else { + if str_eq(kd, "Str") { + let args = native_list_append(args, tok_value(tokens, p)) + } + let p = p + 1 + } + } + } + if str_eq(tok_kind(tokens, p), "RParen") { let p = p + 1 } + } + if str_eq(dname, "decorator") { + if native_list_len(args) >= 2 { + let dkind: String = native_list_get(args, 0) + if str_eq(dkind, "prohibits_outside") { + let has_pending_p = true + let pending_prohibit = native_list_get(args, 1) + } + } + } + let pos = p + } else { + if str_eq(k, "Fn") { + let fname: String = tok_value(tokens, pos + 1) + if has_pending_p { + declare_prohibition(fname, pending_prohibit) + let has_pending_p = false + } + let pos = pos + 2 + } else { + let pos = pos + 1 + } + } + } + } + } +} + // route_method_guard — C boolean prefix guarding on HTTP method, or "" for none. fn route_method_guard(method: String) -> String { if str_eq(method, "") { return "" } @@ -4133,9 +3996,6 @@ fn emit_streaming_preamble(sigs: [Map], source: String) -> Void { if cgi_count >= 1 { let kind = "cgi" } if svc_count >= 1 { let kind = "service" } state_set("__program_kind", kind) - state_set("__cap_violations", "") - state_set("__arity_violations", "") - state_set("__time_violations", "") emit_line("#include ") emit_line("#include ") @@ -4186,7 +4046,7 @@ fn emit_streaming_preamble(sigs: [Map], source: String) -> Void { // sigs: pre-scanned signature list from scan_fn_sigs(tokens) // source: original source string (for string literal lookup) fn codegen_streaming(tokens: [Any], sigs: [Map], source: String) -> String { - let total_tokens: Int = native_list_len(tokens) / 2 + let total_tokens: Int = native_list_len(tokens) / 3 // Emit preamble (forward decls, file-scope lets, #includes) // Arena scope: free intermediate strings built during preamble emission. @@ -4199,6 +4059,11 @@ fn codegen_streaming(tokens: [Any], sigs: [Map], source: String) -> // tokens below via refcount, like `sigs`). If any exist, forward-declare the // generated dispatcher NOW so hand-written fns (e.g. handle_request) may call // it before its definition is emitted after the fn-emit loop. + // Declared constructs: register what the program says its decorators mean, + // before any fn is emitted. Must precede the fn-emit loop — cg_fn reads the + // registry to decide what, if anything, a decorator injects. + scan_declared_decorators(tokens) + let route_records: [Map] = scan_routes(tokens) if program_has_routes(route_records) { emit_line("el_val_t el_route_dispatch(el_val_t method, el_val_t clean, el_val_t path, el_val_t body);") @@ -4537,6 +4402,7 @@ fn codegen_streaming(tokens: [Any], sigs: [Map], source: String) -> // main() delegates to the El-side runner. Everything above this line is // generated glue; all reporting logic lives in runtime/eltest.el. + state_set("__cg_current_fn", "main") emit_line("int main(int _argc, char **_argv) {") emit_line(" el_runtime_init_args(_argc, _argv);") emit_line(" for (int _i = 1; _i < _argc; _i++) {") @@ -4566,6 +4432,7 @@ fn codegen_streaming(tokens: [Any], sigs: [Map], source: String) -> // Emit main() — wrap in arena scope to free intermediate strings. let main_arena_mark: Any = el_arena_push() let kind2: String = state_get("__program_kind") + state_set("__cg_current_fn", "main") emit_line("int main(int _argc, char** _argv) {") emit_line(" el_runtime_init_args(_argc, _argv);") // Cross-cutting concerns declared by a `program` block run BEFORE anything @@ -4651,10 +4518,6 @@ fn codegen_streaming(tokens: [Any], sigs: [Map], source: String) -> emit_line(" return 0;") emit_line("}") emit_blank() - - emit_cap_violations() - emit_arity_violations() - emit_time_violations() el_arena_pop(main_arena_mark) "" diff --git a/lang/el-compiler/src/compiler.el b/lang/el-compiler/src/compiler.el index 14c3ccd..b06405c 100644 --- a/lang/el-compiler/src/compiler.el +++ b/lang/el-compiler/src/compiler.el @@ -414,6 +414,12 @@ fn parse_import_line(trimmed: String, dir: String) -> String { // Accumulates chunks into lists and joins once at the end to avoid the O(n²) // memory growth caused by repeated `prefix = prefix + chunk` concatenation. fn resolve_imports(src_path: String) -> String { + // Only the OUTERMOST call publishes provenance. Nested calls number their + // lines from 1 within themselves, so their spans are meaningless once the + // text is spliced into the parent. + let depth: String = state_get("__elc_prov_depth") + if str_eq(depth, "") { state_set("__elc_prov_depth", "1") } + let is_top: Bool = str_eq(depth, "") let seen_key: String = "__elc_imp__:" + src_path let already: String = state_get(seen_key) if !str_eq(already, "") { return "" } @@ -443,6 +449,7 @@ fn resolve_imports(src_path: String) -> String { // Collect chunks into lists — O(1) amortized per append. // Join once at the end — O(n) single pass. let prefix_chunks: [String] = native_list_empty() + let prefix_paths: [String] = native_list_empty() let body_chunks: [String] = native_list_empty() let i: Int = 0 while i < n { @@ -454,21 +461,54 @@ fn resolve_imports(src_path: String) -> String { // Only check .elh for imported files — never for the entry file itself. let imp_elh_path: String = str_slice(imp_path, 0, str_len(imp_path) - 3) + ".elh" let imp_elh: String = fs_read(imp_elh_path) + // Provenance: record which line range of the combined source came + // from which file, so a diagnostic can name the FILE and not just a + // line in a string that no longer exists on disk. if !str_eq(imp_elh, "") { // Header exists: mark the .el as seen (so it won't be re-inlined // if something else also imports it) and use the header text. let seen_imp_key: String = "__elc_imp__:" + imp_path state_set(seen_imp_key, "1") let prefix_chunks = native_list_append(prefix_chunks, imp_elh) + let prefix_paths = native_list_append(prefix_paths, imp_path) } else { let imp_body: String = resolve_imports(imp_path) let prefix_chunks = native_list_append(prefix_chunks, imp_body) + let prefix_paths = native_list_append(prefix_paths, imp_path) } } else { let body_chunks = native_list_append(body_chunks, line + "\n") } let i = i + 1 } + // Walk the assembled chunks once and publish spans . + // LIMIT: nested imports return a single string, so their internal + // boundaries are already lost by the time we see them -- a definition + // inside a transitively imported file is attributed to the direct import. + // Local, not accumulated in state: a nested call numbers its lines from 1 + // within itself, so letting it append to a shared buffer republishes + // meaningless spans under the parent's name. + let prov: String = "" + let line_at: Int = 1 + let ci: Int = 0 + let nchunks: Int = native_list_len(prefix_chunks) + while ci < nchunks { + let chunk: String = native_list_get(prefix_chunks, ci) + let nlines: Int = str_count_lines(chunk) + let src: String = native_list_get(prefix_paths, ci) + let prov = prov + src + " spans " + native_int_to_str(line_at) + " " + native_int_to_str(line_at + nlines - 1) + "\n" + let line_at = line_at + nlines + let ci = ci + 1 + } + let prov = prov + src_path + " spans " + native_int_to_str(line_at) + " 999999\n" + if is_top { + let prov_out: String = env("EL_RELATIONS_OUT") + if !str_eq(prov_out, "") { + let existing: String = "" + if fs_exists(prov_out) { let existing = fs_read(prov_out) } + fs_write(prov_out, existing + prov) + } + } return str_join(prefix_chunks, "") + str_join(body_chunks, "") } diff --git a/lang/el-compiler/src/lexer.el b/lang/el-compiler/src/lexer.el index a8620d3..358f486 100644 --- a/lang/el-compiler/src/lexer.el +++ b/lang/el-compiler/src/lexer.el @@ -138,13 +138,38 @@ fn lex_is_whitespace(ch: String) -> Bool { // tok_append — append a (kind, value) pair to a flat token list. // Returns the updated list. Gamma combines flat-list + char-code for max savings. +// A token is (kind, value, line). The line comes from state rather than a +// parameter so the ~200 existing tok_append call sites are untouched -- the +// lexer advances __lex_line as it walks, and every token minted takes the line +// it was minted on. +// +// WHY AT ALL: before this a token carried no position, so no diagnostic in El +// could name a place. Every error named a symbol and never a line, and after +// textual inlining there was no way to say which FILE a definition came from. fn tok_append(tokens: [Any], kind: String, value: String) -> [Any] { let tokens = native_list_append(tokens, kind) - native_list_append(tokens, value) + let tokens = native_list_append(tokens, value) + native_list_append(tokens, state_get("__lex_line")) } // -- Keyword lookup ------------------------------------------------------------ +// keyword_kind — the language's reserved spellings. +// +// A grammar is a BASIS: `fn` means function-start because someone said so, and +// nothing derives it. But unlike the other tables moved out this session, this +// one stays code, and the SHOULD gate is why. The keyword set is closed by the +// language definition -- it does not leak the way an allowlist does -- and the +// lexer runs before the program is understood, so a program can never declare +// its own keywords. Externalising it would cost file I/O on every compile and +// buy nothing. +// +// Removed 2026-08-17: sealed, activate, seed, protocol, impl. Reserved in the +// lexer, consumed by no parser or codegen path, and each one stole an +// identifier from users for nothing. `test` LOOKED inert by the same measure +// and is not -- codegen consumes it at 4135 for --test mode, 408 uses in the +// tree. The first measurement checked only parser.el and would have broken all +// of them. fn keyword_kind(word: String) -> String { if word == "let" { return "Let" } if word == "fn" { return "Fn" } @@ -161,14 +186,9 @@ fn keyword_kind(word: String) -> String { if word == "from" { return "From" } if word == "as" { return "As" } if word == "with" { return "With" } - if word == "sealed" { return "Sealed" } - if word == "activate" { return "Activate" } if word == "where" { return "Where" } if word == "test" { return "Test" } - if word == "seed" { return "Seed" } if word == "assert" { return "Assert" } - if word == "protocol" { return "Protocol" } - if word == "impl" { return "Impl" } if word == "retry" { return "Retry" } if word == "times" { return "Times" } if word == "fallback" { return "Fallback" } @@ -521,6 +541,12 @@ fn scan_interp_brace(src: String, start: Int, total: Int) -> Map { // interp_tokens_append_all - copy every (kind, value) pair from flat src list // into flat dst list, skipping the trailing Eof pair that lex() always appends. +// Splices re-lexed interpolation tokens into the stream. This walks the token +// list DIRECTLY rather than through tok_append, so it carries its own copy of +// the stride -- which is why giving tokens a line broke the compiler's second +// generation and not its first: the compiler's own source uses string +// interpolation, so gen1 (built by the old compiler) was fine and gen2 emitted +// a corrupted stream. fn interp_tokens_append_all(dst: [Any], src: [Any]) -> [Any] { let src_len: Int = native_list_len(src) let j = 0 @@ -531,9 +557,11 @@ fn interp_tokens_append_all(dst: [Any], src: [Any]) -> [Any] { let j = src_len } else { let val: String = native_list_get(src, j + 1) + let ln: String = native_list_get(src, j + 2) let result = native_list_append(result, kind) let result = native_list_append(result, val) - let j = j + 2 + let result = native_list_append(result, ln) + let j = j + 3 } } result @@ -764,8 +792,14 @@ fn lex(source: String) -> [Any] { let total: Int = str_len(source) let tokens: [Any] = native_list_empty() let i: Int = 0 + state_set("__lex_line", "1") + let line_no: Int = 1 while i < total { + if str_eq(str_slice(source, i, i + 1), "\n") { + let line_no = line_no + 1 + state_set("__lex_line", native_int_to_str(line_no)) + } let c: Int = str_char_code(source, i) // Skip whitespace (space=32, tab=9, newline=10, CR=13) diff --git a/lang/el-compiler/src/parser.el b/lang/el-compiler/src/parser.el index 68c69d3..f72015f 100644 --- a/lang/el-compiler/src/parser.el +++ b/lang/el-compiler/src/parser.el @@ -17,8 +17,8 @@ // programs. All callers use these helpers -- only these three need updating. fn tok_at(tokens: [Any], pos: Int) -> Map { - let kind: String = native_list_get(tokens, pos * 2) - let value: String = native_list_get(tokens, pos * 2 + 1) + let kind: String = native_list_get(tokens, pos * 3) + let value: String = native_list_get(tokens, pos * 3 + 1) { "kind": kind, "value": value } } @@ -28,25 +28,32 @@ fn tok_kind(tokens: [Any], pos: Int) -> String { // single trailing Eof token returns runtime null (el_list_get OOB -> 0), // which matches no delimiter, letting inner parse loops append AST nodes // forever on malformed input -> unbounded allocation -> OOM. - let n: Int = native_list_len(tokens) / 2 + let n: Int = native_list_len(tokens) / 3 if pos < 0 { return "Eof" } if pos >= n { return "Eof" } - native_list_get(tokens, pos * 2) + native_list_get(tokens, pos * 3) +} + +fn tok_line(tokens: [Any], pos: Int) -> String { + let n: Int = native_list_len(tokens) / 3 + if pos < 0 { return "0" } + if pos >= n { return "0" } + native_list_get(tokens, pos * 3 + 2) } fn tok_value(tokens: [Any], pos: Int) -> String { - let n: Int = native_list_len(tokens) / 2 + let n: Int = native_list_len(tokens) / 3 if pos < 0 { return "" } if pos >= n { return "" } - native_list_get(tokens, pos * 2 + 1) + native_list_get(tokens, pos * 3 + 1) } // parse_progress_fatal — robustness backstop. Called by the token-consuming @@ -1230,7 +1237,7 @@ fn parse_block(tokens: [Any], pos: Int) -> Map { // Runaway backstop: a block can hold at most (token count) statements, since // every iteration consumes >= 1 token. If we exceed that, the cursor has run // off the end without terminating (malformed input) -> fail fast, don't hang. - let blk_total: Int = native_list_len(tokens) / 2 + let blk_total: Int = native_list_len(tokens) / 3 let blk_iters: Int = 0 while running { let blk_iters = blk_iters + 1 @@ -1550,7 +1557,10 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map { let p = r2["pos"] // r2 result map fully consumed — release to free peak heap. el_release(r2) - return make_result({ "stmt": "FnDef", "name": name, "params": params, "body": body, "ret_type": ret_type }, p) + // The definition carries the line it was written on. Without it no + // diagnostic can name a place, and after textual inlining there is no + // way to say which FILE a definition came from. + return make_result({ "stmt": "FnDef", "name": name, "params": params, "body": body, "ret_type": ret_type, "line": tok_line(tokens, pos) }, p) } // type definition: `type Name = { field: Type, ... }` @@ -1842,6 +1852,7 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map { "params": inner["params"], "body": inner["body"], "ret_type": inner["ret_type"], + "line": inner["line"], "decorator": dec_name, "decorators": dlist } @@ -2158,7 +2169,7 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map { fn parse(tokens: [Any]) -> [Map] { // Flat list: 2 entries per token, so divide by 2 for token count. - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let stmts: [Map] = native_list_empty() let pos: Int = 0 let running = true @@ -2201,7 +2212,7 @@ fn parse_one(tokens: [Any], pos: Int) -> Map { // On entry, pos must point at the LBrace token. // Returns the position of the token AFTER the matching RBrace. fn skip_to_rbrace(tokens: [Any], pos: Int) -> Int { - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let p: Int = pos + 1 let depth: Int = 1 let going: Bool = true @@ -2253,7 +2264,7 @@ fn is_stmt_start_kind(k: String) -> Bool { // token that could start a new top-level statement, staying depth-aware // so that braces inside expressions don't fool us. fn skip_expr_to_stmt_boundary(tokens: [Any], pos: Int) -> Int { - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let p: Int = pos let depth: Int = 0 let going: Bool = true @@ -2419,7 +2430,7 @@ fn scan_params_el(tokens: [Any], pos: Int) -> Map { // // Peak memory: O(tokens) with no expression AST allocation. fn scan_fn_sigs_el(tokens: [Any]) -> [Map] { - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let sigs: [Map] = native_list_empty() let pos: Int = 0 let going: Bool = true @@ -2561,7 +2572,7 @@ fn scan_params_c(tokens: [Any], pos: Int) -> Map { // // The scan allocates only small string values per entry, keeping peak RSS low. fn scan_fn_sigs(tokens: [Any]) -> [Map] { - let total: Int = native_list_len(tokens) / 2 + let total: Int = native_list_len(tokens) / 3 let sigs: [Map] = native_list_empty() let pos: Int = 0 let going: Bool = true diff --git a/lang/runtime/el_runtime.c b/lang/runtime/el_runtime.c index 441be58..73c04e0 100644 --- a/lang/runtime/el_runtime.c +++ b/lang/runtime/el_runtime.c @@ -712,6 +712,20 @@ el_val_t el_map_set(el_val_t mapv, el_val_t keyv, el_val_t value) { * happen to look like aligned heap pointers are exceedingly unlikely to land * on a page whose first 4 bytes match either magic. */ +/* el_tagged — THE gate for "is this slot a heap object carrying this tag". + * + * el_val_t carries both integers and tagged heap pointers, so deciding which + * requires checking the value BEFORE dereferencing it. That check was a + * convention every author had to know rather than a gate they had to pass + * through, and the result is measurable: geom_of and mfld_of call + * looks_like_heap_obj and are correct; el_bin_lookup checked only a 4096 floor + * -- no alignment, no small-int, no negative -- and reads EIGHT BYTES BACKWARD + * from the pointer. sha256_hex(50000) therefore compiled clean and segfaulted. + * + * Exported, so the engram siblings stop re-deriving it. Anything that + * dereferences a slot without passing through here is the defect. */ +int el_tagged(el_val_t v, uint32_t magic); + static int looks_like_heap_obj(el_val_t v) { if (v == 0) return 0; int64_t s = (int64_t)v; @@ -722,6 +736,12 @@ static int looks_like_heap_obj(el_val_t v) { return 1; } +int el_tagged(el_val_t v, uint32_t magic) { + if (!looks_like_heap_obj(v)) return 0; + return *(const uint32_t*)(uintptr_t)v == magic; +} + + void el_retain(el_val_t v) { if (!looks_like_heap_obj(v)) return; ElHeader* h = (ElHeader*)(uintptr_t)v; @@ -14379,7 +14399,67 @@ el_val_t engram_ise_log_append(el_val_t content_v){ } } fputs("\"}\n", f); + /* RETENTION (2026-08-17 self-review). The on-graph ISE branch in + * server.el calls engram_prune_telemetry(48h) on every insert, but that + * branch is DEAD in production: ENGRAM_ISE_OFFGRAPH=1 is the live + * setting, so every state event lands here instead — and this path had + * no retention of any kind. Measured: 17.1 MB / 14,305 events over 3.56 + * days = 4.81 MB/day, growing without bound (~1.76 GB/year). The graph + * got its telemetry-growth fix on 2026-07-16; moving telemetry off-graph + * moved the leak rather than closing it. + * + * Byte-bounded rather than time-bounded on purpose: this is a flat + * append-only file with no index, so size is the property that actually + * has to be bounded, and a byte check is O(1) against the handle we + * already hold (ftell) instead of an O(file) timestamp scan per append. + * At the measured rate the 64 MB default retains ~13 days — comfortably + * more history than the 48h the on-graph path kept. + * + * Compaction keeps the TAIL, never the head: engram_dreams_json reads + * the last ~2 MB of this file for dream-recall, so the recent end is the + * end that has a reader. KEEP is held well above that 2 MB window so + * recall is never truncated by a rotation. The honesty rail is + * preserved exactly as before — rotated-out remains "I don't remember", + * never a synthesized dream; this only makes the forgetting bounded and + * explicit instead of deferred forever. */ + long pos = ftell(f); fclose(f); + { + long maxb = 64L*1024L*1024L; + long keepb = 16L*1024L*1024L; + const char* mv = getenv("ENGRAM_ISE_LOG_MAX_BYTES"); + if (mv && *mv) { long v = atol(mv); if (v > 0) maxb = v; } + if (keepb > maxb/2) keepb = maxb/2; + if (pos > 0 && pos > maxb) { + FILE* rf = fopen(path, "rb"); + if (rf) { + if (fseek(rf, pos - keepb, SEEK_SET) == 0) { + char* buf = (char*)malloc((size_t)keepb + 1); + if (buf) { + size_t rd = fread(buf, 1, (size_t)keepb, rf); + buf[rd] = 0; + /* Resume at the first LINE boundary so the tail never + * begins with a half-written JSON record. */ + char* start = memchr(buf, '\n', rd); + start = start ? start + 1 : buf; + size_t keep_n = rd - (size_t)(start - buf); + char tmp[4096]; + snprintf(tmp, sizeof tmp, "%s/state-events.jsonl.tmp", dir); + FILE* wf = fopen(tmp, "wb"); + if (wf) { + int ok = (fwrite(start, 1, keep_n, wf) == keep_n); + fclose(wf); + /* Only replace the live log if the tail was written + * in full — a short write must not destroy history. */ + if (ok) rename(tmp, path); else remove(tmp); + } + free(buf); + } + } + fclose(rf); + } + } + } return EL_INT(1); } @@ -16991,24 +17071,140 @@ void dharma_emit(el_val_t event_type, el_val_t payload) { free(b.buf); } -/* engram_boundary_beat(op_name) — the decorated-fn boundary AUTO-EMIT (VBD seam). - * codegen injects a single call to this at the entry of every @manager/@accessor - * decorated fn, so a decorated op self-reports with ZERO hand-written - * instrumentation in its body: +/* engram_boundary_beat(op_name, construct) — decorated-fn boundary AUTO-EMIT + * (VBD seam). codegen injects a single call to this at the entry of every + * @manager/@accessor decorated fn, so a decorated op self-reports with ZERO + * hand-written instrumentation in its body: * (1) afferent counter++ — the boundary was crossed * (2) engram_chrono_tick() — interoception: the mind senses its own op firing * (3) engram_strengthen(self-anchor)— reinforce the self-activity anchor (an * activation-count/salience bump, NOT a content/edge write — identity * write-protection is untouched) * (4) dharma_emit(neuron.op.) — provenance on the shared bus transport - * (same bus the swarm peers field on); bumps _eg_dharma_emits. */ -el_val_t engram_boundary_beat(el_val_t op_name) { + * (same bus the swarm peers field on); bumps _eg_dharma_emits. + * + * `construct` is the DECORATOR that caused the beat ("manager" / "accessor"), + * carried in the payload. Before it, the beat reported which fn crossed a + * boundary but never which construct put the beat there — so boundary events + * accumulated in the graph with no attribution, and no decorator could ever be + * measured. "Is this construct earning its keep" was an argument; with the + * attribution it is a traversal. The payload is built here rather than at the + * call site so the format has exactly one author. + * + * The construct name comes from a closed set codegen controls, so the + * unescaped snprintf below cannot be injected through. That is NOT true of + * dharma_emit generally — its payloads are hand-concatenated at 39 call sites + * with no escaping, and a value containing a quote silently corrupts the + * event. Fixing that is a separate change; this one does not add to it. */ +/* ── Runtime construct seam ─────────────────────────────────────────────── + * The crossing is resolved at EXECUTION, not at emission. Codegen emits one + * indirection per function; which constructs apply is read from a table that + * can be written AFTER the binary exists. + * + * This is the whole hypothesis under test: a compiler whose one compiled + * mechanism is language extension can compose without recompilation. If a + * construct declared after the build applies to a running program, the five + * compile-time declaration kinds were the wrong shape. + * + * Table format, one binding per line: + * entry|exit + * + * Targets are resolved with dlsym against the running image, so composition is + * bounded by the LINKED SYMBOL SET -- a construct naming a symbol nobody + * linked is skipped, not fatal. That bound is the honest limit on "endless". */ +#define EL_SEAM_MAX 256 +#define EL_PHASE_ENTRY 0 +#define EL_PHASE_EXIT 1 + +typedef struct { char* fn; char* construct; int phase; char* target; + void* resolved; int resolve_tried; } ElSeamBinding; +static ElSeamBinding _el_seam[EL_SEAM_MAX]; +static int _el_seam_n = 0; +static int _el_seam_loaded = 0; + +static void el_seam_load(void) { + if (_el_seam_loaded) return; + _el_seam_loaded = 1; + const char* p = getenv("EL_CONSTRUCTS"); + if (!p || !*p) return; + FILE* f = fopen(p, "r"); + if (!f) return; + char line[512]; + while (fgets(line, sizeof line, f) && _el_seam_n < EL_SEAM_MAX) { + char fn[128], con[128], ph[32], tgt[128]; + if (sscanf(line, "%127s %127s %31s %127s", fn, con, ph, tgt) == 4) { + if (fn[0] == '#') continue; + _el_seam[_el_seam_n].fn = el_strdup(fn); + _el_seam[_el_seam_n].construct = el_strdup(con); + _el_seam[_el_seam_n].phase = (strcmp(ph, "exit") == 0) ? EL_PHASE_EXIT : + (strcmp(ph, "wrap") == 0) ? 2 : EL_PHASE_ENTRY; + _el_seam[_el_seam_n].target = el_strdup(tgt); + _el_seam_n++; + } + } + fclose(f); +} + +/* el_seam_wrap — the seam calls the body itself, so a bound construct can + * control invocation: run it zero times, N times, or around a transaction. + * With no binding it is a direct call through the thunk, which is what the + * unwrapped code did anyway. */ +el_val_t el_seam_wrap(el_val_t fn_v, el_val_t (*body)(void*), void* env) { + if (!_el_seam_loaded) el_seam_load(); + if (_el_seam_n == 0) return body(env); + const char* fn = EL_CSTR(fn_v); + if (!fn) return body(env); + for (int i = 0; i < _el_seam_n; i++) { + if (_el_seam[i].phase != 2) continue; /* 2 = wrap */ + if (strcmp(_el_seam[i].fn, fn) != 0) continue; + if (!_el_seam[i].resolve_tried) { + _el_seam[i].resolved = dlsym(RTLD_DEFAULT, _el_seam[i].target); + _el_seam[i].resolve_tried = 1; + } + if (!_el_seam[i].resolved) continue; + el_val_t (*fp)(el_val_t, el_val_t, el_val_t (*)(void*), void*) = + (el_val_t (*)(el_val_t, el_val_t, el_val_t (*)(void*), void*))_el_seam[i].resolved; + return fp(fn_v, el_wrap_str(el_strdup(_el_seam[i].construct)), body, env); + } + return body(env); +} + +el_val_t el_seam_run(el_val_t fn_v, el_val_t phase_v, el_val_t result) { + if (!_el_seam_loaded) el_seam_load(); + if (_el_seam_n == 0) return result; /* the common path: no bindings */ + const char* fn = EL_CSTR(fn_v); + if (!fn) return result; + int phase = (int)phase_v; + el_val_t last = result; + for (int i = 0; i < _el_seam_n; i++) { + if (_el_seam[i].phase != phase) continue; + if (strcmp(_el_seam[i].fn, fn) != 0) continue; + /* Resolve ONCE. dlsym walks the dynamic symbol table on every call, and + * measured at 6.6x on a hot path with two bindings -- the table scan was + * never the cost. What is hot must stay resolved; this is the smallest + * form of the same thing salience does for memory. */ + if (!_el_seam[i].resolve_tried) { + _el_seam[i].resolved = dlsym(RTLD_DEFAULT, _el_seam[i].target); + _el_seam[i].resolve_tried = 1; + } + void* sym = _el_seam[i].resolved; + if (!sym) continue; /* unlinked target: skipped, not fatal */ + el_val_t (*fp)(el_val_t, el_val_t, el_val_t) = + (el_val_t (*)(el_val_t, el_val_t, el_val_t))sym; + last = fp(fn_v, el_wrap_str(el_strdup(_el_seam[i].construct)), last); + } + return last; +} + +el_val_t engram_boundary_beat(el_val_t op_name, el_val_t construct) { _eg_aff_boundary_ops++; engram_chrono_tick(); engram_strengthen(EL_STR("kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")); - const char* nm = EL_CSTR(op_name); if (!nm) nm = ""; + const char* nm = EL_CSTR(op_name); if (!nm) nm = ""; + const char* ct = EL_CSTR(construct); if (!ct) ct = ""; char ev[160]; snprintf(ev, sizeof ev, "neuron.op.%s", nm); - dharma_emit(el_wrap_str(el_strdup(ev)), EL_STR("")); + char pl[192]; snprintf(pl, sizeof pl, "{\"construct\":\"%s\"}", ct); + dharma_emit(el_wrap_str(el_strdup(ev)), el_wrap_str(el_strdup(pl))); return (el_val_t)0; } @@ -18096,8 +18292,11 @@ static int el_bin_lookup(const void* p, size_t* out_len) { /* Avoid reading off the front of a page on tiny pointers (e.g. NULs * passed in as int-cast values). 4096 is a safe lower bound on any * platform we target. */ - if ((uintptr_t)p < 4096) return 0; - const el_bin_hdr_t* hdr = (const el_bin_hdr_t*)((const char*)p - sizeof(el_bin_hdr_t)); + /* Reads BACKWARD, so the HEADER address is what must be validated -- and a + * 4096 floor alone let sha256_hex(50000) through to a SIGSEGV. */ + const char* hp = (const char*)p - sizeof(el_bin_hdr_t); + if (!looks_like_heap_obj((el_val_t)(uintptr_t)hp)) return 0; + const el_bin_hdr_t* hdr = (const el_bin_hdr_t*)hp; if (hdr->magic != EL_MAGIC_BIN) return 0; *out_len = hdr->length; return 1; @@ -18107,7 +18306,12 @@ static int el_bin_lookup(const void* p, size_t* out_len) { static size_t el_input_len(const char* s) { size_t n; if (el_bin_lookup(s, &n)) return n; - return s ? strlen(s) : 0; + /* The FALLBACK is the hazard, not the tagged lookup. A NULL check does not + * establish that a slot is a pointer: el_val_t carries integers too, so + * strlen() on `sha256_hex(50000)` walks address 50000. Guarding the tagged + * path alone left this untouched and the SIGSEGV unchanged -- measured. */ + if (!looks_like_heap_obj((el_val_t)(uintptr_t)s)) return 0; + return strlen(s); } /* ─── SHA-256 (Brad Conte / public domain) ──────────────────────────────── */ diff --git a/lang/runtime/el_runtime.h b/lang/runtime/el_runtime.h index 716e147..2492c68 100644 --- a/lang/runtime/el_runtime.h +++ b/lang/runtime/el_runtime.h @@ -887,7 +887,10 @@ el_val_t engram_age_field(el_val_t delta_ms); el_val_t engram_age_field_catchup(void); el_val_t engram_chrono_persist_tick(void); el_val_t engram_chrono_tick(void); -el_val_t engram_boundary_beat(el_val_t op_name); /* API-reshape decorator-seam auto-emit */ +el_val_t engram_boundary_beat(el_val_t op_name, el_val_t construct); +int el_tagged(el_val_t v, uint32_t magic); /* the gate: validate a slot BEFORE dereferencing it */ +el_val_t el_seam_run(el_val_t fn_name, el_val_t phase, el_val_t result); /* runtime construct seam */ +el_val_t el_seam_wrap(el_val_t fn_name, el_val_t (*body)(void*), void* env); /* runtime invocation control */ /* API-reshape decorator-seam auto-emit; construct = the decorator that caused the beat */ el_val_t engram_self_anchor_capture(void); el_val_t engram_self_drift_json(void); el_val_t engram_neighbors_json(el_val_t node_id, el_val_t max_depth, el_val_t direction); diff --git a/lang/spec/language.md b/lang/spec/language.md index df79455..ab23710 100644 --- a/lang/spec/language.md +++ b/lang/spec/language.md @@ -460,7 +460,8 @@ The `@` token followed by an identifier attaches a decorator to the next `FnDef` | Decorator | Structural effect | |---|---| | `@manager` | Permits calls to `dharma_emit` / `dharma_field`. Calling either from a non-`@manager` fn emits a `#error` into the generated C — a compile-time failure, not a lint. | -| `@manager`, `@accessor` | Codegen injects one call to `engram_boundary_beat()` at function entry. The decorated op self-reports (chrono tick, afferent counter, self-activity strengthen, dharma bus event) with **zero** hand-written instrumentation in its body. | +| `@manager`, `@accessor` | Codegen injects one call to `engram_boundary_beat(, )` at function entry, where `` is the decorator that caused the beat. The decorated op self-reports (chrono tick, afferent counter, self-activity strengthen, dharma bus event carrying `{"construct":"..."}`) with **zero** hand-written instrumentation in its body. Without the construct argument the graph accumulates boundary events with no attribution, so no construct can be measured. | +| `@decorator(kind, target)` | **Declares a construct.** The decorated `fn`'s name becomes a usable decorator whose meaning is `target`. Codegen reads the declaration; it does not know the construct. Adding a construct is a declaration in the program, not a compiler edit. Two kinds exist: `"injects_at_entry"` calls `target(, )` at entry, result discarded — this is what `@manager`/`@accessor` are, seeded as the compiled-in core. `"guards_at_entry"` calls `target(, )` at entry and a **non-zero return short-circuits the decorated fn and becomes its result**. Guards run before injections (a refused call must not report a crossing) and *every* guard on a fn runs, whereas the topmost injecting construct wins. The compiler knows nothing about authentication, rate limiting or validation: the program points the construct at its own function. | | `@route(path, method, …)` | Records a route into a generated dispatch table. | Decorators with no registered meaning are accepted and ignored. diff --git a/lang/tests/integration/annotation_query.sh b/lang/tests/integration/annotation_query.sh new file mode 100755 index 0000000..5e6cf70 --- /dev/null +++ b/lang/tests/integration/annotation_query.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# annotation_query.sh — a declared type must match what it annotates. +# +# El had annotations and no checking: the annotation fed dispatch and was never +# verified against the value, so a mismatch did not fail, it REINTERPRETED +# MEMORY. let x: Int = "hello" printed 4343631981 (a string pointer used as an +# integer); let s: String = 42 dereferenced address 42. +set -uo pipefail +ELC="${1:?usage: annotation_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } +cd "$LANG_DIR" + +printf 'fn main() { let x: Int = "hello" println("x") }\n' > "$W/a.el" +EL_RELATIONS_OUT="$W/r.txt" "$ELC" "$W/a.el" >/dev/null 2>&1 +out=$(./tools/check/annotations.sh "$W/r.txt" 2>&1); rc=$? +chk "Int annotated on a String literal is caught" "1" "$rc" +chk "and names the variable" "1" "$(echo "$out" | grep -c "'x' is declared Int")" + +printf 'fn main() { let s: String = 42 println(s) }\n' > "$W/b.el" +EL_RELATIONS_OUT="$W/r2.txt" "$ELC" "$W/b.el" >/dev/null 2>&1 +./tools/check/annotations.sh "$W/r2.txt" >/dev/null 2>&1 +chk "String annotated on an Int literal is caught" "1" "$?" + +printf 'fn main() { let n: Int = 42 let s: String = "ok" println(s + int_to_str(n)) }\n' > "$W/c.el" +EL_RELATIONS_OUT="$W/r3.txt" "$ELC" "$W/c.el" >/dev/null 2>&1 +./tools/check/annotations.sh "$W/r3.txt" >/dev/null 2>&1 +chk "correct annotations are clean" "0" "$?" + +EL_RELATIONS_OUT="$W/r4.txt" "$ELC" elc-cli.el >/dev/null 2>&1 +./tools/check/annotations.sh "$W/r4.txt" >/dev/null 2>&1 +chk "the compiler's own source is clean — no false positives" "0" "$?" +echo; echo " 5 assertions, $((5-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/arity_query.sh b/lang/tests/integration/arity_query.sh new file mode 100755 index 0000000..6a9e9b9 --- /dev/null +++ b/lang/tests/integration/arity_query.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Control for arity-from-header: the runtime declares its own surface, so the +# compiler does not carry a second copy of it. +set -uo pipefail +ELC="${1:?usage: arity_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } + +printf 'fn main() {\n println("a", "b")\n}\n' > "$W/bad.el" +EL_RELATIONS_OUT="$W/r1.txt" "$ELC" "$W/bad.el" >/dev/null 2>&1 +chk "the emitter does not adjudicate arity" "0" "$("$ELC" "$W/bad.el" 2>/dev/null | grep -c 'arity error')" +out=$("$LANG_DIR/tools/check/arity.sh" "$W/r1.txt" 2>&1); rc=$? +chk "a wrong-arity call is caught" "1" "$rc" +chk "the expected count is correct" "1" "$(echo "$out" | grep -c "takes 1 arguments, called with 2")" + +printf 'fn main() {\n println("a")\n}\n' > "$W/ok.el" +EL_RELATIONS_OUT="$W/r2.txt" "$ELC" "$W/ok.el" >/dev/null 2>&1 +"$LANG_DIR/tools/check/arity.sh" "$W/r2.txt" >/dev/null 2>&1 +chk "a correct call is clean" "0" "$?" + +# multi-line declarations must not parse as zero params +n=$("$LANG_DIR/tools/check/arity.sh" "$W/r2.txt" | grep -oE '[0-9]+ signatures') +chk "signatures parsed from the header" "503 signatures" "$n" + +echo; echo " 5 assertions, $((5-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/capability_query.sh b/lang/tests/integration/capability_query.sh new file mode 100755 index 0000000..01385f1 --- /dev/null +++ b/lang/tests/integration/capability_query.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# Control for capability-as-policy: the compiler records the program's kind and +# its call graph; the shipped policy file and the checker decide. +set -uo pipefail +ELC="${1:?usage: capability_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } + +cat > "$W/u.el" <<'EOF' +fn leaky() -> Int { + dharma_emit("x", "y") + return 1 +} +fn main() { println("ok") } +EOF +EL_RELATIONS_OUT="$W/r.txt" "$ELC" "$W/u.el" >/dev/null 2>&1 +chk "the emitter does not adjudicate" "0" "$("$ELC" "$W/u.el" 2>/dev/null | grep -c 'capability violation')" +"$LANG_DIR/tools/check/capabilities.sh" "$W/r.txt" > "$W/o.txt" 2>&1; rc=$? +chk "a utility calling a DHARMA primitive is caught" "1" "$rc" +chk "the offending fn is named" "1" "$(grep -c 'called from leaky' "$W/o.txt")" + +cat > "$W/c.el" <<'EOF' +fn quiet() -> Int { return 1 } +fn main() { println("ok") } +EOF +EL_RELATIONS_OUT="$W/r2.txt" "$ELC" "$W/c.el" >/dev/null 2>&1 +"$LANG_DIR/tools/check/capabilities.sh" "$W/r2.txt" >/dev/null 2>&1 +chk "a clean program exits 0" "0" "$?" + +# the policy is DATA: editing it changes enforcement, with no compiler rebuild +printf 'utility prohibits_within println\n' > "$W/policy.rel" +"$LANG_DIR/tools/check/capabilities.sh" "$W/r2.txt" "$W/policy.rel" >/dev/null 2>&1 +chk "editing the policy file changes enforcement, no rebuild" "1" "$?" + +echo; echo " 5 assertions, $((5-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/definitions_query.sh b/lang/tests/integration/definitions_query.sh new file mode 100755 index 0000000..e2fb523 --- /dev/null +++ b/lang/tests/integration/definitions_query.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Control for duplicate-definition detection. +# +# El has no namespacing: import is textual inlining, so two modules defining the +# same name emit two C functions into one translation unit. +set -uo pipefail +ELC="${1:?usage: definitions_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } + +printf 'fn helper() -> Int { return 1 }\n' > "$W/a.el" +printf 'fn helper() -> Int { return 2 }\n' > "$W/b.el" +printf 'import "a.el"\nimport "b.el"\nfn main() { println(int_to_str(helper())) }\n' > "$W/m.el" +EL_RELATIONS_OUT="$W/r.txt" "$ELC" "$W/m.el" >/dev/null 2>&1 +out=$("$LANG_DIR/tools/check/definitions.sh" "$W/r.txt" 2>&1); rc=$? +chk "a collision across modules is caught at El level" "1" "$rc" +chk "the colliding name is reported" "1" "$(echo "$out" | grep -c "'helper' is defined 2 times")" +chk "and the reason is given" "1" "$(echo "$out" | grep -c 'no namespacing')" +chk "both source FILES are named" "1" "$(echo "$out" | grep -c 'a.el:1')" +chk "with file-local line numbers, not combined ones" "1" "$(echo "$out" | grep -c 'b.el:1')" + +printf 'fn only_once() -> Int { return 1 }\nfn main() { println(int_to_str(only_once())) }\n' > "$W/c.el" +EL_RELATIONS_OUT="$W/r2.txt" "$ELC" "$W/c.el" >/dev/null 2>&1 +"$LANG_DIR/tools/check/definitions.sh" "$W/r2.txt" >/dev/null 2>&1 +chk "a clean program exits 0" "0" "$?" +echo; echo " 6 assertions, $((6-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/prohibition_query.sh b/lang/tests/integration/prohibition_query.sh new file mode 100755 index 0000000..924905f --- /dev/null +++ b/lang/tests/integration/prohibition_query.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +# Control for prohibition-as-query: the compiler records, the checker decides. +set -uo pipefail +ELC="${1:?usage: prohibition_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } + +cat > "$W/p.el" <<'EOF' +@decorator("prohibits_outside", "raw_sql") +fn repository() {} + +fn sneaky() -> Int { raw_sql("DROP") return 1 } + +@repository +fn allowed() -> Int { raw_sql("SELECT") return 2 } + +fn main() { println("ok") } +EOF +EL_RELATIONS_OUT="$W/rel.txt" "$ELC" "$W/p.el" >/dev/null 2>&1 +"$LANG_DIR/tools/check/prohibitions.sh" "$W/rel.txt" > "$W/out.txt" 2>&1; rc=$? +chk "a violation outside the boundary is caught" "1" "$rc" +chk "the offending fn is named" "1" "$(grep -c 'sneaky is not one' "$W/out.txt")" +chk "a call inside the boundary is NOT flagged" "0" "$(grep -c 'allowed is not one' "$W/out.txt")" + +cat > "$W/q.el" <<'EOF' +@decorator("prohibits_outside", "raw_sql") +fn repository() {} + +@repository +fn only_allowed() -> Int { raw_sql("SELECT") return 1 } + +fn main() { println("ok") } +EOF +EL_RELATIONS_OUT="$W/rel2.txt" "$ELC" "$W/q.el" >/dev/null 2>&1 +"$LANG_DIR/tools/check/prohibitions.sh" "$W/rel2.txt" >/dev/null 2>&1 +chk "a clean program exits 0" "0" "$?" +echo; echo " 4 assertions, $((4-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/seam_binding.sh b/lang/tests/integration/seam_binding.sh new file mode 100755 index 0000000..58e28c6 --- /dev/null +++ b/lang/tests/integration/seam_binding.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# seam_binding.sh — integration control for the runtime construct seam. +# +# The seam's whole claim is that a construct declared AFTER a binary exists +# applies to that already-built program. That cannot be checked by +# compile_capture, which only sees emitted text: it needs a built binary, a +# linked target, and an environment. Hence a harness rather than a unit test. +# +# usage: seam_binding.sh [lang-dir] +# exit 0 = all assertions held; non-zero = number of failures +set -uo pipefail +ELC="${1:?usage: seam_binding.sh [lang-dir]}" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT +FAILS=0 + +ok() { printf ' ok %s\n' "$1"; } +fail() { printf ' FAIL %s\n expected: %s\n actual: %s\n' "$1" "$2" "$3"; FAILS=$((FAILS+1)); } +check(){ [ "$2" = "$3" ] && ok "$1" || fail "$1" "$2" "$3"; } + +SRCS=$("$LANG_DIR/../scripts/el-runtime-sources.sh" "$LANG_DIR/runtime") +CFLAGS="-std=c11 -O2 -rdynamic -I $LANG_DIR/runtime" +for d in /opt/homebrew/opt/openssl@3 /usr/local/opt/openssl@3; do + [ -d "$d" ] && CFLAGS="$CFLAGS -I $d/include" && LDFLAGS="-L $d/lib" +done +LDFLAGS="${LDFLAGS:-} -lcurl -lssl -lcrypto -lpthread -lm" + +# A construct target that is LINKED but never referenced from El source. +cat > "$WORK/targets.c" <<'EOF' +#include +#include +typedef int64_t el_val_t; +el_val_t observe(el_val_t fn, el_val_t con, el_val_t r){ + printf("SEEN %s/%s\n", (const char*)(intptr_t)fn, (const char*)(intptr_t)con); + return r; /* zero = do not refuse */ +} +el_val_t double_result(el_val_t fn, el_val_t con, el_val_t r){ + (void)fn; (void)con; return r * 2; /* exit: replace the result */ +} +el_val_t refuse(el_val_t fn, el_val_t con, el_val_t r){ + (void)fn; (void)con; (void)r; return 42; /* non-zero = short-circuit */ +} +EOF + +# A program with NO construct anywhere in its source. +cat > "$WORK/prog.el" <<'EOF' +fn work() -> Int { + return 7 +} + +fn main() { + println(int_to_str(work())) +} +EOF + +"$ELC" "$WORK/prog.el" > "$WORK/prog.c" 2>/dev/null +cc $CFLAGS -o "$WORK/prog" "$WORK/prog.c" "$WORK/targets.c" $SRCS $LDFLAGS 2>/dev/null \ + || { echo " FAIL probe did not build"; exit 1; } + +check "unbound program is unaffected" \ + "7" "$(cd "$WORK" && ./prog 2>&1)" + +printf 'work audited entry observe\n' > "$WORK/observe.txt" +check "a construct declared AFTER the build applies" \ + "SEEN work/audited +7" "$(cd "$WORK" && EL_CONSTRUCTS=observe.txt ./prog 2>&1)" + +printf 'work denied entry refuse\n' > "$WORK/refuse.txt" +check "a construct declared after the build can REFUSE" \ + "42" "$(cd "$WORK" && EL_CONSTRUCTS=refuse.txt ./prog 2>&1)" + +printf 'work ghost entry no_such_symbol_anywhere\n' > "$WORK/ghost.txt" +check "an unlinked target is skipped, not fatal" \ + "7" "$(cd "$WORK" && EL_CONSTRUCTS=ghost.txt ./prog 2>&1)" + +printf 'other_fn x entry refuse\n' > "$WORK/other.txt" +check "a binding for a different fn does not fire" \ + "7" "$(cd "$WORK" && EL_CONSTRUCTS=other.txt ./prog 2>&1)" + +printf 'work a entry observe\nwork b entry observe\n' > "$WORK/two.txt" +check "two constructs compose on one crossing" \ + "SEEN work/a +SEEN work/b +7" "$(cd "$WORK" && EL_CONSTRUCTS=two.txt ./prog 2>&1)" + +cat >> "$WORK/targets.c" <<'TGT' +el_val_t thrice(el_val_t fn, el_val_t con, el_val_t (*b)(void*), void* e){ + (void)fn; (void)con; return b(e) + b(e) + b(e); /* wrap: invoke N times */ +} +TGT +printf 'work doubler exit double_result\n' > "$WORK/exit.txt" +check "an EXIT construct declared after the build replaces the result" \ + "14" "$(cd "$WORK" && EL_CONSTRUCTS=exit.txt ./prog 2>&1)" + +echo +echo " 7 assertions, $((7-FAILS)) passed, $FAILS failed" +exit $FAILS diff --git a/lang/tests/integration/tagged_gate.sh b/lang/tests/integration/tagged_gate.sh new file mode 100755 index 0000000..5445dcc --- /dev/null +++ b/lang/tests/integration/tagged_gate.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# tagged_gate.sh — a slot must be validated before it is dereferenced. +# +# el_val_t carries both integers and tagged heap pointers, so "is this a +# pointer" is undecidable without checking first. That check was a CONVENTION +# every author had to know rather than a GATE they had to pass through: +# geom_of, mfld_of call looks_like_heap_obj correct +# el_bin_lookup checked only a 4096 floor read 8 bytes backward +# el_input_len checked only for NULL strlen'd an integer +# sha256_hex(50000) therefore compiled clean and segfaulted (exit 139). +set -uo pipefail +ELC="${1:?usage: tagged_gate.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } +cd "$LANG_DIR" +SRCS=$(../scripts/el-runtime-sources.sh runtime) +CF="-std=c11 -O2 -I runtime"; for d in /opt/homebrew/opt/openssl@3 /usr/local/opt/openssl@3; do [ -d "$d" ] && CF="$CF -I $d/include" && LF="-L $d/lib"; done +LF="${LF:-} -lcurl -lssl -lcrypto -lpthread -lm" + +build(){ "$ELC" "$1" > "$W/t.c" 2>/dev/null && cc $CF -o "$W/t" "$W/t.c" $SRCS $LF 2>/dev/null; } + +printf 'fn main() { let h: String = sha256_hex(50000) println("got " + h) }\n' > "$W/a.el" +build "$W/a.el"; "$W/t" >"$W/o" 2>&1; chk "an integer where a string is expected does not crash" "0" "$?" +chk "and yields the empty-string hash, not memory" "1" "$(grep -c e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 "$W/o")" + +printf 'fn main() { let h: String = sha256_hex(-5) println("got " + h) }\n' > "$W/b.el" +build "$W/b.el"; "$W/t" >/dev/null 2>&1; chk "a NEGATIVE integer does not crash" "0" "$?" + +printf 'fn main() { println(sha256_hex("abc")) }\n' > "$W/c.el" +build "$W/c.el"; out=$("$W/t" 2>&1) +chk "a legitimate string still hashes correctly" "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" "$out" + +chk "the gate is exported, so siblings stop re-deriving it" "1" "$(grep -c 'int *el_tagged(el_val_t' runtime/el_runtime.h)" +echo; echo " 5 assertions, $((5-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/temporal_query.sh b/lang/tests/integration/temporal_query.sh new file mode 100755 index 0000000..0b88bf1 --- /dev/null +++ b/lang/tests/integration/temporal_query.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# Control for temporal adjudication as a query. +# +# The emitter records which illegal combination it saw and still emits a +# TIME_TYPE_ERROR placeholder -- it has to emit SOMETHING for an illegal +# expression. What moved out is the judgment and the wording. +set -uo pipefail +ELC="${1:?usage: temporal_query.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } +cd "$LANG_DIR" + +printf 'fn main() {\n let a: Instant = now()\n let b: Instant = now()\n let c: Instant = a + b\n println("x")\n}\n' > "$W/b.el" +EL_RELATIONS_OUT="$W/r.txt" "$ELC" "$W/b.el" >/dev/null 2>&1 +chk "the illegal combination is recorded, not judged, by the emitter" \ + "1" "$(grep -c 'temporal:instant_plus_instant' "$W/r.txt")" +chk "the emitter no longer authors the message" \ + "0" "$("$ELC" "$W/b.el" 2>/dev/null | grep -c 'is not allowed')" +chk "a placeholder is still emitted for the illegal expression" \ + "1" "$("$ELC" "$W/b.el" 2>/dev/null | grep -c TIME_TYPE_ERROR)" +out=$("./tools/check/temporal.sh" "$W/r.txt" 2>&1); rc=$? +chk "the query judges it" "1" "$rc" +chk "and explains why, from data" "1" "$(echo "$out" | grep -c 'a point plus a point is not a point')" + +printf 'fn main() {\n let a: Instant = now()\n let d: Duration = el_duration_from_nanos(1)\n let c: Instant = a + d\n println("x")\n}\n' > "$W/g.el" +EL_RELATIONS_OUT="$W/r2.txt" "$ELC" "$W/g.el" >/dev/null 2>&1 +"./tools/check/temporal.sh" "$W/r2.txt" >/dev/null 2>&1 +chk "a legal program exits 0" "0" "$?" +echo; echo " 6 assertions, $((6-F)) passed, $F failed"; exit $F diff --git a/lang/tests/integration/temporal_signatures.sh b/lang/tests/integration/temporal_signatures.sh new file mode 100755 index 0000000..7ec9b79 --- /dev/null +++ b/lang/tests/integration/temporal_signatures.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# Control for temporal signatures as data. +# +# Unlike the other checks this one is read BY the compiler, not after it: the +# El-level return type decides which runtime wrapper to emit, and that is +# dispatch, not adjudication. What moved out is the data. +set -uo pipefail +ELC="${1:?usage: temporal_signatures.sh }" +LANG_DIR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +W=$(mktemp -d); trap 'rm -rf "$W"' EXIT; F=0 +chk(){ [ "$2" = "$3" ] && printf ' ok %s\n' "$1" || { printf ' FAIL %s\n expected %s got %s\n' "$1" "$2" "$3"; F=$((F+1)); }; } +cd "$LANG_DIR" + +printf 'fn main() {\n let a = now()\n let b = el_duration_from_nanos(5)\n let c = a + b\n println("ok")\n}\n' > "$W/i.el" +chk "an inferred Instant + Duration dispatches to the typed wrapper" \ + "1" "$("$ELC" "$W/i.el" 2>/dev/null | grep -c el_instant_add_dur)" +chk "with no signature file, the type is unknown and it does not" \ + "0" "$(EL_SIGNATURES=/nonexistent "$ELC" "$W/i.el" 2>/dev/null | grep -c el_instant_add_dur)" + +printf 'fn main() {\n let a: Instant = now()\n let b: Instant = now()\n let c: Instant = a + b\n println("x")\n}\n' > "$W/b.el" +chk "Instant + Instant is still refused" \ + "1" "$("$ELC" "$W/b.el" 2>/dev/null | grep -c 'TIME_TYPE_ERROR: Instant + Instant')" + +printf 'fn main() {\n let a: Instant = now()\n let d: Duration = el_duration_from_nanos(1)\n let c: Instant = a + d\n println("x")\n}\n' > "$W/g.el" +chk "Instant + Duration is allowed" \ + "0" "$("$ELC" "$W/g.el" 2>/dev/null | grep -c TIME_TYPE_ERROR)" + +echo; echo " 4 assertions, $((4-F)) passed, $F failed"; exit $F diff --git a/lang/tests/native/test_compiler.el b/lang/tests/native/test_compiler.el index 84ad5c4..5f2c304 100644 --- a/lang/tests/native/test_compiler.el +++ b/lang/tests/native/test_compiler.el @@ -18,7 +18,9 @@ import "../../el-compiler/src/compiler.el" // ── Lexer helpers ───────────────────────────────────────────────────────────── fn tok_count(tokens: [Any]) -> Int { - native_list_len(tokens) / 2 + // A token is (kind, value, line). This helper carried its own copy of the + // stride, so it escaped a search scoped to the compiler sources. + native_list_len(tokens) / 3 } // ── Codegen helper: capture compile() stdout to a string ───────────────────── @@ -259,22 +261,28 @@ test "lex-multiline-source" { assert tok_kind(tokens, 0) == "Let", "first token is Let" } -test "lex-flat-stride-2-layout" { - // Verify that the flat stride-2 layout: token i has kind at index 2*i, value at 2*i+1 +test "lex-flat-stride-3-layout" { + // A token is (kind, value, line): token i has kind at 3*i, value at 3*i+1, + // line at 3*i+2. Before 2026-08-17 a token carried no position at all, so + // no diagnostic in El could name a place. let tokens: [Any] = lex("fn foo") - // tokens[0] = "Fn", tokens[1] = "fn", tokens[2] = "Ident", tokens[3] = "foo", ... let raw_len: Int = native_list_len(tokens) - assert raw_len == 6, "fn + foo + Eof = 3 tokens = 6 raw entries" - let kind0: String = native_list_get(tokens, 0) - let val0: String = native_list_get(tokens, 1) - let kind1: String = native_list_get(tokens, 2) - let val1: String = native_list_get(tokens, 3) - assert kind0 == "Fn", "raw[0] is Fn kind" - assert val0 == "fn", "raw[1] is fn value" - assert kind1 == "Ident", "raw[2] is Ident kind" - assert val1 == "foo", "raw[3] is foo value" + assert raw_len == 9, "fn + foo + Eof = 3 tokens = 9 raw entries" + assert native_list_get(tokens, 0) == "Fn", "raw[0] is the kind" + assert native_list_get(tokens, 1) == "fn", "raw[1] is the value" + assert native_list_get(tokens, 2) == "1", "raw[2] is the line" + assert native_list_get(tokens, 3) == "Ident", "raw[3] is the next kind" + assert native_list_get(tokens, 5) == "1", "still line 1" } +test "lexer-tracks-line-numbers" { + let tokens: [Any] = lex("fn a\nfn b\nfn c") + assert tok_line(tokens, 0) == "1", "first fn is on line 1" + assert tok_line(tokens, 2) == "2", "second fn is on line 2" + assert tok_line(tokens, 4) == "3", "third fn is on line 3" +} + + // ── Parser tests ────────────────────────────────────────────────────────────── fn get_first_stmt_kind(src: String) -> String { @@ -726,3 +734,264 @@ test "compiler-stdint-include" { let out: String = compile_capture(src) assert str_contains(out, "stdint.h"), "output includes stdint.h" } + +// ── Decorator seam: boundary-beat attribution ──────────────────────────────── +// +// The beat carries the CONSTRUCT that caused it, not only the fn that beat. +// Without the second argument the graph accumulates boundary events with no +// way to attribute them to the decorator responsible, so no construct can ever +// be measured and "is this decorator earning its keep" stays an argument +// instead of a query. + + + +test "decorator-undecorated-fn-has-no-beat" { + let src: String = "fn f() -> Int { return 1 }" + let out: String = compile_capture(src) + assert !str_contains(out, "engram_boundary_beat"), "an undecorated fn does not beat" +} + +// ── Decorator seam: the twelve inert names ─────────────────────────────────── +// +// PINS A KNOWN DEFECT. codegen calls fn_has_decorator for exactly three names +// (manager, accessor, route). Twelve others parse, attach as {name,args}, and +// compile to nothing — including four that look like protection: +// @authenticate (6 uses), @authorize (3), @rate_limit (3), @validate (2). +// +// This test asserts the CURRENT behaviour so that fixing it is a visible +// change rather than a silent one. When a pass wires or rejects these, this +// test flips and that flip is the proof. + +test "decorator-authenticate-compiles-to-nothing" { + let src: String = "@authenticate\nfn f() -> Int { return 1 }" + let out: String = compile_capture(src) + let bare: String = compile_capture("fn f() -> Int { return 1 }") + assert str_eq(out, bare), "KNOWN DEFECT: @authenticate emits identical C to no decorator at all" +} + +// ── Declared constructs ────────────────────────────────────────────────────── +// +// A construct declares its own meaning and codegen reads it. Adding a +// construct is a declaration in the program; it does not touch the compiler. + + +test "declared-construct-name-unknown-to-codegen" { + // The name is arbitrary. Nothing in the compiler mentions it. + let src: String = "@decorator(\"injects_at_entry\", \"engram_boundary_beat\")\nfn zzq_unlikely_name() {}\n@zzq_unlikely_name\nfn f() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "EL_STR(\"zzq_unlikely_name\")"), "an arbitrary construct name works" +} + +test "undeclared-construct-still-injects-nothing" { + let src: String = "@nobody_declared_this\nfn f() -> Int { return 1 }" + let out: String = compile_capture(src) + assert !str_contains(out, "engram_boundary_beat"), "an undeclared construct injects nothing" +} + + +// ── Declared constructs: guards ────────────────────────────────────────────── +// +// A guard is an injection that may refuse. Non-zero return short-circuits the +// decorated fn. This is what @authenticate/@authorize/@rate_limit/@validate +// needed and never had — fourteen applications that read as protection and +// emitted no instruction. + + + + +test "undeclared-guard-emits-nothing" { + let src: String = "@not_a_declared_guard\nfn handler() -> Int { return 7 }" + let out: String = compile_capture(src) + assert !str_contains(out, "if (__g)"), "an undeclared construct guards nothing" +} + +// ── Declared constructs: exit injection and composition ────────────────────── + + + + + +// ── Declared constructs: wraps and prohibitions ────────────────────────────── + + + + + +// ── Runtime seam ───────────────────────────────────────────────────────────── +// +// CONTROL for the finding that a crossing can be resolved at execution rather +// than at emission. Codegen emits one unconditional indirection per fn; which +// constructs apply is read from a table written after the binary exists. + +test "seam-indirection-emitted-on-every-fn" { + let src: String = "fn a() -> Int { return 1 }\nfn b() -> Int { return 2 }" + let out: String = compile_capture(src) + assert str_contains(out, "el_seam_run(EL_STR(\"a\"), 0, 0);"), "fn a carries the indirection" + assert str_contains(out, "el_seam_run(EL_STR(\"b\"), 0, 0);"), "fn b carries the indirection" +} + +test "seam-emitted-without-any-decorator" { + // The point of the seam: source need not mention a construct at all. + let src: String = "fn undecorated() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "el_seam_run"), "an undecorated fn is still bindable at runtime" + assert !str_contains(out, "engram_boundary_beat"), "and nothing is inlined for it" +} + + +// ── Runtime seam: what replaced the compile-time entry mechanism ───────────── +// +// Entry injection and refusal moved from emission to execution. These assert +// the emitted shape; the BEHAVIOUR — that a construct declared after the build +// applies, refuses, composes, and that an unlinked target is skipped — is +// covered by tests/integration/seam_binding.sh, which needs a built binary and +// an environment and therefore cannot be a compile_capture test. + +test "seam-replaces-inlined-entry-injection" { + let src: String = "@manager\nfn m() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "el_seam_run(EL_STR(\"m\")"), "the crossing goes through the seam" + assert !str_contains(out, "engram_boundary_beat(EL_STR(\"m\")"), "nothing is inlined at the crossing any more" +} + +test "seam-entry-is-refusable" { + let src: String = "fn f() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "if (__s) return __s;"), "a bound construct can short-circuit the fn" +} + +test "seam-is-emitted-for-undecorated-fns" { + let src: String = "fn plain() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "el_seam_run(EL_STR(\"plain\")"), "any fn is bindable later, decorated or not" +} + + +// ── Exit crossings resolve at runtime too ──────────────────────────────────── +// +// The wrapper is now UNCONDITIONAL. It has to be: early returns must route +// through something for an exit construct to see them, and codegen cannot know +// which fns will be bound after the binary exists. Measured cost of always +// emitting it: 0.37s -> 0.38s across ten self-compiles. + +test "every-fn-gets-a-body-helper-and-wrapper" { + let src: String = "fn plain(k: Int) -> Int { if k > 0 { return 1 } return 2 }" + let out: String = compile_capture(src) + assert str_contains(out, "static el_val_t __el_body_plain"), "the body is a helper" + assert str_contains(out, "el_val_t plain(el_val_t k) {"), "the visible fn is a wrapper" +} + +test "exit-crossing-goes-through-the-seam" { + let src: String = "fn f() -> Int { return 1 }" + let out: String = compile_capture(src) + assert str_contains(out, "__r = el_seam_run(EL_STR(\"f\"), 1, __r);"), "the exit crossing is resolved at execution and may replace the result" +} + +test "early-returns-route-through-the-exit-seam" { + let src: String = "fn early(k: Int) -> Int { if k > 0 { return 99 } return 1 }" + let out: String = compile_capture(src) + let helper: Int = str_index_of(out, "__el_body_early") + let seam: Int = str_index_of(out, "el_seam_run(EL_STR(\"early\"), 1") + assert helper < seam, "the early return is inside the helper, so it passes through the exit seam" +} + + +// ── Invocation control resolves at runtime ─────────────────────────────────── +// +// Every fn gets an env struct and a thunk, because codegen cannot know which +// fns a wrap construct will be bound to after the binary exists. That the bound +// construct can invoke the body zero or N times is behaviour, so it lives in +// tests/integration/seam_binding.sh. + +test "every-fn-gets-a-closure" { + let src: String = "fn f(k: Int) -> Int { return k }" + let out: String = compile_capture(src) + assert str_contains(out, "struct __env_f { el_val_t k; };"), "captured environment" + assert str_contains(out, "static el_val_t __thunk_f(void* __v)"), "thunk over that environment" + assert str_contains(out, "el_seam_wrap(EL_STR(\"f\"), __thunk_f, &__env)"), "invocation goes through the seam" +} + +test "zero-param-fn-emits-valid-c" { + // An empty struct is a GNU extension and an empty initialiser is C23. + let src: String = "fn noargs() -> Int { return 3 }" + let out: String = compile_capture(src) + assert str_contains(out, "struct __env_noargs { char __e0; };"), "zero-param env has a field" + assert !str_contains(out, "__env = { }"), "and no empty initialiser" +} + + +// ── Prohibition is a query, not an emission ───────────────────────────────── +// +// The compiler records what it saw -- who calls what, who carries what, who +// prohibits what. Whether that is legal is decided by tools/check/prohibitions.sh +// against the emitted relations, at build time. An emitter that also adjudicates +// has to contain every rule anyone will ever want. + +test "compiler-no-longer-emits-prohibition-errors" { + let src: String = "@decorator(\"prohibits_outside\", \"raw_sql\")\nfn repository() {}\nfn sneaky() -> Int { raw_sql(\"DROP\") return 1 }" + let out: String = compile_capture(src) + assert !str_contains(out, "boundary violation"), "the emitter does not adjudicate" +} + +// ── Int return types drive + dispatch ──────────────────────────────────────── +// +// El has one type, so `a + b` must be dispatched from what the operands ARE. +// The 35 Int-returning builtins moved to signatures.rel; the dispatch stayed, +// because choosing between arithmetic and concatenation is emission. + +test "int-returning-builtin-drives-arithmetic-dispatch" { + let src: String = "fn main() { let a = str_len(\"hello\") let b = str_len(\"hi\") let c = a + b println(int_to_str(c)) }" + let out: String = compile_capture(src) + assert str_contains(out, "(a + b)"), "Int + Int is arithmetic" + assert !str_contains(out, "el_str_concat(a, b)"), "and NOT concatenation" +} + +test "string-plus-string-still-concatenates" { + let src: String = "fn main() { let s = \"a\" + \"b\" println(s) }" + let out: String = compile_capture(src) + assert str_contains(out, "el_str_concat"), "String + String still concatenates" +} + +// ── Reserved words that reserved nothing ───────────────────────────────────── +// +// sealed, activate, seed, protocol and impl were keywords in the lexer and were +// consumed by no parser or codegen path. Each stole an identifier from users +// for nothing, and using one silently miscompiled: `let seed = 42` compiled +// clean and produced the wrong value with no diagnostic at any layer. + +test "freed-identifiers-compile-as-identifiers" { + let src: String = "fn main() { let seed = 42 let impl = seed + 1 println(int_to_str(impl)) }" + let out: String = compile_capture(src) + assert str_contains(out, "el_val_t seed"), "seed is an identifier" + assert str_contains(out, "el_val_t impl"), "impl is an identifier" + assert str_contains(out, "(seed + 1)"), "and arithmetic on them dispatches correctly" +} + +test "test-keyword-is-still-reserved" { + // `test` LOOKED inert by the same measure and is not: codegen consumes it + // for --test mode, 408 uses in the tree. Measuring only parser.el would + // have removed it. + let src: String = "fn main() { println(\"x\") }" + let out: String = compile_capture(src) + assert str_contains(out, "int main"), "the suite still compiles, which requires test to remain a keyword" +} + +// ── A bare literal is a magnitude with no axis ─────────────────────────────── +// +// Duration + Int was already refused because an Int carries no unit. Adding one +// to a POINT is worse: it moves the instant by an unspecified amount. The +// asymmetry had no justification; it was simply never written. + +test "instant-plus-bare-int-is-refused" { + let src: String = "fn main() { let t: Instant = now() let u: Instant = t + 3 println(\"x\") }" + let out: String = compile_capture(src) + assert str_contains(out, "TIME_TYPE_ERROR: Instant + Int"), "3 of what?" +} + +test "instant-plus-unit-suffix-is-allowed" { + // .hour supplies the axis, so the magnitude becomes a displacement. + let src: String = "fn main() { let t: Instant = now() let u: Instant = t + 1.hour println(\"x\") }" + let out: String = compile_capture(src) + assert str_contains(out, "el_instant_add_dur"), "a unit suffix makes it a Duration" + assert !str_contains(out, "TIME_TYPE_ERROR"), "and the addition is legal" +} diff --git a/lang/tools/check/annotations.sh b/lang/tools/check/annotations.sh new file mode 100755 index 0000000..34e7ce5 --- /dev/null +++ b/lang/tools/check/annotations.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# annotations.sh — verify that a declared type matches what it annotates. +# +# El had annotations and no checking. The annotation fed dispatch (deciding +# whether `a + b` is arithmetic or concatenation) and was never verified against +# the value, so a mismatch did not fail -- it reinterpreted memory: +# +# let x: Int = "hello" a string pointer used as an integer +# let s: String = 42 address 42 dereferenced as a string +# +# The second is an arbitrary-read primitive if the integer is influenced. +set -uo pipefail +REL="${1:?usage: annotations.sh }" +[ -f "$REL" ] || exit 0 +locate() { + awk -v L="$1" '$2=="spans" && $3<=L && $4>=L {printf "%s:%d", $1, L-$3+1; exit}' "$REL" 2>/dev/null +} +V=0 +while read -r caller _ rest; do + [ "${rest#typemismatch:}" = "$rest" ] && continue + body="${rest#typemismatch:}" + declared="${body%%:*}"; body="${body#*:}" + actual="${body%%:*}"; var="${body#*:}" + printf "type error in %s: '%s' is declared %s but assigned a %s literal — the annotation drives dispatch, so the value will be reinterpreted rather than rejected\n" \ + "$caller" "$var" "$declared" "$actual" + V=$((V+1)) +done < <(sort -u "$REL") +[ "$V" -eq 0 ] && echo "annotations: clean" +exit "$V" diff --git a/lang/tools/check/arity.sh b/lang/tools/check/arity.sh new file mode 100755 index 0000000..c0d7a75 --- /dev/null +++ b/lang/tools/check/arity.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# arity.sh — check call arity against the runtime's OWN declarations. +# +# codegen.el carried builtin_arity(): 344 lines, 300 entries, of which 243 were +# an exact duplicate of el_runtime.h. Measured drift between them was zero -- +# the duplicate had been maintained correctly -- but 199 functions the runtime +# declares had NO entry, so calling them with the wrong argument count produced +# no El-level diagnostic at all. The table was not wrong, it was 40% incomplete. +# +# Deriving from the header fixes the coverage and makes drift impossible. +set -uo pipefail +REL="${1:?usage: arity.sh [runtime-header]}" +HDR="${2:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/runtime/el_runtime.h}" +[ -f "$REL" ] || exit 0 +[ -f "$HDR" ] || { echo "no header: $HDR" >&2; exit 0; } + +SIG=$(mktemp); trap 'rm -f "$SIG"' EXIT +# Declarations may span lines, so join continuations before parsing. Reading +# only the first line silently yields 0 params, and a checker that reports the +# wrong expected count is worse than no checker at all. +sed 's://.*::' "$HDR" | tr '\n' ' ' | sed 's:/\*[^*]*\*/: :g; s/;/;\n/g' | awk ' + /el_val_t[[:space:]]+[a-z0-9_]+[[:space:]]*\(/ { + line=$0 + match(line, /el_val_t[[:space:]]+[a-z0-9_]+/); name=substr(line,RSTART,RLENGTH) + sub(/el_val_t[[:space:]]+/,"",name) + match(line, /\(.*\)/); params=substr(line,RSTART+1,RLENGTH-2) + gsub(/^[[:space:]]+|[[:space:]]+$/,"",params) + if (params=="void" || params=="") n=0 + else { n=1; for(i=1;i<=length(params);i++) if(substr(params,i,1)==",") n++ } + if (line ~ /\.\.\./) n=-1 + print name, n + }' | sort -u > "$SIG" + +V=0 +while read -r callee _ rest; do + [ "${rest#arity:}" = "$rest" ] && continue + actual="${rest#arity:}" + expected=$(awk -v n="$callee" '$1==n {print $2; exit}' "$SIG") + # 60 of 500 runtime decls carry a __ prefix: El's `println` is C's + # `__println`. codegen owns that mapping and its table carried BOTH keys. + # One rule covers every one of them. + [ -n "$expected" ] || expected=$(awk -v n="__$callee" '$1==n {print $2; exit}' "$SIG") + [ -n "$expected" ] || continue # not a runtime builtin + [ "$expected" = "-1" ] && continue # variadic + if [ "$actual" != "$expected" ]; then + printf "arity error: '%s' takes %s arguments, called with %s\n" "$callee" "$expected" "$actual" + V=$((V+1)) + fi +done < <(sort -u "$REL") + +[ "$V" -eq 0 ] && echo "arity: clean ($(wc -l < "$SIG" | tr -d ' ') signatures from the header)" +exit "$V" diff --git a/lang/tools/check/capabilities.rel b/lang/tools/check/capabilities.rel new file mode 100644 index 0000000..f2bab0a --- /dev/null +++ b/lang/tools/check/capabilities.rel @@ -0,0 +1,18 @@ +# capabilities.rel — the capability policy, as shipped data. +# +# A program's tier bounds what it may call. This is policy that comes from +# OUTSIDE the program: a utility cannot be trusted to declare its own +# restrictions, because it would declare none. So unlike prohibits_outside, +# which a program declares about itself, this ships with the language and is +# editable without a compiler release. +# +# Previously: four functions and eighteen string literals inside codegen.el. +# +# prohibits_within + +service prohibits_within llm_call_agentic,llm_register_tool,dharma_emit,dharma_field + +utility prohibits_within dharma_connect,dharma_send,dharma_activate,dharma_emit,dharma_field,dharma_strengthen,dharma_relationship,dharma_peers +utility prohibits_within llm_call,llm_call_system,llm_call_agentic,llm_vision,llm_register_tool,llm_models + +# cgi is unrestricted: self-formation is what a cgi program is for. diff --git a/lang/tools/check/capabilities.sh b/lang/tools/check/capabilities.sh new file mode 100755 index 0000000..e91d31f --- /dev/null +++ b/lang/tools/check/capabilities.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# capabilities.sh — enforce the capability tier as a QUERY over emitted +# relations plus a shipped policy file. The compiler records the program's kind +# and its call graph; deciding what that tier may call is not an emitter's job. +set -uo pipefail +REL="${1:?usage: capabilities.sh [policy]}" +POLICY="${2:-$(dirname "${BASH_SOURCE[0]}")/capabilities.rel}" +[ -f "$REL" ] || exit 0 +KIND=$(grep -m1 '^program calls is_kind:' "$REL" | sed 's/.*is_kind://') +[ -n "$KIND" ] || KIND=utility +V=0 +while read -r kind rel names; do + [ "$kind" = "$KIND" ] && [ "$rel" = "prohibits_within" ] || continue + IFS=',' read -ra NAMES <<< "$names" + for n in "${NAMES[@]}"; do + while read -r caller _ callee; do + [ "$callee" = "$n" ] || continue + printf "capability violation: '%s' programs may not call '%s' (called from %s)\n" "$KIND" "$n" "$caller" + V=$((V+1)) + done < <(sort -u "$REL") + done +done < <(grep -v '^#' "$POLICY" | grep -v '^[[:space:]]*$') +[ "$V" -eq 0 ] && echo "capabilities: clean ($KIND)" +exit "$V" diff --git a/lang/tools/check/definitions.sh b/lang/tools/check/definitions.sh new file mode 100755 index 0000000..64f6834 --- /dev/null +++ b/lang/tools/check/definitions.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# definitions.sh — catch duplicate top-level definitions, and name the files. +# +# El has no namespacing. `import` is textual inlining, so two modules defining +# the same name emit two C functions into one translation unit. cc catches it, +# but reports the generated helpers (__el_body_f, __env_f, __thunk_f) before the +# user's own function, so the first three errors name symbols nobody wrote. +# +# Naming the FILES needed provenance threaded end to end: tokens had no line +# numbers at all, so no diagnostic in El could name a place. Now a token is +# (kind, value, line), FnDef carries its line, and resolve_imports publishes +# which line range of the combined source came from which file. +# +# LIMIT: a nested import returns one string, so a definition inside a +# transitively imported file is attributed to the direct import. +set -uo pipefail +REL="${1:?usage: definitions.sh }" +[ -f "$REL" ] || exit 0 + +# line in the COMBINED source -> "file:line-within-that-file". Reporting the +# combined line against a filename would point at a line that file does not +# have, which is worse than reporting no line at all. +locate() { + awk -v L="$1" '$2=="spans" && $3<=L && $4>=L {printf "%s:%d", $1, L-$3+1; found=1; exit} + END{ if(!found) printf "" }' "$REL" +} + +V=0 +while read -r name; do + lines=$(grep -E "^$name calls defines_at:" "$REL" | sed 's/.*defines_at://' | sort -un) + n=$(echo "$lines" | wc -l | tr -d ' ') + [ "$n" -gt 1 ] || continue + printf "duplicate definition: '%s' is defined %s times — El has no namespacing, so imported modules share one global scope\n" "$name" "$n" + for l in $lines; do + loc=$(locate "$l") + [ -n "$loc" ] && printf " %s\n" "$loc" || printf " combined line %s\n" "$l" + done + V=$((V+1)) +done < <(grep ' calls defines_at:' "$REL" | awk '{print $1}' | sort -u) + +[ "$V" -eq 0 ] && echo "definitions: clean" +exit "$V" diff --git a/lang/tools/check/prohibitions.sh b/lang/tools/check/prohibitions.sh new file mode 100755 index 0000000..0c61c25 --- /dev/null +++ b/lang/tools/check/prohibitions.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +# prohibitions.sh — enforce boundary prohibitions as a QUERY over relations the +# compiler emitted, rather than as a rule the compiler contains. +# +# A prohibition is a containment relation over the call graph: "these calls may +# appear only inside a fn carrying construct C". The compiler's job is to say +# what it saw — who calls what, who carries what, who prohibits what. Deciding +# whether that is legal is a query, and a query does not belong in an emitter. +# +# Detection still happens at BUILD time. What moved is where the rule and the +# checker live, which is what "a #error has no runtime" was hiding. +# +# usage: prohibitions.sh +# exit 0 = clean; exit N = N violations +set -uo pipefail +REL="${1:?usage: prohibitions.sh }" +[ -f "$REL" ] || { echo "no relations file: $REL" >&2; exit 0; } + +V=0 +# construct -> prohibited names +while read -r construct _ rest; do + [ "${rest#prohibits:}" = "$rest" ] && continue + names="${rest#prohibits:}" + IFS=',' read -ra NAMES <<< "$names" + for n in "${NAMES[@]}"; do + # every fn that calls a prohibited name + while read -r caller _ callee; do + [ "$callee" = "$n" ] || continue + # ...must carry the owning construct + if ! grep -qx "$caller calls @$construct" "$REL"; then + printf 'boundary violation: %s may only be called from an @%s fn, but %s is not one\n' \ + "$n" "$construct" "$caller" + V=$((V+1)) + fi + done < <(sort -u "$REL") + done +done < <(sort -u "$REL") + +[ "$V" -eq 0 ] && echo "prohibitions: clean" +exit "$V" diff --git a/lang/tools/check/signatures.rel b/lang/tools/check/signatures.rel new file mode 100644 index 0000000..e7a7a01 --- /dev/null +++ b/lang/tools/check/signatures.rel @@ -0,0 +1,73 @@ +# signatures.rel — El-level return types for runtime builtins. +# +# el_runtime.h declares every builtin as returning el_val_t, because El has ONE +# type. That single type is why the whole seam is cheap, and it is also why the +# header cannot say that now() returns an Instant while unix_seconds() returns +# an Int. The El-level type is real and the C boundary erases it. +# +# So the compiler needs this, and unlike the other checks it needs it at +# EMISSION time: Instant + Duration must become el_instant_add_dur, and that is +# dispatch, not adjudication. What moved here is the DATA -- previously 19 +# hardcoded names across two functions in codegen.el. What stays in the emitter +# is choosing which call to emit, which is an emitter's actual job. +# +# returns + +now returns Instant +el_now_instant returns Instant +unix_seconds returns Instant +unix_millis returns Instant +instant_from_iso8601 returns Instant +el_instant_add_dur returns Instant +el_instant_sub_dur returns Instant + +el_duration_from_nanos returns Duration +duration_seconds returns Duration +duration_millis returns Duration +duration_nanos returns Duration +el_instant_diff returns Duration +el_duration_add returns Duration +el_duration_sub returns Duration +el_duration_scale returns Duration +el_duration_div returns Duration +ttl_cache_age returns Duration + +# Int-returning builtins. Previously 35 hardcoded names in is_int_call(). +# These decide whether `a + b` is arithmetic or concatenation, so the +# compiler reads them at emission time -- dispatch, not adjudication. + +str_len returns Int +str_index_of returns Int +str_to_int returns Int +str_char_code returns Int +str_count returns Int +str_count_chars returns Int +str_count_bytes returns Int +str_count_lines returns Int +str_count_words returns Int +str_count_letters returns Int +str_count_digits returns Int +str_last_index_of returns Int +str_find_chars returns Int +native_list_len returns Int +el_list_len returns Int +len returns Int +json_get_int returns Int +json_array_len returns Int +engram_node_count returns Int +engram_edge_count returns Int +time_now returns Int +time_now_utc returns Int +time_diff returns Int +time_add returns Int +time_from_parts returns Int +el_abs returns Int +el_max returns Int +el_min returns Int +float_to_int returns Int +unix_timestamp returns Int +instant_to_unix_seconds returns Int +instant_to_unix_millis returns Int +duration_to_seconds returns Int +duration_to_millis returns Int +duration_to_nanos returns Int diff --git a/lang/tools/check/temporal.rel b/lang/tools/check/temporal.rel new file mode 100644 index 0000000..50d49c9 --- /dev/null +++ b/lang/tools/check/temporal.rel @@ -0,0 +1,19 @@ +# temporal.rel — the affine algebra of time, as data. +# +# An Instant is a POINT and a Duration is a DISPLACEMENT. Every rule below +# follows from that, and the set is closed because there are only two kinds of +# thing: you may add a displacement to a point, subtract two points to get a +# displacement, and combine displacements. Nothing else is meaningful. +# +# The emitter records which illegal combination it saw; this file says what that +# means and how to say it. +# +# means + +instant_plus_instant means Instant + Instant is not allowed — a point plus a point is not a point. Subtract them for a Duration, or add a Duration. +instant_plus_int means Instant + Int is not allowed — a bare literal is a magnitude with no axis. 3 of what? Adding it to a point moves the instant by an unspecified amount. Use a Duration. +duration_plus_int means Duration + Int is not allowed — an Int carries no unit. Use duration_seconds(n) or N.seconds. +duration_minus_int means Duration - Int is not allowed — an Int carries no unit. +instant_cmp_duration means Instant < Duration is not allowed — a point and a displacement are not on the same scale. +duration_cmp_instant means Duration < Instant is not allowed — a displacement and a point are not on the same scale. +caltime_plus_caltime means CalendarTime + CalendarTime is not allowed — a CalendarTime already projects an Instant under a Calendar. Use cal_to_instant first. diff --git a/lang/tools/check/temporal.sh b/lang/tools/check/temporal.sh new file mode 100755 index 0000000..8452c8d --- /dev/null +++ b/lang/tools/check/temporal.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# temporal.sh — report temporal type violations from emitted relations. +# +# The emitter still has to emit SOMETHING for an illegal expression, so the +# TIME_TYPE_ERROR placeholder stays in the generated C. What moved out is the +# judgment and the wording: codegen records "temporal:instant_plus_instant" and +# this decides what that means. +set -uo pipefail +REL="${1:?usage: temporal.sh [rules]}" +RULES="${2:-$(dirname "${BASH_SOURCE[0]}")/temporal.rel}" +[ -f "$REL" ] || exit 0 +V=0 +while read -r caller _ rest; do + [ "${rest#temporal:}" = "$rest" ] && continue + kind="${rest#temporal:}" + msg=$(awk -v k="$kind" '$1==k && $2=="means" {sub(/^[^ ]+[ ]+means[ ]+/,""); print; exit}' "$RULES") + [ -n "$msg" ] || msg="$kind" + printf 'temporal type error in %s: %s\n' "$caller" "$msg" + V=$((V+1)) +done < <(sort -u "$REL") +[ "$V" -eq 0 ] && echo "temporal: clean" +exit "$V"