remove deprecated rust artifacts and elvm bytecode spec

The Rust bootstrap was archived in 4f3543b and removed from the working
tree in e7a49eb. The bytecode tier was retired in 9fca4dc. What remained
on disk was leftover platform binaries (dist/platform/el-macos-universal,
el-windows-x86_64.exe) that nothing should be invoking, the elvm.md spec
for the retired bytecode tier, and the 8.7GB target/ build cache that
was tracked despite being in .gitignore.

Untracks target/, removes the platform binaries and elvm.md, and updates
spec/language.md so its self-hosting section no longer references the
genesis Rust path. The canonical toolchain is dist/platform/elc against
el-compiler/runtime/el_runtime.{h,c} — one compiler, one runtime, one
language.
This commit is contained in:
Will Anderson
2026-04-30 14:57:59 -05:00
parent 5adc05aa48
commit 04ecd1aafe
5536 changed files with 1 additions and 2274 deletions
-427
View File
@@ -1,427 +0,0 @@
# ELVM — El Virtual Machine Specification
Version 1.0.0 — April 29, 2026
---
## Overview
ELVM is the execution substrate for El programs. It is a stack-based virtual machine that executes JSON-serialized bytecode instructions stored in a typed binary container format. ELVM has four defining properties:
1. **Binary container with semantic header.** The `.elc` file format begins with a 4-byte magic identifier (`ELVM`), a 4-byte version field, and an 8-byte payload length — making every artifact self-describing and parseable without external schema.
2. **JSON-serialized instruction encoding.** The bytecode payload is a JSON array of instruction objects. Each instruction is human-inspectable with standard JSON tools, language-portable, and schema-free. The JSON encoding is the canonical on-disk format; no separate binary encoding exists.
3. **Builtin dispatch architecture.** Function calls resolve first against a runtime builtin dispatch table, then against the user-defined function table. New builtins are added to the dispatch table without recompiling existing programs. The execution model treats builtins and user functions identically from the instruction side.
4. **Self-hosting bootstrap.** A Rust genesis compiler produces ELVM bytecode. That bytecode, when executed by ELVM, runs the El compiler written in El, which produces ELVM bytecode for all subsequent compilation. The VM is the only persistent execution engine.
---
## 1. Binary Container Format
### 1.1 Header
Every `.elc` file begins with a 16-byte header:
```
Offset Size Type Field
────── ───── ──────────────── ──────────────────────────────────────────
0 4 [u8; 4] Magic: b"ELVM"
4 4 u32 little-endian Version: currently 1
8 8 u64 little-endian Payload length in bytes
```
### 1.2 Payload
Immediately following the 16-byte header is the payload: a UTF-8 JSON array encoding a `Vec<Bytecode>` instruction sequence.
```
[instruction, instruction, ...]
```
The payload length field in the header equals the exact byte count of the JSON payload. Bytes beyond `header_size + payload_length` are ignored (reserved for future trailer use).
### 1.3 Legacy Format
Files without the `ELVM` magic header are treated as raw JSON bytecode (no header). This is the legacy format produced by early compiler versions. The VM accepts both formats via the `deserialize_all` entry point.
### 1.4 Identification
The magic bytes `ELVM` (hex: `45 4C 56 4D`) uniquely identify El bytecode containers. Any tool that encounters these bytes at offset 0 may assume the 16-byte header format described above.
---
## 2. Value Types
The VM stack holds `Value` instances of the following types:
| Variant | Rust encoding | Description |
|---------|--------------|-------------|
| `Int(i64)` | 64-bit signed integer | All integer arithmetic |
| `Float(f64)` | IEEE 754 double | Floating-point arithmetic |
| `Str(String)` | UTF-8 string | Text and identifiers |
| `Bool(bool)` | Boolean | Logical conditions |
| `Nil` | Unit | Absent value, uninitialized |
| `List(Vec<Value>)` | Ordered sequence | Arrays, activation results |
| `Map(Vec<(String, Value)>)` | Ordered key-value pairs | Struct instances, JSON objects |
| `ResultOk(Box<Value>)` | Success variant | Fallible operation success |
| `ResultErr(Box<Value>)` | Error variant | Fallible operation failure |
| `Struct { type_name, fields }` | Named struct | Typed record with field list |
Maps use `Vec<(String, Value)>` rather than a hash map to preserve insertion order and remain serialization-friendly.
---
## 3. Instruction Set
### 3.1 JSON Encoding
Each instruction serializes as either a JSON string (no-argument instructions) or a JSON object (instructions with arguments):
```json
"Halt"
"Pop"
{"Push": {"Int": 42}}
{"Push": {"Str": "hello"}}
{"Push": "Nil"}
{"LoadLocal": "x"}
{"StoreLocal": "x"}
{"Call": {"name": "println", "arity": 1}}
{"Jump": 3}
{"JumpIfNot": -2}
```
### 3.2 Stack Instructions
| Instruction | JSON | Stack effect | Description |
|-------------|------|-------------|-------------|
| `Push(v)` | `{"Push": <value>}` | `-- v` | Push constant value |
| `Pop` | `"Pop"` | `v --` | Discard top of stack |
| `Dup` | `"Dup"` | `v -- v v` | Duplicate top of stack |
### 3.3 Arithmetic Instructions
All arithmetic instructions pop two operands and push the result. For binary operations, operands are popped as `(b, a)``b` is popped first (top of stack), `a` is popped second. The operation computes `a OP b`.
| Instruction | JSON | Description |
|-------------|------|-------------|
| `Add` | `"Add"` | `a + b` (Int/Float/Str concatenation) |
| `Sub` | `"Sub"` | `a - b` |
| `Mul` | `"Mul"` | `a * b` |
| `Div` | `"Div"` | `a / b` (Int: integer division; division by zero returns Nil) |
| `Mod` | `"Mod"` | `a % b` |
| `BitAnd` | `"BitAnd"` | `a & b` (Int only) |
| `BitOr` | `"BitOr"` | `a \| b` (Int only) |
| `BitXor` | `"BitXor"` | `a ^ b` (Int only) |
| `BitNot` | `"BitNot"` | `~a` (Int only, unary) |
| `Shl` | `"Shl"` | `a << b` (Int only) |
| `Shr` | `"Shr"` | `a >> b` (Int only) |
**Type promotion:** When one operand is `Int` and the other is `Float`, the `Int` is widened to `Float` before the operation.
### 3.4 Comparison Instructions
Each pops `(b, a)` and pushes `Bool`:
| Instruction | JSON | Description |
|-------------|------|-------------|
| `Eq` | `"Eq"` | `a == b` |
| `NotEq` | `"NotEq"` | `a != b` |
| `Lt` | `"Lt"` | `a < b` |
| `Gt` | `"Gt"` | `a > b` |
| `LtEq` | `"LtEq"` | `a <= b` |
| `GtEq` | `"GtEq"` | `a >= b` |
**Ordering semantics:** Numeric types use natural ordering. Strings use lexicographic ordering. Mixed numeric types promote Int to Float. All other mixed-type comparisons return `Equal` (neither less nor greater).
### 3.5 Logical Instructions
| Instruction | JSON | Description |
|-------------|------|-------------|
| `And` | `"And"` | Pop `(b, a)`, push `Bool(a && b)` |
| `Or` | `"Or"` | Pop `(b, a)`, push `Bool(a \|\| b)` |
| `Not` | `"Not"` | Pop `a`, push `Bool(!a)` |
Logical `And` and `Or` do not short-circuit — both operands are always evaluated before the instruction executes.
### 3.6 Local Variable Instructions
| Instruction | JSON | Description |
|-------------|------|-------------|
| `LoadLocal(name)` | `{"LoadLocal": "name"}` | Push value of local variable; pushes `Nil` if not set |
| `StoreLocal(name)` | `{"StoreLocal": "name"}` | Pop top of stack into local variable |
Local variables are stored in a `HashMap<String, Value>` per call frame. The `StoreLocal` instruction overwrites the slot unconditionally, implementing El's variable shadowing/mutation model.
### 3.7 Function Call Instructions
| Instruction | JSON | Description |
|-------------|------|-------------|
| `Call { name, arity }` | `{"Call": {"name": "fn", "arity": 2}}` | Call function with `arity` arguments |
| `Return` | `"Return"` | Return from current frame |
**Call dispatch sequence:**
1. The `Call` instruction attempts `builtins::dispatch(name, arity, stack, args, global_state)`.
2. If the builtin dispatcher returns `NotBuiltin`, the VM looks up `name` in the function table.
3. If found, the VM saves the current frame (return IP + locals snapshot) to the call stack and sets IP to the function's entry point.
4. If not found, `Nil` is pushed (graceful failure).
**Function table construction:** Before execution begins, the VM scans the entire instruction array for `Push(Int(n))` followed immediately by `StoreLocal("__fn_name")`. Each such pair registers function `name` at entry IP `n`. This is the function registration protocol that the codegen emits for every `fn` definition.
**Return protocol:** `Return` pops the top call frame, restores `saved_locals`, and sets IP to `return_ip`. The return value remains on the stack.
### 3.8 Control Flow Instructions
All jump offsets are signed 32-bit integers relative to the instruction *after* the jump instruction (i.e., the new IP = current IP + 1 + offset).
| Instruction | JSON | Description |
|-------------|------|-------------|
| `Jump(offset)` | `{"Jump": 3}` | Unconditional jump |
| `JumpIf(offset)` | `{"JumpIf": 2}` | Jump if top of stack is `Bool(true)`; pops the value |
| `JumpIfNot(offset)` | `{"JumpIfNot": -5}` | Jump if top of stack is not `Bool(true)`; pops the value |
A positive offset jumps forward; a negative offset jumps backward. An offset of `0` jumps to the instruction immediately following the jump (a no-op equivalent).
### 3.9 Field and Indexing Instructions
| Instruction | JSON | Description |
|-------------|------|-------------|
| `GetField(name)` | `{"GetField": "field"}` | Pop Map or Struct; push value of named field (or Nil) |
| `SetField(name)` | `{"SetField": "field"}` | Pop value, update named field in Map/Struct on top of stack |
| `GetIndex` | `"GetIndex"` | Pop index, pop container; push element at index (or Nil) |
| `BuildMap(n)` | `{"BuildMap": 3}` | Pop 2n values (alternating key, value) and build a Map |
| `BuildList(n)` | `{"BuildList": 4}` | Pop n values and build a List |
| `BuildStruct { type_name, fields }` | `{"BuildStruct": {"type_name": "T", "fields": ["a","b"]}}` | Pop n values, build Struct |
**BuildMap protocol:** The code generator pushes keys and values interleaved: `Push(Str(k1)), Push(v1), Push(Str(k2)), Push(v2), ..., BuildMap(n)`. The `BuildMap` instruction pops `2n` values in reverse order and assembles the pairs, then reverses to restore insertion order.
**GetIndex on strings:** When the container is a `Str`, `GetIndex` with an integer index returns the nth character as a single-character `Str`.
### 3.10 Special Instructions
| Instruction | JSON | Description |
|-------------|------|-------------|
| `Activate { type_name, query }` | `{"Activate": {"type_name": "User", "query": "..."}}` | Semantic graph search; pushes `List` of results |
| `SealedBegin` | `"SealedBegin"` | Mark start of sealed region |
| `SealedEnd` | `"SealedEnd"` | Mark end of sealed region |
| `Reason { query }` | `{"Reason": {"query": "..."}}` | LLM inference call; pushes `Str` response |
| `Parallel { entries }` | `{"Parallel": {"entries": [["name", ip]]}}` | Spawn parallel sub-VMs; pushes `Map` of results |
| `TraceBegin { label }` | `{"TraceBegin": {"label": "..."}}` | Record start time for trace region |
| `TraceEnd { label }` | `{"TraceEnd": {"label": "..."}}` | Print elapsed ms for trace region |
| `ContractCheck { message }` | `{"ContractCheck": {"message": "..."}}` | Assert top of stack is true; exit(1) if not |
| `DeployFn { fn_name, route, target }` | `{"DeployFn": {"fn_name":"...", "route":"...", "target":"..."}}` | POST function to deployment API |
| `Nop` | `"Nop"` | No operation |
| `Halt` | `"Halt"` | Halt execution |
---
## 4. Execution Model
### 4.1 Execution State
The VM maintains the following state during execution:
- **Instruction pointer (IP):** Integer index into the flat instruction array.
- **Value stack:** `Vec<Value>`, grows upward.
- **Locals:** `HashMap<String, Value>` for the current call frame.
- **Call stack:** `Vec<CallFrame>`, each frame holding `{ return_ip: usize, saved_locals: HashMap }`.
- **Global state:** `HashMap<String, String>` shared across the entire program lifetime (accessible via `state_set`/`state_get` builtins).
- **Function table:** `HashMap<String, usize>` mapping function names to entry IPs (populated by the pre-execution scan pass).
### 4.2 Execution Loop
1. Load instruction at IP.
2. Dispatch on instruction variant.
3. Increment IP (unless the instruction set IP explicitly via a jump or call).
4. Repeat until `Halt` or `Return` with empty call stack.
### 4.3 Call Frame Protocol
When `Call` dispatches to a user-defined function:
1. Save `(ip + 1, locals.clone())` as a `CallFrame` onto the call stack.
2. Clear locals (the new frame starts empty; parameters are passed via the stack and stored by the function's prologue).
3. Set IP to the function's entry point.
When `Return` executes:
1. Pop the top `CallFrame`.
2. Restore `locals` from `saved_locals`.
3. Set IP to `return_ip`.
4. The return value remains on the value stack.
### 4.4 Function Prologue Protocol
The codegen emits function prologues as a sequence of `StoreLocal` instructions in reverse parameter order. Because arguments are pushed left-to-right by the caller, and the stack is LIFO, storing in reverse order assigns parameters to their correct names:
```
// fn f(a: Int, b: Int)
// Caller pushes: a, b (b is on top)
StoreLocal("b") // pops b
StoreLocal("a") // pops a
```
### 4.5 Higher-Order Function Protocol
The VM implements three higher-order function helpers that are resolved by name before the function table:
| Name | Behavior |
|------|---------|
| `list_map` | Pops `fn_name: Str` and `list: List`; calls `fn_name` on each element; pushes result `List` |
| `list_filter` | Pops `fn_name: Str` and `list: List`; calls `fn_name` on each element; pushes filtered `List` |
| `list_reduce` | Pops `fn_name: Str`, `init: Value`, and `list: List`; folds; pushes accumulated value |
### 4.6 Parallel Execution
The `Parallel` instruction spawns one thread per entry using `std::thread::spawn`. Each thread runs a sub-VM instance starting at the specified entry IP. All threads complete before the parent continues; results are collected into a `Map` keyed by entry name.
---
## 5. Builtin Dispatch
### 5.1 Architecture
The builtin dispatch system is the primary extension point of the ELVM. When a `Call` instruction is encountered:
1. The VM calls `builtins::dispatch(name, arity, stack, program_args, global_state)`.
2. The dispatcher pattern-matches on `name` to identify the builtin.
3. If matched, it pops `arity` arguments from the stack, performs the operation, and pushes the result.
4. It returns one of: `Handled`, `HttpServe`, `Exit(code)`, or `NotBuiltin`.
### 5.2 Dispatch Result
| Variant | Meaning |
|---------|---------|
| `Handled` | Builtin executed successfully |
| `HttpServe` | Builtin started an HTTP server (non-blocking from VM perspective) |
| `Exit(code)` | Builtin called for process exit |
| `NotBuiltin` | Name not recognized; VM should continue to user function table |
### 5.3 Builtin Categories
The dispatch table handles all builtins described in the El language specification (Section 14). All builtins are registered by name string with no external configuration. Adding a new builtin requires only adding a match arm to the dispatch function.
### 5.4 Engram Integration Builtins
The `Activate` instruction dispatches to `engram_activate_search(type_name, query)`, which performs an HTTP POST to `ENGRAM_URL/search` with a JSON body `{ "query": query, "limit": 20 }`. Results are deserialized into a `List` of `Value` instances.
The `Reason` instruction dispatches to `soma_reason(query)`, which performs an HTTP POST to `SOMA_URL/v1/chat/completions` with the query as the user message.
Both external calls are blocking and synchronous.
---
## 6. Memory Model
### 6.1 Locals
Local variables exist per call frame as a `HashMap<String, Value>`. Frame creation copies the parent's locals map (snapshot semantics). Modifications in a child frame do not affect the parent; the parent's state is restored from the saved snapshot when the child returns.
### 6.2 Global State
The `global_state` map (`HashMap<String, String>`) persists for the entire process lifetime and is accessible from any point in program execution. It is shared by all call frames and all threads in a parallel execution. Access is mediated by the `state_set`, `state_get`, and `state_delete` builtins.
### 6.3 Value Semantics
All values are cloned on push, pop, and frame save. There is no aliasing — mutating a `List` or `Map` value does not affect other bindings to the same original value.
### 6.4 Stack Discipline
The stack is unbounded. Stack underflow (pop from empty stack) returns `Nil` rather than panicking. This preserves program continuity in the face of codegen anomalies but may silently propagate type errors.
---
## 7. Error Handling
### 7.1 Runtime Error Policy
ELVM does not have a trap mechanism for arithmetic errors. Division by zero returns `Nil`. Out-of-bounds array access returns `Nil`. Missing map keys return `Nil`. Missing local variables return `Nil`. This design allows programs to use nil-checks for error detection.
### 7.2 ContractCheck
The `ContractCheck { message }` instruction provides explicit assertion. If the top of the stack is not `Bool(true)`, the VM prints the message to stderr and calls `std::process::exit(1)`.
### 7.3 Fatal Errors
The following conditions produce fatal exits:
- Malformed ELVM header (truncated magic or length).
- Unsupported ELVM version.
- JSON deserialization failure of the payload.
These are reported as `elvm: error: <message>` on stderr before exit.
---
## 8. Source Map Format
Debug builds produce a source map file (`<name>.map.json`) alongside the bytecode. The source map is a JSON array:
```json
[
{ "instruction": 0, "start": 0, "end": 5, "line": 3, "col": 0 },
...
]
```
| Field | Description |
|-------|-------------|
| `instruction` | Bytecode instruction index |
| `start` | Byte offset in source where the expression begins |
| `end` | Byte offset where the expression ends |
| `line` | 1-based source line number |
| `col` | 0-based column number |
Source maps enable debuggers and error reporters to translate bytecode offsets to source positions.
---
## 9. Bootstrap Sequence
### 9.1 Genesis Bootstrap
The bootstrap sequence that produces a self-hosting El toolchain:
1. **Rust genesis compiler** (`engrams/el-compiler/`) reads `lexer.el`, `parser.el`, `codegen.el` and produces `el-compiler.elc` — the self-hosting compiler as an ELVM bytecode file.
2. **ELVM** (`bin/elvm/`) executes `el-compiler.elc`. When invoked on an El source file, it produces an `.elc` output by running the El lexer, parser, and codegen implemented in El.
3. **Verification pass:** The Rust genesis compiler and the self-hosted compiler are both run on the same El source file; their outputs are byte-compared. Identity confirms bootstrapping correctness.
4. **Production toolchain:** The `el` binary (`bin/el/`) incorporates the genesis Rust compiler for the `build-file` command and invokes `elvm` for execution. The self-hosted compiler path is used for project builds.
### 9.2 Invariant
The Rust genesis compiler and the self-hosted El compiler are structural mirrors of each other. Both implement the same lexer, parser, and codegen algorithms. The genesis compiler serves exclusively as a bootstrap vehicle; no production El bytecode is produced by it after initial bootstrap.
---
## 10. CLI Reference
### elvm
```
elvm <file.elc> [args...]
elvm --version
elvm --help
```
Executes a compiled El bytecode file (`.elc`). Arguments after the file path are forwarded to the program via the `args()` builtin.
### el build-file
```
el build-file <file.el> [--target debug|release|prod] [-o output]
```
Compiles a single `.el` file without a project manifest. Useful for scripts and standalone programs.
### el run
```
el run
```
Compiles the project with `debug` target and immediately executes the resulting bytecode.
+1 -1
View File
@@ -671,7 +671,7 @@ The El compiler lives in `el-compiler/src/`:
- `codegen.el` — C emitter
- `compiler.el` — pipeline wiring + `main()` entry
These are concatenated into `elc-combined.el` (single-file bootstrap edition). The genesis Rust compiler at `target/debug/el` was used once to produce the first self-hosted compiler binary at `dist/platform/elc`. From that point forward, `elc` compiles itself and all El programs.
These are concatenated into `elc-combined.el` (single-file bootstrap edition). The bootstrap compiler binary lives at `dist/platform/elc`; from there `elc` compiles itself and all El programs.
### 13.3 C Runtime