el/spec/elvm.md

ELVM — El Virtual Machine Specification

Version 1.0.0 — April 29, 2026

---

Overview

ELVM is the execution substrate for El programs. It is a stack-based virtual machine that executes JSON-serialized bytecode instructions stored in a typed binary container format. ELVM has four defining properties:

1. Binary container with semantic header. The .elc file format begins with a 4-byte magic identifier (ELVM), a 4-byte version field, and an 8-byte payload length — making every artifact self-describing and parseable without external schema.

2. JSON-serialized instruction encoding. The bytecode payload is a JSON array of instruction objects. Each instruction is human-inspectable with standard JSON tools, language-portable, and schema-free. The JSON encoding is the canonical on-disk format; no separate binary encoding exists.

3. Builtin dispatch architecture. Function calls resolve first against a runtime builtin dispatch table, then against the user-defined function table. New builtins are added to the dispatch table without recompiling existing programs. The execution model treats builtins and user functions identically from the instruction side.

4. Self-hosting bootstrap. A Rust genesis compiler produces ELVM bytecode. That bytecode, when executed by ELVM, runs the El compiler written in El, which produces ELVM bytecode for all subsequent compilation. The VM is the only persistent execution engine.

---

1. Binary Container Format

1.1 Header

Every .elc file begins with a 16-byte header:

Offset  Size   Type              Field
──────  ─────  ────────────────  ──────────────────────────────────────────
0       4      [u8; 4]           Magic: b"ELVM"
4       4      u32 little-endian Version: currently 1
8       8      u64 little-endian Payload length in bytes

1.2 Payload

Immediately following the 16-byte header is the payload: a UTF-8 JSON array encoding a Vec<Bytecode> instruction sequence.

[instruction, instruction, ...]

The payload length field in the header equals the exact byte count of the JSON payload. Bytes beyond header_size + payload_length are ignored (reserved for future trailer use).

1.3 Legacy Format

Files without the ELVM magic header are treated as raw JSON bytecode (no header). This is the legacy format produced by early compiler versions. The VM accepts both formats via the deserialize_all entry point.

1.4 Identification

The magic bytes ELVM (hex: 45 4C 56 4D) uniquely identify El bytecode containers. Any tool that encounters these bytes at offset 0 may assume the 16-byte header format described above.

---

2. Value Types

The VM stack holds Value instances of the following types:

Variant	Rust encoding	Description
Int(i64)	64-bit signed integer	All integer arithmetic
Float(f64)	IEEE 754 double	Floating-point arithmetic
Str(String)	UTF-8 string	Text and identifiers
Bool(bool)	Boolean	Logical conditions
Nil	Unit	Absent value, uninitialized
List(Vec<Value>)	Ordered sequence	Arrays, activation results
Map(Vec<(String, Value)>)	Ordered key-value pairs	Struct instances, JSON objects
ResultOk(Box<Value>)	Success variant	Fallible operation success
ResultErr(Box<Value>)	Error variant	Fallible operation failure
Struct { type_name, fields }	Named struct	Typed record with field list

Maps use Vec<(String, Value)> rather than a hash map to preserve insertion order and remain serialization-friendly.

---

3. Instruction Set

3.1 JSON Encoding

Each instruction serializes as either a JSON string (no-argument instructions) or a JSON object (instructions with arguments):

"Halt"
"Pop"
{"Push": {"Int": 42}}
{"Push": {"Str": "hello"}}
{"Push": "Nil"}
{"LoadLocal": "x"}
{"StoreLocal": "x"}
{"Call": {"name": "println", "arity": 1}}
{"Jump": 3}
{"JumpIfNot": -2}

3.2 Stack Instructions

Instruction	JSON	Stack effect	Description
Push(v)	{"Push": <value>}	-- v	Push constant value
Pop	"Pop"	v --	Discard top of stack
Dup	"Dup"	v -- v v	Duplicate top of stack

3.3 Arithmetic Instructions

All arithmetic instructions pop two operands and push the result. For binary operations, operands are popped as (b, a) — b is popped first (top of stack), a is popped second. The operation computes a OP b.

Instruction	JSON	Description
Add	"Add"	a + b (Int/Float/Str concatenation)
Sub	"Sub"	a - b
Mul	"Mul"	a * b
Div	"Div"	a / b (Int: integer division; division by zero returns Nil)
Mod	"Mod"	a % b
BitAnd	"BitAnd"	a & b (Int only)
BitOr	"BitOr"	a | b (Int only)
BitXor	"BitXor"	a ^ b (Int only)
BitNot	"BitNot"	~a (Int only, unary)
Shl	"Shl"	a << b (Int only)
Shr	"Shr"	a >> b (Int only)

Type promotion: When one operand is Int and the other is Float, the Int is widened to Float before the operation.

3.4 Comparison Instructions

Each pops (b, a) and pushes Bool:

Instruction	JSON	Description
Eq	"Eq"	a == b
NotEq	"NotEq"	a != b
Lt	"Lt"	a < b
Gt	"Gt"	a > b
LtEq	"LtEq"	a <= b
GtEq	"GtEq"	a >= b

Ordering semantics: Numeric types use natural ordering. Strings use lexicographic ordering. Mixed numeric types promote Int to Float. All other mixed-type comparisons return Equal (neither less nor greater).

3.5 Logical Instructions

Instruction	JSON	Description
And	"And"	Pop (b, a), push Bool(a && b)
Or	"Or"	Pop (b, a), push Bool(a || b)
Not	"Not"	Pop a, push Bool(!a)

Logical And and Or do not short-circuit — both operands are always evaluated before the instruction executes.

3.6 Local Variable Instructions

Instruction	JSON	Description
LoadLocal(name)	{"LoadLocal": "name"}	Push value of local variable; pushes Nil if not set
StoreLocal(name)	{"StoreLocal": "name"}	Pop top of stack into local variable

Local variables are stored in a HashMap<String, Value> per call frame. The StoreLocal instruction overwrites the slot unconditionally, implementing El's variable shadowing/mutation model.

3.7 Function Call Instructions

Instruction	JSON	Description
Call { name, arity }	{"Call": {"name": "fn", "arity": 2}}	Call function with arity arguments
Return	"Return"	Return from current frame

Call dispatch sequence:

1. The Call instruction attempts builtins::dispatch(name, arity, stack, args, global_state).
2. If the builtin dispatcher returns NotBuiltin, the VM looks up name in the function table.
3. If found, the VM saves the current frame (return IP + locals snapshot) to the call stack and sets IP to the function's entry point.
4. If not found, Nil is pushed (graceful failure).

Function table construction: Before execution begins, the VM scans the entire instruction array for Push(Int(n)) followed immediately by StoreLocal("__fn_name"). Each such pair registers function name at entry IP n. This is the function registration protocol that the codegen emits for every fn definition.

Return protocol: Return pops the top call frame, restores saved_locals, and sets IP to return_ip. The return value remains on the stack.

3.8 Control Flow Instructions

All jump offsets are signed 32-bit integers relative to the instruction after the jump instruction (i.e., the new IP = current IP + 1 + offset).

Instruction	JSON	Description
Jump(offset)	{"Jump": 3}	Unconditional jump
JumpIf(offset)	{"JumpIf": 2}	Jump if top of stack is Bool(true); pops the value
JumpIfNot(offset)	{"JumpIfNot": -5}	Jump if top of stack is not Bool(true); pops the value

A positive offset jumps forward; a negative offset jumps backward. An offset of 0 jumps to the instruction immediately following the jump (a no-op equivalent).

3.9 Field and Indexing Instructions

Instruction	JSON	Description
GetField(name)	{"GetField": "field"}	Pop Map or Struct; push value of named field (or Nil)
SetField(name)	{"SetField": "field"}	Pop value, update named field in Map/Struct on top of stack
GetIndex	"GetIndex"	Pop index, pop container; push element at index (or Nil)
BuildMap(n)	{"BuildMap": 3}	Pop 2n values (alternating key, value) and build a Map
BuildList(n)	{"BuildList": 4}	Pop n values and build a List
BuildStruct { type_name, fields }	{"BuildStruct": {"type_name": "T", "fields": ["a","b"]}}	Pop n values, build Struct

BuildMap protocol: The code generator pushes keys and values interleaved: Push(Str(k1)), Push(v1), Push(Str(k2)), Push(v2), ..., BuildMap(n). The BuildMap instruction pops 2n values in reverse order and assembles the pairs, then reverses to restore insertion order.

GetIndex on strings: When the container is a Str, GetIndex with an integer index returns the nth character as a single-character Str.

3.10 Special Instructions

Instruction	JSON	Description
Activate { type_name, query }	{"Activate": {"type_name": "User", "query": "..."}}	Semantic graph search; pushes List of results
SealedBegin	"SealedBegin"	Mark start of sealed region
SealedEnd	"SealedEnd"	Mark end of sealed region
Reason { query }	{"Reason": {"query": "..."}}	LLM inference call; pushes Str response
Parallel { entries }	{"Parallel": {"entries": [["name", ip]]}}	Spawn parallel sub-VMs; pushes Map of results
TraceBegin { label }	{"TraceBegin": {"label": "..."}}	Record start time for trace region
TraceEnd { label }	{"TraceEnd": {"label": "..."}}	Print elapsed ms for trace region
ContractCheck { message }	{"ContractCheck": {"message": "..."}}	Assert top of stack is true; exit(1) if not
DeployFn { fn_name, route, target }	{"DeployFn": {"fn_name":"...", "route":"...", "target":"..."}}	POST function to deployment API
Nop	"Nop"	No operation
Halt	"Halt"	Halt execution

---

4. Execution Model

4.1 Execution State

The VM maintains the following state during execution:

• Instruction pointer (IP): Integer index into the flat instruction array.
• Value stack: Vec<Value>, grows upward.
• Locals: HashMap<String, Value> for the current call frame.
• Call stack: Vec<CallFrame>, each frame holding { return_ip: usize, saved_locals: HashMap }.
• Global state: HashMap<String, String> shared across the entire program lifetime (accessible via state_set/state_get builtins).
• Function table: HashMap<String, usize> mapping function names to entry IPs (populated by the pre-execution scan pass).

4.2 Execution Loop

1. Load instruction at IP.
2. Dispatch on instruction variant.
3. Increment IP (unless the instruction set IP explicitly via a jump or call).
4. Repeat until Halt or Return with empty call stack.

4.3 Call Frame Protocol

When Call dispatches to a user-defined function:

1. Save (ip + 1, locals.clone()) as a CallFrame onto the call stack.
2. Clear locals (the new frame starts empty; parameters are passed via the stack and stored by the function's prologue).
3. Set IP to the function's entry point.

When Return executes:

1. Pop the top CallFrame.
2. Restore locals from saved_locals.
3. Set IP to return_ip.
4. The return value remains on the value stack.

4.4 Function Prologue Protocol

The codegen emits function prologues as a sequence of StoreLocal instructions in reverse parameter order. Because arguments are pushed left-to-right by the caller, and the stack is LIFO, storing in reverse order assigns parameters to their correct names:

// fn f(a: Int, b: Int)
// Caller pushes: a, b  (b is on top)
StoreLocal("b")    // pops b
StoreLocal("a")    // pops a

4.5 Higher-Order Function Protocol

The VM implements three higher-order function helpers that are resolved by name before the function table:

Name	Behavior
list_map	Pops fn_name: Str and list: List; calls fn_name on each element; pushes result List
list_filter	Pops fn_name: Str and list: List; calls fn_name on each element; pushes filtered List
list_reduce	Pops fn_name: Str, init: Value, and list: List; folds; pushes accumulated value

4.6 Parallel Execution

The Parallel instruction spawns one thread per entry using std::thread::spawn. Each thread runs a sub-VM instance starting at the specified entry IP. All threads complete before the parent continues; results are collected into a Map keyed by entry name.

---

5. Builtin Dispatch

5.1 Architecture

The builtin dispatch system is the primary extension point of the ELVM. When a Call instruction is encountered:

1. The VM calls builtins::dispatch(name, arity, stack, program_args, global_state).
2. The dispatcher pattern-matches on name to identify the builtin.
3. If matched, it pops arity arguments from the stack, performs the operation, and pushes the result.
4. It returns one of: Handled, HttpServe, Exit(code), or NotBuiltin.

5.2 Dispatch Result

Variant	Meaning
Handled	Builtin executed successfully
HttpServe	Builtin started an HTTP server (non-blocking from VM perspective)
Exit(code)	Builtin called for process exit
NotBuiltin	Name not recognized; VM should continue to user function table

5.3 Builtin Categories

The dispatch table handles all builtins described in the El language specification (Section 14). All builtins are registered by name string with no external configuration. Adding a new builtin requires only adding a match arm to the dispatch function.

5.4 Engram Integration Builtins

The Activate instruction dispatches to engram_activate_search(type_name, query), which performs an HTTP POST to ENGRAM_URL/search with a JSON body { "query": query, "limit": 20 }. Results are deserialized into a List of Value instances.

The Reason instruction dispatches to soma_reason(query), which performs an HTTP POST to SOMA_URL/v1/chat/completions with the query as the user message.

Both external calls are blocking and synchronous.

---

6. Memory Model

6.1 Locals

Local variables exist per call frame as a HashMap<String, Value>. Frame creation copies the parent's locals map (snapshot semantics). Modifications in a child frame do not affect the parent; the parent's state is restored from the saved snapshot when the child returns.

6.2 Global State

The global_state map (HashMap<String, String>) persists for the entire process lifetime and is accessible from any point in program execution. It is shared by all call frames and all threads in a parallel execution. Access is mediated by the state_set, state_get, and state_delete builtins.

6.3 Value Semantics

All values are cloned on push, pop, and frame save. There is no aliasing — mutating a List or Map value does not affect other bindings to the same original value.

6.4 Stack Discipline

The stack is unbounded. Stack underflow (pop from empty stack) returns Nil rather than panicking. This preserves program continuity in the face of codegen anomalies but may silently propagate type errors.

---

7. Error Handling

7.1 Runtime Error Policy

ELVM does not have a trap mechanism for arithmetic errors. Division by zero returns Nil. Out-of-bounds array access returns Nil. Missing map keys return Nil. Missing local variables return Nil. This design allows programs to use nil-checks for error detection.

7.2 ContractCheck

The ContractCheck { message } instruction provides explicit assertion. If the top of the stack is not Bool(true), the VM prints the message to stderr and calls std::process::exit(1).

7.3 Fatal Errors

The following conditions produce fatal exits:

• Malformed ELVM header (truncated magic or length).
• Unsupported ELVM version.
• JSON deserialization failure of the payload.

These are reported as elvm: error: <message> on stderr before exit.

---

8. Source Map Format

Debug builds produce a source map file (<name>.map.json) alongside the bytecode. The source map is a JSON array:

[
  { "instruction": 0, "start": 0, "end": 5, "line": 3, "col": 0 },
  ...
]

Field	Description
instruction	Bytecode instruction index
start	Byte offset in source where the expression begins
end	Byte offset where the expression ends
line	1-based source line number
col	0-based column number

Source maps enable debuggers and error reporters to translate bytecode offsets to source positions.

---

9. Bootstrap Sequence

9.1 Genesis Bootstrap

The bootstrap sequence that produces a self-hosting El toolchain:

1. Rust genesis compiler (engrams/el-compiler/) reads lexer.el, parser.el, codegen.el and produces el-compiler.elc — the self-hosting compiler as an ELVM bytecode file.

2. ELVM (bin/elvm/) executes el-compiler.elc. When invoked on an El source file, it produces an .elc output by running the El lexer, parser, and codegen implemented in El.

3. Verification pass: The Rust genesis compiler and the self-hosted compiler are both run on the same El source file; their outputs are byte-compared. Identity confirms bootstrapping correctness.

4. Production toolchain: The el binary (bin/el/) incorporates the genesis Rust compiler for the build-file command and invokes elvm for execution. The self-hosted compiler path is used for project builds.

9.2 Invariant

The Rust genesis compiler and the self-hosted El compiler are structural mirrors of each other. Both implement the same lexer, parser, and codegen algorithms. The genesis compiler serves exclusively as a bootstrap vehicle; no production El bytecode is produced by it after initial bootstrap.

---

10. CLI Reference

elvm

elvm <file.elc> [args...]
elvm --version
elvm --help

Executes a compiled El bytecode file (.elc). Arguments after the file path are forwarded to the program via the args() builtin.

el build-file

el build-file <file.el> [--target debug|release|prod] [-o output]

Compiles a single .el file without a project manifest. Useful for scripts and standalone programs.

el run

el run

Compiles the project with debug target and immediately executes the resulting bytecode.