963 lines
30 KiB
Markdown
963 lines
30 KiB
Markdown
# El Language Specification
|
|
|
|
Version 1.0.0 — April 29, 2026
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
El is a statically-typed, compiled programming language designed as the execution substrate for the Neuron agent runtime. El is self-hosting: the El compiler is written in El, compiled to ELVM bytecode, and executed by the El Virtual Machine. A Rust genesis compiler bootstraps the first iteration; all subsequent compilation is performed by the self-hosted compiler.
|
|
|
|
El has four defining properties:
|
|
|
|
1. **Self-hosting compiler.** The compiler (`lexer.el`, `parser.el`, `codegen.el`) is written in El, produces ELVM bytecode, and runs on the ELVM. The genesis Rust compiler (`el-compiler` crate) is used only for bootstrapping.
|
|
|
|
2. **First-class application identity.** The `app` block is a language-level construct, not a library call. It declares service name, version, typed configuration schema, secrets, and feature flags — all resolved by the runtime before the program body executes.
|
|
|
|
3. **Graph-native builtins.** Knowledge graph operations (`graph_compile`, `graph_traverse`, `graph_write_node`, `graph_write_edge`) are language primitives dispatched by the VM, not library imports.
|
|
|
|
4. **Sealed artifact target.** The `prod` compilation target produces AES-256-GCM encrypted, BLAKE3-authenticated bytecode containers — quantum-resistant sealed artifacts that are indistinguishable from random bytes without the deployment key.
|
|
|
|
---
|
|
|
|
## 1. Lexical Structure
|
|
|
|
### 1.1 Source Encoding
|
|
|
|
El source files are UTF-8 encoded. The canonical extension is `.el`.
|
|
|
|
### 1.2 Comments
|
|
|
|
```
|
|
// Single-line comment — extends to end of line
|
|
```
|
|
|
|
Block comments are not supported. Use `//` on each line.
|
|
|
|
### 1.3 Whitespace
|
|
|
|
Spaces, tabs, newlines (`\n`), and carriage returns (`\r`) are whitespace. Whitespace is not significant except as a token separator.
|
|
|
|
### 1.4 Identifiers
|
|
|
|
```
|
|
identifier = (alpha | '_') (alnum | '_')*
|
|
```
|
|
|
|
Identifiers are case-sensitive. Identifiers beginning with `__` (double underscore) are reserved for compiler-generated names.
|
|
|
|
### 1.5 Keywords
|
|
|
|
The following words are reserved and cannot be used as identifiers:
|
|
|
|
```
|
|
let fn type enum match return if else for in while
|
|
import from as with sealed activate where test seed
|
|
assert protocol impl retry times fallback reason parallel
|
|
trace requires deploy to via target true false app
|
|
version config secrets flags
|
|
```
|
|
|
|
### 1.6 Token Types
|
|
|
|
| Token | Pattern |
|
|
|-------|---------|
|
|
| `Int` | `[0-9]+` |
|
|
| `Float` | `[0-9]+ '.' [0-9]+` |
|
|
| `Str` | `'"' (char | escape)* '"'` |
|
|
| `Bool` | `true` or `false` |
|
|
| `Ident` | identifier (not a keyword) |
|
|
| `Let` `Fn` `Type` etc. | keyword tokens |
|
|
| `EqEq` | `==` |
|
|
| `NotEq` | `!=` |
|
|
| `LtEq` | `<=` |
|
|
| `GtEq` | `>=` |
|
|
| `Arrow` | `->` |
|
|
| `FatArrow` | `=>` |
|
|
| `ColonColon` | `::` |
|
|
| `And` | `&&` |
|
|
| `Or` | `\|\|` |
|
|
| `PipeOp` | `\|>` |
|
|
|
|
### 1.7 String Escape Sequences
|
|
|
|
| Sequence | Character |
|
|
|----------|-----------|
|
|
| `\n` | Newline (U+000A) |
|
|
| `\t` | Tab (U+0009) |
|
|
| `\r` | Carriage return (U+000D) |
|
|
| `\"` | Double quote |
|
|
| `\\` | Backslash |
|
|
| `\0` | Null byte |
|
|
|
|
---
|
|
|
|
## 2. Type System
|
|
|
|
### 2.1 Primitive Types
|
|
|
|
| Type | Description | Example literals |
|
|
|------|-------------|-----------------|
|
|
| `Int` | 64-bit signed integer | `42`, `-7`, `1_000` |
|
|
| `Float` | 64-bit IEEE 754 double | `3.14`, `0.5` |
|
|
| `String` | UTF-8 string | `"hello"` |
|
|
| `Bool` | Boolean | `true`, `false` |
|
|
| `Uuid` | RFC 4122 UUID | (runtime-produced only) |
|
|
| `Void` | Unit type; no value | — |
|
|
| `Any` | Dynamically-typed value | (for generic containers) |
|
|
|
|
### 2.2 Composite Types
|
|
|
|
| Type form | Description |
|
|
|-----------|-------------|
|
|
| `[T]` | Array (ordered sequence) of `T` |
|
|
| `T?` | Optional `T` — may be absent |
|
|
| `Map<K, V>` | Key-value map with string keys |
|
|
| `List` | Untyped list (dynamic, used in builtins) |
|
|
| `Named` | User-defined struct or enum |
|
|
|
|
### 2.3 Type Inference
|
|
|
|
The compiler infers types for `let` bindings without annotation:
|
|
|
|
```
|
|
let x = 42 // inferred: Int
|
|
let s = "hello" // inferred: String
|
|
let b = true // inferred: Bool
|
|
```
|
|
|
|
Function parameter types and return types must always be annotated. Function signatures are specification.
|
|
|
|
### 2.4 Type Coercions
|
|
|
|
- `Int` is implicitly coercible to `Float`.
|
|
- `Float` is not coercible to `Int` (use `float_to_int` builtin).
|
|
- `T` is assignable to `T?` (non-optional is a subtype of optional).
|
|
- `String + String` performs concatenation via the `+` operator overload.
|
|
|
|
### 2.5 Optional Type and Null Coalescing
|
|
|
|
`T?` denotes an optional value. The null-coalescing operator `??` returns the left operand if it is not nil, otherwise the right:
|
|
|
|
```
|
|
let name: String? = maybe_get_name()
|
|
let display: String = name ?? "unknown"
|
|
```
|
|
|
|
The `?` postfix operator on a function call propagates `nil` outward (early return of `nil` if the subexpression is nil):
|
|
|
|
```
|
|
let val = some_optional_fn()?
|
|
```
|
|
|
|
### 2.6 Type Casting
|
|
|
|
The `as` keyword performs explicit type casts at runtime:
|
|
|
|
```
|
|
let n: Int = 42
|
|
let f: Float = n as Float
|
|
let s: String = f as String
|
|
```
|
|
|
|
Casting to `String` invokes the value's string representation. Casting numeric types performs the standard numeric conversion. Invalid casts produce `Nil`.
|
|
|
|
---
|
|
|
|
## 3. Variables and Bindings
|
|
|
|
### 3.1 Let Bindings
|
|
|
|
```
|
|
let name: Type = expression
|
|
let name = expression // type inferred
|
|
```
|
|
|
|
All bindings are block-scoped. Bindings are immutable by default. Shadowing is permitted: a new `let` in the same scope with the same name creates a new binding that shadows the previous one. This is the mechanism for mutation in El — the VM's `StoreLocal` instruction overwrites the slot for the name.
|
|
|
|
```
|
|
let count: Int = 0
|
|
let count = count + 1 // shadows previous binding; effective mutation
|
|
```
|
|
|
|
### 3.2 Scope
|
|
|
|
Bindings are valid from the point of declaration to the end of the enclosing block. Function bodies, `if` branches, `for` bodies, and `while` bodies each introduce a new scope.
|
|
|
|
---
|
|
|
|
## 4. Functions
|
|
|
|
### 4.1 Function Definition
|
|
|
|
```
|
|
fn name(param1: Type1, param2: Type2) -> ReturnType {
|
|
// body
|
|
return expression
|
|
}
|
|
```
|
|
|
|
Functions are first-class values. A function definition emits a jump over the function body and registers the entry IP in the VM's function table via a `Push(Int(entry)) StoreLocal("__fn_name")` stanza.
|
|
|
|
### 4.2 Function Type
|
|
|
|
The type of a function is expressed as:
|
|
|
|
```
|
|
fn(Type1, Type2) -> ReturnType
|
|
```
|
|
|
|
Functions can be passed as values and stored in variables:
|
|
|
|
```
|
|
let f: fn(Int) -> Int = double
|
|
```
|
|
|
|
### 4.3 Return Statement
|
|
|
|
```
|
|
return expression
|
|
```
|
|
|
|
An implicit `return nil` is appended by the compiler if no explicit return is present at the end of a function body.
|
|
|
|
### 4.4 Calling Convention
|
|
|
|
Arguments are pushed left-to-right onto the stack. The function body pops parameters in reverse order (right-to-left) using `StoreLocal`. Return values are left on the stack top when `Return` executes.
|
|
|
|
---
|
|
|
|
## 5. Control Flow
|
|
|
|
### 5.1 If/Else
|
|
|
|
```
|
|
if condition {
|
|
// then branch
|
|
} else {
|
|
// else branch
|
|
}
|
|
```
|
|
|
|
Both branches must produce the same type when used as an expression. The `else` branch is optional; its absence produces `Void`. If/else chains use `else if`:
|
|
|
|
```
|
|
if x > 0 {
|
|
"positive"
|
|
} else if x < 0 {
|
|
"negative"
|
|
} else {
|
|
"zero"
|
|
}
|
|
```
|
|
|
|
### 5.2 While Loops
|
|
|
|
```
|
|
while condition {
|
|
// body
|
|
}
|
|
```
|
|
|
|
Condition is evaluated before each iteration. The loop exits when the condition is `false`.
|
|
|
|
### 5.3 For Loops
|
|
|
|
```
|
|
for item in collection {
|
|
// body
|
|
}
|
|
```
|
|
|
|
`collection` must be a `List` or `[T]`. The compiler desugars `for` into: compute list, store length, initialize counter at 0, loop while counter < length, load element at counter, execute body, increment counter, jump back.
|
|
|
|
### 5.4 Match Expressions
|
|
|
|
```
|
|
match expression {
|
|
Pattern1 => result_expr1
|
|
Pattern2 => result_expr2
|
|
_ => default_expr
|
|
}
|
|
```
|
|
|
|
Pattern forms:
|
|
|
|
| Pattern | Meaning |
|
|
|---------|---------|
|
|
| `_` | Wildcard — always matches |
|
|
| `name` | Binding — captures subject into `name` |
|
|
| `42` | Integer literal |
|
|
| `"str"` | String literal |
|
|
| `true` / `false` | Boolean literal |
|
|
| `EnumName::Variant` | Unit enum variant (future) |
|
|
| `EnumName::Variant(binding)` | Payload-bearing variant (future) |
|
|
|
|
All arms must produce the same type. A `match` with no matching arm evaluates to `nil`.
|
|
|
|
---
|
|
|
|
## 6. Data Structures
|
|
|
|
### 6.1 Struct Types
|
|
|
|
```
|
|
type TypeName {
|
|
field1: Type1
|
|
field2: Type2
|
|
}
|
|
```
|
|
|
|
Struct types are registered in the type environment at compile time. Field access is `value.field_name`, checked at compile time. The VM represents struct instances as `Value::Struct { type_name, fields }`.
|
|
|
|
### 6.2 Enum Types
|
|
|
|
```
|
|
enum EnumName {
|
|
Variant1
|
|
Variant2
|
|
VariantWithPayload(PayloadType)
|
|
}
|
|
```
|
|
|
|
Variants without parentheses carry no payload. Variants with parentheses carry exactly one value of the given type. Enum variants are referenced as `EnumName::Variant`.
|
|
|
|
### 6.3 Array Literals
|
|
|
|
```
|
|
let numbers: [Int] = [1, 2, 3]
|
|
let empty: [String] = []
|
|
```
|
|
|
|
### 6.4 Map Literals
|
|
|
|
```
|
|
let m: Map<String, Any> = { "key1": value1, "key2": value2 }
|
|
```
|
|
|
|
Map literals use string keys and `Any` values. The VM represents maps as `Value::Map(Vec<(String, Value)>)`, preserving insertion order.
|
|
|
|
### 6.5 Field Access and Index Access
|
|
|
|
```
|
|
let field = struct_value.field_name
|
|
let elem = array[0]
|
|
let val = map["key"]
|
|
```
|
|
|
|
Index expressions on arrays require an `Int` index. Bounds violations return `nil`. String indexing `s[n]` returns the nth character as a `String`.
|
|
|
|
---
|
|
|
|
## 7. Operators
|
|
|
|
### 7.1 Arithmetic Operators
|
|
|
|
| Operator | Types | Result |
|
|
|----------|-------|--------|
|
|
| `+` | Int, Float, String | Same as operands (String: concatenation) |
|
|
| `-` | Int, Float | Same |
|
|
| `*` | Int, Float | Same |
|
|
| `/` | Int, Float | Same (integer division for Int) |
|
|
| `%` | Int, Float | Modulo |
|
|
|
|
### 7.2 Bitwise Operators
|
|
|
|
| Operator | Types | Result |
|
|
|----------|-------|--------|
|
|
| `&` | Int | Bitwise AND |
|
|
| `\|` | Int | Bitwise OR |
|
|
| `^` | Int | Bitwise XOR |
|
|
| `~` | Int | Bitwise NOT (unary) |
|
|
| `<<` | Int | Left shift |
|
|
| `>>` | Int | Right shift |
|
|
|
|
### 7.3 Comparison Operators
|
|
|
|
| Operator | Result |
|
|
|----------|--------|
|
|
| `==` | Bool |
|
|
| `!=` | Bool |
|
|
| `<` `>` `<=` `>=` | Bool |
|
|
|
|
### 7.4 Logical Operators
|
|
|
|
| Operator | Result |
|
|
|----------|--------|
|
|
| `&&` | Bool |
|
|
| `\|\|` | Bool |
|
|
| `!` | Bool (unary) |
|
|
|
|
### 7.5 Special Operators
|
|
|
|
| Operator | Meaning |
|
|
|----------|---------|
|
|
| `??` | Null coalescing: left if not nil, else right |
|
|
| `?` (postfix) | Optional propagation: return nil if subexpression is nil |
|
|
| `as` | Type cast |
|
|
| `\|>` | Pipe: `x \|> f` desugars to `f(x)` |
|
|
|
|
### 7.6 Operator Precedence (high to low)
|
|
|
|
1. `!` `~` (unary), postfix `?`
|
|
2. `*` `/` `%`
|
|
3. `+` `-`
|
|
4. `<<` `>>`
|
|
5. `&`
|
|
6. `^`
|
|
7. `|`
|
|
8. `<` `>` `<=` `>=`
|
|
9. `==` `!=`
|
|
10. `&&`
|
|
11. `||`
|
|
12. `??`
|
|
13. `|>`
|
|
|
|
---
|
|
|
|
## 8. Module System
|
|
|
|
### 8.1 Import
|
|
|
|
```
|
|
import "filename.el"
|
|
```
|
|
|
|
Imports all top-level bindings from the named file into the current scope. The path is relative to the importing file's directory. Import cycles are not permitted.
|
|
|
|
### 8.2 From-Import
|
|
|
|
```
|
|
from module_name import { Name1, Name2 }
|
|
```
|
|
|
|
Imports named symbols from a module. The parser records the module name and symbol list; the linker resolves them at build time.
|
|
|
|
---
|
|
|
|
## 9. The App Block
|
|
|
|
The `app` block is a first-class language construct that declares service identity. It must appear at top level, before the program body.
|
|
|
|
```
|
|
app "service-name" {
|
|
version "1.0.0"
|
|
|
|
config {
|
|
KEY: Type = default_value
|
|
prod {
|
|
KEY = "production-override"
|
|
}
|
|
}
|
|
|
|
secrets {
|
|
SECRET_NAME: Type
|
|
}
|
|
|
|
flags {
|
|
feature_name: Bool = false
|
|
}
|
|
}
|
|
```
|
|
|
|
### 9.1 Config Block
|
|
|
|
Config entries declare typed configuration keys with optional defaults. Environment variables with the same name override defaults. The `prod { }` sub-block provides environment-specific overrides applied when `NEURON_ENV=prod`.
|
|
|
|
Config values are accessed via the `config(key)` builtin:
|
|
|
|
```
|
|
let api_url: String = config("NEURON_API_URL")
|
|
```
|
|
|
|
### 9.2 Secrets Block
|
|
|
|
Secrets entries declare required secret values. The runtime loads them from environment variables or from `~/.neuron/secrets.json`. Secrets are redacted from all trace output.
|
|
|
|
Secrets are accessed via the `secret(key)` builtin:
|
|
|
|
```
|
|
let token: String = secret("NEURON_TOKEN")
|
|
```
|
|
|
|
### 9.3 Flags Block
|
|
|
|
Feature flags declare boolean runtime switches. Flags are accessed via `flag(name)`:
|
|
|
|
```
|
|
let enabled: Bool = flag("bidirectional_ctx")
|
|
```
|
|
|
|
### 9.4 Runtime Resolution
|
|
|
|
When the El runtime encounters an `app` block, it:
|
|
|
|
1. Parses the app block from the source before compilation begins (the `parse_app_block` function in `bin/el/src/main.rs`).
|
|
2. Resolves the active environment from `NEURON_ENV` (default: `"dev"`).
|
|
3. Applies environment-specific config overrides.
|
|
4. Loads secrets from environment variables, falling back to the secrets file.
|
|
5. Populates thread-local state: `APP_SERVICE`, `APP_VERSION`, `APP_CONFIG`, `APP_SECRETS`, `APP_FLAGS`, `APP_INSTANCE`.
|
|
6. Registers secrets in the `SECRET_VALUES` set for redaction.
|
|
|
|
The program body executes after resolution completes. `config()`, `secret()`, and `flag()` builtins read from the thread-local state.
|
|
|
|
---
|
|
|
|
## 10. The Sealed Block
|
|
|
|
```
|
|
sealed {
|
|
let api_key: String = env("API_KEY")
|
|
// sensitive operations
|
|
}
|
|
```
|
|
|
|
The `sealed {}` construct marks a code region as containing sensitive material. In debug builds, it emits `SealedBegin` and `SealedEnd` bytecode markers that signal the debugger not to expose values from this region. In prod builds, the entire artifact is AES-256-GCM encrypted, making the `sealed {}` annotation redundant but preserved for documentation and tooling.
|
|
|
|
---
|
|
|
|
## 11. The Activate Construct
|
|
|
|
```
|
|
activate TypeName where "semantic query string"
|
|
```
|
|
|
|
`activate` is a first-class language construct that performs a spreading activation query over the connected Engram knowledge graph and returns a typed array of results.
|
|
|
|
At runtime, the ELVM dispatches an `Activate { type_name, query }` instruction to the Engram HTTP API at `ENGRAM_URL` (default `http://localhost:8742`). The API performs semantic search over graph nodes and returns matching nodes.
|
|
|
|
The result type is always `[TypeName]`:
|
|
|
|
```
|
|
let users: [User] = activate User where "recent premium subscribers"
|
|
```
|
|
|
|
When no Engram instance is connected, `activate` returns an empty array.
|
|
|
|
---
|
|
|
|
## 12. Graph Builtins
|
|
|
|
The following builtins provide direct access to the Engram knowledge graph. They are VM primitives dispatched by name without imports.
|
|
|
|
| Builtin | Signature | Description |
|
|
|---------|-----------|-------------|
|
|
| `graph_compile` | `(query: String, depth: Int) -> String` | Compile graph context for LLM injection |
|
|
| `graph_traverse` | `(node_id: String, depth: Int) -> List` | BFS from node, return activated nodes |
|
|
| `graph_write_node` | `(label: String, content: String, tier: String, tags: [String]) -> String` | Create a node, return UUID |
|
|
| `graph_write_edge` | `(from_id: String, to_id: String, relation: String, weight: Float) -> Bool` | Create an edge |
|
|
| `graph_search` | `(query: String, limit: Int) -> List` | Full-text search over nodes |
|
|
| `graph_get_node` | `(node_id: String) -> Map<String, Any>?` | Fetch a single node by ID |
|
|
| `graph_activate` | `(seeds: [String], depth: Int, limit: Int) -> List` | Spreading activation from seed set |
|
|
|
|
---
|
|
|
|
## 13. HTTP Builtins
|
|
|
|
| Builtin | Signature | Description |
|
|
|---------|-----------|-------------|
|
|
| `http_get` | `(url: String) -> String` | HTTP GET, returns response body |
|
|
| `http_post` | `(url: String, body: String) -> String` | HTTP POST with JSON body |
|
|
| `http_put` | `(url: String, body: String) -> String` | HTTP PUT with JSON body |
|
|
| `http_delete` | `(url: String) -> String` | HTTP DELETE |
|
|
| `http_serve` | `(handler_fn: String) -> Void` | Start HTTP server on configured port |
|
|
| `http_serve_on` | `(port: Int, handler_fn: String) -> Void` | Start HTTP server on specified port |
|
|
|
|
HTTP builtins use blocking I/O. The runtime dispatches them synchronously. The handler function receives a request map and must return a response map.
|
|
|
|
---
|
|
|
|
## 14. Standard Library Builtins
|
|
|
|
### 14.1 String Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `str_len(s)` | Length in characters |
|
|
| `str_slice(s, start, end)` | Substring |
|
|
| `str_starts_with(s, prefix)` | Boolean |
|
|
| `str_ends_with(s, suffix)` | Boolean |
|
|
| `str_contains(s, substr)` | Boolean |
|
|
| `str_replace(s, from, to)` | Replace first occurrence |
|
|
| `str_split(s, delim)` | Split into List |
|
|
| `str_join(list, delim)` | Join List into String |
|
|
| `str_trim(s)` | Strip leading/trailing whitespace |
|
|
| `str_upper(s)` | Uppercase |
|
|
| `str_lower(s)` | Lowercase |
|
|
| `str_pad_left(s, width, pad)` | Left-pad with pad character |
|
|
| `str_pad_right(s, width, pad)` | Right-pad with pad character |
|
|
| `str_format(template, data)` | `{key}` interpolation from map |
|
|
| `str_char_at(s, idx)` | Character at index |
|
|
| `str_char_code(s, idx)` | Unicode code point at index |
|
|
| `str_from_char_code(code)` | Character from code point |
|
|
|
|
### 14.2 Integer Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `int_to_str(n)` | Integer to string |
|
|
| `str_to_int(s)` | Parse integer |
|
|
| `int_to_float(n)` | Widen to float |
|
|
| `abs(n)` | Absolute value |
|
|
| `min(a, b)` | Minimum |
|
|
| `max(a, b)` | Maximum |
|
|
|
|
### 14.3 Float Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `float_to_str(f)` | Float to string |
|
|
| `float_to_int(f)` | Truncate to integer |
|
|
| `str_to_float(s)` | Parse float |
|
|
| `format_float(f, decimals)` | Format to N decimal places |
|
|
| `math_sin(f)` | Sine |
|
|
| `math_cos(f)` | Cosine |
|
|
| `math_sqrt(f)` | Square root |
|
|
| `math_pi()` | π constant |
|
|
| `decimal_round(f, places)` | Round to N decimal places |
|
|
|
|
### 14.4 List Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `list_len(l)` | Length |
|
|
| `list_get(l, idx)` | Element at index |
|
|
| `list_append(l, v)` | Append, return new list |
|
|
| `list_push(l, v)` | Alias for `list_append` |
|
|
| `list_new()` | Empty list |
|
|
| `list_join(l, delim)` | Join to string |
|
|
| `list_range(start, end)` | Integer range |
|
|
| `list_map(l, fn_name)` | Map over list |
|
|
| `list_filter(l, fn_name)` | Filter list |
|
|
| `list_reduce(l, init, fn_name)` | Reduce list |
|
|
| `list_peek_last(l)` | Last element without removing |
|
|
|
|
### 14.5 JSON Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `json_parse(s)` | Parse JSON string to value |
|
|
| `json_stringify(v)` | Serialize value to JSON |
|
|
| `json_get_string(json, key)` | Extract string field |
|
|
| `json_get_int(json, key)` | Extract integer field |
|
|
| `json_get_float(json, key)` | Extract float field |
|
|
| `json_get_raw(json, key)` | Extract raw JSON sub-object |
|
|
|
|
### 14.6 I/O Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `println(s)` | Print line to stdout |
|
|
| `print(s)` | Print without newline |
|
|
| `env(key)` | Read environment variable |
|
|
| `getpid()` | Current process ID |
|
|
| `args()` | Command-line arguments as List |
|
|
| `exit(code)` | Exit process |
|
|
|
|
### 14.7 File System Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `fs_read(path)` | Read file contents as String |
|
|
| `fs_write(path, content)` | Write string to file, return Bool |
|
|
| `fs_mkdir(path)` | Create directory |
|
|
| `fs_list(path)` | List directory entries as List |
|
|
| `fs_exists(path)` | Boolean |
|
|
|
|
### 14.8 Time Operations
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `time_now_utc()` | Current time as Unix seconds |
|
|
| `time_format(ts, format)` | Format timestamp ("ISO", "RFC") |
|
|
| `time_to_parts(ts)` | Decompose into year/month/day/etc. |
|
|
| `time_from_parts(secs, nanos, tz)` | Construct timestamp |
|
|
| `time_add(ts, amount, unit)` | Add duration ("day", "hour", etc.) |
|
|
| `time_diff(ts1, ts2, unit)` | Difference in units |
|
|
|
|
### 14.9 State Operations (Global Mutable State)
|
|
|
|
The VM maintains a global key-value string store accessible within a process lifetime:
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `state_set(key, value)` | Store string value |
|
|
| `state_get(key)` | Retrieve string value |
|
|
| `state_delete(key)` | Delete key |
|
|
|
|
### 14.10 Color/Terminal Formatting
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `color_bold(s)` | Bold ANSI formatting |
|
|
| `color_dim(s)` | Dim ANSI formatting |
|
|
| `color_red(s)` | Red text |
|
|
| `color_green(s)` | Green text |
|
|
| `color_yellow(s)` | Yellow text |
|
|
| `color_cyan(s)` | Cyan text |
|
|
|
|
### 14.11 Native List Primitives (Self-Hosting Compiler)
|
|
|
|
These primitives are used by the self-hosting compiler internals and are not available in user programs:
|
|
|
|
| Builtin | Description |
|
|
|---------|-------------|
|
|
| `native_list_empty()` | Create empty list |
|
|
| `native_list_append(l, v)` | Append to list |
|
|
| `native_list_get(l, idx)` | Element at index |
|
|
| `native_list_len(l)` | List length |
|
|
| `native_string_chars(s)` | Split string into character list |
|
|
| `native_string_contains(s, substr)` | Substring test |
|
|
| `native_str_to_int(s)` | Parse integer |
|
|
| `native_int_to_str(n)` | Format integer |
|
|
|
|
---
|
|
|
|
## 15. Compilation Model
|
|
|
|
### 15.1 Pipeline
|
|
|
|
```
|
|
source.el
|
|
→ [Lexer] → token list
|
|
→ [Parser] → AST (list of statement maps)
|
|
→ [Codegen] → JSON bytecode string
|
|
→ [Linker] → resolved imports
|
|
→ [Wrapper] → ELVM binary container (.elc)
|
|
→ [Sealer] → encrypted sealed artifact (.sealed) [prod only]
|
|
```
|
|
|
|
### 15.2 Self-Hosting Architecture
|
|
|
|
The El compiler is written in El:
|
|
|
|
- `el-compiler/src/lexer.el` — tokenizer
|
|
- `el-compiler/src/parser.el` — recursive descent parser
|
|
- `el-compiler/src/codegen.el` — bytecode emitter
|
|
|
|
These files are compiled by the Rust genesis compiler (`engrams/el-compiler/`) to produce the self-hosting compiler binary. Once bootstrapped, the self-hosting compiler compiles itself and all subsequent El programs.
|
|
|
|
The genesis Rust compiler implements the same pipeline in Rust (`el-parser`, `el-compiler` crates) as a structural mirror of the El source. Both produce identical bytecode for valid El programs.
|
|
|
|
### 15.3 Compilation Targets
|
|
|
|
| Target | Artifact | Behavior |
|
|
|--------|----------|----------|
|
|
| `debug` | `.elc` + `.map.json` | Full source maps, no dead-code elimination, type errors are warnings |
|
|
| `release` | `.elc` | No source maps, minor dead-code pruning, type errors are warnings |
|
|
| `prod` | `.sealed` | AES-256-GCM encrypted, type errors are fatal, no debug info |
|
|
|
|
### 15.4 Incremental Builds
|
|
|
|
The build system tracks a BLAKE3 hash of every source file in `.el/build-cache.json`. Only changed files and their dependents are recompiled.
|
|
|
|
---
|
|
|
|
## 16. Sealed Artifact Format
|
|
|
|
### 16.1 Purpose
|
|
|
|
The `prod` target produces sealed artifacts — bytecode containers encrypted with AES-256-GCM and authenticated with BLAKE3. Without the deployment key, the artifact is indistinguishable from random bytes. Decompilers and static analysis tools receive AES-GCM ciphertext.
|
|
|
|
### 16.2 Wire Format
|
|
|
|
```
|
|
Offset Size Field
|
|
────── ────── ──────────────────────────────────────────────────
|
|
0 8 Magic: b"ENGRAM01"
|
|
8 2 Format version: u16 big-endian (currently 1)
|
|
10 * JSON body: SealedArtifact
|
|
```
|
|
|
|
SealedArtifact JSON:
|
|
|
|
```json
|
|
{
|
|
"algorithm_id": "aes256gcm-v1",
|
|
"signature": "<base64 BLAKE3 keyed MAC>",
|
|
"encapsulated_key": "<base64 K XOR BLAKE3(deployment_material)>",
|
|
"nonce": "<base64 12-byte AES-GCM nonce>",
|
|
"ciphertext": "<base64 AES-256-GCM ciphertext + 128-bit auth tag>",
|
|
"deployment_fingerprint": "<base64 BLAKE3(binding_key) or null>"
|
|
}
|
|
```
|
|
|
|
### 16.3 Sealing Process
|
|
|
|
1. Generate a cryptographically random 256-bit symmetric key K.
|
|
2. Encrypt: `ciphertext = AES-256-GCM(K, nonce=random_96bit, plaintext=bytecode)`.
|
|
3. Derive binding hash: `H = BLAKE3(deployment_material)`.
|
|
4. Encapsulate: `encapsulated_key = K XOR H`.
|
|
5. Compute MAC: `signature = BLAKE3-keyed(K, algorithm_id ‖ nonce ‖ ciphertext)`.
|
|
6. Serialize: `ENGRAM01 ‖ version_u16be ‖ JSON(artifact)`.
|
|
|
|
### 16.4 Deployment Binding Modes
|
|
|
|
| Mode | Description | Security |
|
|
|------|-------------|----------|
|
|
| `EnvironmentKey(var)` | Key from environment variable | High |
|
|
| `MachineFingerprint` | Key from hostname + OS + architecture | Medium |
|
|
| `None` | Zero vector (development only) | None |
|
|
|
|
### 16.5 Security Properties
|
|
|
|
AES-256 provides 128-bit post-quantum security under Grover's algorithm. The `algorithm_id` field supports forward migration to ML-KEM (CRYSTALS-Kyber) without format changes.
|
|
|
|
---
|
|
|
|
## 17. ELVM Integration
|
|
|
|
El compiles to ELVM bytecode. See the ELVM specification (`elvm.md`) for the complete instruction set and execution model. Key integration points:
|
|
|
|
- The El codegen emits function registrations as `Push(Int(entry_ip)) StoreLocal("__fn_name")` stanzas that the VM's scan pass collects into the function table.
|
|
- User-defined functions are called via `Call { name, arity }` which the VM resolves first against the builtin dispatch table, then against the function table.
|
|
- The `app` block is parsed from source by the `el` binary before compilation; it does not appear in the bytecode directly.
|
|
- The `activate` construct compiles to an `Activate { type_name, query }` instruction.
|
|
- The `reason` construct compiles to a `Reason { query }` instruction.
|
|
- The `parallel` block compiles to a `Parallel { entries }` instruction.
|
|
- The `deploy` construct compiles to a `DeployFn { fn_name, route, target }` instruction.
|
|
|
|
---
|
|
|
|
## 18. Package System
|
|
|
|
### 18.1 Project Manifest — `manifest.el`
|
|
|
|
The project manifest is an El file — everything is El. The file is named `manifest.el`
|
|
and lives at the project root. It uses El block syntax: space-separated declarations,
|
|
no equals signs, strings in `"..."`, integers as bare numbers, arrays as `[...]`.
|
|
|
|
```el
|
|
// manifest.el
|
|
package "my-service" {
|
|
version "0.1.0"
|
|
description "What this does"
|
|
authors ["Will Anderson <will@neurontechnologies.ai>"]
|
|
edition "2026"
|
|
}
|
|
|
|
dependencies {
|
|
engram-http "1.2"
|
|
some-local { path "../some-local" }
|
|
}
|
|
|
|
build {
|
|
target "prod"
|
|
entry "src/main.el"
|
|
output "dist/"
|
|
seal_key "env:ENGRAM_SEAL_KEY"
|
|
}
|
|
|
|
cross {
|
|
targets ["x86_64-linux", "aarch64-linux", "aarch64-macos", "wasm32"]
|
|
}
|
|
```
|
|
|
|
Rules:
|
|
- String values use `"..."` — no equals sign
|
|
- Integer values are bare numbers — no equals sign
|
|
- Arrays use `[...]`
|
|
- Block sections use `{ }` — no `[section]` headers
|
|
- Only include sections that are relevant to the project
|
|
- The `app` section is for native desktop apps (el-ui); it sets window dimensions
|
|
|
|
### 18.2 CLI Reference
|
|
|
|
```
|
|
el new <name> scaffold new project
|
|
el add <pkg>[@ver] add dependency
|
|
el remove <pkg> remove dependency
|
|
el update update all deps
|
|
el build [--target prod] build project
|
|
el build --cross build for all cross targets
|
|
el run build debug and run
|
|
el test run tests
|
|
el check type-check only
|
|
el fmt format source
|
|
el clean clean artifacts
|
|
el publish publish to registry
|
|
el search <query> search registry
|
|
el seal <artifact> seal existing artifact
|
|
el unseal <artifact> decrypt sealed artifact
|
|
el build-file <file.el> compile single file
|
|
```
|
|
|
|
---
|
|
|
|
## 19. Grammar (EBNF)
|
|
|
|
```ebnf
|
|
program = (app_block | stmt)* EOF
|
|
|
|
app_block = "app" STRING "{" app_entry* "}"
|
|
app_entry = "version" STRING
|
|
| "config" "{" config_entry* "}"
|
|
| "secrets" "{" secret_entry* "}"
|
|
| "flags" "{" flag_entry* "}"
|
|
config_entry = IDENT ":" type_expr ("=" expr)?
|
|
| IDENT "{" (IDENT "=" expr)* "}" // env overlay
|
|
secret_entry = IDENT ":" type_expr
|
|
flag_entry = IDENT ":" "Bool" ("=" expr)?
|
|
|
|
stmt = let_stmt
|
|
| return_stmt
|
|
| fn_def
|
|
| type_def
|
|
| enum_def
|
|
| import_stmt
|
|
| while_stmt
|
|
| for_stmt
|
|
| expr_stmt
|
|
|
|
let_stmt = "let" IDENT (":" type_expr)? "=" expr ";"?
|
|
return_stmt = "return" expr? ";"?
|
|
fn_def = "fn" IDENT "(" param_list ")" "->" type_expr "{" stmt* "}"
|
|
type_def = "type" IDENT "{" (IDENT ":" type_expr ","?)* "}"
|
|
enum_def = "enum" IDENT "{" variant* "}"
|
|
variant = IDENT ("(" type_expr ")")? ","?
|
|
import_stmt = "import" STRING ";"?
|
|
| "from" IDENT "import" "{" (IDENT ","?)* "}" ";"?
|
|
while_stmt = "while" expr "{" stmt* "}"
|
|
for_stmt = "for" IDENT "in" expr "{" stmt* "}"
|
|
expr_stmt = expr ";"?
|
|
|
|
param_list = (param ("," param)*)?
|
|
param = IDENT ":" type_expr
|
|
|
|
type_expr = IDENT
|
|
| "[" type_expr "]"
|
|
| type_expr "?"
|
|
| "Map" "<" type_expr "," type_expr ">"
|
|
| "fn" "(" (type_expr ("," type_expr)*)? ")" "->" type_expr
|
|
|
|
expr = null_coalesce_expr
|
|
null_coalesce_expr = or_expr ("??" or_expr)*
|
|
or_expr = and_expr ("||" and_expr)*
|
|
and_expr = eq_expr ("&&" eq_expr)*
|
|
eq_expr = cmp_expr (("==" | "!=") cmp_expr)*
|
|
cmp_expr = bitwise_expr (("<" | ">" | "<=" | ">=") bitwise_expr)*
|
|
bitwise_expr = add_expr (("&" | "|" | "^" | "<<" | ">>") add_expr)*
|
|
add_expr = mul_expr (("+" | "-") mul_expr)*
|
|
mul_expr = unary_expr (("*" | "/" | "%") unary_expr)*
|
|
unary_expr = "!" unary_expr | "~" unary_expr | postfix_expr
|
|
postfix_expr = primary ("." IDENT | "(" arg_list ")" | "[" expr "]" | "?" | "as" type_expr)*
|
|
|
|
primary = INT | FLOAT | STRING | BOOL
|
|
| "(" expr ")"
|
|
| "[" arg_list "]"
|
|
| "{" (STRING ":" expr ","?)* "}"
|
|
| "if" expr "{" stmt* "}" ("else" "{" stmt* "}")?
|
|
| "match" expr "{" match_arm* "}"
|
|
| "while" expr "{" stmt* "}"
|
|
| "for" IDENT "in" expr "{" stmt* "}"
|
|
| "activate" IDENT "where" STRING
|
|
| "sealed" "{" stmt* "}"
|
|
| "parallel" "{" (IDENT ":" expr ","?)* "}"
|
|
| "reason" STRING
|
|
| IDENT ("::" IDENT)*
|
|
|
|
arg_list = (expr ("," expr)*)?
|
|
match_arm = pattern "=>" expr ","?
|
|
pattern = "_" | IDENT | INT | STRING | BOOL
|
|
```
|