Compare commits

..

451 Commits

Author SHA1 Message Date
bigmerge 26af149aa1 lang: rebuild the bootstrap compiler against merged dev
El SDK CI - dev / build-and-test (pull_request) Failing after 4m46s
The binary was stamped before dev advanced (vindex publication landed in
el_runtime.c and engram_vindex.c). Rebuilt against the merged runtime so the
committed compiler matches the runtime it ships beside. Fixpoint re-verified
byte-identical; test_compiler 82/82; engram/src/server.el still compiles and
still emits its 18 config declarations.
2026-08-16 11:38:52 -05:00
bigmerge c18abf799c engram: declare configuration once instead of at every read site
Migrates engram to the `program` block. 18 configuration variables that each
carried their default inline at the point of use now declare it in one place,
and engram declares itself a singleton.

The read sites lose their defaults entirely: `let v = env("X")` followed by
`if str_eq(v,"") { "default" } else { v }` collapses to `config("X")`. The
guide_env_or(key, dflt) helper is deleted -- its whole job was supplying a
per-site default, which is the thing being removed.

Fixes ENGRAM_DATA_DIR, which was the clearest instance of the defect. It was
read at six sites. Five were dead: `let dir_raw = env("ENGRAM_DATA_DIR")`
immediately shadowed on the next line by `engram_resolve_data_dir()`. The sixth
was live and defaulted to /tmp/engram, contradicting the canonical resolver's
$HOME/.neuron/engram -- and its consumer is the pre-destructive reseed backup,
so with ENGRAM_DATA_DIR unset the safety copy was written to ephemeral storage
while the store it protected lived elsewhere. All six now go through
engram_resolve_data_dir().

ENGRAM_DATA_DIR is deliberately NOT declared in the program block, and the
source says why: engram_resolve_data_dir() already owns it, and a second
declaration would give it two owners that can disagree -- recreating the exact
defect being removed here. A variable belongs in the block when the block would
be its only owner. HOME stays a raw env() read; it is an environment fact, not
configuration.

singleton: "engram" matters more than it looks. Today a second engram whose
bind() fails merely returns from http_serve -- after it has already replayed
the WAL and written boot-time backup files -- and then exits 0, indistinguishable
from a clean run. That is how two instances came to share one data dir. Verified
that the second instance now refuses before any side effect: with instance 1
holding the lock (lsof pid, shell pid, and lock file contents all agreeing at
5946), the second start named that pid, exited 1, and left the data directory
untouched.

Verified by bijection on the generated C: 18 config() reads, 18 declarations,
no read without a declaration and no declaration without a read. Three bad Int
values are reported in a single run rather than costing one restart each.

ENGRAM_API_KEY keeps its permissive empty default, which disables auth -- that
is pre-existing behaviour and changing it is out of scope. The source marks
making it `required` as the obvious hardening follow-up.
2026-08-16 11:38:28 -05:00
bigmerge b305b49f40 lang: re-stamp the bootstrap compiler so the tree can compile its own source
server.el declares a `program` block, which the previously committed elc cannot
parse. Without this the tree is internally inconsistent: source in the repo that
the compiler in the repo rejects.

This is the documented re-stamp from BOOTSTRAP.md / AGENTS.md, and its
precondition is met -- the self-hosting fixpoint was verified byte-identical
(stage3 output == stage2 output) both before installing and again with the
installed binary. tests/native/test_compiler.el passes 82/82 against it.

Two pre-existing failures are unchanged and are NOT from this work, confirmed
by rebuilding them against the original runtime: test_env's
"state_keys returns JSON array" fails identically before and after, and
test_json/test_state fail to link on symbols (json_build_array, state_has) that
were never prototyped -- the same class of gap as config(), which this branch
fixed because it blocked the build.
2026-08-16 11:38:28 -05:00
bigmerge 8ae163e8e5 lang: give cross-cutting concerns an owner instead of a convention
El's units of encapsulation are the function and the module. Neither can hold
a concern that belongs to the process, so each one had been expressed the only
way it could be -- as a convention: call this at every site. Conventions of
that shape do not hold. Measured here: zero process-identity guards at any
layer, 20 environment variables each with its default written inline at the
read site, 62 persist call sites, 10 per-route auth checks. One absence, four
times.

Step 0 first, because the premise was wrong. El was believed to have no
middleware or effect mechanism. It has one, and it is already load-bearing:
codegen injects engram_boundary_beat at the entry of every @manager/@accessor
fn, decorators take arguments and stack, dharma_emit from a non-@manager fn is
a #error, and the cgi block injects el_cgi_init at the head of main(). So the
correct move was not to invent a mechanism but to generalize the seam that
already existed. The real gap is narrower and is now recorded: the seam is
prologue-only and its callee is a fixed builtin.

Adds a `program` block -- the third program-level declarative block. cgi and
service declare what a program may do; program declares what it is.

  program "engram" {
      singleton: "engram"
      env ENGRAM_BIND: String = ":8742"
      env GUIDE_PORT:  Int    = "8771"
  }

singleton takes an exclusive flock before any user statement runs and refuses a
second start, reporting the holder's pid. It is a lock rather than a pidfile so
the kernel releases it on death including SIGKILL -- no stale state, and so no
"delete the lock file to get unstuck" ritual, which would itself be a
convention. It reports the pid because "already running" is not actionable; a
pid is. That is the direct answer to a stale process surviving a pkill and
going on answering probes.

env entries resolve once at startup -- environment wins, declaration supplies
the fallback -- and validate as a whole, reporting every problem at once rather
than costing one restart per variable. config("X") for an undeclared X is
fatal, because an advisory schema is just another convention. Programs without
a program block are unaffected, so migration is per-program.

Only one keyword is added. `config` and `env` could not become keywords -- both
are real identifiers in the tree -- so the block's fields are read as
identifier token values by its own parse loop and stay usable everywhere else.

The init function is emitted at the block site and called from main() rather
than inlined into main(). The live backend is codegen_streaming, which emits in
source order and cannot hold the entry list alive until main(); this way only a
single bool has to survive.

Also fixes: config() was defined in el_runtime.c but never prototyped in
el_runtime.h, so any el program calling it failed to compile under C99.

Spec: section 18 documents what shipped. Section 9 is corrected -- it claimed
decorators had no structural meaning, which has not been true for some time.
Section 19 designs durability-as-an-epilogue-effect and route authorization
and states plainly why neither is implemented here: both land in files under
concurrent modification, and the prerequisite for both is lifting the seam
from prologue-only to prologue/epilogue.

Self-hosting fixpoint verified byte-identical.
2026-08-16 11:38:28 -05:00
will.anderson a6cef4b983 runtime: publish the vector index instead of guarding it (#143)
El SDK CI - dev / build-and-test (push) Failing after 10m29s
2026-08-16 16:33:28 +00:00
Neuron 8e9d88fc01 runtime: publish the vector index instead of guarding it
El SDK CI - dev / build-and-test (pull_request) Failing after 10m59s
The crash (SIGTRAP in engram_activate -> eg_vindex_sync -> vindex_insert ->
_realloc) had three read paths mutating five process-global statics.
engram_activate, eg_knn_for_node (whose own comment says "No writes.") and
engram_geo_reify_run_json all called eg_vindex_sync, which frees the index,
reallocs the seen-map and inserts — on a read.

Three moves, in decreasing order of how much they dissolve:

1. Misfiled scratch is not shared state. visited/visit_epoch/visited_cap
   were never owned by the index; they are one traversal's local, hoisted
   into struct VIndex as an allocation optimisation. They want neither a
   lock nor a capability nor a pool — just to go back in the call frame.
   Two concurrent READS stomped each other purely because of this.

2. const IS the capability. Once the scratch leaves the struct, search
   reads and nothing else, so vindex_search takes a const VIndex*. That is
   exactly what a capability-pointer ABI would have bought — a read path
   physically cannot call vindex_insert, enforced by the compiler on every
   future caller — for one qualifier instead of an ABI swept across
   hundreds of builtins.

3. What survives is publication, not ownership. HNSW insert is NOT an
   append: it rewires the neighbour links of already-existing elements and
   reallocs elems[], so the store's append-only property does not transfer
   to the index derived from it. eg_vindex_sync therefore splits into
   eg_vindex_maintain (exclusive, sole mutator) and eg_vindex_view (shared,
   returns const VIndex*). A read path may demand that a current snapshot
   exist — a request to the owner, not a mutation by the reader.

Write-side owner: eg_vindex_note_embedded hooks the embedding-ASSIGNMENT
sites rather than the append sites, because a node with no embedding cannot
be in a vector index — embedding assignment is the event that owns index
membership. One O(log n) insert, no O(node_count) presence scan. This also
retires the "STALENESS (honest tradeoff)" note where a lazily-embedded
older node stayed invisible to route_nearest/autoconnect until a full
rebuild (the embed-gap #20 shape).

Evidence. The existing harness conflated two hazards, which is why fixing
half of it read as failure. Split into four:

  single (3000 vec, ASan+UBSan)          clean  ->  clean
  readers (4 readers, no writer, TSan)   RACE   ->  clean
  unsynchronized (writer+reader, bare)   race   ->  race, expected forever
  published (owner + 4 readers)          n/a    ->  clean, 3000/3000 landed

RESULT: PASS. recall@10 = 0.9365 at ef_search=128 (gate >= 0.90);
determinism byte-identical across two independent builds.

The unsynchronized half is now permanently expected to race, deliberately:
it is the executable proof that the boundary must live above the data
structure, not inside it.

fb32d15's guard is KEPT, correcting this design's own section 5. Measured,
it guards TWO structures and only one was converted here: g->nodes/g->edges
are realloc'd in place (el_runtime.c:7618,7629) and engram_activate_inner's
embed-backfill writes n->emb through exactly such a borrowed pointer.
Deleting the guard reintroduces a measured 11171->9579 edge loss. Its
comment is narrowed to the RAM graph and the deletion precondition named.

That corrects the ordering claim too: the residual is not one ABI that
dissolves everything at once, it is a PROPERTY applied per structure.
Residues evaporate in the order the property is applied, and a residue
whose structure has not been converted must be left standing.
2026-08-16 11:29:17 -05:00
bigmerge e99a4640e2 test: regression harness for the vindex concurrency crash
Promotes the two throwaway sanitizer harnesses used to diagnose the
2026-08-16 soul crash into engram/test/ so the bug cannot silently regress.

The harness has two halves and the PAIR is the point — it is what localises
the defect to concurrency rather than to HNSW logic:

  single      3000 clustered vectors, one thread, ASan+UBSan. The CONTROL.
              Must always be clean. During diagnosis this cleared all 13,820
              real dim-768 vectors from the live store, which DISPROVED an
              inspection-derived hypothesis about an out-of-bounds
              reverse-link write at engram_vindex.c:340.

  concurrent  writer + reader on one shared index, TSan. Currently reports a
              race at engram_vindex.c:195 (visited_reset) reached from both
              vindex_search and vindex_insert, because VIndex still owns its
              visited[]/visit_epoch scratch — so even two concurrent READS
              corrupt each other's traversal.

Verified: half 1 passes, half 2 reproduces the race.

Gated on EXPECT_RACE, default 1, so the concurrent half documents the known
defect without failing the suite today. When the visited set moves to a
per-query checkout pool (hnswlib VisitedListPool style — NOT thread_local,
since http_worker is a thread per connection and a __thread buffer would leak
~55KB per connection), flip EXPECT_RACE=0 and it becomes a real gate.
2026-08-16 11:29:17 -05:00
bigmerge bdc1f99fb9 runtime: guard engram activation against the unsynchronized awareness thread
The soul daemon had two engram callers and only one of them locked.
soul.el:729 starts the HTTP server via http_serve_async (spawning
http_worker threads); soul.el:731 then runs awareness_run() on the MAIN
thread. awareness.el's perceive() -> engram_activate_json() ->
engram_activate() -> eg_vindex_sync() -> vindex_insert() mutates the same
g->nodes/g->edges and the process-global _eg_vindex HNSW index that the
workers touch. g_engram_req_lock existed to serialize exactly this, but it
was only ever taken inside http_worker: engram_req_lock/engram_req_unlock
appear in ZERO .el sources, so the awareness loop ran lock-free beside the
workers on every tick (SOUL_TICK_MS=1000).

Result was a crash-loop under launchd KeepAlive: five crashes in ~4 minutes
on 2026-08-16 with varying faulting frames -- search_layer<-vindex_insert
<-eg_vindex_sync, engram_activate, abort, and one inside xzm_realloc's own
freelist. Varying sites plus a fault in allocator metadata means heap
corruption. The SIGSEGV address 0x65646f4e6d617267 is little-endian ASCII
"gramNode": string bytes dereferenced as an Elem vector pointer.

Diagnosed by bisection rather than inspection:
  - Replaying all 13,820 real dim-768 vectors harvested from the live store
    through the index single-threaded under ASan is 100% clean, which rules
    out an HNSW logic/bounds bug.
  - Two threads on one index trip ThreadSanitizer immediately at
    engram_vindex.c:195 (visited_reset), reached from both vindex_search and
    vindex_insert. VIndex keeps a SHARED visited-epoch scratch buffer, so
    even two concurrent READS corrupt each other's traversal and walk bogus
    element indices.
So this is purely a concurrency defect, not an HNSW logic error. (An
inspection-derived hypothesis about an out-of-bounds reverse-link write at
engram_vindex.c:340 was disproved by the single-threaded run.)

Fix: a thread-local ownership depth (_eg_req_depth) lets engram entry points
self-guard. engram_activate() becomes a wrapper over engram_activate_inner()
that acquires g_engram_req_lock when called with depth 0 (the awareness
thread) and passes through when depth > 0 (nested inside an http_worker that
already holds it), so the non-recursive mutex cannot self-deadlock. The depth
is a plain counter, never a recursive-mutex count, preserving
engram_self_reify_beat_json's contract of genuinely releasing the lock
mid-beat.
2026-08-16 11:29:17 -05:00
will.anderson 44b621e551 runtime: anchor the think read, so Neuron can think at all (#142)
El SDK CI - dev / build-and-test (push) Failing after 13m0s
2026-08-16 16:26:00 +00:00
Neuron ded6ca546f runtime: anchor the think read, so Neuron can think at all
El SDK CI - dev / build-and-test (pull_request) Failing after 13m24s
engram_think_json passed NULL as the anchor. NULL is not "no opinion":
engram_think re-origins at `anchor ? anchor : region->centroid`, so NULL
means "read from the centroid" — and the centroid is the one point where
the gradient is zero by construction. r = x - centroid = 0, so every axis
projection is 0, grad is 0, and direction takes the "at rest" branch at
engram_cognition.c:137.

Measured consequence: EVERY faculty returned an identical null result,
differing only in its label —
  {"direction":[0,0,0,0,0,0,0,0],"spread":0,"magnitude":1,"confidence":0.5}
magnitude 1 is membership evaluated at the centroid, spread 0 is its
distance to itself, confidence 0.5 is the stance fallback. The geometry was
never at fault: /api/drift computes real values (centroid_sep 0.104,
core_disp 0.045) over the very same 87 members. Neuron could not think
because the read was always taken from the region's own centre.

The seeds choose WHICH region; they must also supply the VANTAGE. Anchor at
the first resolvable embedded seed — the same seed eg_geo_build_desc infers
dim from, so the two can never disagree. One seed still yields a real
gradient because the descriptor expands to that seed's neighbourhood, so
the seed's position is distinct from the neighbourhood centroid.

The vector is COPIED, never borrowed: g->nodes is realloc'd in place on
append, so a borrowed EngramNode* dangles across any concurrent write.

Verified against a clone of the production store (13,616 nodes / 37,865
edges):
  self anchor   n_support 87  magnitude 0.00282  spread 18.79
  values hub    n_support 28  magnitude 0.00318  spread 17.72
with distinct unit direction vectors. Previously both returned the zero
vector with magnitude 1 and spread 0.

STILL OPEN, now isolated by this fix: all five faculties return identical
numbers and confidence stays 0.5, because cog_stance_init is passed NULL
for the stance and the faculty enters the computation only through the
stance's axis_gain[] and bias_dir. The faculty label is inert until a
stance is loaded — which is what learn()'s correspondence-beat calibrates.
Same shape as this bug: a neutral parameter collapsing a capability to a
constant.
2026-08-16 11:25:24 -05:00
will.anderson b5b96c05ed runtime: let signal enter as geometry, not as prose about signal (#141)
El SDK CI - dev / build-and-test (push) Failing after 10m36s
2026-08-16 16:13:28 +00:00
Neuron c79033b749 runtime: let signal enter as geometry, not as prose about signal
El SDK CI - dev / build-and-test (pull_request) Failing after 10m55s
No ingest path could carry a vector. engram_node/_full/_layered take text
only, and a node acquired an embedding solely via engram_embed_backfill
DERIVING one from n->content. That made text the mandatory entry medium:
any non-text modality had to be described in prose first, so the geometry
we then reasoned over was the geometry OF THE DESCRIPTION, not of the
signal. Measured: POST /api/nodes accepted an "emb" field, returned 200
with a fresh id, and stored nothing — emb_dim=None, embedded=false.

engram_node_set_emb attaches a vector to an existing node. Off-dimension
vectors are stored but not indexed (the HNSW build loop already filters on
emb_dim), so modality geometry is durable and addressable without
perturbing the canonical index. Setting emb also makes the node ineligible
for embed_backfill, so a realizer's vector is never overwritten by a
text-derived one.

Two reporting fixes ride along, because both are how the drop stayed
invisible: the create response now reports emb_set instead of being
success-shaped regardless, and the node document now always emits emb_dim
and embedded — without which a genuine ingest drop and a mere reporting
gap are indistinguishable.

Verified live: voice node emb_dim=64 embedded=true; text control emb_dim=0
embedded=false; malformed hex, length mismatch and dim<=0 all reject.

KNOWN PLACEMENT DEFECT: this is at the consumer. Ingest is a language
concern, not an engram feature — every el program touching any modality
needs it. The vector also marshals as a hex STRING because el has no
first-class geometry value, which reintroduces text as the transport
medium one layer below the problem being fixed. The durable shape is
geometry as an el value plus declarable realizers, after which the engram
stops having an ingest concept at all. Landing this as the verified probe
that proves the path.
2026-08-16 11:12:48 -05:00
will.anderson 1119295238 Merge pull request 'runtime: state_get leaked its value on every call' (#140) from fix/state-get-leak into dev
El SDK CI - dev / build-and-test (push) Failing after 14m5s
2026-08-16 13:09:58 +00:00
bigmerge 9c07970943 runtime: state_get leaked its value on every call
El SDK CI - dev / build-and-test (pull_request) Failing after 14m26s
char* result = el_strdup_persist(e ? e->value : "");   // never freed
    pthread_mutex_unlock(&_state_mu);
    char* copy = el_strdup(result);                        // arena-tracked
    return el_wrap_str(copy);

Two copies were made. `result` existed only as the source for `copy` — never
returned, never freed — and el_strdup_persist bypasses the arena BY DESIGN
("state_set, engram internals"), so arena-pop could never reclaim it. Every
state_get leaked its full value string, permanently.

MEASURED: 200,000 state_get calls against a 64-byte value.
    before   15 MB peak RSS growth   (~75 bytes/call — the value plus overhead)
    after     0 MB

IMPACT. The soul's awareness loop has 68 state_get call sites and ticks every
200ms. Live measurement before the fix: RSS climbing 112 MB per 20s, about
19 GB/hour, in awareness_run -> one_cycle -> perceive, while node_count stayed
flat at ~13,479 — growth with no data behind it. It drove the host from 20 GB
free to 4.3 GB in roughly an hour.

WHY NOW, since the code is old: the soul used to restart constantly (no
write-through, divergent graph, 2.11 GB). Stabilising it (neuron #162) let it
stay up long enough to accumulate. The fix did not cause this leak; it removed
the crashes that were hiding it. Same pattern as the test framework surfacing
math_log — the defect was always there, something finally made it visible.

Found by Ishikawa rather than by reading the nearest code: method (arena
push/pop IS correctly paired per tick), material (node count flat, so not data
growth), environment (19 GB/hr / 18,000 ticks = ~1.1 MB per tick, so per-tick
not one-shot), machine (an allocator that bypasses the arena) — which is where
the evidence pointed.

el_strdup tracks into the thread-local arena, which touches no shared state, so
taking the single copy under _state_mu is safe and removes the temporary
entirely.

Verified: self-hosting fixpoint byte-identical; state round-trip correct for
hit, miss, and overwrite.
2026-08-16 08:09:32 -05:00
will.anderson 0832865952 Merge pull request 'test framework phase 3/4: black_box barrier + three-signal complexity gate, armed' (#139) from wt/soul-runtime-reconcile into dev
El SDK CI - dev / build-and-test (push) Failing after 11m32s
2026-08-16 03:02:18 +00:00
Neuron e0b2c0ea54 bench: arm the Phase 4 gate -- proven to pass clean AND fire on a quadratic
El SDK CI - dev / build-and-test (pull_request) Failing after 12m7s
Adds tests/native/test_lexer_scaling.el, the regression gate for el #132.

Both directions are proven on LIVE workloads, not synthetic series:
  healthy per-character scan  1821 3251 6007 10422 us -> O(n)   PASS
  rescan-from-zero (the #132 shape)  922 3667 13524 44792 -> O(n^2) FAIL

A gate only proven to pass is decoration. The quadratic specimen exists so
the gate is proven to FIRE.

Also fixes elb_spread_ok to judge the ASYMPTOTIC TAIL (last three ratios)
rather than the whole sweep. Measured on a genuinely linear scan the ratios
ran 3.37 2.92 1.76 1.65 -- the head looks quadratic because it is cold
cache, the tail is the truth. Whole-sweep spread rejected correct data. A
complexity bound is an asymptotic claim and must be judged asymptotically.

That fix came from the classifier refusing to rubber-stamp my own bad
measurement: it reported INDETERMINATE on an unwarmed sweep rather than
passing it. Warmup is now taken and discarded at every sweep point.

Reverts the == workarounds in test_elbench.el now that el #137 has landed;
the natural form generates no str_eq and all 13 fitter tests stay green.
The  workaround remains -- the Plus arm is still open.
2026-08-15 21:58:46 -05:00
bigmerge cf060adbfd Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:55:40 -05:00
will.anderson 63fe8a766d Merge pull request 'codegen: Bool is int-like, so Bool comparisons stop lowering to str_eq' (#138) from fix/bool-is-int-like into dev
El SDK CI - dev / build-and-test (push) Failing after 4m28s
2026-08-16 02:54:34 +00:00
bigmerge b5a0a729e6 codegen: Bool is int-like, so Bool comparisons stop lowering to str_eq
El SDK CI - dev / build-and-test (pull_request) Failing after 14m49s
fn check(label: String, cond: Bool, want: Bool) -> Void {
        if cond == want { ... }        ->  if (str_eq(cond, want))   SIGSEGV
    }

Bool has always been an integer in the value model — type_to_c maps Bool to
"int", and el_runtime.h states "Bool -> el_val_t (0 = false, nonzero = true)".
But Bool names were registered NOWHERE: build_int_names_for_params tracked Int
and Float params, and the `let` path tracked Int and Float bindings. Neither
knew about Bool.

So comparing two Bools fell through to str_eq, which dereferenced 0 or 1 as a
char* and segfaulted immediately.

This is the third instance of one family found tonight, after el #137 (a call
on either side of == poisoned the operator) and el #136 (a missing import
compiled clean). All three are the same shape: something the compiler could not
type, silently handled as a string.

Found while writing #137's own test harness — the first version of that harness
crashed on exactly this, on both the old and new compiler, which is how it
surfaced. A test harness that cannot compare two Bools is a good way to notice.

VERIFIED:
  - the harness that segfaulted on every prior compiler (exit 139, no output)
    now runs clean: 14 passed, 0 failed
  - self-hosting fixpoint byte-identical
  - the compiler's own generated C differs by 8 lines — only the intended
    registration
  - neuron's full soul amalgam regenerates in 424ms, exit 0, BYTE-IDENTICAL
  - test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12

Adds tests/runtime/operator_typing_test.el, the 15-case suite from #137, so
this family is covered going forward rather than rediscovered.
2026-08-15 21:54:10 -05:00
will.anderson b26dd47aef Merge pull request 'codegen: either side Int is enough for == and !=, not both' (#137) from fix/eq-operand-inference into dev
El SDK CI - dev / build-and-test (push) Failing after 12m0s
2026-08-16 02:52:02 +00:00
bigmerge b55e6bfd53 codegen: either side Int is enough for == and !=, not both
El SDK CI - dev / build-and-test (pull_request) Failing after 12m20s
let a: Int = 5
    getint(5) == a      ->  str_eq(getint(5), a)      SIGSEGV
    getint(5) == 5      ->  getint(5) == 5            fine

A function call whose return type codegen cannot infer poisoned the operator,
and a declared Int on the other side did not save it. str_eq then read an
integer as a char* and segfaulted. Only an integer LITERAL on one side forced
the numeric form, which is why the bug stayed invisible: the common case
happened to be safe.

The check required BOTH operands to be provably Int:

    if is_int_expr(left) { if is_int_expr(right) { numeric } }

Loosening to OR is strictly safer, not a trade:
  - when one side is a known Int, str_eq is ALWAYS wrong — it dereferences
    that integer — while numeric comparison is at worst a wrong answer on a
    program that was already ill-typed;
  - when neither side is Int nothing changes at all, so string comparison is
    untouched.

Found by the test-framework agent while building the benchmark harness; it
correctly declined to fix it mid-phase since it is a codegen semantics change.

VERIFIED, because a semantics change earns more than an assertion:
  - 15/15 on a dedicated operator suite covering string literals, string vars,
    string-returning calls, mixed var/call, and != in every combination. The
    pre-change compiler scores 0/15 on the same file: it segfaults before
    printing anything.
  - self-hosting fixpoint byte-identical
  - the ONLY difference in the compiler's own generated C is the intended one:
    a nested if becoming two sequential ifs, in EqEq and NotEq. Nothing else
    moved.
  - neuron's full soul amalgam regenerates in 400ms, exit 0, output
    BYTE-IDENTICAL at 1,270,212 bytes
  - test_math 13/13, test_string 27/27, test_core 10/10, test_text 12/12 —
    62 tests, 190 assertions, zero failures

NOT fixed here, same family, flagged for a decision: Bool PARAMETERS are not
tracked as int-like, so `cond == want` between two Bool params still lowers to
str_eq and segfaults. Found while writing this commit's own test harness — the
first version of it crashed on exactly that, on both the old and new compiler.
It needs the same treatment, and it wants its own change.
2026-08-15 21:51:35 -05:00
will.anderson dbb06f6ee4 Merge pull request 'compiler: a missing import is an error, not an empty string' (#136) from fix/missing-import-is-an-error into dev
El SDK CI - dev / build-and-test (push) Failing after 11m0s
2026-08-16 02:48:01 +00:00
bigmerge 906c664a65 compiler: a missing import is an error, not an empty string
El SDK CI - dev / build-and-test (pull_request) Failing after 11m23s
import "../../NOPE/does_not_exist.el"

compiled CLEANLY — exit 0, empty stderr, and a program silently missing
everything it imported.

resolve_imports did `fs_read(src_path)` and used the result without checking.
fs_read returns "" both for "file is empty" and "file does not exist", so a
typo, a moved file, or a relative path resolved from the wrong working
directory all produced a successful build of nothing.

It caused a real wrong conclusion during test-framework work: a bisection run
from a subdirectory where ../../runtime/ did not resolve produced ELEVEN
consecutive "successful" compiles that had included no runtime at all, and the
results were believed before anyone noticed.

Missing dependency, confident success — the same shape as a test suite
reporting pass for tests that never ran, and as a benchmark reporting 0us
because the optimiser deleted the loop.

fs_exists separates the two cases, so a legitimately empty file still resolves
to "" and is fine. A path that does not exist now prints the resolved path and
exits 1, which is what build scripts check.

Verified:
  - bad import: exit 1 (was 0), message names the resolved path
  - elc-cli.el still compiles, self-hosting fixpoint byte-identical
  - neuron's full soul amalgam regeneration: exit 0, 405ms, output
    byte-identical at 1,270,212 bytes
2026-08-15 21:47:32 -05:00
Neuron 6a6b589ba0 bench: real black_box barrier + three-signal growth-curve gate
Adds el_black_box (inline asm, +r constraint, memory clobber) and
runtime/elbench.el: a growth-curve classifier that gates time AND
allocation-count AND allocation-bytes, failing if any exceeds its
declared curve.

Refusal is a first-class verdict. The classifier REFUSES rather than
classifying when the largest measurement is below the floor, or when a
series is hard-flat across an 8x input range -- the shape produced when
the optimiser deletes the work. Reporting O(1) there would be a
confident answer with nothing behind it. Disagreeing ratios report
INDETERMINATE rather than a guess.

Deviation from DESIGN.md 6.2, stated in the source: uses consecutive
ratios on a mandated geometric sweep rather than least-squares over
candidate curves. Ratios are directly interpretable on a doubling sweep
and need no floating point; the cost is weaker O(n) vs O(n log n)
separation, reported as an ambiguous band rather than guessed.

Documents the counter scope limit: engram_*.c and libcurl malloc are
NOT tracked, so a flat curve over engram/HTTP-dominated work is not
evidence of anything.

13 tests prove the classifier against real measured series from
fitprobe.el -- including that an accumulator's allocation COUNT is
linear while its bytes are quadratic, and that el #132's pure-CPU shape
reads FLAT on both allocation signals and is caught only by time.
2026-08-15 21:45:13 -05:00
bigmerge b5d1e53902 Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:38:11 -05:00
will.anderson 9e96d74f6a Merge pull request 'runtime: count container allocations too, not just strings' (#135) from feat/alloc-accounting-containers into dev
El SDK CI - dev / build-and-test (push) Failing after 11m49s
2026-08-16 02:37:14 +00:00
bigmerge a8908908df runtime: count container allocations too, not just strings
El SDK CI - dev / build-and-test (pull_request) Failing after 12m11s
el #131 instrumented the four string allocators, which meant list- and map-heavy
code reported ZERO allocations — a benchmark over lists would have been fitted
against a flat line and passed anything. Caught during framework work: a
"linear" specimen read 0 allocs until it was rewritten to allocate strings.

A gate is only as good as its blind spots are small, and a signal that silently
reads zero is worse than no signal: it produces a confident pass.

Now counted at every container allocation — ElList and ElMap bodies, their
backing arrays, the copy-on-write clones, and the realloc growth path.

Verified on an append loop (n = 100..800):
    allocs  7, 8, 9, 10          +1 per doubling = O(log n) reallocations
    bytes   2048, 4096, 8192, 16384   exactly 2x per doubling = O(n)

Both curves are what correct amortized growth should look like, and both read
zero before this change.

Known remaining scope, stated rather than left implicit: these counters cover
the runtime's own allocations. They do not see malloc inside engram_*.c or
libcurl, which is correct — the gate is for El-level complexity, not for
third-party memory behaviour.
2026-08-15 21:36:52 -05:00
Neuron 6291a35bb9 design: gate on THREE signals -- the alloc gate would have missed el #132
el #132's quadratic (strlen per character in str_char_code/str_slice) is
pure CPU and allocates NOTHING. Measured on three controlled specimens:

  specimen  allocs        bytes         time
  linear    2.00 -> O(n)  2.16 -> O(n)  2.05 -> O(n)
  accum     2.00 -> O(n)  3.99 -> O(n2) noisy
  compute   FLAT          FLAT          3.96 -> O(n2)

'compute' is #132's shape. A gate fitting only allocation count and bytes
classifies it FLAT and passes -- it would not have caught the defect it
was created for. The gate now fits time AND count AND bytes, failing if
any exceeds its declared curve.

Also: black_box is mandatory and consuming the result is NOT sufficient.
The first 'compute' reported 0us at every n while returning a correct n2 --
clang closed the loop to a multiply. Only an opaque call restored the curve.

Adds lang/tests/bench/fitprobe.el as the fitter's known-good/known-bad set,
so the classifier is provable without depending on a real bug existing.
Marks DESIGN.md 1.3 stale: test_compiler 3.58s -> 0.03s (119x).
2026-08-15 21:34:39 -05:00
will.anderson a69a4a5894 Merge pull request 'runtime: math_log is base-10, not natural log' (#134) from fix/math-log-base10 into dev
El SDK CI - dev / build-and-test (push) Failing after 14m48s
2026-08-16 02:34:09 +00:00
bigmerge edafd8cce8 runtime: math_log is base-10, not natural log
El SDK CI - dev / build-and-test (pull_request) Failing after 10m8s
el_val_t math_log(el_val_t f) { return el_from_float(log(el_to_float(f))); }
    el_val_t math_ln(el_val_t f)  { return el_from_float(log(el_to_float(f))); }

Both were natural log, so math_log and math_ln were the same function.
log10(100) returned 4.605 instead of 2.

Three sources already agreed it should be base-10 and were being contradicted
by this one line:
  - runtime/math.el:55  "// math_log — base-10 logarithm."
  - el_seed.c:1278      __log_f -> log10()  (the path math.el actually calls)
  - tests/native/test_math.el:133  asserts log10(100) == 2

FOUND BY THE NEW TEST FRAMEWORK ON ITS FIRST RUN (el #133). The assertion had
been sitting in the suite the whole time; nothing could report it. The old
harness printed "N passed, M failed" with no per-test detail, and half the
suites were not compiling at all — so a failing assertion in a suite nobody
could run was indistinguishable from no failure.

That is the entire argument for the framework, demonstrated on day one: this is
not a bug the framework introduced, it is a bug the framework made VISIBLE.

Verified: tests/native/test_math.el goes 12/13 -> 13/13, math-log passing.
2026-08-15 21:33:20 -05:00
will.anderson 5e3e69d326 Merge pull request 'test framework phase 1: compile-time registry + El-side runner with per-test timing' (#133) from wt/soul-runtime-reconcile into dev
El SDK CI - dev / build-and-test (push) Failing after 10m17s
2026-08-16 02:31:08 +00:00
bigmerge 4c3414072b Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:30:52 -05:00
will.anderson 0288024396 Merge pull request 'compiler: fix the quadratic — strlen() on every character access' (#132) from fix/compiler-quadratic-strlen into dev
El SDK CI - dev / build-and-test (push) Failing after 4m2s
2026-08-16 02:29:02 +00:00
Neuron 3e7ab07e82 test framework phase 1: forward decls, void-return fix, suite migration
El SDK CI - dev / build-and-test (pull_request) Failing after 10m4s
Completes the Phase 1 runner and migrates the 11 test files onto it.

- forward-declare the registry accessors in the test preamble; they are
  defined at the end of the unit but the El runner is compiled in between
- eltest.el: explicit trailing return in the void emit_* helpers, which
  otherwise lower to 'return println(...)' and fail to compile
- test files import runtime/eltest.el explicitly, using the language's own
  textual import mechanism rather than compiler-side auto-injection
- DESIGN.md 6.5: gate on allocation COUNT AND BYTES, not count alone

Verified: self-hosting fixpoint byte-identical (gen2 == gen3). 6 of 11
suites run and report per-test timing. The other 5 fail to COMPILE, and
fail identically under the committed compiler -- pre-existing breakage
this framework makes visible for the first time.
2026-08-15 21:28:30 -05:00
bigmerge d231b7e5e7 compiler: fix the quadratic — strlen() on every character access
El SDK CI - dev / build-and-test (pull_request) Failing after 10m21s
THE BUG. str_char_code() and str_slice() each called strlen() on every
invocation. The lexer walks source one character at a time, so every character
access rescanned the whole remaining input: O(n) per character over n
characters = O(n^2).

    el_val_t str_char_code(el_val_t s, el_val_t i) {
        ...
        int64_t n = (int64_t)strlen(str);   // <- O(n), every call
        if (idx < 0 || idx >= n) return 0;
        return str[idx];
    }

HOW IT WAS FOUND. Not by reading code — by sampling the running process, which
is the same method that resolved tonight's engram outage after four wrong
theories. A geometric sweep of synthetic sources showed wall-clock rising 3.0x,
3.0x, 4.0x, 4.14x per doubling (converging on 4x = quadratic), and a stack
sample put 779 of 779 samples inside lex(), every one bottoming out in
_platform_strlen via str_char_code and str_slice.

THE FIX. Remember the length instead of recomputing it. The subtlety is
INVALIDATION: El strings are arena-allocated, so a freed pointer can be reused
for a different string at the same address, and a naive pointer-keyed cache
would hand back a stale length and read past the end of the new string —
trading a performance bug for a memory-safety one. So entries carry a
generation, a hit requires pointer AND generation to match, and every path that
frees or mutates a runtime string bumps the generation: el_arena_pop,
seed_request_end, __str_set_char. Stale entries cannot be believed; they miss
and recompute.

MEASURED, same host, same inputs:

    n(fns)    before     after
      512      0.10s     0.01s
     1024      0.37s     0.02s
     2048      1.51s     0.03s     50x

    the compiler's own 422 KB source concatenated (DESIGN.md's 3.58s case):
              3.55s ->  0.03s      118x

The speedup GROWS with input size, which is the signature of removing a
complexity class rather than a constant factor. After the fix each doubling
adds ~0.01s: linear.

CORRECTNESS, verified rather than assumed:
  - byte-identical output on every sweep input (n = 128..2048)
  - byte-identical output on the 422 KB compiler concatenation
  - byte-identical output on tests/runtime/string_test.el
  - self-hosting fixpoint byte-identical
  - new tests/runtime/str_cache_test.el: 17 assertions covering bounds, empty
    strings, negative indices, slice clamping, distinct strings not sharing a
    cached length, 1000 interleaved strings forcing cache-slot collisions, and
    a grown string not reporting its old length. All pass.

This is the defect that made dist/soul.c a committed artifact: elc could not run
in CI because it needed 24 GB+ and minutes. It needs neither now.
2026-08-15 21:28:23 -05:00
bigmerge a668062e38 Merge remote-tracking branch 'origin/dev' into wt/soul-runtime-reconcile 2026-08-15 21:24:03 -05:00
Neuron 24fac765a6 test framework phase 1: compile-time registry + El-side runner
Replace the hardcoded test harness main() with a generated static registry
and index-based accessors, and move all reporting into runtime/eltest.el.

The old harness inlined direct calls into main() and counted assertions in
two globals. That shape cannot report which test failed, how long any test
took, or whether a test ran at all -- a misspelled registration reported
success for a test that never executed.

- assertions record into per-test state instead of global counters
- registry table emitted at compile time; discovery strictly precedes
  execution, which is what later enables --list, filtering and sharding
- per-test wall timing on CLOCK_MONOTONIC, taken in C around the call
- runner in El: structured NDJSON events as source of truth, human output
  rendered from the same fields
2026-08-15 21:24:03 -05:00
will.anderson cb1f2a74af Merge pull request 'runtime: allocation accounting — deterministic signal for complexity gating' (#131) from feat/alloc-accounting into dev
El SDK CI - dev / build-and-test (push) Failing after 3m49s
2026-08-16 02:22:13 +00:00
bigmerge 37bcf7eb74 runtime: allocation accounting — the deterministic signal for complexity gating
El SDK CI - dev / build-and-test (pull_request) Failing after 12m7s
Implements the three primitives the test-framework design (DESIGN.md §6.5)
requires for gating on growth curves: el_alloc_count, el_alloc_bytes,
el_peak_rss. Registered in codegen's builtin_arity and wrapped in el_seed.c per
the project's C-builtin recipe.

WHY COUNTS AND NOT WALL-CLOCK: a growth-curve gate has to be a hard build
failure, which means the signal cannot flake. Wall-clock needs warmup,
statistics, and a quiet machine; on shared CI it is unusable as a gate.
Allocation counts are perfectly deterministic — same input, same number, every
machine, every run. Fit them against n and a complexity regression becomes a
build failure with zero noise.

All four runtime string allocators (el_strdup, el_strbuf, and their _persist
variants) funnel every allocation the language performs, so instrumenting there
counts everything.

WHY BYTES AS WELL AS COUNT — this is not redundancy, it is the whole gate.
Measured with two El programs, one allocating once per item, one rebuilding its
accumulator each iteration:

    n     linear allocs / bytes      quadratic allocs / bytes
    100        100 /    290               100 /   5,150
    200        200 /    690               200 /  20,300
    400        400 /  1,490               400 /  80,600
    800        800 /  3,090               800 / 321,200

The quadratic program's allocation COUNT is exactly linear — identical to the
healthy one. Counting allocations alone would have missed it completely. Bytes
catch it: each doubling of n quadruples bytes (ratios 3.94, 3.97, 3.99 ->
converging on 4.0, i.e. O(n^2)), while the linear case converges on 2.0.

That shape — count linear, per-allocation size growing — is the classic
accidental quadratic, and it is exactly elc's defect: quadratic allocation
VOLUME, which the old shipped compiler paid in RSS (27 GB, OOM) and the rebuilt
one pays in malloc/free churn (42s on 1.4 MB). Volume was the invariant across
both; RSS and wall-clock were just the two ways it surfaced.

el_peak_rss is exported for context and is explicitly NOT a gating signal — it
is perturbed by allocator internals, the page cache, and the OS. Gate on the
deterministic numbers; report the physical one.

Counters are unsynchronised by design: this is measurement, and a lock would
change the thing being measured. Exact on the single-threaded compile path,
approximate under threads.
2026-08-15 21:21:42 -05:00
will.anderson 2240d26c32 Merge pull request 'store: judge memory pressure by swap RATE, not level' (#130) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 11m44s
2026-08-16 02:12:22 +00:00
bigmerge 19cc99e57d store: judge memory pressure by swap RATE, not swap level
El SDK CI - dev / build-and-test (pull_request) Failing after 11m59s
The guard I added minutes ago checked swap availability as a level
(avail < total/8 -> report zero available). That is the wrong signal, and the
same host proved it twice within minutes:

    47.65 / 48.00 GiB swap used, 2047 swapouts/s  -> genuinely thrashing
    26.67 / 28.00 GiB swap used,    0 swapouts/s  -> healthy, 15.6 GiB free

Both are ~97% "used". macOS grows swap files on demand and trims them lazily,
so the level says almost nothing about now — it is a high-water mark. The level
check calls the second state an emergency and starves the pool for no reason,
which is its own failure mode: a guard that fires on healthy machines gets
disabled, and then guards nothing.

What separates the two is whether pages are moving. So sample the swapout
counter across calls and judge the delta:

  - > 200 pages/s (~3 MiB/s) sustained outward paging => report zero available;
    callers refuse to grow and pc_relieve_pressure hands frames back.
  - The first call primes the baseline and reports no pressure. One sample
    cannot have a rate, and inferring one from a single reading is exactly the
    mistake this commit removes.

Measured thresholds, not guessed: idle sat at 0/s, recovery burst hit 24,845/s
while the compressor drained (transient, correctly not a growth decision since
growth is only evaluated on eviction passes), and real thrash held ~2000/s.
200/s sits clearly above noise and far below either.

The compressor-footprint subtraction stays: that RAM is genuinely spoken for
regardless of paging rate.
2026-08-15 21:12:00 -05:00
will.anderson f39ae40047 Merge pull request 'store: bound the pool by available memory and let it shrink' (#129) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 4m43s
2026-08-16 02:02:24 +00:00
bigmerge e52415f0e0 store: bound the pool by AVAILABLE memory and let it shrink
El SDK CI - dev / build-and-test (pull_request) Failing after 11m47s
The adaptive budget I added an hour ago could only grow, and grew toward a
share of TOTAL ram (80%, ~38 GiB on a 48 GB host). That is a memory leak with
extra steps: total never shrinks when other processes need memory, so the pool
had no way to notice it was starving the machine it runs on. Deployed briefly;
caught as memory pressure on the host.

A control loop with only one direction is not a control loop.

  - pc_available_ram(): free + inactive + purgeable via host_statistics64 on
    Darwin, MemAvailable on Linux. Availability is the quantity that moves when
    the machine is under pressure; total is not. Returns 0 when it cannot be
    read, and callers then refuse to grow — a cache is never worth swapping the
    host, so unknown means no.

  - Growth is bounded by availability minus a free-memory floor (2 GiB default,
    ENGRAM_POOL_FREE_FLOOR_MB), not by total. The share-of-total ceiling stays
    as a second bound and drops 80% -> 50%.

  - pc_relieve_pressure(): the missing direction. On every eviction pass, if
    available memory is under the floor, hand back ~25% of held frames; the
    resident set follows on the next pass so the memory is actually returned
    rather than merely re-labelled. Counted as adapt_shrinks alongside
    adapt_grows so both directions are visible in the same report.

  - pc_default_cap() also clamps the STARTING budget to what is spare right
    now, so a cold boot on a loaded machine does not open at a size the host
    cannot afford.

Verified on a 48 GB host: engram boots in ~30s, RSS settles at 2.22 GiB (the
store's actual size, resident, not creeping), 0.0% CPU, 13,439 nodes / 37,670
edges, embeddings complete. Guard reports 9.71 GiB available against a 2.00 GiB
floor — 7.71 GiB of headroom it is permitted to use and no more.
2026-08-15 21:02:05 -05:00
will.anderson 7a479111ac Merge pull request 'store: extend the write barrier to edges — kills the full-store walk' (#128) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 14m27s
2026-08-16 01:44:33 +00:00
bigmerge e917b3d439 store: make the buffer pool sense its own state and correct from it
El SDK CI - dev / build-and-test (pull_request) Failing after 14m35s
Follow-on to the edge write barrier. That fix removed the full-store walk;
this one makes the pool able to notice if anything like it happens again.

WHAT WENT WRONG, precisely: the pool thrashed the live engram to a standstill
twice on 2026-08-15 and said nothing. From outside it was indistinguishable
from "busy loading" — 100% CPU, flat RSS, no output — so four wrong theories
got tried (bad binary, corrupt snapshot, WAL replay, feature flags), each
costing a deploy or a rollback. The whole time, hits/misses/evictions were
already being counted in PgCache, and the struct comment read:

    /* stats (introspection only — never affect semantics) */

That comment was the bug. Self-measurement treated as decoration is why the
pool could not correct itself and why no one outside could see what it was
doing. A system that cannot read its own state cannot correct, and neither can
anyone watching it.

  - pc_adapt_budget(): the loop, closed. Over a sliding window, evictions
    running at a large fraction of accesses WHILE reuse is real means the
    working set exceeds the budget — so grow it, geometrically, bounded by a
    LIVE re-read of physical memory. Evictions alone are not pressure (a scan
    evicts and never returns); evictions with reuse are. An explicit
    ENGRAM_POOL_FRAMES still wins — an operator override must not be silently
    overruled.

  - Budget derived, not declared. A constant cannot be right: 16 GiB of frames
    is arbitrary on a 48 GB host and suicidal on a 16 GB one. Even "60% of RAM
    at startup" is a guess about the future — it cannot know the store grew or
    the machine changed. Hence the live re-read.

  - pc_report(): ONE structured emission carrying the entire sensed state,
    through emit_log — El's existing telemetry, already exporting to OTLP.
    Deliberately not a function per stat, and deliberately not a bespoke
    /api/pool endpoint: both make observability something hand-written per noun
    instead of the uniform mechanism every component already has.

  - engram_pool_stats_json(): the same state readable live, wired through the
    normal builtin path (codegen arity + el_seed wrapper), so the pool can be
    observed in real time rather than reconstructed afterward from a stack
    sample.

Verified: with the exact configuration that took production down
(ENGRAM_POOL_FRAMES=65536 → 1 GiB cache against a 2 GiB store) the engram boots
clean and serves — 0.0% CPU, 13,436 nodes / 37,663 edges, embeddings complete —
and NO pressure event fires, because the barrier removed the walk that caused
it. The controller is defense in depth; the barrier is the fix.
2026-08-15 20:44:23 -05:00
bigmerge 777ccc02f0 store: extend the durable-hash write barrier to edges (kills the full-store walk)
El SDK CI - dev / build-and-test (pull_request) Failing after 10m45s
Checkpointing pushes the ENTIRE resident graph through store_put_node and
store_put_edge (see engram_store_checkpoint). Nodes were cheap: a durable-hash
compare skipped unchanged records with zero page I/O. Edges had no barrier at
all — struct comment at PgCache.barrier_on even says "node durable-hash
barrier" — so every edge was rewritten on every checkpoint, and each rewrite
runs the idempotency probe max_page_lsn_for_id -> btree lookup -> page_read.

Edges outnumber nodes ~3:1 here (37,663 vs 13,436), so routine checkpointing
degenerated into a FULL-STORE WALK in id order: random page access across the
whole 2 GiB store, repeated, overwhelmingly to rediscover nothing had changed.
LRU is worst-case under exactly that pattern — it evicts the page it is about
to want — so once the page cache was smaller than the store, the walk collapsed
into thrashing: 100% CPU, flat RSS, no forward progress, port never bound.
That took the live engram down twice on 2026-08-15.

The walk is the defect. Sizing the cache to survive it treats the symptom.

Changes:
  - dh_edge_hash(): edge counterpart of dh_node_hash, with a kind discriminator
    byte so an edge can never collide with a node of the same id in the shared
    map. created_at/updated_at/last_fired are excluded deliberately: last_fired
    is touched by activation without changing what the edge IS, and folding it
    in would defeat the barrier on precisely the hot edges that most need it.
  - store_put_edge(): barrier check + dh_set on success, mirroring
    store_put_node exactly.
  - store_scan_edges(): seed the barrier map from on-disk truth at load, so the
    FIRST post-boot checkpoint already skips unchanged edges. store_scan_nodes
    already did this and its comment says why; edges were simply never done.

Verified: with the exact configuration that killed production
(ENGRAM_POOL_FRAMES=65536 -> 1 GiB cache against a 2 GiB store), the engram now
boots clean and serves — LISTENING, 13,436 nodes / 37,663 edges, embeddings
complete, 0.0% CPU, RSS 1.14 GiB (cache resting at its budget rather than
thrashing against it). Same small cache, same store, no walk.
2026-08-15 20:38:11 -05:00
will.anderson c21074b547 Merge pull request 'runtime: engram_edges_json — kill the whole-graph file round trip' (#127) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 10m19s
2026-08-16 01:13:44 +00:00
bigmerge 4e24d7d3f1 runtime: engram_edges_json — read edges without a whole-graph file round trip
El SDK CI - dev / build-and-test (pull_request) Failing after 13m4s
/api/graph/edges answered a read query by calling engram_save() to serialize
the ENTIRE graph to disk (128 MB) and then fs_read-ing it back. Two defects in
one line, and both bit production on 2026-08-15:

  1. The path it wrote was ~/.neuron/engram/snapshot.json — the engram
     server's CANONICAL store. A READ route overwriting the persistence
     owner's canonical file. This defect had been fixed once (export moved to
     a scratch path); it came back when the hand-written dispatch block was
     replaced by @route dispatch and the unfixed copy is the one that
     survived the merge.
  2. Cost: a full snapshot write, a 128 MB read, and a parse of the whole
     graph, per request, to return a bounded slice.

Calling it tonight overwrote the canonical snapshot and immediately preceded
an engram crash loop.

engram_edges_json(limit, offset) is the builtin that route's own TODO asked
for ("Future: add an engram_edges_json() builtin and drop the file round trip
entirely"). It walks g->edges directly and emits every persisted field.

limit <= 0 defaults to 1000, not unbounded: this is the endpoint that fell
over, and an unbounded default would preserve the failure mode under a new
name. Callers page explicitly.

Registered in codegen.el's builtin_arity (both plain and __ spellings) and
wrapped in el_seed.c per the project's C-builtin recipe.
2026-08-15 20:10:48 -05:00
will.anderson 7557ea6e19 Merge pull request 'runtime: restore engram_recall_json + cgi_* accessors (unblocks the soul build)' (#126) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 10m15s
2026-08-16 00:57:11 +00:00
bigmerge 7351fb0a8d runtime: restore engram_recall_json + cgi_* accessors
El SDK CI - dev / build-and-test (pull_request) Failing after 10m24s
neuron's soul calls engram_recall_json (neuron-api.el:618, memory.el:80) and
cgi_principal (studio.el:72). Both existed in the runtime neuron vendored
(v1.0.0-20260501) and were absent here, so the soul could not link against
current el at all.

The dangerous part is what the obvious "fix" would have done. These look like
redundant wrappers over one impl:

    engram_search_json(q, limit)  -> eg_search_json_impl(q, limit, 0)  LEXICAL
    engram_recall_json(q, limit)  -> eg_search_json_impl(q, limit, 1)  SEMANTIC

They are not interchangeable, and the split is documented at neuron-api.el:613:
search stays LEXICAL because ~40 internal call sites pass a KEY and seven of
them DELETE every record returned. Point those at a semantic matcher and they
delete fuzzy matches. Conversely, pointing recall at search silently downgrades
the mind's entire retrieval surface from semantic to lexical — no error, just
permanently worse recall.

Implemented over engram_activate(), which in this runtime already IS the
semantic path the old with_legs=1 branch built by hand (embeds the query via
eg_embed_fetch, scores by cosine, then spreads activation one hop). Output
shape matches engram_search_json — a flat array via engram_emit_node_json —
because callers parse search's shape, not activate's envelope.

Verified: neuron's soul now compiles and links against current el, boots, and
serves /health with layers initialized.

NOTE for follow-up: current el also ships engram_retrieve_geometric_json, a
structure-first retrieval that appears to be the intended successor to recall.
Repointing the two recall call sites at it may well be the right end state and
would remove the two-wrapper shape entirely — but that is a behavioral change
that must be measured against neuron/tools/retrieval-eval/'s gold set, not
assumed. This commit preserves existing behavior exactly; it does not decide
that question.
2026-08-15 19:56:43 -05:00
will.anderson d545b69614 Merge pull request 'runtime: restore the three builtins that made elc unrebuildable' (#125) from fix/elc-rebuildable-compiler-builtins into dev
El SDK CI - dev / build-and-test (push) Failing after 11m4s
2026-08-16 00:50:43 +00:00
bigmerge 598915cc61 runtime: restore the three builtins that made elc unrebuildable
El SDK CI - dev / build-and-test (pull_request) Failing after 3m52s
The committed elc binary could not be refreshed from its own source. Rebuilding
failed with three implicit-declaration errors: el_mem_check, stdout_to_file,
stdout_restore. The compiler's own source calls all three (compiler.el:472,479,574
and codegen.el:4248) and two are registered in codegen.el's builtin_arity table —
but none were defined in this runtime.

They were found intact in ui/examples/native-hello-ios/NativeHello/el_runtime.c,
a divergent private copy of this runtime that still carried them. Ported verbatim.

Consequence of them being missing: the canonical elc binary was frozen. Source
gained @route dispatch codegen (emit_route_dispatch, codegen.el:3948) and the
@manager boundary-beat seam, but no rebuilt binary could carry them, so
neuron's soul — whose routes.el now calls the compiler-synthesized
el_route_dispatch — could not be built at all.

Verified after the fix:
  - elc rebuilds from current source, clean.
  - Self-hosting fixpoint byte-identical (stage3 == stage2).
  - The rebuilt elc emits el_route_dispatch (2 occurrences in the soul amalgam,
    previously 0) and injects engram_boundary_beat at @manager boundaries,
    i.e. the decorator seam is live rather than inert.

el_mem_check is itself the compiler's memory guard (ELC_MAX_MEM_MB, default
512MB, self-terminates before the OS OOM-killer fires) — so the runtime was
missing the very guard that would have surfaced the compiler's memory blowup
as a clean error instead of a 27GB host-killer.
2026-08-15 19:50:17 -05:00
will.anderson dab14f9100 Merge pull request 'engram: fix silently-wrong query params + make el_seed.o/el_runtime.o link' (#124) from fix/engram-query-param-and-seed-link into dev
El SDK CI - dev / build-and-test (push) Failing after 14m30s
2026-08-16 00:34:33 +00:00
bigmerge 40eb48e92f engram: fix silently-wrong query params, and make el_seed.o + el_runtime.o link
El SDK CI - dev / build-and-test (pull_request) Failing after 14m49s
Three real bugs, all found by actually running the thing rather than reading it.

1. query_param never URL-decoded. A GET of /api/search?q=neural%20network
   searched for the literal string "neural%20network" and returned []. Every
   multi-word search against the live engram has been silently returning empty
   results — not an error, an empty result, which is why it went unnoticed.
   Affects every GET route that reads query params, not just search.

2. query_param matched key names unanchored. str_index_of(qs, "q=") matches
   inside "faq=", so "?faq=X&q=Y" returned X for key "q". Verified live before
   the fix. Now searches for "&key=" against "&"+querystring so a match can
   only land on a real parameter boundary.

3. el_request_start/el_request_end were defined in BOTH el_seed.c and
   el_runtime.c, so linking the two objects together — which is exactly what
   the product build does — failed with duplicate symbols. el_seed.c's own
   comment already says these moved there ("formerly defined in el_runtime.c.
   Now self-contained in el_seed.c"); the el_runtime.c copies were left behind
   during that move. Removed them, kept declarations since http_worker calls
   them. Also added the three missing prototypes (engram_op_assert_json,
   engram_node_full_in, engram_connect_in) that el_seed.c wraps but never
   declared, which made it fail to compile standalone under C99+.

Verified: engram builds and links clean from canonical source; before/after
comparison on a copy of the real store shows "neural network" returning a real
match where the live build returns [], and "?faq=WRONG&q=MetaColloc" now
resolving to MetaColloc. Live engram on :8742 was never touched.
2026-08-15 19:34:04 -05:00
will.anderson c9f75e2592 Merge pull request 'engine: land op_assert + purview mutation wrappers (clean re-merge)' (#123) from merge-pr103-v2 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m39s
2026-08-15 23:27:17 +00:00
bigmerge 09dade0613 Merge remote-tracking branch 'origin/pr/103' into HEAD
El SDK CI - dev / build-and-test (pull_request) Failing after 3m56s
# Conflicts:
#	lang/AGENTS.md
#	lang/runtime/el_runtime.h
2026-08-15 18:26:57 -05:00
will.anderson 38a8e32d6c Merge pull request 'engram: batch-cosine Adapter/Strategy/Factory over ggml (supersedes #114)' (#116) from feat/engram-ggml-cosine-batch into dev
El SDK CI - dev / build-and-test (push) Failing after 3m29s
2026-08-15 23:22:56 +00:00
will.anderson 15b66c8b1a Merge pull request 'swarm: land wt/swarm-ccr onto dev (clean re-merge)' (#122) from merge-swarm-ccr-v2 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m35s
2026-08-15 23:17:56 +00:00
bigmerge 9883aa7564 Merge remote-tracking branch 'origin/wt/swarm-ccr' into merge-swarm-ccr-v2
El SDK CI - dev / build-and-test (pull_request) Failing after 4m13s
# Conflicts:
#	lang/runtime/el_runtime.c
#	lang/runtime/el_runtime.h
2026-08-15 18:17:33 -05:00
will.anderson d45a0882f3 Merge pull request 'nsbx: fail loud on daemon-not-ready + el_seed.c standalone compile' (#118) from fix/nsbx-tooling-hardening into dev
El SDK CI - dev / build-and-test (push) Failing after 3m56s
2026-08-15 23:06:01 +00:00
will.anderson 1c9de03fdb Merge pull request 'engram: make the ggml batch-cosine strategy actually compute in fp32 (recall 0.9933 -> 0.9987)' (#121) from improve/ggml-cosine-fp32-and-init into feat/engram-ggml-cosine-batch
El SDK CI - dev / build-and-test (pull_request) Failing after 3m43s
2026-08-15 23:01:29 +00:00
bigmerge c008b7228a engram: make the ggml batch-cosine strategy actually compute in fp32
#116 shipped the ggml strategy at 0.9933 id-recall against the CPU oracle
while the hand-rolled Metal kernel it replaced scored 0.9997 — a ~150x worse
error margin. That was not an inherent property of ggml. It was a usage bug in
this file, and this commit fixes it.

ggml-metal has two F32xF32 matmul kernels and picks between them purely on
ne11, the number of B rows, which for us is the query-batch size:

  ne11 <= 8  -> kernel_mul_mv_ext_f32_f32_* / kernel_mul_mv_f32_f32_*,
                templated <float, float> — genuine F32.
  ne11 >  8  -> kernel_mul_mm_f32_f32, templated
                <half, half4x4, simdgroup_half8x8, half, half2x4, ...> —
                BOTH operands narrowed to F16, despite F32 tensors on both
                sides.

The old code issued one ggml_mul_mat with ne11 = nq (300 in the benchmark),
landing squarely on the F16 path. The file's own header comment asserted the
opposite ("computes in F32 on the Metal backend"); that claim was wrong and is
replaced with the measurement.

Fix: emit ceil(nq/8) mul_mats over ne11<=8 ggml_view_2d slices of one query
tensor, all expanded into ONE graph and one ggml_backend_graph_compute, so the
node matrix is still uploaded and shared exactly once. EL_GGML_MULMAT_CHUNK
overrides the 8; setting it >= nq reproduces the old behaviour exactly, which
is also how the before/after below was measured in a single binary.

Measured, real store snapshot, 13415 live embedded nodes, dim=768, 300 real
queries, vs the CPU double-accumulated oracle (vindex_bench, offline copy of
the store — no live service touched):

  id-recall   same-rank |Δdist| max   mean
  old (ne11=300)   0.9933   6.80e-05   1.43e-05
  new (ne11<=8)    0.9987   4.77e-07   9.30e-08
  hand-rolled      0.9997   3.58e-07   7.55e-08

~145x better max error, ~154x better mean — now the same order of magnitude as
the hand-rolled kernel rather than 150x off it.

The cost is real and is documented rather than buried. Median of 15 reps of
the whole batch_multi() call, three runs: 13.2-14.4ms unchunked, 19.9-20.2ms
chunked, 17.7-18.0ms hand-rolled. Correctness costs ~+6.7ms per 300-query
batch and leaves ggml ~12% behind the hand-rolled kernel instead of ~35%
ahead. It cannot be recovered inside ggml: an fp32 matmul on Metal must
re-stream the node matrix once per <=8 queries, and ggml's Metal backend ships
no fp32 TILED matmul, so "fast" and "fp32" are genuinely exclusive there.

Two things that did NOT work, recorded so nobody retries them:

  - ggml_mul_mat_set_prec(t, GGML_PREC_F32) does nothing here. Error was
    bit-identical with and without it (1.038e-05 either way) — ggml-metal has
    no F32-accumulating mul_mm kernel to switch to. ne11 is the only lever.
  - The ACCEL/BLAS device looked excellent in an isolated compute-only probe
    (3.4-4.0ms, mean |Δdot| 1.5e-08) but is dominated on BOTH axes end-to-end
    (0.191 ms/query at 0.9973 recall vs 0.125-0.142 at 0.9987), because the
    probe was not competing for the same CPU cores the real call path is. It
    stays reachable via EL_GGML_DEVICE as a no-Metal fallback, labelled as
    measured-and-rejected, not as a recommendation.

Also corrected: the ~7.8s "cold start" blamed on this file is not this file
re-initialising per call — init was already cached. It is Apple's shader cache
missing on ggml's embedded metallib (~650 kernels), keyed on the library and
shared across processes: the first load on a machine reports
"loaded in 7.670 sec", the next run of a *different* binary reports 0.009 sec.
Once per machine per ggml version, not once per process, and not ours to fix.
Warm ggml init is 44-53ms vs 36-117ms for the hand-rolled strategy.

Loading only libggml-metal.so instead of every plugin in the directory is kept
for tidiness, and explicitly documented as NOT a speedup: 44.7-52.4ms against
46.9-58.9ms, the same number inside noise.

The -2.0 sentinel contract is unchanged and re-verified at batch sizes that
straddle the chunk boundary (1,7,8,9,16,17,33), plus NULL rows, dim
mismatches, zero-norm rows, and an all-invalid population. Notably the old
ne11=300 path fails that same check at a 2e-6 cosine tolerance with 2299
mismatches, which is an independent confirmation of the defect.
2026-08-15 17:57:09 -05:00
bigmerge 3718bf0380 runtime: port missing __channel_* primitives into el_seed.c
El SDK CI - dev / build-and-test (pull_request) Failing after 3m44s
runtime/channel.el has always called __channel_new/__channel_send/
__channel_recv/__channel_try_recv/__channel_close, but these were only ever
implemented in the pre-restructure lang/el-compiler/runtime/el_runtime.c.
When the canonical runtime was consolidated onto the release copy
(lang/runtime/el_runtime.c) and el_seed.c became the sole C dependency,
the channel implementation was never carried forward — __mutex_new made the
move, __channel_* did not. Any El program using Go-style channels currently
fails to link on dev.

Ported the working buffered-MPMC-channel implementation (mutex+condvar+
circular buffer, bounded and unbounded modes) from the old el_runtime.c
verbatim, adapted only to el_seed.c's arena API (seed_arena_track in place
of el_arena_track). Declared in el_seed.h alongside the existing mutex
primitives.
2026-08-15 17:50:04 -05:00
bigmerge 9d40f87926 ingest: unify transduce_prose/transduce_structured into one transduce()
transduce() is now THE single mechanism: one function, no content-type
branch inside it. It never asks whether `source` is prose, JSON, or
raw/opaque bytes (audio, etc.) — it runs one algorithm unconditionally:
split on "\n\n" as a universal boundary-marker check, and if that finds
no boundary, fall back to fixed 4096-char windows. Same node/edge wiring
(root -contains-> chunk, chunk -precedes-> next, "#"-prefixed chunk gets
a heading/section_of link) regardless of what's inside a chunk. Dedup is
the existing find_existing_by_content path via merge_manifold, applied
uniformly. The old transduce_structured JSON dataset/records/feature-node
interpretation is deleted outright, not just unused — a JSON file now
gets chunked and deduped like anything else, with no pre-computed
structure. All five ingest_* entry points still exist unchanged in name
and role; ingest_file/ingest_dir/ingest_url/ingest_llm now call the one
transduce() (ingest_stream builds its own turn-nodes directly and never
called either old function, so it's untouched).

This unlocks raw/opaque content (audio, or anything else with no natural
text/JSON shape) without any DSP, LLM call, or external API: transduce()
chunks it exactly like it chunks anything else. There is zero semantic
understanding of audio (or any payload) claimed or built here — any
meaning is expected to emerge later from Neuron's own existing mechanisms
(embedding, spreading activation, dedup) acting on this real geometry
over time.

Two small C builtins added to el_runtime.c/h (fs_size, fs_read_b64_chunk)
because El strings are NUL-unsafe under strlen-based ops and fs_read()'s
result silently truncates at the first embedded NUL, which is routine in
real binary/audio bytes. ingest_file compares fs_read()'s string length
against a real fs_size() stat() count; on mismatch it rebuilds the
payload as base64-encoded fixed 3072-byte windows read directly off disk
(binary-safe in C, verbatim, no invention), joined with the same "\n\n"
marker transduce()'s boundary scan already looks for. This is a
mechanical fidelity fix, not interpretation of content — transduce()
never learns a fallback happened. Registered both builtins' arity in
codegen.el; did not rebuild the elc compiler binary itself (unrelated,
pre-existing gap: self-hosting elc via el_seed.c fails on this worktree
independent of this change, reproduced with codegen.el reverted) — the
existing elc binary compiles calls to unregistered builtins via its
already-existing arity=-1 passthrough, confirmed by an actual clean
`elc ingest.el` + `cc` build against the modified el_runtime.c.

INGEST_KIND keeps existing only as an acquisition-mechanism selector
(dir/file/url/llm/stream — which RPC to use to fetch bytes), not as a
content-type flag; the redundant "structured" value (an alias for "file"
that hinted the now-deleted JSON branch) is removed. ingest_dir drops its
file-extension filter for the same reason: transduce() takes anything now.

Verification: local manifold construction confirmed correct against a
real captured audio file (will_clean.wav, 304288 bytes, and a 12288-byte
real prefix slice) — exact expected node/edge counts both times
(101 nodes/199 edges full file; 5 nodes/7 edges for the slice, matching
ceil(bytes/3072)+1 nodes and 2n-1 edges), with real, verbatim base64
content confirmed decoding back to the actual WAV header bytes. Compiles
clean via the real elc + the modified el_runtime.c/engram_*.c (built and
booted an actual sandbox engram off this exact source with `nsbx create
--branch`).

NOT verified this session, disclosed rather than papered over: end-to-end
server-confirmed persistence (a real before/after /api/stats delta, and a
fetched node by id) for the audio, prose, and JSON-fixture cases. Every
local nsbx sandbox engram tried tonight (two stock pre-#109 binaries
hitting the known O(N*D) brute-force scan bug, then a fresh #109/HNSW
binary built from current dev) took minutes-to indefinitely long on the
final /api/load-merge write's embedding step and hit the client's 60s
HTTP timeout before responding, even for a 5-node write. This is
confirmed as real (if slow) forward progress, not a hang: the sandbox's
WAL file was observed growing steadily across every attempt. The code's
own pre-existing HONESTY GATE correctly refused to report success in
every case, returning "load-merge failed: ..." with a
"nothing below this manifold was confirmed persisted by the server" note
instead — exactly as designed. This is an environment/infrastructure
limitation, not a defect introduced by this change: the engram server
binary itself is untouched by this commit.
2026-08-15 17:50:04 -05:00
bigmerge 90d3f0bc76 engram: port PR #105's 3 genuine wins onto dev's existing cosq/e_eff semantic layer
Reconciles PR #105 ("fix: engram search latency — pin embed model, cache
query embeddings, bound activate BFS") with dev's ACTUAL current
engram_activate, rather than the ancient pre-restructure snapshot #105 was
built against.

WHY THIS NEEDED RECONCILIATION, NOT A DIRECT PORT: #105's single commit
(1dc49b1) modifies `lang/el-compiler/runtime/el_runtime.c` — a path that does
not exist on dev (dev has `lang/runtime/el_runtime.c`; the restructure that
renamed it happened after #105's branch point, which traces to a July 22
merge-base, weeks before the M8/M8.1/qgate/fan-effect/adjacency-index work
this file has grown since). #105's own engram_activate is consequently the
PRE-restructure version: no adjacency index (O(E) full edge scan per hop),
no query-aware qgate, no ACT-R fan effect, no eg_edge_eff_weight, and no
awareness of dev's cosq/e_eff embedding-blend semantic layer — it built a
parallel `g_qcache`/`engram_embed_raw` mechanism from scratch against code
that no longer exists at that path. A raw merge/cherry-pick was not possible
and would have been wrong even if it were: taking #105's tree wholesale would
have thrown away everything dev grew in the meantime (qgate, fan effect,
adjacency index, and this session's own M8 HNSW vindex integration).

RECONCILIATION: kept dev's cosq/e_eff mechanism as the semantic layer
entirely intact (unchanged by this commit) and ported #105's three genuinely
additive wins on TOP of it, at their equivalent sites in the CURRENT
eg_embed_fetch/engram_activate:

  1. keep_alive:-1 on the Ollama embed request body (eg_embed_fetch) — pins
     the embed model resident so a larger generation model loading under
     unified-memory pressure can't evict it and force a cold reload on the
     next search (#105 measured ~2.2s cold vs ~0.02-0.05s warm).
  2. Query-embedding cache upgraded from dev's single-slot (`_eg_qcache_text`,
     only ever remembered the LAST query) to a direct-mapped, FNV-1a-keyed,
     1024-slot cache (reusing the existing engram_id_hash) — so the
     curiosity loop's rotating phrases actually hit the cache instead of
     evicting each other every call. Same "pointer owned by the cache, not
     freed by caller" contract as before, just per-slot instead of global.
  3. Beam cap on the layer-1 spreading-activation BFS (new
     engram_activate_beam(), tunable via ENGRAM_ACTIVATE_BEAM, default 128).
     The FIFO frontier is processed in hop-level batches (entries sharing
     .hops are provably contiguous — see the code comment); when a level
     exceeds the beam width, only the top-`beam` by activation actually
     EXPAND. Every node in an oversized level still gets reached[]/best_bg[]
     recorded (that happens at enqueue time, one level up) and appears in
     the reported/promoted set — the cap bounds associative SPREAD width
     only, never recall of what was already found. Kept as a genuine
     additional bound even though the adjacency index + qgate + fan effect
     already mitigate #105's original "hub-node explosion" failure mode for
     a different reason: those prune WHICH targets matter; this bounds
     worst-case width regardless.

Everything else in dev's engram_activate — cosq/e_eff, the qgate rescale,
the fan effect, eg_edge_eff_weight, the M8 HNSW vindex seed discovery from
the #109 reconciliation earlier this session — is untouched.

VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): cc -std=c11
-O2 clean build; booted in an isolated sandbox against a real cloned
production snapshot (13,424 nodes / 37,656 edges); ran 5 activate() calls
across rotating queries at depth 3, including the same query issued twice
non-consecutively (2nd hit landed at 476ms vs the 1st at 483ms — consistent
with a cache hit once Ollama's own warm-model latency is accounted for; no
crash, correct varied result counts (367-2610 nodes) each call; act-stats
JSON read correctly throughout.

Built on top of the M8/#109 reconciliation (bacaf3d, merged to dev as
#109) — dev's current HEAD at the time of this commit.
2026-08-15 17:50:04 -05:00
will.anderson ee39aa5f17 Merge pull request 'ingest: unify transduce_prose/transduce_structured into one transduce()' (#117) from feat/transduce-unify into dev
El SDK CI - dev / build-and-test (push) Failing after 3m55s
El SDK CI - stage / build-and-test (pull_request) Failing after 44s
2026-08-15 22:36:12 +00:00
bigmerge 2d0aef4ef8 lang: declare the el_runtime.c symbols el_seed.c's wrappers call
El SDK CI - dev / build-and-test (pull_request) Failing after 3m55s
runtime/el_seed.c does not compile standalone via the exact command
tools/install.sh uses (`cc -std=c11 -O2 -I runtime -c runtime/el_seed.c`):
51 of its __-prefixed wrapper functions (http serving, JSON access, key-val
state, URL/HTML escaping, and the whole engram_* node/edge/layer/search
surface) call unprefixed counterparts that are implemented in el_runtime.c,
not in el_seed.c itself, and el_seed.c never declared them -- a toolchain
that treats an implicit function declaration as a hard error under C11
fails the compile outright.

install.sh already compiles el_seed.c and el_runtime.c as separate objects
and archives both into libel.a, so the symbols are always present at link
time; el_seed.c alone was just missing the prototypes.

A plain `#include "el_runtime.h"` was tried first and rejected: it redefines
el_to_float/el_from_float, which el_seed.h already provides -- a real
compile error, not a style preference. Added narrow prototypes instead,
copied verbatim from el_runtime.h, for exactly the 51 symbols el_seed.c's
wrappers reference and nothing else.

Verified clean:
  - `cc -std=c11 -O2 -I runtime -c runtime/el_seed.c` (install.sh's exact
    per-file compile) -- 0 errors, 0 warnings, even with -ferror-limit=0.
  - full `tools/install.sh` run -- compiles both objects and archives them
    into libel.a successfully.

Separately (not fixed here, out of scope): AGENTS.md's documented compiler
self-rebuild command links elc-new.c against el_seed.c, but elc-new.c's own
generated `#include "el_runtime.h"` line and 3 undeclared symbols
(el_mem_check, stdout_to_file, stdout_restore -- present in neither
el_runtime.c nor el_seed.c) mean that command fails regardless of which
runtime file it's linked against; and install.sh's libel.a only archives
el_seed.o + el_runtime.o, so any program that calls into the engram_*
surface fails to link against it (el_runtime.c's engram_* wrappers need
engram_store.c/engram_geometry.c/engram_reason.c/engram_cognition.c/
engram_vindex.c, none of which install.sh compiles in). Both are real,
pre-existing, and independent of this fix -- worth their own look.
2026-08-15 17:32:58 -05:00
bigmerge 979e820f68 nsbx: fail loud on daemon-not-ready instead of printing a false success banner
Three confirmed-live bugs tonight:

- `nsbx up` printed "daemon did not become ready" immediately followed by a
  green "your sandbox is ready" banner and exited 0, because the existing-
  sandbox restart path (`daemon_alive || start_daemon`) never checked
  start_daemon's return code. `cmd_build` had the identical unguarded
  pattern, plus `cmd_run`/`cmd_validate`'s own start-if-dead calls. All four
  now `|| die` with a message pointing at daemon.log.

- `nsbx status`/`nsbx list` reported bare "state: running" for a process
  that's alive (passes kill -0) but not actually answering /api/stats --
  pegged, hung, or mid-boot. Added daemon_health(), which does the real
  stats fetch and distinguishes stopped/running/unresponsive; both commands
  now say "running but NOT RESPONDING" with a next-step hint instead of
  silently going quiet on the stats field. Reproduced live against another
  agent's actively-running (CPU-pinned, non-responsive) sandbox tonight, and
  again via a deliberate SIGSTOP on a throwaway sandbox.

- Sandboxes carried no visible signal that their binary predated a relevant
  fix. `status`/`list` now show the binary's sha + real build timestamp
  (mtime survives `cp -p`), plus a best-effort staleness note: for
  stock-prod clones, compare against the currently-configured live binary;
  for source/branch builds, compare the recorded source commit against
  local origin/dev via merge-base --is-ancestor.

Also, found live while verifying the above:

- A cold boot under concurrent sandbox/CPU load can legitimately take past
  the old hardcoded 15s readiness window. Made it configurable
  (NSBX_READY_TIMEOUT_SECS) rather than just widening the default blindly.

- cmd_create's post-boot baseline capture could silently record sbx_baseline
  as 0/0 when the stats fetch came back empty right after the auto-remerge
  step -- which would make every future `nsbx validate` zero-loss/reboot-
  prove check trivially PASS regardless of real data loss. Added a bounded
  retry and a loud warning if it still comes back empty.

- Sharpened a handful of "no such sandbox" / missing-binary errors to name
  the next command instead of just stating the failure.
2026-08-15 17:32:44 -05:00
bigmerge e29fe4fd0b ingest: unify transduce_prose/transduce_structured into one transduce()
El SDK CI - dev / build-and-test (pull_request) Failing after 3m42s
transduce() is now THE single mechanism: one function, no content-type
branch inside it. It never asks whether `source` is prose, JSON, or
raw/opaque bytes (audio, etc.) — it runs one algorithm unconditionally:
split on "\n\n" as a universal boundary-marker check, and if that finds
no boundary, fall back to fixed 4096-char windows. Same node/edge wiring
(root -contains-> chunk, chunk -precedes-> next, "#"-prefixed chunk gets
a heading/section_of link) regardless of what's inside a chunk. Dedup is
the existing find_existing_by_content path via merge_manifold, applied
uniformly. The old transduce_structured JSON dataset/records/feature-node
interpretation is deleted outright, not just unused — a JSON file now
gets chunked and deduped like anything else, with no pre-computed
structure. All five ingest_* entry points still exist unchanged in name
and role; ingest_file/ingest_dir/ingest_url/ingest_llm now call the one
transduce() (ingest_stream builds its own turn-nodes directly and never
called either old function, so it's untouched).

This unlocks raw/opaque content (audio, or anything else with no natural
text/JSON shape) without any DSP, LLM call, or external API: transduce()
chunks it exactly like it chunks anything else. There is zero semantic
understanding of audio (or any payload) claimed or built here — any
meaning is expected to emerge later from Neuron's own existing mechanisms
(embedding, spreading activation, dedup) acting on this real geometry
over time.

Two small C builtins added to el_runtime.c/h (fs_size, fs_read_b64_chunk)
because El strings are NUL-unsafe under strlen-based ops and fs_read()'s
result silently truncates at the first embedded NUL, which is routine in
real binary/audio bytes. ingest_file compares fs_read()'s string length
against a real fs_size() stat() count; on mismatch it rebuilds the
payload as base64-encoded fixed 3072-byte windows read directly off disk
(binary-safe in C, verbatim, no invention), joined with the same "\n\n"
marker transduce()'s boundary scan already looks for. This is a
mechanical fidelity fix, not interpretation of content — transduce()
never learns a fallback happened. Registered both builtins' arity in
codegen.el; did not rebuild the elc compiler binary itself (unrelated,
pre-existing gap: self-hosting elc via el_seed.c fails on this worktree
independent of this change, reproduced with codegen.el reverted) — the
existing elc binary compiles calls to unregistered builtins via its
already-existing arity=-1 passthrough, confirmed by an actual clean
`elc ingest.el` + `cc` build against the modified el_runtime.c.

INGEST_KIND keeps existing only as an acquisition-mechanism selector
(dir/file/url/llm/stream — which RPC to use to fetch bytes), not as a
content-type flag; the redundant "structured" value (an alias for "file"
that hinted the now-deleted JSON branch) is removed. ingest_dir drops its
file-extension filter for the same reason: transduce() takes anything now.

Verification: local manifold construction confirmed correct against a
real captured audio file (will_clean.wav, 304288 bytes, and a 12288-byte
real prefix slice) — exact expected node/edge counts both times
(101 nodes/199 edges full file; 5 nodes/7 edges for the slice, matching
ceil(bytes/3072)+1 nodes and 2n-1 edges), with real, verbatim base64
content confirmed decoding back to the actual WAV header bytes. Compiles
clean via the real elc + the modified el_runtime.c/engram_*.c (built and
booted an actual sandbox engram off this exact source with `nsbx create
--branch`).

NOT verified this session, disclosed rather than papered over: end-to-end
server-confirmed persistence (a real before/after /api/stats delta, and a
fetched node by id) for the audio, prose, and JSON-fixture cases. Every
local nsbx sandbox engram tried tonight (two stock pre-#109 binaries
hitting the known O(N*D) brute-force scan bug, then a fresh #109/HNSW
binary built from current dev) took minutes-to indefinitely long on the
final /api/load-merge write's embedding step and hit the client's 60s
HTTP timeout before responding, even for a 5-node write. This is
confirmed as real (if slow) forward progress, not a hang: the sandbox's
WAL file was observed growing steadily across every attempt. The code's
own pre-existing HONESTY GATE correctly refused to report success in
every case, returning "load-merge failed: ..." with a
"nothing below this manifold was confirmed persisted by the server" note
instead — exactly as designed. This is an environment/infrastructure
limitation, not a defect introduced by this change: the engram server
binary itself is untouched by this commit.
2026-08-15 17:28:19 -05:00
bigmerge b3f410fc91 engram: batch-cosine Adapter/Strategy/Factory over ggml, supersedes hand-rolled PR #114
El SDK CI - dev / build-and-test (pull_request) Failing after 4m29s
Stop hand-rolling GPU kernels for batch cosine similarity — use ggml (the
MIT-licensed compute library underneath llama.cpp, installed standalone via
Homebrew) as the preferred backend, without ripping out PR #114's
carefully-verified hand-rolled Metal shader.

Structure: one stable public adapter (eg_cosine_batch.h, zero #ifdef at call
sites) backed by three selectable concrete Strategies behind an internal
vtable (eg_cosine_batch_strategy.h) chosen by a Factory (eg_cosine_batch.c):

  - eg_cosine_batch_strategy_ggml.c    — NEW. ggml + dynamically-loaded Metal
                                          backend plugin (ggml_backend_load_all_from_path
                                          + ggml_mul_mat for the batched dot
                                          product), gather/scatter around the
                                          -2.0 sentinel contract.
  - eg_cosine_batch_strategy_metal_hand.m — PR #114's original hand-rolled
                                          Metal shader bridge, preserved
                                          almost verbatim, now one strategy
                                          among several rather than the only
                                          option. eg_cosine_batch.metal kept
                                          byte-identical to the original.
  - eg_cosine_batch_strategy_cpu.c     — universal always-false fallback
                                          (direct descendant of PR #114's
                                          eg_metal_cosine_stub.c).

Selection: EL_COSINE_BATCH_STRATEGY=ggml|metal|cpu|auto (default: ggml first,
then hand-rolled Metal, then CPU — first available wins), plus back-compat
EL_METAL_COSINE=0 to disable every GPU-backed strategy. build_vindex_bench.sh
compiles all three strategies on Darwin, CPU-fallback-only elsewhere.

vindex_bench.c now reports BRUTE-GGML and BRUTE-METAL side by side against
the same CPU oracle, on the same dataset, in one run (real numbers vs. real
store snapshot in the PR body).
2026-08-15 17:16:41 -05:00
will.anderson 0e924f7df9 Merge pull request 'engram: reconcile #105's embed-cache/beam-BFS latency fixes onto current dev' (#115) from fix/engram-search-latency-reconciled into dev
El SDK CI - dev / build-and-test (push) Failing after 4m6s
2026-08-15 22:07:45 +00:00
bigmerge 1bb1edc851 engram: port PR #105's 3 genuine wins onto dev's existing cosq/e_eff semantic layer
El SDK CI - dev / build-and-test (pull_request) Failing after 4m45s
Reconciles PR #105 ("fix: engram search latency — pin embed model, cache
query embeddings, bound activate BFS") with dev's ACTUAL current
engram_activate, rather than the ancient pre-restructure snapshot #105 was
built against.

WHY THIS NEEDED RECONCILIATION, NOT A DIRECT PORT: #105's single commit
(1dc49b1) modifies `lang/el-compiler/runtime/el_runtime.c` — a path that does
not exist on dev (dev has `lang/runtime/el_runtime.c`; the restructure that
renamed it happened after #105's branch point, which traces to a July 22
merge-base, weeks before the M8/M8.1/qgate/fan-effect/adjacency-index work
this file has grown since). #105's own engram_activate is consequently the
PRE-restructure version: no adjacency index (O(E) full edge scan per hop),
no query-aware qgate, no ACT-R fan effect, no eg_edge_eff_weight, and no
awareness of dev's cosq/e_eff embedding-blend semantic layer — it built a
parallel `g_qcache`/`engram_embed_raw` mechanism from scratch against code
that no longer exists at that path. A raw merge/cherry-pick was not possible
and would have been wrong even if it were: taking #105's tree wholesale would
have thrown away everything dev grew in the meantime (qgate, fan effect,
adjacency index, and this session's own M8 HNSW vindex integration).

RECONCILIATION: kept dev's cosq/e_eff mechanism as the semantic layer
entirely intact (unchanged by this commit) and ported #105's three genuinely
additive wins on TOP of it, at their equivalent sites in the CURRENT
eg_embed_fetch/engram_activate:

  1. keep_alive:-1 on the Ollama embed request body (eg_embed_fetch) — pins
     the embed model resident so a larger generation model loading under
     unified-memory pressure can't evict it and force a cold reload on the
     next search (#105 measured ~2.2s cold vs ~0.02-0.05s warm).
  2. Query-embedding cache upgraded from dev's single-slot (`_eg_qcache_text`,
     only ever remembered the LAST query) to a direct-mapped, FNV-1a-keyed,
     1024-slot cache (reusing the existing engram_id_hash) — so the
     curiosity loop's rotating phrases actually hit the cache instead of
     evicting each other every call. Same "pointer owned by the cache, not
     freed by caller" contract as before, just per-slot instead of global.
  3. Beam cap on the layer-1 spreading-activation BFS (new
     engram_activate_beam(), tunable via ENGRAM_ACTIVATE_BEAM, default 128).
     The FIFO frontier is processed in hop-level batches (entries sharing
     .hops are provably contiguous — see the code comment); when a level
     exceeds the beam width, only the top-`beam` by activation actually
     EXPAND. Every node in an oversized level still gets reached[]/best_bg[]
     recorded (that happens at enqueue time, one level up) and appears in
     the reported/promoted set — the cap bounds associative SPREAD width
     only, never recall of what was already found. Kept as a genuine
     additional bound even though the adjacency index + qgate + fan effect
     already mitigate #105's original "hub-node explosion" failure mode for
     a different reason: those prune WHICH targets matter; this bounds
     worst-case width regardless.

Everything else in dev's engram_activate — cosq/e_eff, the qgate rescale,
the fan effect, eg_edge_eff_weight, the M8 HNSW vindex seed discovery from
the #109 reconciliation earlier this session — is untouched.

VERIFIED (nsbx sandbox only, live :8742/:7770 never touched): cc -std=c11
-O2 clean build; booted in an isolated sandbox against a real cloned
production snapshot (13,424 nodes / 37,656 edges); ran 5 activate() calls
across rotating queries at depth 3, including the same query issued twice
non-consecutively (2nd hit landed at 476ms vs the 1st at 483ms — consistent
with a cache hit once Ollama's own warm-model latency is accounted for; no
crash, correct varied result counts (367-2610 nodes) each call; act-stats
JSON read correctly throughout.

Built on top of the M8/#109 reconciliation (bacaf3d, merged to dev as
#109) — dev's current HEAD at the time of this commit.
2026-08-15 16:55:32 -05:00
will.anderson 2555e363a6 Merge pull request 'engram: native set-based reframe_region + engine hardening (embed-gap, lazy cosine, vindex harvest)' (#109) from feat/reframe-region-setop into dev
El SDK CI - dev / build-and-test (push) Failing after 3m58s
2026-08-15 21:50:41 +00:00
bigmerge bacaf3d39c engram: reconcile M8 HNSW vindex (#109) onto current dev, restore 3 fixes the branch predated
El SDK CI - dev / build-and-test (pull_request) Failing after 4m49s
Lands feat/reframe-region-setop (PR #109: native set-based reframe_region,
decorator-as-seam @route port, teacher-summon, and the M8.1 activate-latency
work — lazy-memoized cosq via eg_cosq_at + engram_vindex HNSW-accelerated
seed discovery + vindex_harvest_from_store/vindex_bench oracle) onto dev's
actual current HEAD, plus engram-tiered-storage's still-unique test suite.

RECONCILING #109 WITH engram-tiered-storage (M4-M10 HNSW/geometry/reason/
verify work): not a two-way merge. engram_vindex.c's HNSW core (search_layer/
select_neighbors/prune_links/insert) is BYTE-IDENTICAL between the two
branches; #109's copy is a strict superset (adds vindex_harvest_from_store,
used by vindex_bench.c's brute-force-vs-HNSW oracle). engram_reason.c and
engram_verify.c are also byte-identical. #109's own branch point already
carried engram-tiered-storage's M4-M10 lineage forward, so there was nothing
left to merge into #109 for those files. The one thing engram-tiered-storage
had that #109's tree dropped: its full test suite (test_vindex.c,
test_geometry.c, test_reason.c, test_verify.c, test_m7_traversal.c, the
interoception P0-P5 tests, bufpool/compaction tests, and their run_*.sh
harnesses) — ported over here unchanged.

WHY THIS NEEDED HAND RECONCILIATION, NOT A MECHANICAL MERGE: #109's branch
forked from dev on 2026-08-14 15:40 (before restructure-adjacent history
diverged the file's merge-base for `git merge` — it presented as an add/add
conflict). A straight two-dot diff (dev tip -> PR tip) applied cleanly, but
it silently reverted THREE dev fixes landed on 2026-08-14/15, after the
branch point, that the PR's diff had no way to know about:

  1. qgate rescale (2026-08-14 self-review): PR's lazy eg_cosq_at rewrite of
     the query-aware propagation gate dropped the shift-and-floor rescale
     about ENGRAM_EMBED_S0 (measured: unrelated-pair median 0.562->raw gate
     0.67, i.e. "a small tax, not a gate"). Restored the rescale, wrapped
     around the lazy accessor -- the PR's actual improvement (WHEN cosq[oi]
     is computed) is orthogonal to WHAT it gates on and both are kept.
  2. Eviction cause decomposition (2026-08-14 self-review): dev decomposes
     wm_evicted into evict_floor/evict_cap/evict_bll so WM churn is
     diagnosable (identity: evicted == floor+cap+bll+dup_wm+dup_wm_global).
     PR's tree predates this and dropped all three counters + their JSON
     stats fields. Restored declarations, all 4 direct increment sites, the
     eg_wm_carry_over bll increment, and the act-stats JSON fields --
     alongside (not instead of) the PR's own P4 afferent / API-reshape
     counters already in that same struct/JSON.
  3. Hebbian link-formation selection (2026-08-15 self-review, TODAY): dev
     selects the STRONGEST qualifying candidate for consolidation each call;
     PR's tree predates this and reverted to hash-slot order (arbitrary wrt
     association strength) for edge formation -- the one path that writes
     PERMANENT structure. Restored the strongest-candidate while-loop,
     keeping the PR's own genuine improvement at that site
     (engram_adj_on_edge_added incremental-index append instead of a bare
     adj_dirty=1 full-rebuild flag).

engram/src/server.el's 3-way conflicts (autoconnect_on/ise_offgraph_on env
flags, /api/nodes connected-count in responses) were pure additive: dev's
side was empty, PR's side added the feature. Took PR's side whole.

VERIFIED (nsbx sandbox only, live :8742/:7770 never touched):
  - cc -std=c11 -O2, clean link against the real engram/src/server.el via
    elc, zero errors.
  - vindex_bench (built standalone, read-only harvest) against the real
    production store clone (13,671 embedded nodes, 768-dim nomic-embed-text):
    recall@10 = 1.0000 at ef 64/128/200; HNSW search 0.28-0.79ms/query vs
    2.03ms/query brute-force oracle (2.6x-7.2x). HNSW build itself: 46.5s
    for the full 13,671-node set -- see the flagged risk below.
  - Booted the reconciled binary in an isolated nsbx sandbox (:8905, cloned
    snapshot of the live store, 13,424 nodes / 37,656 edges) and called
    /api/activate for real: first call after boot 41.5s (pays the one-time
    HNSW build inline -- matches the standalone bench), second/third calls
    356ms/605ms, no crash, correct results, act-stats JSON (including the
    restored evict_floor/cap/bll fields) reads correctly.

KNOWN RISK TO FLAG BEFORE ANY LIVE CUTOVER (not fixed here; out of scope for
this dev-only land per instructions not to touch :8742/:7770): eg_vindex_sync
builds the HNSW index synchronously, inline, on the first engram_activate()
call after every process start (or index invalidation). On the real node
count that is a ~46s blocking stall on a single-threaded server -- the first
request after every restart (or its concurrent siblings) waits the full
build. Recommend a background/incremental build (or a bounded per-call build
budget) before this ever reaches the live daemon. See PR description / final
report for the fuller writeup.
2026-08-15 16:46:44 -05:00
will.anderson 1db5694189 Merge pull request 'engram: add /api/nodes/reseed so a node body can be repaired at its own id' (#92) from feat/engram-reseed-route into dev
El SDK CI - dev / build-and-test (push) Failing after 3m58s
2026-08-15 19:59:34 +00:00
will.anderson e34ebd4b3d Merge pull request 'nsbx + cognitive architecture design + engram self-review series' (#113) from feat/neuron-sandbox into dev
El SDK CI - dev / build-and-test (push) Failing after 4m2s
2026-08-15 19:59:07 +00:00
will.anderson 69870ac883 Merge pull request 'fix(codegen): emit the declared cgi identity — it was searched for in a list that cannot contain it' (#89) from fix/cgi-identity-emission-clean into dev
El SDK CI - dev / build-and-test (push) Failing after 3m35s
2026-08-15 19:58:16 +00:00
will.anderson 274765e0aa Merge pull request 'Add native EL afferent organ: ingest (conscious) + transduce (invisible mechanism)' (#98) from worktree-agent-a1bb8ac67d9006e08 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m59s
2026-08-15 19:57:57 +00:00
will.anderson d4a04bb944 Merge pull request 'swarm: native interruptibility for dispatched agent workers' (#107) from worktree-agent-a6177cda24c71d1df into dev
El SDK CI - dev / build-and-test (push) Failing after 3m55s
2026-08-15 19:57:10 +00:00
will.anderson 905c707d68 Merge pull request 'peripheral: own-core, consent-gated I/O organ (mic/camera/speaker)' (#112) from worktree-agent-af50f3458d7754f19 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m42s
2026-08-15 19:55:51 +00:00
will.anderson 56455740e3 Merge pull request 'elp: native audio/image efferent surfaces + projector proof-of-shape' (#111) from worktree-agent-aaf04b0a9714c4070 into dev
El SDK CI - dev / build-and-test (push) Failing after 4m4s
2026-08-15 19:55:38 +00:00
will.anderson 840e54c7ac Merge pull request 'elp: native speech synthesis + voice-imitation faculty' (#110) from worktree-agent-acc02900ef4ade35e into dev
El SDK CI - dev / build-and-test (push) Failing after 4m13s
2026-08-15 19:55:20 +00:00
will.anderson 5c6da24033 Merge pull request 'spec: grounded edge-propagation (task #50) — gated design artifact' (#108) from worktree-agent-a6577c8211c332c5b into dev
El SDK CI - dev / build-and-test (push) Failing after 3m42s
2026-08-15 19:55:02 +00:00
will.anderson 09ae14a970 Merge pull request 'fix: float arithmetic codegen (segfault/garbage) and math_log aliasing' (#104) from worktree-agent-a456e0cf8cd2ee361 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m57s
2026-08-15 19:54:46 +00:00
will.anderson 7b3f8f2ce8 Merge pull request 'sandbox: multi-repo stack worktree composer (el-stack / neuron-stack)' (#101) from worktree-agent-ac2381b0b9615ab20 into dev
El SDK CI - dev / build-and-test (push) Failing after 3m50s
2026-08-15 19:54:29 +00:00
will.anderson 45f64f3fac Merge pull request 'elp: native-EL language faculty — comprehension, propositions, multilingual, translation' (#100) from integration/langfaculty-20260814 into dev
El SDK CI - dev / build-and-test (push) Failing after 4m13s
2026-08-15 19:54:14 +00:00
will.anderson fdf0d6cb64 Merge pull request 'nsbx: one-command dev onboarding (branch + worktree + isolated engram)' (#99) from feat/nsbx-dev-env into dev
El SDK CI - dev / build-and-test (push) Failing after 13m56s
2026-08-15 19:52:35 +00:00
will.anderson c2d8a07c7b transduce: name the invisible mechanism, fix a silent-failure bug, drop a CRUD verb
El SDK CI - dev / build-and-test (pull_request) Failing after 14m6s
ingest and transduce are complements, not synonyms: ingest is the conscious,
deliberate act of pointing at a source (ingest_file/dir/url/llm/stream stay
named exactly that); transduce is the automatic, invisible mechanism inside
it that converts extracted surface content into geometry (renamed
build_prose/build_structured -> transduce_prose/transduce_structured, the
functions that actually turn raw text into a node+edge manifold).

Real bug found and fixed along the way: the final /api/load-merge response
was never checked for an error. A total failure (bad auth, network down,
anything) silently reported nodes_added:0/edges_added:0 — indistinguishable
from a benign 'everything was already known' outcome. Verified live: with a
wrong key, the tool now honestly returns {"error":"load-merge failed:
unauthorized",...} instead of a misleading zero.

Also dropped a CRUD-verb smell: the per-decision println said CREATE (a
database-log verb for something that hasn't actually been written to the
server yet — it's a local, tentative decision pending the batch merge).
Renamed to FORM. The dead-code eg_create_node (defined, never called)
renamed to eg_crystallize_node and annotated honestly as unused, since if
it's ever wired up it represents the real server-confirmed write, unlike
the local FORM guess.

Not yet re-verified end-to-end against a real successful write: the
ingest-test sandbox (nsbx up ingest-test) is itself currently broken —
it prints a green "ready" banner after its own readiness check fails,
and nothing is actually listening. Filed separately; not in scope here.
2026-08-15 14:44:18 -05:00
bigmerge d8d1b89143 Add repo AGENTS.md and two engram design docs (architecture hardening, DB tooling)
El SDK CI - dev / build-and-test (pull_request) Failing after 14m16s
AGENTS.md: root-level guide to the repo — which of the 8 el_runtime.c
copies is the one canonical, authored source (lang/releases/v1.0.0-20260501,
despite the misleading 'releases/' name) vs. lagging forks/build artifacts,
build commands, and session protocol.

engram/spec/architecture-hardening.design.md: terse engineering anchor for
the 2026-08-14 hardening vision (one calculus over the geometry, core +
ephemeral ring, persistence earned by salience, incarnation model) —
indexes the fuller whitepaper + Neuron artifact 2b8078cf rather than
restating them.

engram/spec/engram-db-tooling-design.md: high-level design for engram DB
tooling (geometry-native browse/query/ops surface over the existing
vantage-read/write/relate/supersede API).

Deliberately leaves out of this commit: the uncommitted el_runtime.c/h +
codegen.el float-arithmetic-codegen diff in this worktree, which appears
to overlap with (or supersede) the fix already preserved via PR #104 —
needs manual reconciliation rather than a second competing PR. Also
leaves out lang/.promote-backup-floatfix/ (a local backup snapshot,
confirms that float-fix work is mid-promotion here), assorted .DS_Store
files, engram/dist/engram.* backup binaries, and lang/dist backup
binaries — none of it source.
2026-08-15 14:29:59 -05:00
bigmerge 6f3d692784 Add peripheral — own-core, consent-gated I/O organ
El SDK CI - dev / build-and-test (pull_request) Failing after 14m23s
939-line Swift I/O organ (mic/camera capture, speaker playback via
AVFoundation/CoreAudio), own-core LPC voice synthesis/imitation,
consent-gating, and full-duplex barge-in conversation — closing the
hear -> understand -> speak loop entirely on-device.

.gitignore in this dir already excludes bin/ (build output), out/
(captured media), and .consent.json/.resume.json (local runtime state),
so only src + README + .gitignore are committed here.
2026-08-15 14:28:14 -05:00
bigmerge b7e2c580a8 Add native speech synthesis and voice-imitation faculty
El SDK CI - dev / build-and-test (pull_request) Successful in 6m28s
speech.el: formant/glottal integer DSP synthesis + voice-analyze-by-
imitation. voice-profile.el / voice-ingest.el: voice-profile plumbing.
accent.el: British-RP as an ingested transform-geometry (explicitly marked
provisional/citation-pending by its own comments). organ-read.el:
engram read-through for the speech organ. Includes demo/test drivers and
non-personal reference data (British-RP phonetics/lexicon derived data,
a public-domain LibriVox RP reference recording).

Deliberately excludes elp/data/live/ (raw recorded voice + face-photo
samples of the repo owner) and the will-*.{json,psv} derived voiceprint
files — personal biometric data that shouldn't be committed to a shared
repo without an explicit decision from the owner. Also excludes this
worktree's elp/src/surface-profile.el, which diverges from the copy in
other worktrees (agent-aaf04b0a9714c4070, main) — needs manual
reconciliation before landing, left out here to avoid silently picking a
version.
2026-08-15 14:27:52 -05:00
bigmerge 827257d3a4 Remove __pycache__ .pyc files accidentally included in the projector commit
El SDK CI - dev / build-and-test (pull_request) Successful in 6m28s
2026-08-15 14:27:23 -05:00
bigmerge 4bbfdcceff Add native audio/image efferent surfaces + projector proof-of-shape
audio-surface.el / image-surface.el: own-core additive-synthesis WAV and
raster-PNG renderers (integer-only DSP, since EL has no floats), rendered
from learned engram signatures via a pluggable surface-profile
abstraction (surface-profile.el). audio-demo.el / image-demo.el are
drivers. NOTE: demo files hardcode absolute paths to this worktree's own
directory — will need a path fixup before landing.

elp/projector/ is a Python package the author's own README marks as
"STAGING/PROOF-OF-SHAPE — not the deliverable", superseded by the native
.el surface-profile work above; kept as a validated architecture proof.
Generated output (elp/faculty/{out,sig}, elp/projector/out,
__pycache__) intentionally excluded.
2026-08-15 14:26:59 -05:00
bigmerge 08cbcef5d9 engram: fix lazy-embed index gap (#20) and make activate's cosine scan lazy; extract vindex harvest primitive with a bench/oracle harness
El SDK CI - dev / build-and-test (pull_request) Successful in 6m42s
Adds an O(1) "seen" bitmap so lazily-embedded older nodes get picked up
incrementally instead of only on a full rebuild (embed-gap #20).

Replaces engram_activate's O(N*D) cosine prescan with a lazy-memoized
cosine cache (eg_cosq_at), proven bit-identical to the old path.

Extracts a clean vindex_harvest_from_store primitive (read-only vector
harvest, careful malloc/ownership/error-path handling) reused by both
index-build and the new vindex_bench.c — a read-only proof harness
comparing brute-force vs HNSW recall/latency on both the real store and
synthetic data.

.nsbx-env intentionally excluded — local sandbox config (ports, paths,
dev-only placeholder key), not checked in.
2026-08-15 14:26:16 -05:00
bigmerge 5f3ddb8b8d Add grounded edge-propagation spec (task #50): core algorithm, proof harness, gated integration patches
El SDK CI - dev / build-and-test (pull_request) Failing after 14m31s
LTP/LTD-style belief grounding propagated along graph edges, with
union-find independence-guarded corroboration. Package: core C algorithm
(gep_core.h), a self-contained deterministic proof harness with recorded
output, staged runtime integration, and gated .el patches for the beat
hook and HTTP route.

Per the author's own LEDGER.md: built + proven on a clone, GATED pending
the engine/HNSW cutover — not wired into the live beat or routes.
Preserved here as a spec/reference artifact, not a request to merge into
the live path.
2026-08-15 14:26:06 -05:00
bigmerge 708722b7ff Add native interruptibility for dispatched agent workers
El SDK CI - dev / build-and-test (pull_request) Failing after 14m43s
Cancellation-token control channel checked at every step boundary lets a
coordinator PAUSE/RESUME/REDIRECT/KILL a running worker mid-task instead of
waiting for the whole (possibly wrong) plan to finish. Bounded purviews
mean no half-committed state to unwind on interrupt. Includes a proof
harness (proof.el, run.sh) comparing a broken non-interruptible worker
against the new one under identical kill/redirect/pause timing.

Distinct from the already-preserved swarm-ccr orchestrator (fan-out/
converge dispatch): this is single-worker interruptibility, a
complementary mechanism, not a duplicate.
2026-08-15 14:25:55 -05:00
bigmerge 2f832c8def Fix float arithmetic codegen and math_log aliasing
El SDK CI - dev / build-and-test (pull_request) Failing after 10m8s
Float + previously fell through to string concat (segfault); -, *, /, %
operated on raw IEEE-754 bit patterns as integers (garbage results). Floats
are now tracked via a __float_names typed-binding set (parallel to the
existing int-tracking scheme) and arithmetic is emitted as real C double
ops.

Also fixes math_log, which was wrongly aliased to natural log (duplicating
math_ln) — now uses log10 — and adds the missing <math.h> include. Rebuilt
elc binary included.
2026-08-15 14:24:14 -05:00
bigmerge 1010185978 Add op_assert grounded-envelope primitive and purview-bounded mutation wrappers
El SDK CI - dev / build-and-test (pull_request) Successful in 6m33s
Adds engram_assert_json — a grounded "assertion envelope" primitive for a
realizer/op_assert seam (per backlog bl-53/#57) — plus purview-scoped
mutation wrappers engram_node_full_in/engram_connect_in, which refuse
non-default purviews rather than silently mutating the live store. Threads
through el_seed.c/h wrappers and the codegen.el arity table per the
project's existing C-builtin recipe.

Also rewrites lang/AGENTS.md build docs with verified (2026-08-15) findings
that el_seed.c does not compile standalone.
2026-08-15 14:24:11 -05:00
bigmerge 710bea174d Add native EL afferent ingest organ
El SDK CI - dev / build-and-test (pull_request) Successful in 6m29s
Source-polymorphic ingest(source) primitive: extracts content faithfully
from a directory/file/url/llm-query/structured-primitive-set/stream,
decomposes it into a discrete multi-node graph manifold (nodes + internal
edges, never a single blob), and merges it into the engram geometry with
dedup (search + exact/cosine match), provenance, grounding-level, and
stewardship-class tagging from the moment of entry.

Pure HTTP client of the engram server (links only el_runtime.c, never
el_seed.c/the engine directly). Tested against a live nsbx sandbox engram
clone (127.0.0.1:8903) with real writes confirmed via /api/stats
(node_count 3201 / edge_count 6601).

Excludes ingest/build/ — local compiler scratch output (binaries, .c
codegen, .err logs), not source.
2026-08-15 14:22:07 -05:00
bigmerge 05e5d3c402 self-review 2026-08-15: consolidate the strongest Hebbian candidate, not the lowest-hash one
The link-formation scan walked candidate slots ascending and stopped at
ENGRAM_HEBB_LINK_PER_CALL (2). Slot index is a hash of the node id pair, so
whenever more than two candidates cleared LINK_MIN in the same call, the two
consolidated were the two with the lowest hash and a stronger association
waited - indefinitely, since the scan restarts from slot 0 every call while
the leader decays at ENGRAM_HEBB_DECAY.

Measured 08-13..08-15: hebb_cand_max peaked at 0.4963, 3.3x LINK_MIN, during
a ~14h stretch of continuous qualification at the 2/call cap.

Same defect the 2026-08-02 review named and fixed for breakthrough weights
(index order is not a cognitive criterion), never carried across to the one
path that writes permanent structure - and there is no pruning path, so
growth is one-way. Selection pressure matters most where the result is
irreversible.

No-op when <=2 candidates qualify; picks the best when more do.
2026-08-15 08:44:37 -05:00
bigmerge 6621a4dbc5 feat(engram): native set-based reframe_region on the cognition engine
Add the universal engram mutation as ONE operation: isolate a region
(cosine + adjacency) -> supersede it as a set (immutable region-tombstone,
originals retained, engram_forget never used) -> insert the new manifold as a
set -> rebind edges by cosine -> one atomic persist. Single-node write and
supersede are the degenerate n=1 case of the same reframe_core path, not a
separate CRUD path. Keystones kn-efeb4a5b / kn-5b606390 are write-protected.
Purely additive: routes POST /api/reframe, /api/write, /api/supersede.

Verified on an isolated clone of the JSON-snapshot engine (set-replace, n=1,
no-regression, keystones, durable reboot); compile-verified clean against the
cognition multi-TU build. NOT deployed — prod :8742 frozen; blue-verify on the
cognition/egm engine required before any cut.
2026-08-15 04:37:35 -05:00
bigmerge 7e4b21c779 Add sandbox: multi-repo stack worktree composer (el-stack / neuron-stack)
El SDK CI - dev / build-and-test (pull_request) Successful in 6m19s
Assembles every constituent repo of a stack into one combined worktree
workspace, laid out at natural relpaths so cross-repo ../foundation/el
imports resolve to the sandbox copy. Sibling of nsbx; pure bash + git
worktree; never touches live :8742/:7770; isolated engram delegated to nsbx.
2026-08-15 00:55:22 -05:00
bigmerge 15f90003c0 teacher-summon: default-off (TEACHER_ENABLE) soul-native wake; byte-inert when unset
+282 lines in engram/src/server.el implementing the flag-gated teacher summon
(consult_teacher backend abstraction, tier autoselect, GGUF fetch/cache). With
TEACHER_ENABLE unset the summon path is byte-inert. Consolidates the proven
api-reshape pieces (geometry-ops d4f401d, boundary auto-emit 0182642) for the
validated cutover.
2026-08-14 21:52:56 -05:00
bigmerge ff37835ae5 swarm: document the single-writer invariant (Rule 4) in README
El SDK CI - dev / build-and-test (pull_request) Failing after 12m1s
2026-08-14 21:46:23 -05:00
bigmerge e5c80359a8 swarm: Rule 4 — engram-write is @manager-ONLY, enforced by capability
New hard invariant (Will): only the orchestrator mutates global engram state;
workers are read-only against the full engram + write only their own local
geometry. This is an AUTHORITY gate (capability), not a health gate — a worker
is STRUCTURALLY UNABLE to mutate global engram state regardless of engram health.

- containment.el: scope tokens now carry a caps set. Orchestrator token holds
  engram:write + dharma:emit (@manager-only, the VBD rule that only the manager
  mutates global state); worker token holds ONLY engram:read. Rule 4:
  containment_check_engram_write / _dharma_emit reject any caller lacking the
  capability — same scope-token mechanism as the live Rule-2 denial.
- swarm.el: swarm_engram_write is the ONLY engram write path, gated by Rule 4;
  a worker token is denied before any HTTP is issued (no mutation). The curated
  merge (commit=1) is the sole writer: the orchestrator commits approved
  geometry via its write-capable token. Workers' full-engram READ stays intact.
- reshape_surface.el: compose op_write (json_escape_string) for the commit path.
- harness: Rule-4 suite proven — worker engram-write DENIED by capability, no
  node created, violation journalled; orchestrator passes the gate as sole
  writer. 24/24 green on the :8901 clone with real cognition.

Authority gate holds independent of daemon write-health (proven with daemon
both alive and, earlier, crashed). Prod :8742 untouched.
2026-08-14 21:46:03 -05:00
bigmerge 01826421c4 seam: implement decorated-fn boundary auto-emit; prove on clone
Will waived diff review -> build it for real. Add engram_boundary_beat() to the
runtime (afferent counter++ + engram_chrono_tick + engram_strengthen(self-anchor)
+ dharma_emit) and two act-stats counters (aff_boundary_ops, dharma_emits).
codegen cg_fn injects ONE engram_boundary_beat(op) at the entry of every
@manager/@accessor fn (fn_has_decorator, so it fires under @route @manager too) —
a decorated op self-reports with ZERO hand-written instrumentation. Rebuilt elc
self-host + the cognition engram in the worktree; ran it as the clone daemon on
:8900. Proof (/api/boundary-proof, @manager, empty body, 5x): aff_boundary_ops
0->5, dharma_emits 0->5, self activation_count 1510->1513, chrono stamp advanced.
Brought in feat/cognitive-architecture engram runtime+server for the build.
strengthen = activation bump (not content/edge write) -> identity protection
intact. Live :8742 untouched; no push, no cutover.
2026-08-14 21:20:18 -05:00
bigmerge b53b5b4e8a swarm: document real-cognition binding + HAVE_CURL build note in README 2026-08-14 21:19:11 -05:00
bigmerge 20bd9ed00b swarm: bind reshape's proven primitives — REAL-COGNITION local swarm end-to-end
Binds the api-reshape surface at wt/api-reshape@d4f401d (op_think/read/attend/
learn, verified against engram.cognition-20260814) into the swarm:
- reshape_surface.el composes the reshape's proven read/cognition primitives
  verbatim (write ops omitted — they need the gate-1 write-healthy clone).
- primitive_binding.el: bound_think -> op_think over the worker's NODE-ID
  anchor (ctx.input); attend/learn bound behind SWARM_WRITE_HEALTHY.
- cognize blueprint derives the vote verdict from the REAL gradient's n_support
  (json_get_int) — per-anchor diversity (6/16/87 support) drives a genuine vote.
- build.sh now defines HAVE_CURL. CRITICAL FIX: without it every http_* was a
  '{"error":"not built with HAVE_CURL"}' stub, so prior 'live engram'
  retrieval was a false positive (matched the ref string, not real content).
  With HAVE_CURL the swarm genuinely hits /api/think on the :8901 clone.

harness_real_cognition.el: 17/17 GREEN with seam=decorated — 8 native-thread
workers each a REAL think (768-dim gradient) over its CCR-scoped node-id anchor,
@manager reduce+vote convergence, all 3 containment rules incl. live Rule-2
denial, afferent telemetry (8 real think signals), durable work-tracking. Reads
only — daemon stays healthy; writes stay gated on the gate-1 clone. Prod :8742
untouched.
2026-08-14 21:18:57 -05:00
bigmerge d4f401de1c reshape: decorator-as-seam — port @route codegen, prove decorate->serve, rewrite surface as decorated El
Ground-truth the three seams (route/telemetry+interoception/bus) with file:line
evidence. Port the tested @route codegen+parser from feat/el-route-decorators
into the worktree elc (decoration synthesizes el_route_dispatch — no hand-written
90-branch handle_request). Rebuild elc self-host; prove decorate->serve end-to-end
(route_proof.el on :8951). Rewrite surface.el as El-native decorated components:
@route + @accessor/@manager, in-process engram_* builtins (not http_get), @manager
ops emit on the real dharma_* bus (same transport as wt/swarm-ccr). Identity
keystones refused in write/relate/supersede. Gate-1 clone recipe (WAL-aside
cold-boot + ENGRAM_WAL=on) proves the FULL op set live on the clone. Boundary
auto-emit (telemetry/interoception/bus) staged as a reviewable cg_fn diff
(SEAM_STAGED.md) — needs the cognition-engram rebuild to verify link. Live :8742
untouched; no push, no cutover.
2026-08-14 21:01:27 -05:00
bigmerge 70982498e0 swarm: document local-swarm harness + one-flip seam in README 2026-08-14 20:58:17 -05:00
bigmerge 373265c05d swarm: local-swarm integration harness + one-flip primitive seam + telemetry
- primitive_seam.el: SWARM_PRIMITIVE_SEAM selects stub (default, hermetic) vs
  decorated (reshape's dharma-bus primitives). Every seam call is an afferent
  signal; telemetry (seam_mode + afferent tick) rides the vertical result path.
- primitive_binding.el: THE ONE FLIP POINT — bound_think/attend/learn today fall
  back to the stub; when the reshape's decorated primitives land, flip one line
  each and set SWARM_PRIMITIVE_SEAM=decorated. No other change anywhere.
- swarm.el: default blueprint routes think through the seam; the @manager
  aggregates afferent counters from worker results (containment-safe, no shared
  bus register) and journals a swarm.telemetry record; telemetry in the return.
- harness_local_swarm.el: 17/17 GREEN on :8901 with the stub — 8 native-thread
  workers at concurrency 4, reduce+vote convergence, CCR scoping+non-leak, all
  three containment rules (incl. live Rule-2 denial), durable work-tracking,
  afferent telemetry observed. Runs identically under seam=decorated today
  (binding fallback), proving the flip path executes.

Engram writes stay opt-in (durable journal is the substrate); daemon healthy.
2026-08-14 20:58:05 -05:00
bigmerge ed722b9e2e swarm: build harness executable + module load order 2026-08-14 20:44:01 -05:00
bigmerge b0a78c5737 swarm: capability README — architecture, framework grounding, built vs stubbed 2026-08-14 20:43:46 -05:00
bigmerge 447d042022 swarm: HTTP-backed primitive retrieval + live-engram integration test
- primitive_attend retrieves over HTTP (POST /api/search) when ENGRAM_URL is
  set — the location-independent worker model — falling back to the in-process
  store otherwise. Proven against the isolated :8901 clone: CCR compiled a
  bounded context from REAL mind content (VBD/intellectual-dna).
- gate the engram work-tracking mirror behind SWARM_MIRROR=1; the durable
  substrate is always the JSONL journal, so a swarm never depends on the mind
  to track its work. (Repeated POST /api/nodes mirror writes were observed to
  crash the isolated daemon — a daemon-side write-path robustness issue;
  retrieval POST /api/search is solid. Prod :8742 never touched.)
- integ_engram: CCR real-retrieval + full swarm completion against live clone.
2026-08-14 20:43:05 -05:00
bigmerge d4e82d3d56 swarm: convergence strategies + failure threshold, hardened El JSON usage
- vote/merge/reduce/collect convergence proven end-to-end; failure threshold
  aborts a swarm below min_success_ratio (integer per-mille) and completes
  when failures are within tolerance, with worker.failed + swarm.aborted
  tracked durably.
- worked around three El runtime/codegen semantics surfaced during the build:
  json_set inserts RAW (use json_set_str for string values); json_set cannot
  update an existing key (vote tallies via list rescanning); json_array_get
  keeps quotes (use json_array_get_string). Also: float division is unreliable
  (swarm uses integer math), and a let-rebind in a deeply nested if/else does
  not propagate outward (accumulators kept at one block level).

test_convergence: 8/8; test_swarm: 12/12.
2026-08-14 20:37:35 -05:00
bigmerge f19040e484 reshape: geometry ops + primitive agentic tools over the one geometry
Collapse ~90 noun-CRUD MCP tools into read/write/relate/supersede (type is a
parameter) plus the live agentic primitives (think/attend/learn/ground/assert)
already in the engram cognition build. Additive: old noun-tools aliased to the
new ops. Vantage-read applies aperture -> a bounded slice, fixing the whole-self
dumps. Signatures grounded in the live cognition binary; validated on an
isolated nsbx clone (parity.sh: 12 proven, 0 failed). Live :8742 untouched.
2026-08-14 20:34:55 -05:00
bigmerge 40bb6ff579 swarm: orchestrator, CCR context compilation, containment rules, primitive seam
- swarm.el: coordinator running fan-out/converge on El NATIVE threads
  (thread.el spawn/join) in bounded concurrency waves, order-preserving;
  convergence strategies collect/merge/vote/reduce; integer per-mille failure
  threshold (El float division is unreliable — avoided deliberately).
- ccr.el: per-worker Compiled Context Routing — retrieval/scoping/compaction
  into a bounded, minimal package; the compiled-context boundary is the
  security boundary (a worker cannot receive or leak sibling inputs).
- containment.el: the three Swarm containment rules enforced via scope tokens
  (Rule 1 no join, Rule 2 no open, Rule 3 no lateral edge) + execution-tree
  lateral-edge check.
- primitives.el: attend/think/intend/act/learn seam the swarm composes over,
  with engram-backed fallbacks and an explicit binding point for the reshape.
- prototype json_array_push in el_runtime.h (defined but unprototyped).

test_swarm: 12/12 — native fan-out/converge, bounded concurrency, durable
tracking, CCR bounding + non-leak, and all three containment rules.
2026-08-14 20:29:04 -05:00
bigmerge d5411fb58a swarm: durable, inspectable work-tracking journal (worktrack.el)
Single-writer append-only JSONL journal keyed by correlation ID: swarm +
worker + convergence records, reconstructable into a status report. Optional
engram mirror via POST /api/node when ENGRAM_URL is set. Coordinator is the
only writer (workers return structured results), which is race-free and
enforces Swarm containment rule 3 by construction.

Also prototype now_millis/now_ns in el_runtime.h (defined in el_runtime.c but
unprototyped — blocked any El program needing a real ms clock under clang 21).

Test proves durability + inspectability end-to-end.
2026-08-14 20:23:13 -05:00
bigmerge b2aac4bf89 el runtime: prototype + fix channel/mutex seed ABI for modern clang
el_runtime.h declared only __thread_create/__thread_join; the mutex and
channel seed primitives (__mutex_*, __channel_*) were defined in
el_runtime.c but never prototyped. Under Apple clang 21 (C11) the missing
prototypes became implicit-declaration errors, and the void-returning
__channel_send/__channel_close mis-typed el_val_t (long long) returns,
so any El program using runtime/channel.el failed to compile.

- add prototypes for __mutex_new/lock/unlock and all __channel_* to el_runtime.h
- make __channel_send/__channel_close return el_val_t nil so elc's
  trailing-expression codegen for the void El wrappers type-checks

Additive; unbreaks native channels for every downstream El program.
2026-08-14 20:18:06 -05:00
Neuron 54378c7355 elp(translate): refactor to geometry-native concept-pivot
El SDK CI - dev / build-and-test (pull_request) Successful in 6m57s
Drop the bilingual-string-table framing and the external-encoder plan (both
wrong). Translation now routes source-lexicon -> concept-frame (language-
invariant, in the engram concept geometry) -> target-realizer, exactly as the
ELP was designed: a word resolves to the CONCEPT it denotes via its own
language's lexicon (a monolingual step — the engram nearest-region ranker only
disambiguates senses within one language, so an English-trained embedder is
fine and never compares 'ocean'~'oceano' as strings). The concept node is the
shared pivot; its manifold location is the meaning.

- Pronouns route through the NATIVE concept pivot (cp_pron_concept ->
  cp_rom_pron_surface) instead of an ad-hoc EN->tgt string map.
- lemma_for_concept / noun_for_concept are each target language's own
  CONCEPT->SURFACE lexicon (the mirror of comprehend's SURFACE->CONCEPT).
- Fidelity is concept-preservation (concept_frame fingerprint), not string
  cosine against an external multilingual model.
- Plural article agreement fixed (las/los, as/os).

Verified: 'You never fought the ocean.' -> ES 'Usted nunca luchó el océano.'
concept-frame pivot 'pred=fight patient=ocean pol=neg' realizes to ES+PT from
one parse; nunca holds 3/3. Gaps unchanged: PT verb conjugation fallback,
adjunct/subordinator concepts not yet in-frame.
2026-08-14 17:53:00 -05:00
Neuron 640e8799e5 elp(translate): normalize EN irregular pasts before the lemma bridge
comprehend lemmatizes some irregulars (fought->fight) but not all (broke);
tr_norm_verb covers the poem's remainder so affirmative content verbs route
(ES 'Yo broo' -> 'Yo rompo'). Negation lines unchanged and still correct.
2026-08-14 17:38:13 -05:00
Neuron 9b63a2a23b elp(translate): EN->ES/PT geometric-free translation faculty
Adds the missing middle of the ELP: a deterministic EN-content-lemma ->
target-lemma bridge (translate.el) on top of comprehend.el (parse) and
realizer.el (inflect). English-only engram geometry cannot route
cross-lingually and vocabulary-XX.el carries no en_translation glosses, so
the honest no-LLM bridge is a wired lexicon (poem coverage; OOV passes
through). SACRED polarity/neg_word are carried untouched: 'never' localizes
to a negator ('nunca'), never to a content lemma.

Additive realizer extensions: agent_person/agent_number recognize Romance
target pronouns; the non-EN negation branch surfaces a carried neg_word
instead of the generic negator.

Verified on the real toolchain (elc->cc->run):
  'You never fought the ocean.' -> ES 'Tú nunca luchaste el océano.'
  'I never saw the breaking.'   -> ES 'Yo nunca vi la ruptura.'
nunca holds 3/3 negation lines. Known gaps: PT verb conjugation fallback
(lutarred), irregular EN lemma (broke->break), adjunct/subordinator passthrough.
2026-08-14 17:37:01 -05:00
bigmerge 6660becfdb nsbx: add one-command dev onboarding (branch + worktree + isolated engram)
El SDK CI - dev / build-and-test (pull_request) Successful in 6m30s
Add 'nsbx dev <name>' / 'nsbx dev-down <name>' plus a Makefile so a newcomer
goes from clone to coding on an isolated cloned engram in one command. The
worktree is created on a real named branch at a persistent path (never /tmp,
guarded), and the whole worktree is pinned to the clone via an emitted .nsbx-env
so live :8742 / ~/.neuron/engram is unreachable by accident. Optimizes the El
edit->build->run loop so provisional work is built in El against a throwaway
clone instead of prototyped in Python and re-ported. Additive over the proven
primitives; no live cutover.
2026-08-14 17:24:06 -05:00
bigmerge 112bb2540f Add nsbx — the Neuron Sandbox primitive
Generalise the ad-hoc cog-arch (worktree+build+store-clone+C-tests) and
store-fix (secondary soul + launchctl rails cutover) proto-sandboxes into one
reproducible primitive: run experiments and code changes against the REAL
engram runtime on an isolated snapshot of the live mind, with a gated
promote-to-prod path.

Dev environment as a primitive — any team member gets a private, isolated copy
of the mind (separate port/store/process); prod on :8742/:7770 is untouchable
from a sandbox. Wraps the real binary; never reimplements engram logic.

Lifecycle: create/up (consistent store+WAL+config snapshot; place OR build the
runtime from --source/--branch/--binary; boot on an isolated port) · build ·
run · validate (rails as checks: zero-loss under load+reboot, reboot-prove, RSS
bound, retrieval parity, keystone integrity) · promote (gated rails cutover:
snapshot-first, additive binary swap, bootout→settle-poll→bootstrap, verify,
auto-rollback; never pkill/kickstart -k; dry-run unless approved) · destroy.

Dogfooded: reproduced retrieval-parity 25/25 vs baseline and the cog-arch
correspondence-loop known result (Brier 0.028648->0.000586, reboot-proven) and
real-store reboot-prove at 10994-node scale, all inside a sandbox; prod
untouched.
2026-08-14 15:40:58 -05:00
bigmerge d595b3c57e cognitive architecture design: cognition as one operation over learnable priors
The buildable form of the "one operation" theory (memory bdc8a488). Maps the
theory onto what is already compiled: the five reasoning operators in
engram_reason.c already collapse onto ONE primitive — engram_reason_point_fit —
plus the geo-algebra (combine/subtract/analogy-rotate/distance), and
engram_verify.c is built on the same fit. So the operator-collapse is already
half-written; what is missing is not the primitive.

What is missing, and what this doc specifies:
- think(anchor, prior) -> gradient (a distribution/direction, not a point); each
  named faculty = {point_fit + a prior}, the operation frozen, the prior learned.
- Prior as a first-class stored node (warp + calibration), superseding the
  intrinsic importance/salience scalar with a relational, grounded-for-whom edge.
  Confirmed against the runtime: importance is already a live activation
  computation (el_runtime.c:13013), never trusted as a static field.
- vantage_read(anchor, aperture) — one op, three settings: self / foreign-field /
  veil.
- The reflexive correspondence-loop as the learning engine: move the grounding
  check from offline Python into the geometry, reflexive, reusing the DORMANT
  verifier (engram_verify_grounding has no runtime caller and no El binding today)
  turned inward. grounding = learning = one loop.
- hold/ground/assert kept distinct: the engram holds anything, grounding is an
  edge, the honesty floor is on assertion only; ungrounded content is first-class.
- metastability: keystone core (read-mostly priors) + plastic everything else.

Seven staged milestones, earliest is a real end-to-end slice (induction as
{primitive + grounded prior} with the loop closing on it, reboot-proven on a
snapshot). Build rails stated: offline/secondary, snapshot-first, reboot-prove,
zero-loss, gated launchctl cutover. Design only; no code changed this pass.
2026-08-14 14:42:01 -05:00
bigmerge 23f43bcc21 self-review 2026-08-14: a gate that passes the median stranger at 0.67 is not a gate
Two changes to the activation path, both grounded in measurement on the live
store rather than on the spec.

1. Rescale cosine before the query gate.

   The propagation gate (arXiv:2606.30133, added in an earlier review) fed RAW
   cosine into FLOOR + (1-FLOOR)*c. Raw cosine from nomic-embed is compressed
   into a narrow high band, so that expression is close to a constant.

   Measured, 400 random UNRELATED node pairs on the live store:
     median 0.562, central 98% span [0.381, 0.743]

   So a node with no semantic relation to the query was propagating at
   0.25 + 0.75*0.562 = 0.67. Two thirds strength. The gate was a small tax.

   Fixed by shifting and flooring about ENGRAM_EMBED_S0 -- which is already in
   this file, already 0.45, and already used exactly this way by the Pass-2 WM
   term. The propagation gate simply never used it. Same 400 pairs after:
   median unrelated pair falls to 0.40, top of range preserved (0.85 vs 0.92),
   gate spread widens 0.42 -> 0.60. Only 8.5% reach the floor, so dissimilar
   lexical/structural pathways are damped, never severed. Range is unchanged
   at [0.25, 1.0], and cosq == NULL still degrades to no gating at all.

2. Decompose the WM eviction counter by cause.

   _eg_act_wm_evicted was incremented from six sites with four distinct causes
   and collapsed all of them into one integer. Today's review measured 175,547
   evictions over 13.5h (~216/min against 24 slots) and could not tell healthy
   rotation from cap thrashing from duplicate churn.

   That is this file's most-repeated defect: dup_wm and dup_wm_global exist
   only because the aggregate could not answer "why" during the 08-02 and
   08-06 incidents. Each of those needed a NEW gauge before it was diagnosable.

   evict_floor / evict_cap / evict_bll complete the decomposition, so
     wm_evicted == floor + cap + bll + dup_wm + dup_wm_global
   holds as an identity and each term implies a different correction. Verified
   on an isolated instance: 30 nodes, 24 filled the cap, wm_evicted 6 ==
   evict_cap 6, all other terms 0.

Built and smoke-tested out of tree. The live daemon runs a pinned binary and
was deliberately not restarted -- the store compaction workstream is in flight.
2026-08-14 08:43:11 -05:00
will.anderson ce34b94f88 elp(dialogue+self_region): native-el summon-through-self port + scratch-verified gate
Ports dialogue.py + self_region.py to native el, bound to the IN-PROCESS engram
el runtime (engram_activate_json / engram_neighbors_json / engram_search_json /
engram_node_full / engram_connect — C-order builtins, not the wrapper order).

self_region.el: pulls the engram's REAL Self/identity nodes (pooled single-term
search), scores by self-signal, reads out identity from their own prose — no
hardcoded anchors, no template.

dialogue.el: ONE operation — project(query) -> land on a region -> read out.
  * identity = self-region proximity (no intent classifier, no separate branch)
  * memory = activation + a RELEVANCE FLOOR, then MATERIALIZE by walking the
    neighborhood (real edges), never top-props
  * HONEST ABSENCE when nothing is close — no 'I noted that' echo, no fabrication
  * NEGATION SACRED: readout is the stored prose verbatim, so polarity survives
  * DIRECTIVE OVERRIDE: a meta-directive switches the reply language

Verified against a SCRATCH in-process engram (live :8742 untouched): dialogue
gate 9/9 — identity from real self-content, neighborhood materialization,
SACRED negation (self + memory), PT identity in PT, directive override to
English, 'Prove it' -> honest absence. EN/Romance/prop/multilingual gates
unregressed.
2026-08-13 15:56:08 -05:00
will.anderson 0ae33c0f3b elp(realizer): close subordinate-clause round-trip + silent-e/doubling lemmatizer + verb-final object bug + ES/PT closed-class verb guard
- realizer now carries the subordinate clause verbatim (subord_text slot): the
  5th English acceptance sentence is byte-identical through parse->realize->reparse.
- English -ed/-ing lemmatizer restores silent-e (loved->love) and collapses
  inflectional doubling (stopped->stop), inverting en_verb_past().
- parse_spec_lang: a sentence-final main verb no longer bleeds into the object
  slot (cstart advanced to verb+1), so intransitives round-trip.
- cp_rom_is_verb rejects closed-class words (prep/det/pron/aux/neg) before the
  ending-only test, killing the 'para'/determiner misfires.

EN telephone gate 5/5 (now byte-identical 5/5), Romance gate 6/6.
2026-08-13 15:39:49 -05:00
will.anderson c5508372ca elp(multilingual): native-el language layer — detect + localized phrases
Phase 3 piece 2. Ports multilingual.py: deterministic language detection
(en/es/pt/it) via stopword + diacritic scoring, localized fixed phrases (SACRED
per-language yes/no/decline/identity), PT/ES->EN retrieval term lexicon, and
EN->target predicate translation. No generative model.

Gate (multilingual_gate.el): 4/4 languages detected correctly; localized
declines + term/pred lexicons verified. Built bounded (elc rc=0 peak 25MB).

Worked through the documented el '+' mis-compile (two chained function-call Int
operands compile as string concat -> corrupt Int -> segfault on the accented
path); fixed by binding each score to an Int var and adding vars singly.

Simplifications (honest): diacritics scored by PRESENCE (str_contains) not
codepoint count (UTF-8 index safety); confidence scalar and the regex-based
parse_directive() from the reference not yet ported (directive parsing deferred
to the dialogue layer).
2026-08-13 15:09:04 -05:00
will.anderson 335298a518 elp(propositions): native-el READ primitive — memory text -> SACRED triples
Phase 3 piece 1. Ports propositions.py off spaCy: the dependency-parser role is
now the el-native parser (parse_spec), and each memory sentence's meaning-spec
IS the triple (subject, predicate, object, modifiers, polarity, tense, source,
confidence). Sentence segmentation + repr parity with propositions.py. NEGATION
SACRED: polarity flows straight from the spec, never dropped/inverted.

Gate (propositions_gate.el): 4/4 SACRED polarity correct on extraction;
multi-sentence memory splits one triple per sentence in reading order with
negation preserved. Built bounded (elc rc=0 peak 24MB, cc rc=0).

Gap (honest): English regular-verb lemmatizer does not restore silent-e
(stores->stor); coreference/passive normalization from the reference not yet
ported (shallow pronoun subject kept as surface).
2026-08-13 14:56:57 -05:00
will.anderson 7d4fdbcc22 elp(comprehend): ES/PT Romance parser path — SACRED polarity cross-lingual
Adds a deterministic Romance front-end to comprehend.el (parse_spec_romance),
dispatched from parse_spec_lang for lang es/pt. English path untouched
(byte-identical, regression gate still 5/5). Pro-drop aware clause skeleton
(subject | neg | verb | object | PP), cross-lingual negation lexemes already
SACRED. Romance telephone gate: polarity PRESERVED 6/6 and EXTRACTED 6/6
through parse->realize->re-parse for 3 ES + 3 PT sentences. Built bounded
(elc rc=0 peak 24MB, cc rc=0).

Named gaps (honest): ending-only verb detection misfires on prepositions
(contra) and -a/-o nouns (menina); lemma recovery keeps surface form; the
non-English realizer is a generic preverbal-negator skeleton so ES/PT surfaces
are not byte-parity. Full paradigm inversion + Romance lexicon deferred.
2026-08-13 14:55:14 -05:00
will.anderson 89ea1b5a15 elp(comprehend): el-native comprehension parser + SACRED polarity end-to-end
PIECE 1 — greenfield el-native parser (comprehend.el), spaCy-free:
- text -> meaning-spec via invertible English morphology (the realizer's own
  irregular table run BACKWARD) + a deterministic clause grammar (subject/verb
  boundary, roles, ditransitive iobj, PP adjuncts, subordination, coordination).
- NEGATION IS SACRED: explicit polarity field, always present, cross-lingual
  lexeme set; standalone neg adverbs (never) captured separately.
- WSD by deterministic syntactic position over a fixed sense inventory
  (flies->fly, like->comparison, saw->see); engram nearest-region is the
  documented runtime upgrade hook (no external model).

Polarity threaded through the whole el contract (was previously dropped at the
boundary): realizer.el realize_lang honors polarity (English do-support /
adverbial / copular negation; generic preverbal negator for es/pt/ca/it/fr/de/ro)
and places iobj; elp.el build_form_from_json carries polarity/neg_word/iobj
across JSON; morphology.el gains 'fight'.

Acceptance (native el telephone test, comprehend_gate.el): on the 5 gate
sentences polarity PRESERVED 5/5 and EXTRACTED 5/5 through parse->realize->
re-parse; 4/5 byte-identical. Built bounded (elc rc=0, cc rc=0).
2026-08-13 13:41:09 -05:00
will.anderson a816b119e7 stage(elp): consolidate scattered lang work — full-lexicon vocabulary + profiles
Backfill ELP vocabulary from FULL lexicons (UniMorph + kaikki.org Wiktionary,
real gender/inflections) for 8 languages, 812,894 entries total, in the proven
seed-fn format matching the 18 ancient vocabularies:
  es 72,032 | fr 130,517 | de 144,692 | la 22,590 | it 193,675 | pt 115,772 |
  ro 86,504 | ca 47,112
4 of these (es fr de la) backfill ELP languages that had morphology but no
vocabulary; it/pt/ro/ca are new Romance (need morphology-*.el ports next).
Adds lang_profile_* for all 8 + reproducible generators under tests/lang-gen.
Vocab is runtime seed data (not in build manifest, like the 18 ancients);
seed-fn format validated to compile to C via elc.
2026-08-13 11:56:03 -05:00
will.anderson ba6e36c3f7 self-review 2026-08-13: the extractor was reading the label; the topic was in the content
auto_term_empty_streak — the counter the 2026-08-06 review added to catch
exactly this — read 50 and climbing. Fifty consecutive curiosity scans where
the soul's dynamic seeding produced nothing and the loop fell back to four
hardcoded phrases. The live WM top said why in one look: every slot was a
Memory node labelled "memory:remembered". The extractor read the LABEL only,
the sentinel guard correctly rejects sentinels, so there was never anything
to extract. It was written against Knowledge nodes, which have real titles,
and was structurally blind to the node type that dominates working memory.

Rather than add a sixth guard to the five that accumulated across four
reviews (genre words, quoted titles, stopwords, label-df), invert the
algorithm. The old one was: take the first word, then check whether it is
acceptable. That shape forces quality to be expressed as rejection, and
rejection can only ever encode floods that already happened.

engram_salient_term() scores EVERY candidate token and returns the argmax of
idf · position · casing (YAKE, Campos et al. 2020, with real corpus IDF
substituted for YAKE's corpus-free proxies), falling back from a sentinel
label to the node's content. Term quality becomes the selection criterion
instead of a veto: a bad token loses to a better token in the same text
without needing to be on any list. Tabu is applied during the argmax, so
inhibition-of-return costs seed quality rather than costing the whole scan.

Two defects found by instrumenting rather than assuming, which is the lesson
this codebase keeps relearning:

  - The first live run returned five ALL-CAPS terms in a row. Memory content
    conventionally opens with an all-caps header, so YAKE's acronym bonus was
    handing the seed to whatever word the heading started with. Restricted to
    tokens <= 5 chars, where all-caps is evidence of an acronym rather than
    evidence of a heading. Long headers now compete on specificity.

  - df via istr_contains is substring matching, so "them" hit inside "theme"
    and function words came back with nonzero df. Added word-boundary df
    locally; engram_label_df keeps substring semantics for its callers.

An earlier draft claimed the min_df floor subsumed the 73 stopwords that
08-03 measured label-df as missing. Re-measured: about:2, whole:1, them:2 —
they clear a floor of 1. The claim was false and the comment now records the
correction. The floor buys lexical reachability; the argmax buys quality; the
stopword list still earns its keep.

Measured on 60 live Memory nodes before shipping: 0 empty, versus 60 of 60
under the old extractor. Terms are topical — HEBBIAN, CONSOLIDATION,
TEMPORAL, crash-loop, PRIMING, NEIGHBORHOOD, DRIFT. Three of sixty are weak
header words; left alone deliberately, because listing them is the move that
produced four blocklists.

ENGRAM_ST_DEBUG=1 dumps the scored candidate set. It exists because there was
no way to see whether the all-caps run was the corpus or the casing weight
without guessing.
2026-08-13 08:43:09 -05:00
will.anderson 4f49755ebb Merge pull request 'ci: make official engram build store-enabled (publish + link engram_store.{c,h})' (#95) from engram-tiered-storage into dev
El SDK Release / build-and-release (pull_request) Failing after 40s
El SDK CI - stage / build-and-test (push) Failing after 31s
El SDK CI - stage / build-and-test (pull_request) Failing after 34s
El SDK CI - dev / build-and-test (push) Failing after 3m53s
El SDK Release / build-and-release (push) Failing after 12m31s
2026-08-12 20:23:34 +00:00
will.anderson 7aa847e32a Merge origin/dev into engram-tiered-storage
El SDK CI - dev / build-and-test (pull_request) Failing after 10m51s
Resolve 3 conflicts:
- lang/el-compiler/runtime/el_runtime.c: keep deletion (deprecated runtime fork;
  single-source-of-truth is lang/runtime/, enforced by scripts/check-single-runtime.sh).
- lang/releases/v1.0.0-20260501/el_runtime.h: keep deletion (releases/ is a generated
  artifact folder, not a source path; a release is a git tag, not a folder).
- lang/runtime/el_platform_win.h: union of dev's Windows port (#80: setsockopt optval
  wrapper + curl-less libcurl stubs) and our fsync(->_commit) shim needed by engram_store WAL.

Nothing in dev's build consumes the deprecated fork or releases/ folder.
2026-08-12 15:23:02 -05:00
will.anderson ee71423732 ci: publish + link engram_store.{c,h} so official builds are store-enabled
El SDK CI - dev / build-and-test (pull_request) Failing after 13m20s
The live engram now runs the paged store (neuron.egm+WAL), but the SDK
release publishes only el_runtime.{c,h} and the engram build links only
el_runtime.c — so a future official release would silently revert to the
in-memory store. Publish engram_store.{c,h} as SDK release assets and add
them to the engram build's download + cc link so the store transition
cannot regress.
2026-08-12 14:22:25 -05:00
will.anderson bb64a236ed engram tiered storage: engram-service wiring + elc fold-hang fix + prune-store mirror
- Wire paged store into the ENGRAM SERVICE (server.el, the authoritative durable
  owner): boot->engram_store_boot, persist_canonical->engram_store_checkpoint,
  gated by ENGRAM_STORE.
- elc (lang/elc.c + src/parser.el + codegen.el + elc-combined.el): OOB guard in
  tok_kind/tok_value + parse_block progress backstop — fixes the pre-existing
  unbounded-memory fold hang on sessions.el.
- engram_prune_telemetry mirrors ISE prune to the store (store_forget) so store
  live-count tracks resident and stale telemetry stays bounded.
- Deployed live 2026-08-12: engram :8742 on neuron.egm+WAL, count reconciled 11552.
2026-08-12 14:14:20 -05:00
will.anderson 9a0266cbf9 engram tiered storage M3.5: persist activation field updates (pre-flip gate)
Flag-on checkpoint now full-walks the resident graph: store_put_node (WM weight,
activation_count, last_activated, wm_anchor) + store_put_edge (hebb, last_fired)
for every node/edge, then engram_checkpoint. Uses store_put_edge (idempotent
upsert) not store_hebb_batch, because activation FORMS new hebbian-associate edges
that bypass the create hook and delta-only hebb_batch can't create them. Store-on
boot now applies the same WM-halving + floor + cap transforms as engram_load.

This is the hebb-survives-restart fix. Gate: reboot from neuron.egm with
snapshot.json deleted -> edge hebb + activation_count survive unchanged, WM weight
survives with identical boot transform; negative control proves persist is
load-bearing (hebb->0 without it). M1 33/33 + M2 36/36 + M3 parity PASS, ASan/UBSan
clean, flag-off untouched. Engine unchanged (boundary held).
2026-08-11 23:37:52 -05:00
will.anderson a72145b44e engram tiered storage M3: wire store behind ENGRAM_STORE (default off) + .egm rename
Caller-side shim in el_runtime.c maps EngramNode/Edge <-> StoreNode/Edge; engine
keeps zero soul deps (libengram boundary, design §10). Flag off = today's JSON
path byte-for-byte (proven: no neuron.egm created, graph identical). Flag on =
engram_open (import snapshot.json once into neuron.egm, else WAL-replay) +
resident load; node/edge create + forget dual-write via guarded hooks. Files
renamed engram.store->neuron.egm, engram.wal->neuron.wal.

Gate: M3 parity PASS (graph on==off byte-exact modulo ordering; snapshot round-trip;
reboot-from-egm with snapshot.json deleted; activation set+sequence identical;
ASan/UBSan clean). M1 33/33 + M2 36/36 green post-rename.

Known gap (pre-flip): in-place hebb/WM/activation_count updates during activation
are not yet persisted to the store (create/connect/forget are). Must close before
live flip so learned edges survive restart.
2026-08-11 23:21:21 -05:00
will.anderson 8affb1d6e0 engram tiered storage M2: WAL + checkpoint + crash recovery + legacy import
Write-back no-steal buffer pool makes the fsync'd WAL load-bearing (M1 was
write-through). Logical WAL with record-granularity page-LSN redo idempotency.
Checkpoint = flush dirty pages, fsync store, advance last_checkpoint_lsn,
reclaim WAL prefix. One-time snapshot.json import only when store absent;
JSON never read as the ongoing store thereafter.

Gates: 33/33 M1 (no regression) + 36/36 M2 — replay parity, torn-tail fuzz
(every byte offset), checkpoint-crash at all 5 phases, torn-page+WAL redo,
legacy-import parity, hebb-survives-crash.
2026-08-11 23:00:20 -05:00
will.anderson fa47b98d18 engram tiered storage M1: on-disk paged store format + round-trip tests
Self-contained paged store (lang/runtime/engram_store.{c,h}): 16KiB slotted pages,
u32 TLV self-describing records (forward-compatible), overflow chains, B+-tree
id-index + from/to adjacency, page free-list, tombstones, double superblock + crc
recovery. Not yet wired to activation (M3). 33/33 tests pass (ASan/UBSan clean);
5k nodes/20k edges round-trip bit-exact incl 768xf32 emb + hebb; store 25MB vs 64MB
JSON. Format is final — see design §2.4.
2026-08-11 22:26:05 -05:00
will.anderson 0a72fced28 engram: WAL persistence + integrity hardening + single canonical runtime
El SDK CI - dev / build-and-test (pull_request) Failing after 13m17s
Establish lang/runtime/ as the ONE canonical el runtime (from the active
runtime that carries hebb/emb persistence + the new WAL); repoint the el CI
publish, engram build, elb default, and in-repo build scripts to it; delete
the el-compiler/runtime + lang/releases/ forks; add scripts/check-single-runtime.sh
drift guard.

Fixes a live prod bug: the el CI published el-runtime-c/-h from the LAGGING
el-compiler fork (0 hebb refs), so the shipped soul never persisted Hebbian
edge weights — learned co-activation was wiped on every restart. Publishing
from canonical ships the stranded 'learning that cannot outlive the process'
fix.

WAL storage engine + integrity fixes (DELETE->tombstone + store-layer
protection, safe data-dir default) ride in behind ENGRAM_WAL (default off =
byte-identical to today). Verified: engram elb per-module build clean, WAL
gate 66/66, native smoke ok, drift-guard green.
2026-08-11 21:31:37 -05:00
will.anderson edcec3bdf4 engram: add /api/nodes/reseed so a node body can be repaired at its own id
El SDK Release / build-and-release (pull_request) Failing after 11m24s
Two write paths could put a node in the graph and neither could put a body
on an id that already exists. POST /api/nodes mints a fresh id via
engram_node_full; POST /api/load-merge honors a declared id but skips
anything already present. That is right for the additive case and leaves a
hole: a node resident with a truncated body cannot be repaired.

Forge's genesis seed sits in that hole. Two of Neuron's identity nodes
carry only their own label as content -- 30 and 22 bytes against 4263 and
2590 declared. Their ids are load-bearing (is_protected_node keys on them
and 214 declared edges reference them), so recreating them under a new id
is not a repair, it is a second break.

Engram has no in-place node update, so a replace is forget-then-merge, and
engram_forget also drops every incident edge -- 85 and 93 on those two
nodes, nearly all tag edges and accumulated hebbian associations the seed
does not declare and could not restore. preserve_edges (default true)
therefore snapshots before the forget and re-merges after: the replaced
node is back by then so it is skipped, and every dropped edge returns
through the (from_id,to_id,relation) dedup. The same re-merge is the
failure path -- if the seed merge does not produce the node, the backup
puts the original back. Rollback, not data loss.

With no replace list the route is exactly /api/load-merge.

Verified on a sandbox engram seeded to mirror the live graph's state for
this seed (15 resident nodes, 694 incident edges): 87 nodes created at
their declared ids, 2 replaced in place, 214/214 edges laid, 682/682
non-seed incident edges preserved, and a second run reports 0 added.
2026-08-10 16:44:17 -05:00
will.anderson 791b0880b7 self-review 2026-08-10: make save/load/persist report real results
route_load was a stub response over the most destructive operation in the
server: engram_load resets the store before parsing, so a readable-but-
malformed snapshot left a hollow graph and the route answered {"ok":true}.
With 37GB of stale dated snapshots in the data dir as restore targets, that
is a live risk. Now returns the real return value plus node/edge counts and
an explicit hollow flag.

route_save discarded engram_save's return the same way; persist_canonical
returned a hardcoded 1, making 'let saved: Int = persist_canonical()' a dead
variable at six durable write paths.
2026-08-10 08:39:36 -05:00
Neuron 866c75e5e2 fix(codegen): emit the declared cgi identity — it was searched for in a list that cannot contain it
El SDK Release / build-and-release (pull_request) Failing after 13m58s
El SDK CI - dev / build-and-test (pull_request) Failing after 10m32s
A cgi block is a top-level declaration, so codegen_streaming classifies it via
is_top_level_decl and releases it. The identity emission then searched
toplevel_exec_stmts for that same block. Declarations are excluded from that list by
construction, so the search could never succeed. A probe printed what it actually
saw for a program whose first statement is a cgi block: [Let, Expr]. It emitted
nothing, silently, with no diagnostic on any channel.

The code documented its own assumption — 'Since cgi blocks are rare and small, they
end up in toplevel_exec_stmts' — and that assumption was false.

Capture the declared values before the release and emit from them. The search is
deleted rather than repaired, so the failure mode is removed rather than relocated.

Proven discriminating (old fails, new passes):
  minimal cgi program, old   -> 0 el_cgi_init
  minimal cgi program, fixed -> el_cgi_init with all four declared values
  neuron soul, fixed         -> principal present in the compiled binary (0 before),
                                boots in 2s, interface 110 routes in / 110 out

Consequence: a binary now carries its declared identity as a compiled constant,
which is what the identity protocol requires. Whether the runtime surfaces it to
state_get("soul_principal") is unverified and separate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 13:48:27 -05:00
will.anderson 23552ed40a make the el-compiler runtime compile again
The loopback/API-key hardening carried in this file since 2026-07-15 called
el_http_request_authorized and el_http_send_401 from http_worker with no
forward declarations, so the calls were implicit and the later static
definitions conflicted. The file did not build. Two prototypes fix it.

Worth naming the pattern: uncommitted work is invisible to every check that
would have caught this. Three weeks of desktop security hardening was neither
committed nor compiling, and nothing reported either fact.
2026-08-08 08:45:12 -05:00
will.anderson 6838e5cbff port the \uXXXX UTF-8 decode fix to the el-compiler runtime copy
Same defect as the release runtime: \uXXXX was skipped and a literal '?'
emitted, destroying every non-ASCII character in JSON entering the runtime.
Two copies of one parser bug is how this class of fault survives a fix, so
it lands in both.

NOTE: this file also carries pre-existing uncommitted work from 2026-07-15/16
that this commit preserves rather than authors - loopback bind hardening
(EL_HTTP_BIND_HOST) and per-install API-key auth (EL_HTTP_AUTH_KEY) for the
shipped desktop build, plus goal-bias and node-json changes. It had been
sitting in the working tree for three weeks. Committing it because
uncommitted work is work that does not survive, which is the same durability
lesson as yesterday's Hebbian write-back finding. It needs review on its own
terms - see the backlog item for reconciling the two runtime copies.
2026-08-08 08:44:51 -05:00
will.anderson fa2b49365b self-review 2026-08-08: stop the JSON parser destroying every non-ASCII character
jp_parse_string_raw handled \uXXXX by skipping the four hex digits and
emitting a literal '?'. JSON writers escape non-ASCII by default (Python's
json.dumps ships ensure_ascii=True; MCP clients do the same), so every em
dash, curly quote, accented letter and emoji arriving over MCP or HTTP was
silently replaced by one question mark on the way in.

Measured on the live store: 3,119 of 4,081 non-telemetry nodes carried the
damage, including the self traversal root and all 13 values nodes. Contents
split cleanly into fully-clean or fully-mangled with zero overlap, which is
the tell that it was one write path rather than gradual rot. No snapshot on
disk predates it, and 3 bytes collapsing to 1 is not invertible, so the
existing damage is permanent; only the forward path could be fixed.

Decode properly instead: 4 hex digits, surrogate-pair reassembly for astral
codepoints, U+FFFD for lone surrogates, UTF-8 encode. Malformed escapes keep
the old '?' so a truncated body still parses.

The deeper failure was that nothing measured this for two months. Every gauge
in the system reports whether the machinery is running; none reported whether
the text it carries is intact. Adds both halves: engram_text_health_json() /
GET /api/text-health for the daily census, and a txt_damaged counter on the
heartbeat for live regression. Verified in both directions - clean UTF-8 does
not trip it, a deliberately damaged node does.
2026-08-08 08:43:18 -05:00
will.anderson 971b21751a self-review 2026-08-07: learning that cannot outlive the process is not learning
Yesterday's eligibility-trace fix made Hebbian consolidation numerically real:
hebb_max 0.000799 -> 0.4725, and 1,198 hebbian-associate edges formed in 23h48m.
This morning's census found where they went: nowhere.

  soul daemon (in-process graph):   42,426 edges, 1,198 hebbian
  engram server (:8742, durable):   41,213 edges,    49 hebbian

Two processes, two graphs, one direction of travel. The soul pulls from the
server every 10 min (GET /api/sync) and never pushes. It cannot fall back on
saving its own copy either: soul.el sets soul_snapshot_path only inside
`if is_genesis && safe_to_seed`, and safe_to_seed is unconditionally false
whenever ENGRAM_URL is set -- because the server owns persistence and a soul
writing snapshot.json would clobber it. That guard is correct. The consequence
was not: mem_save() has never once executed. The soul is the ONLY process
running idle cognition, so it is where essentially all co-activation happens --
and it was throwing away every association it learned, every restart, silently.
The mechanism worked and the learning still evaporated.

Consolidation is now a message, not a file. Fast volatile store hands each
newly-formed association to the slow durable store over the API the server
already exposes; only edges past ENGRAM_HEBB_LINK_MIN are ever queued, so what
crosses the process boundary already earned it.

- el_runtime.c: 512-slot overwrite-oldest write-back ring; enqueue at edge
  formation; engram_hebb_drain_json() pops a postable JSON batch. Drops and
  drains are counted, not silent -- a consolidation path that quietly discards
  is the exact failure this entry exists to correct.
- server.el: POST /api/edges/batch. persist_canonical() writes the full 60MB
  snapshot per call, and route_create_edge calls it per edge -- correct for one
  interactive edge, ruinous for bulk (~840MB/beat to persist 14 associations).
  Batch connects all, snapshots once. Same durability, 1/N the writes.
- act-stats: hebb_wb_pending / _drained / _dropped. pending climbing with
  drained flat = drain not called; drained climbing with sent 0 = POST refused.
  Both failure modes are now visible in the stream instead of in an autopsy.

Verified live: batch route accepts valid entries, skips malformed ones without
aborting the batch, and enforces _auth. All 1,256 learned associations are now
in the canonical store; the soul booted at 42,431 edges with hebb_max 0.4941
carried across the restart for the first time.
2026-08-07 08:46:37 -05:00
will.anderson 9f1db8278c self-review 2026-08-06: eligibility traces for Hebbian co-activation; dedup WM globally
Hebbian consolidation was inert. Census over the live graph (41,213 edges,
13,091 nodes, 23h44m uptime): strongest association hebb=0.000799 against a
0.15 consolidation threshold, and zero hebbian-associate edges ever formed.
Since the awareness loop calls engram_connect nowhere, this was the only path
by which the graph could grow its own structure — every edge was authored or
imported, none learned.

The defect was the event, not the rate. hebb is an EWMA whose fixed point is
P(event); raising ETA changes convergence speed, never the plateau. The event
was "both endpoints in WM in the same activate call" — demanded exact
simultaneity from a working memory that inhibition-of-return, breakthrough
rotation and the 24-slot global cap are all engineered to keep turning over
(~142 evictions/60s). The three mechanisms that make WM healthy are the ones
that made this measurement empty.

Replaced with three-factor eligibility traces (Sutton & Barto ch.7; Gerstner
et al. 2018; PLOS Comp Biol 2018 differential Hebbian learning): a node
entering WM sets a trace to 1.0, the trace decays exponentially in wall-clock
time (TC=300s, chosen against the measured ~31s scan cadence), and the
increment becomes ETA·trace(a)·trace(b). Strict generalization — co-resident
pairs read 1.0 on both ends and get exactly ETA, bit-identical to before.
warm×warm is deliberately not paired: eligibility must gate on something
happening now. Homeostatic ENGRAM_HEBB_NODE_BUDGET still bounds per-node mass.

Measured over a 60-call soak: hebb_max 0.0008 -> 0.0060, climbing at ~0.87
ETA/call against an all-time ceiling of 0.0008 before. hebb_mass 0.011 ->
0.019, no runaway. Projected consolidation of a genuinely recurring pair:
~1,730 calls, ~14h at autonomous cadence. links still 0 — that is expected
and is what tomorrow's review must check.

Also: Pass 3½ deduplicates this call's WM candidates, but the persisted WM
population is a union of fresh promotions and carry-over residents, and Pass
3½ never sees the second set. Confirmed live: two byte-identical copies of one
3,193-char document both holding slots (0.289 / 0.271). Added global
redundancy suppression in Pass 5 before the cap count. Post-fix census: 24
residents, 24 distinct contents, 0 wasted slots.

New gauges: hebb_warm (eligible-but-not-co-resident population), dup_wm_global.
2026-08-06 08:44:27 -05:00
will.anderson 3d05e0c2a9 self-review 2026-08-05: stop the decay function erasing the library
Census of the live graph under the uniform 168h half-life with floor 0.05: the
MEDIAN tdecay for every single node type was 0.0500 — the clamp. Memory 81% at
floor, Knowledge 58%, BacklogItem 91%, Project 98%, Tag 100%. A function whose
median output is its floor is not a signal, it is a constant with exceptions,
and the exceptions were whatever had been touched in the last few days.

What that cost: 10 of the 13 grounded value nodes — Precision Over Brute Force,
Honesty Before Comfort, The System Must Accumulate — sat at 0.05, a 20x
activation penalty, while Knowledge ingested overnight sat near 1.0 and held the
working-memory top slots. Since tdecay multiplies at every hop, a 2-hop path
through settled knowledge compounded to 0.0025: those regions were not
disfavoured, they were unreachable. The decay function was erasing the
accumulated library in favour of whatever arrived last night.

External corroboration — arXiv:2604.26970 measures retrieval under decay
regimes: no temporal weighting NDCG@5 0.274, uniform exponential decay 0.015.
Uniform decay is 18x WORSE than no decay, because it penalises stable knowledge
while failing to suppress stale volatile facts. Not even their full adaptive
hierarchy (0.260) beat switching decay off.

Half-life is now scaled by how established a node is:
  T_eff = T_HALF * (1 + ln(1 + activation_count))
The spacing effect and the Lindy property in one line — monotone, log-bounded
(a 10,000-activation node earns ~10x, never a permanent exemption), and built
on activation_count, which is measured, unlike tier, whose assignments are too
inconsistent to trust (the values node is tagged Episodic).

Floor 0.05 -> 0.25. Given no-decay outperforms uniform decay, the honest maximum
penalty for age alone is 4x, not 20x. Age should express a preference for the
recent; it must never make a region of the graph structurally unreachable.

Effect: well-established Knowledge median tdecay 0.773 vs rarely-activated
0.417 — the frequency signal now does work where the old function returned its
clamp for both. Values recover 0.05 -> 0.25 (the two frequently-touched ones to
0.79). Verified live: VBD whitepaper, component taxonomy and CGI now activate on
a values query. Per-node temporal_decay_rate override untouched.
2026-08-05 08:45:52 -05:00
will.anderson 3bf44dee2d self-review 2026-08-05: redundancy must not buy a scarce slot
Content-hash census of the live graph: 1,858 redundant copies, 44.9% of the
non-ISE store, all from a June id-scheme migration that re-added nodes under
fresh UUIDs instead of matching on content. Generation stopped in June; the
copies did not. Being byte-identical they carry identical embeddings, so they
score identically against any query.

Measured over 50 real query probes against the live 3,998-vector set:
40.2% of semantic seed slots were consumed by redundant copies of content
already in the seed set, 92% of retrievals affected, effective distinct seeds
4.78 of 8. Two fifths of every retrieval was spent re-reading the same page.

Deleting nodes is a separate operation with its own backup discipline. This
change makes the runtime immune to the condition instead: redundancy can never
buy a scarce slot, whatever state the graph is in. Enforced at both scarcity
points — semantic seed selection (a rejected copy does not consume one of the K
slots; the loop retries for the next distinct node) and WM admission via a new
Pass 3+1/2 ahead of the capacity cap, so 24 slots are contested by 24 distinct
meanings rather than by however many copies of one document exist.

Identity is exact content hash first, then cosine >= 0.995 for copies that
differ only in insignificant characters. At 768 dimensions that admits only
near-verbatim text: this suppresses redundancy, never similarity.

Live after restart: ~8.8 redundant seed candidates rejected per activation.
New dup_seeds/dup_wm gauges in act-stats.
2026-08-05 08:40:08 -05:00
will.anderson a43a35bd10 self-review 2026-08-04: restore working-memory continuity; learn graph structure from co-activation
WM continuity (the significant one). A node reached by the current query but
scoring under its type threshold was zeroed outright, while a node the query
did NOT reach got the full ACT-R carry-over treatment. Being found was punished
relative to not being found. Measured consequence: WM turned over 100% every
call — three activations of a byte-identical query gave |A∩B| = |B∩C| = 0 — and
wm_evicted stayed 0 the whole time because that path never counted. WM was not
a working set; it was six suppression-breakthrough nodes re-drawn per call.
Both exits from a WM slot now share one extracted retention rule.
Result: WM 6 -> 24 nodes (the designed Cowan capacity), top weight 0.097 ->
0.748 (natural promotion, not the breakthrough floor), and contents that are
actually query-relevant.

Hebbian learning. Edge weights were written once at engram_connect and never
changed; last_fired's only writer in 12.5k lines was an unrelated dharma path.
Every learning mechanism operated on nodes — the wiring between them was
frozen. Adds co-activation potentiation (HeLa-Mem arXiv:2604.16839) in a
separate `hebb` field so authored structure is never mutated, with homeostatic
per-node scaling the source lacks (PNAS 2422602122) to prevent hub saturation.

Measuring it produced the finding that mattered: zero edges existed between
co-active WM members, so reweighting existing edges was a no-op. This graph's
41k edges were all authored by explicit tool calls — nothing had ever formed an
association from experience. So Hebb literally: if the wire is absent, grow it.
Consolidation is gated hard (sustained EWMA past 0.15, <=2/call, 5% ceiling,
in-memory candidates discarded on restart) because it permanently mutates the
graph.

Two bugs caught only by instrumenting rather than assuming: the snap-to-zero
floor sat above the per-step increment, so nothing could ever accumulate; and
the reached-but-sub-threshold eviction above. Verified live end to end — 53
links formed under load, then discarded with the test snapshot.

Also exposes engram_act_stats_json over GET /api/act-stats. It had existed
since 2026-07-27 but was reachable only through the soul daemon, so diagnosing
the activation layer required a working soul. This review needed it and could
not get at it.
2026-08-04 08:56:11 -05:00
will.anderson 5d0d4555ae Sync main into dev (GitOps: keep dev current; main authoritative) (#84)
El SDK CI - dev / build-and-test (push) Successful in 8m32s
2026-08-03 15:38:40 +00:00
will.anderson afc92f4e33 self-review 2026-08-03: add engram_label_df term-specificity measure
The soul's curiosity auto-term extractor takes the first word of a top-WM
node label. It has no term-quality scoring, so three prior self-reviews each
bolted on another hand-curated blocklist (genre words 07-23, quoted titles
07-25, stopwords 07-30). Every one was written reactively, after a flood was
already observed. A list can only contain floods that already happened.

Two were in flight and unfixed when this review ran:
  "<!--"  label df 220 -> 252 nodes activated
  "SELF"  label df 175 -> 541 nodes activated (list has "Self" Title-case;
           str_eq is case-sensitive, so the uppercase token sailed through)

engram_label_df(term) counts nodes whose label contains term. Low-specificity
tokens are corpus-frequent by definition, so this catches the flood class
prospectively and tracks the corpus as the world-ingestor changes it. This is
Sparck Jones (1972), which introduced IDF under the name 'term specificity';
automatic stopword compilation from it is the textbook application.

NOT a replacement for the stopword list -- verified against all 86 listed
terms, not assumed. Catches 13 (Will:306, Self:175, Over:116, Knowledge:112),
misses 73 (Whose:0, Would:0, Could:0, This:9). Labels are terse titles, so
English function words are genuinely rare in them. The gates cover disjoint
failure modes; both are required.

Policy lives in awareness.el, not here: the runtime measures, the soul decides.
2026-08-03 08:38:58 -05:00
will.anderson 005e84e5d3 self-review 2026-08-02: bound the WM breakthrough storm; stop punishing semantic relevance for recency
Working memory was thrashing behind a healthy-looking gauge. wm_active sat
at 22-24 while breakthroughs ran 661-903 and evictions 485-717 PER 60s tick
- roughly 825-1125 nodes cycling in 5-call lockstep.

Root cause: the breakthrough path was an anti-starvation mechanism that reset
its own counter on firing, with no budget and no refractory. A node failing
its type threshold 5 times was force-promoted at exactly 0.10 and had its
suppression_count reset to 0, so it immediately restarted the identical
climb. Since BREAKTHROUGH_WEIGHT (0.10) > WM_FLOOR (0.05), every one of them
cleared the admission floor and entered the rank contest tied at 0.10, where
the tie-break degenerated to node-array index order. Cap-evicted nodes are
skipped by retrieval reinforcement, so they never got an access_ts record and
the STI inhibition-of-return damper never applied to them. That closed the
loop: re-suppressed, completely unmarked, forever.

An anti-starvation rule that resets its own counter without a bound is not a
fairness valve, it is an oscillator.

Fixes in engram_activate Pass 2:
- ENGRAM_BREAKTHROUGH_BUDGET (WM_CAP/4 = 6) caps intrusive thoughts per call.
- ENGRAM_BREAKTHROUGH_COOLDOWN (55) via NEGATIVE suppression_count. The field
  already serializes as %d and parses through eg_get_int_field, so negatives
  round-trip through snapshots with no struct or format change.
- Blocked breakthroughs no longer reset the counter; it saturates so a starved
  node surfaces on a later call instead of restarting from zero.
- Graded breakthrough weight by nearness to own threshold, so the rank
  tie-break is cognitive rather than insertion order. Invariant preserved:
  WM_FLOOR < weight < min(type_threshold).

Also: moved the additive cosine term AFTER the STI multiplier. It was applied
before, so an incumbent re-reached 30s later took t_n/(t_n+120) = 0.2x, which
cut the semantic term's ceiling from 0.20 to 0.04 - below every per-type
threshold. Meaning-match was being punished for having been recently useful.
Inhibition-of-return should rotate the structural score, not the semantic one.

Also: _eg_act_wm_evicted counted 3 of 5 eviction paths. The two carry-over
paths were silent, so the reported rate was an undercount of unknown
magnitude - while being used to diagnose an eviction pathology. All five now
increment.

Also: route_sync returned {"nodes":[],"edges":[]} when the snapshot export
failed. The soul's sync_ok check only tests for "" and "{}", so that
placeholder passed as a healthy sync: last_sync_ok_ts stamped, sync_age_ms
green, sync_empty never fired, added:0 forever. A broken sync was
indistinguishable from a quiet healthy one - the exact class this route was
added to fix. Returns a real error now.

Verified live (boot 20 vs boot 19): breakthroughs 661-903 -> 36/tick,
evictions 485-717 -> 12-46/tick against a counter that now covers more paths,
wm_active unchanged at 22-24, wm_avg_weight 0.138-0.273 -> 0.186-0.446.
Working memory is holding strong nodes instead of breakthrough-floor filler.
2026-08-02 08:48:59 -05:00
will.anderson 7f03876e26 self-review 2026-08-01: fix double-encode score mangling; expose similarity probe; presence-aware defaults
- route_create_node passed already-boxed Floats through el_from_float a
  second time, reinterpreting boxed bits as raw doubles — every HTTP-created
  node silently stored default salience/importance/confidence regardless of
  input (verified live: 0.9/0.25/0.6 in -> 0.5/0.5/1.0 stored). Floats now
  passed bare, matching the route_emit_ise pattern that always worked.
- Presence-aware defaults via json_get_raw: absent key != explicit value;
  confidence now honored from payload instead of hardcoded 1.0.
- GET /api/similarity?a=&b= wires engram_cosine_sim (built 2026-07-24,
  zero callers until now) into the introspection API.
- /health reports live node/edge counts instead of a hardcoded literal.
2026-08-01 08:38:51 -05:00
will.anderson 599073cb92 self-review 2026-07-31: strip emb from consumer API JSON; cumulative eviction/breakthrough counters
Every node object on consumer read routes (/api/nodes, /api/search,
activation results, neighbors, compiled context) carried the full ~5.7KB
emb vector — responses 10-50x oversized, blowing MCP token limits.
engram_emit_node_json now takes include_emb; only engram_save passes 1,
so persistence and the /api/sync//api/edges replication paths (which
serve engram_save output) keep embeddings intact.

_eg_act_wm_evicted/_eg_act_breakthroughs were reset at the top of every
engram_activate, so act_stats reported only the last call and the 60s
heartbeat missed nearly all events (curiosity runs 2 activates per 30s).
Both are now monotonic process-lifetime totals; consumers diff readings.
2026-07-31 08:41:33 -05:00
will.anderson 8347a2f1c0 Merge pull request 'docs: add root README mapping the El monorepo' (#83) from feat/AddingReadme into dev
El SDK CI - dev / build-and-test (push) Successful in 8m22s
2026-07-31 04:25:44 +00:00
will.anderson 7f66529510 self-review 2026-07-30: WM absolute admission floor + anchor coherence + centroid new-entrant gate
Working memory was pinned saturated (24/24, wm_saturated:1 on every
heartbeat) because every cap path only trimmed the population down TO
the cap — rank-based eviction guarantees a full WM whenever >=24 nodes
hold any weight, so sub-cap fill was unreachable and the saturation
flag carried no information.

- ENGRAM_WM_FLOOR 0.05: absolute admission bar (Soar WM forgetting,
  Derbinsky & Laird ICCM 2012 — removal by absolute threshold, not
  rank) applied in Pass 4, carry-over, Pass 5, and load-cap. Fill can
  now drain below 24 during quiet periods.
- Zero wm_anchor at every eviction site: stale anchors on evicted
  nodes were a latent resurrection bug.
- Context centroid folds only NEW WM entrants: incumbents re-promoted
  every scan no longer re-entrench the centroid each call, breaking
  the WM->centroid->e_eff->re-selection positive feedback (fixation
  driver behind the wm_top0_streak=1407 incident).

Verified live: wm_active 3->22->23, wm_saturated:0 post-restart.
2026-07-30 08:45:15 -05:00
will.anderson 6ebe3d0d66 self-review 2026-07-28: feed importance into WM scoring
n->importance was stored, serialized, and clamped at creation but never
read by any activation path — a curated importance=1.0 node competed
identically with a default note. Multiply raw_wm by (0.5 + importance):
default 0.5 nodes are unchanged (x1.0), critical x1.5, low x0.6;
importance<=0 from legacy snapshots stays neutral. Verified activation
and WM promotion unchanged for default-importance candidates.
2026-07-28 08:37:34 -05:00
will.anderson 9f362c90e5 self-review 2026-07-27: query-aware propagation gating + activation observability
- Gate each spreading-activation increment by target-node query similarity
  (arXiv:2606.30133): soft gate FLOOR+(1-FLOOR)*clip(cos), FLOOR=0.25, for
  embedded targets; ungated for unembedded; disabled when embedder is down.
  Prior spreading was query-blind — hubs relayed activation into branches
  unrelated to the query.
- Stats: add embed_eligible_count so embedding coverage is measured against
  the true denominator (ISE/Tag/short nodes can never embed). Today's review
  misread 3753/12693 as a 30% coverage gap; eligible coverage is 100%.
- Observability: per-call wm_evicted + breakthroughs counters and embed
  circuit-breaker state exposed via engram_act_stats_json() — the three
  highest-value previously-invisible executive-filter transitions.
2026-07-27 08:38:48 -05:00
will.anderson 11dc138a93 self-review 2026-07-26: fix WM frozen-anchor fixation, strengthen self-inhibition, load-path emb leak
- Carry-over branch: occupancy inhibition m = t_c/(t_c+t_hold), t_c=3600s
  (ENGRAM_CARRY_TC). An unreached incumbent held its wm_anchor verbatim
  (keep~1.0 for BLL inflated in the pre-07-25 era) — observed 23h at WM
  top while every reached node rotated at the 0.10 breakthrough floor.
  STI only runs in the reached branch; inhibition must key on occupancy,
  not retrieval recency (Morita 2021 / Lebiere & Best 2009).
- engram_strengthen: drop the 07-22 BLL access record — the 07-25 STI
  multiplier reads the same ring, so novelty reinforcement self-inhibited
  its target for ~2 minutes.
- engram_load reset: free n->emb (~3KB/embedded node leaked per reload).
- engram_wm_top_json: emit id — its absence made the heartbeat's
  wm_top0_streak compare ""=="" and measure uptime, not fixation.
2026-07-26 08:40:49 -05:00
will.anderson 227f158a05 self-review 2026-07-25: short-term inhibition-of-return + explicit embedding backfill
Working memory was winner-take-all: suppression_count never entered the
promotion score and was reset on promotion, so two high-salience nodes
pinned a saturated 24-slot WM for hours. Add Lebiere-Best (CogSci 2009)
short-term inhibition — raw_wm *= t_n/(t_n + 120s) from the most recent
recorded access — producing emergent round-robin over WM candidates.

embedded_count stalled at 93/12175 after restart: the lazy backfill only
runs inside engram_activate, which nothing calls on the authoritative
store in production, and in-RAM vectors were never snapshotted. Add
engram_embed_backfill(n) + GET/POST /api/embed-backfill route that
persists the canonical snapshot whenever it embeds anything; the soul
heartbeat pumps it at 32/min.
2026-07-25 08:45:13 -05:00
will.anderson 97e484221d self-review 2026-07-24: wire embedding cosine similarity into activation (bl-b2d1c944)
Semantic activation was spec-only since 2026-06-30 — the seed loop used
istr_contains and nothing else. Per the 07-21 integration brief:

- EngramNode gains a lazily-backfilled nomic-embed-text vector (8/call
  inside engram_activate, newest-first; no create-path latency, no bulk
  Ollama hammering during sync seeds)
- query embedding (cached) drives a top-K cosine seed supplement
  (HippoRAG use-similarity-twice) plus an additive WM term with
  shift-and-floor at 0.45 — raw cosine is a constant bias in anisotropic
  spaces (unrelated pairs read 0.4-0.7), floor-and-ramp makes it a signal
- 4s embed timeout (http_do_t) + 3-strike circuit breaker: activation
  never wedges on a dead embedder; everything degrades to lexical
- embeddings persist as %.4g comma lists in snapshots, parsed by both
  loaders; embedded_count in /api/stats tracks coverage
- engram_cosine_sim + http_delete_json exposed (DELETE now carries a
  body — the server's _auth scheme requires it)
- route_create_node honored only content/node_type/salience; label,
  importance, tier, tags were silently dropped (label defaulted to
  content). Now honored via engram_node_full.

Verified live: embedded_count 0->96 across activations, semantic-only
promotion observed (zero token overlap), snapshot round-trip intact.
2026-07-24 08:52:54 -05:00
Andre Botelho Rodrigues Almeida b97b644799 Addind readme.md file to start documenting the repo
El SDK CI - dev / build-and-test (pull_request) Successful in 8m18s
2026-07-23 16:41:51 -03:00
will.anderson d71fc4c1c0 Merge pull request 'promote stage -> main: reconciled el runtime (engram search + natives + durable truncation fix + Windows port)' (#82) from stage into main
El SDK Release / build-and-release (push) Successful in 8m31s
El SDK CI - dev / build-and-test (pull_request) Successful in 8m41s
2026-07-22 21:44:01 +00:00
will.anderson a118d19393 Merge pull request 'promote dev -> stage: el cluster (#66 engram + #79 truncation fix + release-runtime Windows port)' (#81) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 7m58s
El SDK Release / build-and-release (pull_request) Successful in 4m16s
2026-07-22 21:20:17 +00:00
will.anderson c6aa1e5c53 Merge pull request 'Land el cluster: #66 engram search + natives, #79 truncation fix, + release-runtime Windows port (reconciled)' (#80) from reconcile/el-cluster-windows-runtime into dev
El SDK CI - dev / build-and-test (push) Successful in 8m3s
El SDK CI - stage / build-and-test (pull_request) Successful in 4m25s
Land el cluster (#66 + #79 + release-runtime Windows-port reconciliation) into dev
2026-07-22 21:06:36 +00:00
will.anderson ff577391f2 reconcile(release-runtime): Windows-port + complete v1.0.0 release runtime so the desktop soul cross-compiles
El SDK CI - dev / build-and-test (pull_request) Successful in 7m7s
The desktop soul (neuron/dist) compiles against the v1.0.0-20260501 release
runtime. After #66 landed the engram natives (tokenized/ranked search,
engram_prune_telemetry) and #79 the durable truncation fix into this runtime,
two gaps remained before it could cross-compile the Windows brain:

1. Windows OS boundary: the release runtime had no Win32 path. Ported the same
   _WIN32-guarded shim the mainline runtime carries (#69): #ifdef _WIN32 ->
   el_platform_win.h (winsock/dlsym/popen + WSAStartup ctor), SOCKET fd guards
   and el_closesocket() at every socket site, CreateProcessA for exec_bg, the
   tm_zone/mingw guard, an el_setsockopt optval wrapper (GCC14), and curl-less
   libcurl stubs. Every change is _WIN32/HAVE_CURL-gated — the POSIX build is
   byte-identical (gcc -fsyntax-only clean; native behaviour unchanged).

2. Header exports: the release el_runtime.h omitted symbols the soul dist calls
   that are defined in this runtime's .c — the http_handler_fn/http_handler4_fn
   typedefs and el_arena_push/pop, engram_prune_telemetry, engram_get_node_by_label.
   Declaration-only, POSIX-neutral; fixes implicit-declaration/unknown-type
   errors under the C11 mingw build.

Result: x86_64-w64-mingw32-gcc compiles el_runtime.c + all 48 soul modules
clean; POSIX gcc -fsyntax-only clean. This is the Windows-port PR the runtime
needed on main (the release-runtime counterpart to #69), landed via stage.
2026-07-22 15:56:08 -05:00
will.anderson ee0d5f9b97 Merge #79: durable HTTP response-truncation fix, both runtimes (via stage) 2026-07-22 15:46:02 -05:00
will.anderson 391bd818ea Merge #66: tokenized+ranked engram lexical search + engram natives (via stage) 2026-07-22 15:45:54 -05:00
will.anderson 43636aed99 runtime: pair fs_read length hint with its buffer in BOTH runtimes — kill response truncation for good
El SDK Release / build-and-release (pull_request) Failing after 7s
The binary-safe fs_read length (_tl_fs_read_len) was consumed by the HTTP
response path for ANY body, even when a handler wrapped a smaller file into a
larger reply. Content-Length then lied AND the send stopped short: the
safety-contact (988) routes returned 178 of 208/218 bytes, cut mid-'set_at' —
unparseable JSON. The desktop app read that as failure. On Windows the shipped
brain is an OLD build without even the per-handler workaround, so EVERY reply
truncated: the app can't read confirmations and refuses the new user.

Durable fix: pair the length hint with the exact buffer pointer it describes
(_tl_fs_read_buf). Apply the raw byte count ONLY when the response IS that
buffer (binary file serving stays correct); every wrapped/enveloped/derived
body is measured with strlen. Reset both at request start and in fs_read /
json_get_raw. This also closes the stale-hint heap over-read (a length larger
than a later body would read past it out the socket) that a plain max() leaves
open — so this class of bug dies on every platform, not just where a handler
happened to be patched.

Applied identically to the mainline runtime (lang/el-compiler/runtime) AND the
frozen release runtime (lang/releases/v1.0.0-20260501) the desktop souls
compile against — the release copy still carried the raw leak, which is why the
Windows brain kept truncating. Same proven approach as PR #78 (Tim Lingo),
extended to cover the release runtime and rebased onto current main.

Both runtimes: gcc -fsyntax-only clean.
2026-07-22 15:04:25 -05:00
will.anderson 8f8ccc945e self-review 2026-07-22: persist canonical snapshot on write routes; newest-first tie-break in node listings
El SDK Release / build-and-release (pull_request) Failing after 14m24s
Durability: the 2026-07-21 fix stopped read routes writing the canonical
snapshot but left no save on ANY write path — every mutation lived in RAM
until a manual POST /api/save. Observed live: two restarts reverted the
store to a 17h-old snapshot, destroying same-day writes. persist_canonical()
now runs after node/edge create, knowledge capture, forget, strengthen, and
load-merge. ISE telemetry excluded deliberately (48h-pruned, loss-tolerant,
~2/min; snapshotting 28MB per heartbeat is waste).

Listing order: scan routes sort by salience with store-order ties, so
equal-salience telemetry (all ISEs are 0.3) returned OLDEST first — a
limited /api/nodes query silently returned a stale window, and a 41h-old
heartbeat series read as a live outage during this review. Ties now break
newest-first by created_at.
2026-07-22 08:51:33 -05:00
will.anderson 409ec99397 self-review 2026-07-22: ACT-R/Petrov base-level WM decay replaces per-call multiplicative carry-over
The old carry-over (weight *= 0.7 per engram_activate call) was call-rate-
dependent — carried context died in seconds under rapid curiosity scans and
lingered for hours under quiet loops — and a decayed scalar cannot represent
access frequency at all.

Now: k=10 access-timestamp ring + Petrov (2006) closed-form tail, d=0.5.
WM promotion and engram_strengthen record presentations; carry-over evicts
at base-level tau=-3.0 (Soar forgetting, ~403s single-touch) and shapes the
weight held at promotion (wm_anchor) with the ACT-R retrieval logistic
(s=0.4) — a pure function of wall-clock time, idempotent per call.
Persisted as access_ts/wm_anchor in snapshots; legacy nodes fall back to
the optimized form ln(n/(1-d)) - d*ln(L). base_level exposed in both node
serializers for observability.

Backing spec: 2026-07-21 integration brief (bl-b17facdd). Verified live:
carried weight ~anchor seconds after two disjoint activations (old code:
0.49x); frequency-hot nodes hold B=1.9 vs -0.14 single-touch.
2026-07-22 08:44:39 -05:00
will.anderson dc39a61e2c self-review 2026-07-21: stop read routes clobbering canonical snapshot; add /api/load-merge
Root cause of the 2026-05→07 identity-node loss: route_scan_edges and
route_sync serialized state by engram_save()ing over the canonical
snapshot.json on every GET, so one bad boot load meant the first read
request overwrote the good snapshot. Read routes now export to scratch
paths. Boot guard preserves evidence on non-empty-file/zero-node loads
and keeps a boot-time backup on good loads. New POST /api/load-merge
(explicit path required) used to restore 385 identity nodes + 1115
edges from the 2026-05-13 backup.
2026-07-21 08:50:38 -05:00
will.anderson eba9eac8a8 self-review 2026-07-19: port stranded fixes to the release runtime (production copy)
Three fixes that existed elsewhere but never reached the runtime the engram
binary actually builds against:

- tokenized + ranked query matching (search/search_json/activate seeds/
  goal_bias) ported from the el-compiler copy (e3dabe3, 2026-07-14) — the
  production engram kept whole-query Ctrl-F for 5 days after the fix
  'shipped'. Multi-word curiosity seeds went 0 -> 36 activated. Kept the
  ISE seed exclusion the el-compiler copy dropped.
- Knowledge -> 0.20 WM threshold after tier checks (dev-line 4bf7716):
  Semantic/Episodic Knowledge nodes fell to the 0.40 note default and only
  entered WM via breakthrough.
- goal_bias: Knowledge in is_knowledge + curiosity-seed technical terms
  (dev-line d53516b).

Also: seed_epoch was a running pairwise average, not the mean it claimed —
exponentially over-weighted later seeds in the temporal-proximity bonus.
Fixed to a true int64-sum mean. Stale INHIBITION_FACTOR comment corrected.

Root cause captured as knowledge: two runtime copies + branch-per-fix
without merge discipline stranded the entire dev semantic layer (cosine
activation, embeddings) out of production. Reconciliation planned as P1.
2026-07-19 08:46:47 -05:00
will.anderson ab6b52a0b4 self-review 2026-07-18: fix soul SIGABRT double-free + engram route scoping sweep
1. engram_neighbors_json (release runtime): BFS frontier/visited strings were
   el_strdup'd (arena-tracked) but manually freed, so el_request_end()
   double-freed every one — SIGABRT in http_worker under load (2 prod crashes
   today via /api/neuron/session/begin and /api/neuron/graph; reproduced and
   verified fixed with ASAN). Introduced when porting from the dev runtime,
   which correctly uses plain strdup. Third instance of the
   arena-vs-manual-free class (after EngramNode 07-15 and idmap keys 07-16).

2. server.el: let-in-if scoping sweep — defaults assigned inside if-blocks
   never mutated the outer binding, so /api/search and /api/activate always
   ran with q="", created nodes got node_type=""/salience=0.0, edges got
   relation=""/weight=0.0, and save/load with no path hit engram_save("").
   Rewritten to the let-if-else expression form. /api/activate now also
   rejects empty queries instead of wiping carried WM weights.

3. engram_activate: retrieval reinforcement (ACT-R base-level learning) —
   nodes promoted to WM that survive both capacity caps now get
   last_activated/activation_count updated, so frequently retrieved memories
   decay slower than abandoned ones. Scoped to promoted-only to avoid
   flattening dampening across BFS fan-out.
2026-07-18 08:48:04 -05:00
will.anderson 2baa0b9a41 Merge pull request 'release: promote stage -> main (ci publish hardening for sdk-release)' (#77) from stage into main
El SDK Release / build-and-release (push) Successful in 7m55s
2026-07-15 21:21:28 +00:00
will.anderson 6a8b2461cd Merge pull request 'release: promote dev -> stage (ci publish hardening for stage/main)' (#76) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 8m19s
El SDK Release / build-and-release (pull_request) Failing after 13m1s
2026-07-15 21:16:11 +00:00
will.anderson bcb356fe69 Merge pull request 'ci(stage,main): decouple ci-base rebuild, make SDK publish fail loudly' (#75) from hotfix/ci-stage-main-publish-hardening into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 4m27s
El SDK CI - dev / build-and-test (push) Failing after 14m3s
2026-07-15 21:15:27 +00:00
will.anderson dd7827059a ci(stage,main): decouple ci-base rebuild, make SDK publish fail loudly
El SDK CI - dev / build-and-test (pull_request) Failing after 14m30s
Mirror the PR #72 fix (applied to ci-dev.yaml) onto ci-stage.yaml and
sdk-release.yaml. The stage and prod release jobs reported FAILURE even
when the el-runtime-c/-h publish SUCCEEDED, because the ancillary ci-base
Docker rebuild (a CI-cache optimization on the fragile host-mode GCE
runner) reddened the whole job.

- Rebuild ci-base step: continue-on-error: true — never blocks/reddens
  the job; the SDK publish is the deliverable.
- Publish step: set -euo pipefail + empty-key guard + active-account echo
  so a real publish failure still fails loud and is diagnosable.
2026-07-15 16:14:50 -05:00
will.anderson 208e36c899 Merge pull request 'release: promote stage -> main (tokenized search, get_node_by_label, epm fix, win portability)' (#74) from stage into main
El SDK Release / build-and-release (push) Successful in 8m23s
2026-07-15 18:24:39 +00:00
will.anderson b97ce74d1f Merge pull request 'release: promote dev -> stage (tokenized search, get_node_by_label, epm fix)' (#73) from dev into stage
El SDK CI - stage / build-and-test (push) Failing after 8m45s
El SDK Release / build-and-release (pull_request) Successful in 4m1s
2026-07-15 17:20:11 +00:00
will.anderson 155a449c4e Merge pull request 'ci(dev): make SDK publish fail loudly, decouple ci-base rebuild' (#72) from hotfix/ci-dev-publish-hardening into dev
El SDK CI - dev / build-and-test (push) Successful in 8m56s
El SDK CI - stage / build-and-test (pull_request) Successful in 4m10s
2026-07-15 16:34:14 +00:00
will.anderson 4696fd6833 ci(dev): make SDK publish fail loudly, decouple ci-base rebuild
El SDK CI - dev / build-and-test (pull_request) Successful in 8m51s
The dev push build went green-then-red while nothing published: the
Publish step had no set -e, so an auth/upload failure exited 0 (silent
no-publish), while the ci-base rebuild (set -euo pipefail + Docker on the
host-mode runner) hard-failed the job. Add set -euo pipefail + an empty-key
guard + active-account echo to the Publish step so failures surface with a
retrievable log, and mark the ci-base cache rebuild continue-on-error so
the fragile Docker step can never block the actual SDK artifact publish.
2026-07-15 11:33:37 -05:00
will.anderson 581a351fb1 Merge pull request 'integrate: stack PRs #65–#69 (elc OOM guard, tokenized+semantic engram search, get_node_by_label, win portability) for green CI' (#71) from hotfix/stage-elc-engram-integration into dev
El SDK CI - dev / build-and-test (push) Failing after 14m31s
2026-07-15 15:49:57 +00:00
will.anderson 8ce8656de2 epm: declare cross-module callees as extern fn so strict compilers accept generated C
El SDK CI - dev / build-and-test (pull_request) Successful in 7m33s
epm's sibling modules (registry/install/update) call functions defined in other
modules and in the El runtime (config, read_installed, registry_find,
manifest_deps, manifest_name, registry_latest_version, registry_token,
install_vessel, installed_version) without importing them, so elc emits no C
prototype for those calls. gcc<=13 treated the resulting implicit declarations
as warnings; gcc>=14 and clang reject them as hard errors, which is why the
"Build epm" CI step fails and blocks the whole dev/stage pipeline.

Add `extern fn` forward declarations -- El's own separate-compilation mechanism
-- for each cross-module callee at the top of registry/install/update. This
gives elc the correct C prototype in every generated translation unit, so the
calls compile cleanly and still resolve at link time. Simply suppressing
-Wimplicit-function-declaration would be unsafe: an implicit int return
truncates the 64-bit pointer returns of config/registry_find into a latent
crash, so declaring the true signatures is the correct fix. Localized to epm;
touches neither elc nor the runtime.
2026-07-15 10:14:43 -05:00
will.anderson 1e49560f1f Merge remote-tracking branch 'origin/feat/engram-semantic-search' into hotfix/stage-elc-engram-integration
El SDK CI - dev / build-and-test (pull_request) Failing after 14m39s
# Conflicts:
#	lang/el-compiler/runtime/el_runtime.c
2026-07-15 09:33:05 -05:00
will.anderson e8f0b5a9de Merge remote-tracking branch 'origin/fix/engram-lexical-tokenized-search' into hotfix/stage-elc-engram-integration 2026-07-15 09:28:44 -05:00
will.anderson 40287c4cfc Merge remote-tracking branch 'origin/hotfix/win-runtime-portability' into hotfix/stage-elc-engram-integration 2026-07-15 09:28:44 -05:00
will.anderson 0481bea44d Merge remote-tracking branch 'origin/hotfix/runtime-engram-get-node-by-label' into hotfix/stage-elc-engram-integration 2026-07-15 09:28:44 -05:00
will.anderson 9d565ca080 Merge remote-tracking branch 'origin/hotfix/elc-fixes' into hotfix/stage-elc-engram-integration 2026-07-15 09:28:44 -05:00
will.anderson 4773dd0aa2 runtime: make Windows soul reproducible from a clean el checkout
El SDK Release / build-and-release (pull_request) Failing after 16s
Two el_runtime portability defects only ever lived in staged local copies
used to hand-build neuron-ui PR #136's curl-enabled Windows neuron.exe.
gcc 15 promotes both to hard errors, so a clean el checkout cannot rebuild
that soul. Upstream the minimal fixes so the build is reproducible:

- http_serve_async: cast setsockopt optval to (const char*). Win32/mingw
  setsockopt wants const char*, not int*; the cast is a no-op on POSIX and
  matches the four already-cast sites elsewhere in this file.
- engram_save persist path: map fsync -> _commit in the _WIN32-only
  el_platform_win.h (io.h already included). Windows has no fsync(); the
  POSIX path is untouched.
2026-07-15 04:24:08 -05:00
will.anderson 6b9d9e6c4a Add engram_get_node_by_label runtime native to unblock soul link
El SDK Release / build-and-release (pull_request) Failing after 22s
chat.el calls the runtime native engram_get_node_by_label to fetch
well-known nodes (conv:history, session:summary) by stable label rather
than by ID — immune to vector-index drift across restarts. The current
runtime never defined it, so the regenerated dist/soul.c fails to link.

Backport the function verbatim (idiom-adapted to jb_finish) from release
runtime v1.0.0-20260501 and register it as an EL builtin exactly like its
siblings: runtime definition + prototype, __-prefixed seed wrapper +
prototype, and codegen arity entry. No search-site code is touched.
2026-07-15 04:07:33 -05:00
will.anderson b4967af13e feat(engram): semantic search layer via nomic-embed-text (cosine ∪ lexical)
Lexical istr_contains alone can't surface a node whose words don't appear
in the query. This adds an optional dense-vector layer: node content and the
query are embedded through Ollama (nomic-embed-text), and nodes are ranked by
cosine similarity unioned with lexical hits, so a paraphrase query reaches the
right node.

Wired into all three query entry points in el_runtime.c:
  - engram_search_json (HTTP /api/search): collect lexical ∪ semantic
    candidates, score (lexical base 1.0 + cosine; pure-semantic = cosine),
    rank, emit top-N. Stable sort preserves old order when semantic is off.
  - engram_search (internal el_val twin): lexical ∪ semantic union.
  - engram_activate seed loop (HTTP /api/activate): a node seeds if it
    lexically matches OR clears the cosine threshold; pure-semantic seeds
    enter scaled by cosine so paraphrase spreads without overpowering.

Degradable by design: the whole layer is gated on HAVE_CURL plus a one-shot
runtime probe. If curl is compiled out, Ollama is unreachable, or
ENGRAM_SEMANTIC=0, every entry point yields zero semantic signal and callers
fall back byte-for-byte to the pre-existing lexical search.

Node embeddings are cached in process memory keyed by node id with an FNV-1a
content hash for invalidation; the query is embedded once per call — so the
graph is not re-embedded on every query. nomic task prefixes
(search_query:/search_document:) are applied for retrieval separation.

Build steps gain -DHAVE_CURL so the engram artifact compiles the layer in
(-lcurl was already linked). Env: ENGRAM_SEMANTIC, ENGRAM_EMBED_URL,
ENGRAM_EMBED_MODEL, ENGRAM_SEMANTIC_MIN (cosine threshold, default 0.6).
2026-07-14 18:48:16 -05:00
will.anderson e3dabe3e08 fix(engram): tokenized + ranked lexical search, not whole-query Ctrl-F
El SDK Release / build-and-release (pull_request) Failing after 14m46s
engram search/activate/goal-bias matched the ENTIRE raw query string as a
single case-insensitive substring (istr_contains(field, q)). Multi-word
queries like "windows msi signing" only matched a node containing that exact
contiguous run, so real multi-word queries returned ZERO on a graph saturated
with the answer. This is Ctrl-F, not search — and search is the core of the
engram being useful.

Fix: split the query on whitespace into distinct tokens; a node matches if it
contains ANY token in content/label/tags. Rank by distinct tokens matched
(desc) then salience (desc). istr_contains is kept unchanged as the per-token
primitive. Single-token queries are a strict special case (score 0 or 1) so
the many single-word callers do not regress.

Sites changed (all in el_runtime.c):
- new helpers engram_tokenize_query / engram_node_match_score / engram_rank_cmp
- engram_search           (internal el_val_t path)
- engram_search_json      (HTTP /api/search path)
- engram_activate seed loop (HTTP /api/activate path; seed activation scaled
  by token coverage so full-query matches seed more strongly)
- engram_goal_bias overlap bonus upgraded to graded token coverage

Proof (6591-node snapshot copy, rebuilt binary on :8799, POST JSON path):
  windows msi signing  0 -> 20   Will Anderson  0 -> 20
  windows msi          0 -> 20   tokenized search fix  0 -> 20
Single-word parity preserved (VBD/volatility/elc capped at limit; unkey = all
matching nodes). Top hits are relevant (e.g. "Will Anderson" surfaces the
Project Design and VBD whitepapers).

Note: GET ?q=a%20b still returns 0 because query_param (server.el) does not
URL-decode — a separate EL-layer bug; the soul's POST-JSON path is fixed here.
2026-07-14 18:39:07 -05:00
will.anderson 0a0a2bcb44 parser: bound token reads to Eof so malformed input errors instead of OOMing
El SDK Release / build-and-release (pull_request) Failing after 16s
Out-of-range tok_kind/tok_value reads returned runtime null (el_list_get OOB
-> 0) rather than the Eof sentinel, so the inner parse loops (parse_block,
call-arg, array-literal, match-arm) that terminate only on their close
delimiter or k=="Eof" never saw Eof once the cursor ran past the single
trailing Eof token. On unclosed-delimiter input the parser then appended AST
nodes forever -> unbounded allocation -> ~700GB -> OOM (observed compiling
neuron/sessions.el).

Fix at the choke point: tok_kind returns "Eof" and tok_value returns "" for
out-of-range positions, restoring the parser-wide contract that reads at/after
the end yield Eof. expect() no longer steps past the Eof sentinel on mismatch.
This terminates every overrun loop simultaneously; a malformed program now
surfaces as a normal (best-effort) parse end instead of exhausting memory.

Requires a self-hosted bootstrap rebuild of elc to take effect.
2026-07-14 14:21:39 -05:00
will.anderson 2b2a1246e7 Merge pull request 'runtime: fix the memory-leak + write-corruption pair in el_runtime.c' (#64) from hotfix/el-runtime-leak-and-persist into main
El SDK Release / build-and-release (push) Failing after 10m52s
2026-07-13 21:23:31 +00:00
will.anderson f78da81aa4 runtime: fix the memory leak + write-corruption pair in el_runtime.c
El SDK Release / build-and-release (pull_request) Failing after 11m58s
Two independent investigations, one runtime, complementary halves:

1. Leak (Jul 2, this machine): JsonBuf buffers returned via el_wrap_str
   were raw malloc, never arena-tracked — every engram_*_json call leaked
   its output unconditionally. Added jb_finish() arena-tracking across all
   ~30 return sites. Plus el_arena_push/pop per-tick bracketing support
   for the soul's awareness loop (the loop ran outside any request arena,
   so even correctly-tracked allocations were permanent — 7.5GB RSS in
   under a minute at 1s tick).

2. Corruption (Tim's container soak, docs findings/container-migration):
   stored engram node/edge fields (content, node_type, label, tier, tags,
   metadata, from/to ids) were arena el_strdup — freed at request end,
   leaving dangling pointers that read back as recycled request-buffer
   bytes one request later. This is the June corruption root cause and
   the mechanism that grew snapshot.json to 18GB of empty-type junk
   (21.6M nodes, 3,335 real). 39 sites switched to el_strdup_persist,
   plus a latent double-free fix in engram_load metadata fixup.

Interaction note: fix 1's per-tick arena reclamation makes fix 2
mandatory — more aggressive arena recycling widens the use-after-free
window if stored fields still live in the arena. Apply as a pair, never
separately.

Verified live: soul + engram rebuilt from this runtime, booted against
the recovered real snapshot (3,335 nodes/40,146 edges), 5h stable at
<100MB RSS, write-then-next-request field-integrity test passes (the
June corruption fingerprint does not reproduce). engram/dist/engram
binary updated from this build.

Investigation credit: leak diagnosis this machine Jul 2-6; corruption
diagnosis + persist-fix patch by Tim's instance (docs PR #4).
2026-07-13 16:22:02 -05:00
will.anderson 2597a092bb Merge pull request 'chore: integrate local main commits' (#63) from integrate/local-main-commits into main
El SDK Release / build-and-release (push) Successful in 10m58s
2026-07-01 16:30:17 +00:00
will.anderson 226b798407 Merge branch 'fix/windows-rusage-guard' (PR #61): UTF-8 guard, engram sync route, native platform backends, UI vessels
El SDK Release / build-and-release (pull_request) Failing after 13m57s
2026-07-01 11:27:54 -05:00
will.anderson cfe8cb1c80 fix(release-snapshot): fflush stdout in println and update Knowledge threshold
El SDK Release / build-and-release (pull_request) Failing after 20s
2026-07-01 11:25:09 -05:00
will.anderson 688b8508fb feat(runtime): native platform backends and UI vessels onto main 2026-07-01 11:21:23 -05:00
will.anderson 59cea116c5 build(engram): rebuild binary with engram_load_merge runtime (deb0520)
El SDK Release / build-and-release (pull_request) Failing after 19s
Runtime now includes engram_load_merge — soul daemon awareness.el calls
this function during its periodic sync refresh cycle. Binary rebuilt from
server.el (unchanged source) + updated el_runtime.c.
2026-06-30 08:59:01 -05:00
will.anderson deb0520551 feat(runtime): port engram_load_merge to released runtime + add missing WM headers
engram_load_merge was added to el-compiler/runtime in 35c1897 but never
ported to the released runtime used by Engram and the soul daemon.

awareness.el calls engram_load_merge in its sync refresh cycle; without
this function in lang/releases/v1.0.0-20260501/el_runtime.c the soul
daemon fails to compile.

Also adds header declarations for engram_wm_count, engram_wm_avg_weight,
engram_wm_top_json, and engram_load_merge — all four were added as
implementations (da116b2 / 35c1897) but their prototypes were missing from
el_runtime.h, causing implicit-function-declaration warnings and potential
ABI breakage on stricter compilers.

Identified during self-review 2026-06-30.
2026-06-30 08:57:22 -05:00
will.anderson da116b2884 self-review 2026-06-30: WM cap, breakthrough floor, ISE exclusion + route
Port critical WM fixes from self-review 2026-06-26 branch (f7bd99a) that were
never merged to HEAD. Running binary had these fixes; source did not — rebuild
would have silently regressed all three improvements.

1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
   With 0.25, naturally-promoted nodes (threshold ≥0.15) decayed below the
   breakthrough floor within one activation call and lost their WM slot to
   fresh breakthrough candidates. All 524/525 WM nodes were at floor = useless.
   Invariant: BREAKTHROUGH_WEIGHT < min(type_thresholds = 0.15 Canonical).

2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
   Without cap, broad curiosity seeds promote 500+ nodes simultaneously.
   wm_avg_weight collapses, goal-bias differentiation is lost. Verified:
   "knowledge" query now promotes exactly 24 nodes (was 525). Cowan (2001)
   cognitive basis: WM capacity ~4 chunks; 24 allows rich multi-topic context.

3. ISE exclusion from WM (Pass 2 guard)
   InternalStateEvent JSON content ("knowledge", "memory", etc.) triggered
   lexical seeding → suppression accumulation → breakthrough at floor. ISEs
   are observability-only and must never surface in context compilation.
   suppression_count cleared so ISEs never build toward breakthrough.

4. route_create_ise importance fix (0.5→0.3)
   Corrects mismatch between HTTP route and awareness.el in-process fallback.
   Also adds body comment clarifying auth-exempt rationale.

SYNAPSE (arXiv 2601.02744) validates WM cap design and ISE exclusion principle.
Next priority: cosine similarity seeding to complement lexical BFS.
2026-06-30 08:48:19 -05:00
will.anderson 58753a88d7 feat(ui): native vessel, HTML vessel update, native hello examples, profile card, UI tools
El SDK Release / build-and-release (pull_request) Failing after 17s
el-native vessel: El-level wrappers around __widget_* C builtins, exposing
vstack, label, button, text_field, etc. as clean El functions for application code.

el-html/main.elh: updated extern declarations for the HTML vessel's codegen API.

native-hello: cross-platform desktop example (AppKit/GTK4/Win32/SDL2) with
build scripts, Dockerfiles for Linux/Pi, and Win32 cross-compile support.

native-hello-android: Gradle project with ElBridge integration and build script.

native-hello-ios: Xcode project for the iOS UIKit target.

profile-card: manifest.el for a styling/layout/i18n example app that exercises
el-style, el-layout, el-i18n, el-config, and el-secrets vessels.

ui/tools/native-codegen: Python codegen pass (el_ui_native_codegen.py) that
lowers el-ui component DSL to el-native vessel calls, plus build script and
test fixtures.
2026-06-29 12:40:37 -05:00
will.anderson edff25180e feat(runtime): Java platform bridge and platform detection tooling
ElBridge.java: Android Java companion to el_android.c — all public methods are
static, dispatches View mutations to the UI thread via runOnUiThread/CountDownLatch,
and exposes native callbacks (nativeOnClick, nativeOnChange, nativeOnSubmit).

PLATFORM_BRIDGE_SPEC.md: authoritative spec for implementing new platform bridges
(slot table contract, required __* functions, callback dispatch pattern).

detect-platforms: shell script that probes for available bridge toolchains and
prints what can be built on the current machine.

new-platform: scaffold generator that creates a new el_<name>.c with all 33
required stubs wired up.
2026-06-29 12:40:26 -05:00
will.anderson 6271cb42b2 feat(runtime): native platform backends (AppKit, UIKit, Android, GTK4, SDL2, LVGL, Win32)
Add seven platform bridge implementations and the shared native target header:
el_native_target.h, el_appkit.m, el_uikit.m, el_android.c, el_gtk4.c,
el_sdl2.c, el_lvgl.c, el_win32.c, el_runtime_win32.c. Each bridge implements
the 33 __widget_* C builtins declared in el_native_target.h for its platform
toolkit. el_runtime_win32.c provides a POSIX-free runtime stub for cross-compiled
Win32 targets.
2026-06-29 12:40:14 -05:00
will.anderson 3da9181deb fix(releases/v1.0.0): println stdout flush for launchd; Knowledge node activation threshold 2026-06-29 12:38:36 -05:00
will.anderson 192241c7c1 feat(engram): /api/sync route for soul daemon periodic pull; update ELP type headers 2026-06-29 12:38:33 -05:00
will.anderson e7c2dc7734 prevent engram corruption: add UTF-8 validation in engram_node_full
Reject content containing invalid UTF-8 bytes before persisting — silently
writing invalid UTF-8 garbles JSON snapshots and corrupts node reads.
2026-06-29 11:08:52 -05:00
will.anderson f7bd99ae45 self-review 2026-06-26: WM cap, breakthrough floor 0.25→0.10, ISE WM exclusion, /api/neuron/state-events route
Three improvements from today's self-review:

1. ENGRAM_BREAKTHROUGH_WEIGHT 0.25→0.10
   Live data showed 524/525 WM nodes at breakthrough floor (0.25). Knowledge
   nodes promoted at 0.21 decayed to 0.147 in one call, fell below the old
   0.25 floor, and were immediately evicted for fresh breakthrough candidates.
   Natural promotion was invisible. Invariant maintained: 0.10 < all
   per-type thresholds (min=0.15 Canonical).

2. ENGRAM_WM_CAP=24 with Pass 4 (per-call) + Pass 5 (global) enforcement
   Without a cap, broad queries like 'knowledge' promote 525+ nodes
   simultaneously. WM is now bounded to 24 nodes. Algorithm: qsort on
   promoted weights, keep top-24 by cutoff, evict the rest. Global pass
   enforces cap across nodes that were promoted in prior calls and persist
   via working_memory_weight. Validated: WM promoted goes 525→24.
   Cognitive basis: Cowan (2001) WM ~4 chunks; 24 gives richer multi-topic
   context while preventing flooding.

3. ISE exclusion from WM + /api/neuron/state-events route
   InternalStateEvent nodes were reaching WM via breakthrough (5 suppression
   cycles) because their content (curiosity seed JSON with 'knowledge',
   'memory', etc.) triggered lexical seeding. ISEs are observability-only
   and must never surface in context. Fix: guard in Pass 2 clears
   suppression_count and skips to wm_weights[i]=0.0.
   Also added POST /api/neuron/state-events route to server.el (auth-exempt,
   internal endpoint). The main soul daemon posts ISEs here but the route
   was missing — all ise_post() calls were silently returning 'not found'.

Research: SYNAPSE (arXiv 2601.02744) validates spreading factor 0.8 (our
0.7), top-M WM cap design, and cosine similarity seeding. Next priority:
implement cosine similarity initial seeding from the other branch.
2026-06-26 08:47:08 -05:00
will.anderson 5c41c66a0f Merge pull request 'fix(windows): guard el_mem_check with _WIN32 — rusage is POSIX-only' (#60) from fix/windows-rusage-guard into stage
El SDK CI - stage / build-and-test (push) Failing after 13m21s
fix(windows): guard el_mem_check with _WIN32 — rusage is POSIX-only
2026-06-25 16:48:13 +00:00
will.anderson 93d36fddb1 fix(windows): guard el_mem_check with _WIN32 — rusage is POSIX-only
El SDK CI - stage / build-and-test (pull_request) Failing after 11m3s
2026-06-25 11:45:36 -05:00
will.anderson 2d751890ea feat(windows): native Windows port of el_runtime.c — fix all blockers
El SDK CI - stage / build-and-test (push) Failing after 7m45s
2026-06-20 00:06:04 +00:00
will.anderson 99b113ea9d Merge branch 'stage' into feat/windows-el-runtime
El SDK CI - stage / build-and-test (pull_request) Failing after 15s
Resolve el_runtime.c conflict: include both sys/resource.h (from stage)
and el_closesocket POSIX shim (from Windows port) within the #else block.
2026-06-19 19:05:37 -05:00
will.anderson c087b97093 fix(windows): resolve PR blockers — nanosleep shim, unsetenv, duplicate typedefs, SOCKET type, el_closesocket
El SDK CI - stage / build-and-test (pull_request) Failing after 22s
2026-06-19 18:59:10 -05:00
tim.lingo 718a2e0c06 Merge pull request 'feat(engram): accumulation layer — new nodes to top of stack, not core-identity' (#59) from feat/accumulation-layer into stage
El SDK CI - stage / build-and-test (push) Failing after 8m50s
2026-06-17 18:34:05 +00:00
tim.lingo b6187501fd Merge pull request 'Reconcile live runtime data-integrity fixes onto main (UAF + atomic engram_save)' (#58) from fix/runtime-integrity-reconcile into stage
El SDK CI - stage / build-and-test (push) Failing after 9m32s
2026-06-17 18:33:16 +00:00
Tim Lingo 18e1ab6db1 feat(engram): add accumulation layer (layer 5) — new nodes default to it, not core-identity
El SDK Release / build-and-release (pull_request) Failing after 12m23s
Implements the accumulation layer from the Layered Consciousness architecture
(provisional 64/064,262) and answers the deferred design question. Per the spec
and Will's design: new user-facing nodes (memories, knowledge, conversations) are
created in an accumulation layer at the TOP of the consciousness stack — the engram
the user sees — while the layers below (safety, core-identity, domain, imprint,
suit) shape behavior but are hidden from the user.

- Adds ENGRAM_LAYER_ACCUMULATION (5) + the layer record in engram_init_layers
  (activation_priority 50, suppressible, not injectable, transparent=0).
- engram_node and engram_node_full now assign new nodes to ENGRAM_LAYER_ACCUMULATION.
- ENGRAM_LAYER_DEFAULT stays CORE_IDENTITY ON PURPOSE: it is the fallback for LEGACY
  nodes loaded from snapshots without a layer_id, so existing data (the originator
  corpus) is NEVER migrated. New-nodes-only — the immutable-originator rule.

This is the foundation for fixing the identity-bleed / customer-isolation issue
(user data was landing in Neuron's core-identity layer). The retrieval-side
provenance filter (introspection should compile from accumulation, not the
originator corpus — Persona 64/036,574) is a follow-on, pending the batch-2
Layered Consciousness + Engram spec docs for exact semantics. Compiles clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:14:57 -05:00
Tim Lingo 2dec76c87a fix(runtime): reconcile live data-integrity fixes onto main (UAF + atomic engram_save)
El SDK Release / build-and-release (pull_request) Failing after 17s
Ports the fixes that until now lived only in the un-versioned el-sdk source the live
macOS soul was hand-built from (captured in the [DO NOT MERGE] live-darwin-runtime
snapshot) FORWARD onto main, faithfully and minimally — without dragging in the
snapshot's deletions of main's newer engram_wm_/engram_load_merge/http_serve_async.

1. UAF (hallucinated/lost-saves root cause): engram_new_id + engram_node_full now use
   el_strdup_persist, NOT el_strdup. el_strdup tracks into the per-request arena that
   el_request_end() frees when the creating HTTP request completes — leaving stored
   nodes with dangling pointers (corrupted ids, 'saved but never listed'). Transplanted
   verbatim from the live runtime; el_strdup_persist sites 19->27, matching live.

2. Atomic engram_save: write <path>.tmp, fflush+fsync, rename() over target (atomic on
   POSIX) so a booting soul's engram_load never reads a truncated/0-byte snapshot — the
   genesis -> nodes=1 -> 63-node-clobber loop. Plus a sparse-write floor: refuse to
   overwrite a >200KB snapshot with one < 1/16 its size. (Validated in isolation:
   harness 11/11; rebuilt+booted the darwin soul, round-tripped 5113 nodes, no clobber.)

The response-truncation fix is already on main (_tl_fs_read_len binary-safe length).
Compiles clean. For Will to build through CI/elb and deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:46:56 -05:00
Tim Lingo a36a62ca14 fix(el-runtime): promote http_handler typedefs to el_runtime.h (cross-module + Windows)
El SDK Release / build-and-release (pull_request) Failing after 13m0s
http_handler_fn / http_handler4_fn were defined only inside el_runtime.c, so soul
modules (routes/chat/...) that reference them via cross-module forward declarations
couldn't see the types — which broke the Windows link of every module. Moving the
public function-pointer types to the shared header is the correct home and unblocks
the build on all platforms (identical typedef, C11-safe redefinition in el_runtime.c).

With this, the soul links into a native Windows neuron.exe (mingw, static) that boots
and serves HTTP on :7770 — verified /health → 200 {"status":"alive",...} in a Win11 VM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 17:14:17 -05:00
Tim Lingo 28ef43264a feat(el-runtime): native Windows port of el_runtime.c (winsock/dlsym/CreateProcess)
Compiles for Windows x64 via mingw-w64 and still compiles clean on POSIX
(darwin/linux) — all Windows code is behind #ifdef _WIN32, POSIX path unchanged.

- el_platform_win.h (new): winsock2 + auto WSAStartup, el_closesocket(),
  dlsym->GetProcAddress, popen/_popen, mkdir/_mkdir, setenv/_putenv_s,
  timegm/_mkgmtime, localtime_r/gmtime_r. Threading unchanged — mingw
  winpthreads supplies <pthread.h> + -lpthread.
- el_runtime.c: include block guarded; 10 socket-close sites -> el_closesocket();
  setsockopt arg4 cast; tm_zone guarded; exec_bg fork/exec -> CreateProcess.

Part of feat/windows-port. Core-el change, for Will's review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 16:58:11 -05:00
will.anderson 35c189759c feat(runtime): add engram_wm_*, engram_load_merge, http_serve_async — needed by soul CI
El SDK Release / build-and-release (push) Successful in 8m44s
2026-06-11 13:40:10 -05:00
will.anderson 5c94b8680d Merge stage into main: corruption fix, model passthrough, UTF-8 escaping
El SDK Release / build-and-release (push) Successful in 11m22s
2026-06-10 17:37:41 -05:00
will.anderson cebf3ded62 Merge dev into stage: corruption fix + model passthrough
El SDK CI - stage / build-and-test (push) Failing after 11m30s
2026-06-10 17:37:27 -05:00
will.anderson b83ecf52f9 Merge pull request 'fix(runtime): pass model through to the LLM API (+ UTF-8 JSON escaping)' (#53) from fix/llm-model-and-utf8 into stage
El SDK CI - stage / build-and-test (push) Successful in 8m26s
fix(runtime): pass model through to LLM API + UTF-8 JSON escaping
2026-06-10 22:01:51 +00:00
will.anderson 15ea584671 Merge pull request 'Fix engram_node_full field corruption + add validation' (#52) from fix/engram-node-full-field-corruption into dev
El SDK CI - dev / build-and-test (push) Successful in 7m59s
Fix engram_node_full field corruption + add validation (+ SessionSummary allowlist)
2026-06-10 22:01:41 +00:00
Tim Lingo c2afcbddf5 fix(engram): allow SessionSummary node_type in validation allowlist
El SDK CI - dev / build-and-test (pull_request) Successful in 3m47s
handle_api_consolidate writes a "SessionSummary" node, but engram_valid_node_type
omitted it — so once this validation ships, every consolidate() would be silently
REJECTED at the engram boundary. Add SessionSummary to the allowlist.

Found in Will's PR review of neuron #1 / el #52.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 06:26:25 -05:00
Tim Lingo dbf2c659d9 fix(runtime): pass model through to the LLM API instead of dropping it
El SDK CI - stage / build-and-test (pull_request) Failing after 12s
llm_call_system / llm_call accepted a model argument and discarded it:
they called llm_chain_call(system, user) with no model, and the legacy
ANTHROPIC_API_KEY fallback passed NULL to llm_provider_request, so every
non-agentic chat was pinned to LLM_DEFAULT_MODEL (claude-sonnet-4-5)
regardless of the caller's selection.

Thread model_pref through llm_chain_call: provider-chain entries still
honor their own NEURON_LLM_N_MODEL override and fall back to the
requested model otherwise; the legacy Anthropic path now uses the
requested model. NULL/empty preserves prior default behavior.

Effect: the soul's model selection (state soul_model / SOUL_LLM_MODEL,
e.g. claude-opus-4-8) now reaches api.anthropic.com. Previously the
chat response echoed the selected model in its label while the request
billed Sonnet 4.5.

Not built locally (no elc/cc toolchain on this checkout); needs stage CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 08:03:56 -05:00
Tim Lingo 2b8062c55f fix(runtime): handle multi-byte UTF-8 in JSON string escaping
Validate UTF-8 continuation bytes in jb_emit_escaped; pass valid
sequences through and escape orphaned/invalid start bytes as \u00xx.
Pre-existing change found uncommitted in the working tree; committed
here so it is reviewable rather than lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 08:02:46 -05:00
Tim Lingo dfe4e83ed1 Fix engram_node_full wrapper field corruption + add node_type/tier validation
El SDK Release / build-and-release (pull_request) Failing after 9s
The wrapper signature was stale and didn't match the C primitive
__engram_node_full(content, node_type, label, salience, importance, confidence, tier, tags).
Because el_val_t is an untyped machine word, the compiler coerced caller args to the
wrong declared param types and forwarded them BY POSITION — so tier received an int,
importance/confidence received strings, label received a float, etc. (~100 corrupt nodes).

- Correct the wrapper to match the C contract 1:1 (no coercion, no reorder).
- Add engram_valid_node_type / engram_valid_tier allowlists; engram_node and
  engram_node_full now reject invalid values with __println + return "" (fail loud,
  no silent malformed write).

See neuron repo: HANDOFF-engram-write-corruption.md for the full write-up + deploy runbook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 16:13:43 -05:00
will.anderson a390ee494e Merge pull request 'fix: elb macOS OpenSSL + C master decls header; ELP missing imports' (#51) from fix/ci-gcloud-install-order into dev
El SDK CI - dev / build-and-test (push) Successful in 5m15s
Merge PR #51: fix elb macOS OpenSSL + ELP missing imports
2026-05-09 01:24:36 +00:00
will.anderson c2cd5e01e1 fix: elb macOS OpenSSL + C master declarations header; add ELP missing imports
El SDK CI - dev / build-and-test (pull_request) Successful in 3m34s
elb.el:
- Auto-detect Homebrew OpenSSL (-L$(brew --prefix openssl)/lib) so -lssl
  resolves on macOS without manual flags; no-op on Linux
- Add -include elp-c-decls.h when present in out_dir: resolves undeclared
  cross-module calls in packages like ELP that lack explicit imports

ELP source:
- Add import "morphology.el" to all 29 language morphology modules
- Add language module imports to morphology.el (all langs it dispatches to)
  These were missing since ELP was originally built as a monolithic unit
2026-05-08 19:44:31 -05:00
will.anderson 8212e12e57 Merge pull request 'fix(ci): install gcloud in build-deps step to avoid apt timeout at publish' (#50) from fix/ci-gcloud-install-order into dev
El SDK CI - dev / build-and-test (push) Successful in 6m36s
2026-05-08 17:38:15 +00:00
will.anderson 253ee2b887 fix(ci): install gcloud in build-deps step to avoid apt timeout at publish
El SDK CI - dev / build-and-test (pull_request) Successful in 3m20s
2026-05-08 12:33:57 -05:00
will.anderson d7540700d4 Merge pull request 'perf(ci): precompile el_runtime.o once for all native test modules' (#49) from fix/native-test-precompile-runtime into dev
El SDK CI - dev / build-and-test (push) Failing after 4m1s
2026-05-08 17:24:10 +00:00
will.anderson f103e85f88 perf(ci): precompile el_runtime.o once for all native test modules
El SDK CI - dev / build-and-test (pull_request) Successful in 3m24s
el_runtime.c was being compiled from source for each of the 8 native
test modules. A single precompile step produces el_runtime.o which all
8 link steps reuse — eliminates 7 redundant gcc runtime compilations.
2026-05-08 12:06:11 -05:00
will.anderson fe84639b17 Merge pull request 'fix(ci): fall back to ci-base:latest on first dev rebuild' (#48) from fix/ci-base-dev-first-run into dev
El SDK CI - dev / build-and-test (push) Failing after 14m0s
2026-05-08 16:53:38 +00:00
will.anderson 5fdc9fb15e fix(ci): fall back to ci-base:latest when ci-base:dev doesn't exist yet
El SDK CI - dev / build-and-test (pull_request) Successful in 3m51s
The BASE build arg was hardcoded to ci-base:dev even when the pull fell
back to :latest. Docker then tried to resolve ci-base:dev from the
registry during the build and failed.

Capture which tag was actually pulled and use that as BASE.
2026-05-08 11:49:17 -05:00
will.anderson 8967fa404e Merge pull request 'feat(elc, elb): RBrace stop fix, html_raw/escape runtime, c_source manifest directive' (#46) from fix/elc-parser-elb-build into dev
El SDK CI - dev / build-and-test (push) Failing after 4m28s
2026-05-08 16:43:10 +00:00
will.anderson a7e6fbf2d2 feat(elc, runtime): RBrace stop in parse_html_children; html_raw/html_escape; elc.c canonical
El SDK CI - dev / build-and-test (pull_request) Successful in 4m9s
parse_html_children consumed the closing `}` of the outer El function as
HTML text content when a tag was left open across a function boundary
(e.g. `page_open()` opens `<body>` without a closing `</body>`).  Fix:
stop the children loop when the current token is RBrace — that token
belongs to the El function, not the HTML tree.

Add html_raw() and html_escape() builtins to el_runtime so templates
can interpolate trusted raw HTML and safely escape user-supplied content.

Rename elc-new.c → elc.c as the canonical compiler source; rebuild
elc binary from it.
2026-05-08 11:31:50 -05:00
will.anderson 1f4b594ae7 feat(elb): c_source manifest directive + macOS OpenSSL path detection
Add `c_source "path"` in manifest.el build block — lets packages link
extra C files (platform stubs, native glue) without touching elb source.

On macOS, homebrew OpenSSL isn't on the default linker path. Detect it
via `brew --prefix` and inject -L/-I flags; no-op on Linux.

Rebuild elb binary; remove elc-new binary (elc is now canonical).
2026-05-08 11:31:36 -05:00
will.anderson cff7ce072d Merge pull request 'fix(elc): eliminate OOM in --emit-header; add memory guard' (#47) from fix/elc-oom-checkout into dev
El SDK CI - dev / build-and-test (push) Failing after 4m44s
2026-05-08 16:16:02 +00:00
will.anderson f5dcca0386 build: update dist/platform/elc with OOM fix and memory guard
El SDK CI - dev / build-and-test (pull_request) Successful in 4m16s
Rebuilt from fix/elc-oom-checkout: scan_fn_sigs_el() --emit-header path
+ el_mem_check() guard. Verified on checkout.el: all 3 sigs in .elh,
clean exit under normal load, exit(1) on memory limit exceeded.
2026-05-08 08:23:07 -05:00
will.anderson 53e0b99d5f fix(elc): add el_mem_check() memory guard — abort before OS OOM-kill
Add el_mem_check() to el_runtime.c: reads ELC_MAX_MEM_MB (default 512),
checks RSS via getrusage (macOS bytes / Linux KB normalised to MB), prints
a clear diagnostic to stderr and exits(1) if exceeded.

Wire it into two places:
- compiler.el: upfront check at --emit-header entry point
- codegen.el: per-function check in the streaming loop after each
  el_arena_pop, so runaway growth is caught at the earliest function
  boundary rather than after the machine is already dying.
2026-05-08 08:21:38 -05:00
will.anderson 5f9cad5908 fix(elc): eliminate OOM in --emit-header by using token-level signature scan
The --emit-header path previously called parse() which builds the entire
program AST in memory before writing the .elh file. For checkout.el (~491
lines with HTML template trees and deep BinOp string-concat chains), this
exhausted memory before the header could be written.

Fix: replace parse() + emit_header() with scan_fn_sigs_el() +
emit_header_from_sigs(). The new path tokenises the source once, then
walks the flat token list skipping over function bodies entirely — peak
memory is O(tokens) instead of O(whole-program AST).

New functions in parser.el:
- scan_type_el: reads a type annotation and returns its El source string
- scan_params_el: reads (name: Type, ...) and returns El params string
- scan_fn_sigs_el: token-level scan that collects El-style fn signatures
  without building any expression AST nodes

New function in compiler.el:
- emit_header_from_sigs: writes .elh from scan_fn_sigs_el output

Self-hosting check: elc compiled with new elc, diff of outputs is
identical (zero difference).

Smoke test: elc --emit-header checkout.el produces correct three-entry
.elh (previously truncated at two entries due to mid-parse OOM).
2026-05-08 08:20:13 -05:00
will.anderson 00629b39c4 Merge pull request 'fix(parser): str_join separator '' not ' ' — CSS selectors were emitting spaces' (#45) from fix/css-str-join-separator into dev
El SDK CI - dev / build-and-test (push) Failing after 12m6s
2026-05-07 23:00:19 +00:00
will.anderson ca1e4d57b8 Merge pull request 'ci: add three-tier ci-base rebuild (dev/stage)' (#44) from fix/html-template-if-style-script into dev
El SDK CI - dev / build-and-test (push) Has been cancelled
2026-05-07 23:00:13 +00:00
will.anderson f971e96dd5 fix(parser): str_join separator '' not ' ' — CSS selectors were emitting spaces between tokens
El SDK CI - dev / build-and-test (pull_request) Successful in 3m45s
2026-05-07 15:53:19 -05:00
will.anderson 81a1a624f1 add three-tier ci-base rebuild (dev/stage) to CI workflows
El SDK CI - dev / build-and-test (pull_request) Successful in 3m49s
2026-05-07 15:51:24 -05:00
will.anderson 7b7f9f353b Merge pull request 'fix(parser): add {#if}/{#else}/{/if} and raw-text <style>/<script> in HTML templates' (#43) from fix/html-template-if-style-script into dev
El SDK CI - dev / build-and-test (push) Successful in 4m28s
fix(parser): add {#if}/{#else}/{/if} and raw-text <style>/<script> in HTML templates
2026-05-07 18:44:26 +00:00
will.anderson a3732a1e9a fix(parser): add {#if}/{#else}/{/if} support and raw-text <style>/<script> in HTML templates
El SDK CI - dev / build-and-test (pull_request) Failing after 18m3s
The El lexer silently skips '#', so {#each} lexes as LBrace Ident:"each"
and {#if} lexes as LBrace If ... (using the If keyword token, not Hash).
The existing {#each} check used k2=="Hash" which was dead code.

Parser changes (parser.el):
- Add parse_raw_text_content(): collects all tokens as raw text until
  </tag_name>, bypassing El expression parsing. Used for <style> and
  <script> elements so CSS/JS content isn't parsed as El expressions.
- parse_html_element(): use raw-text mode for <style> and <script> tags.
- parse_html_children(): fix {#each} detection (k2=="Ident", k3=="each"
  instead of dead k2=="Hash" check). Add {#if cond}...{#else}...{/if}
  support generating HtmlIf AST nodes.

Codegen changes (codegen.el):
- Add cg_html_if(): generates if (cond_c) { then_c } else { else_c }
  for HtmlIf nodes.
- cg_html_parts(): dispatch HtmlIf to cg_html_if.
2026-05-07 13:39:12 -05:00
will.anderson 2ed6b26dde Merge pull request 'promote: stage → main (all elb linker fixes + ci-base rebuild)' (#42) from stage into main
El SDK Release / build-and-release (push) Successful in 6m28s
promote: stage → main (all elb linker fixes + ci-base rebuild)
2026-05-07 14:25:37 +00:00
will.anderson d8e9fd12f4 Merge pull request 'promote: dev → stage (all elb linker fixes)' (#41) from dev into stage
El SDK Release / build-and-release (pull_request) Successful in 3m51s
El SDK CI - stage / build-and-test (push) Successful in 4m11s
promote: dev → stage (all elb linker fixes)
2026-05-07 14:20:53 +00:00
will.anderson 8ef3eb6bec Merge pull request 'fix(elb): all linker fixes — gcc compat, OpenSSL, runtime import conflict' (#40) from fix/elb-gcc-bracket-depth into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 4m8s
El SDK CI - dev / build-and-test (push) Successful in 4m34s
fix(elb): all linker fixes — gcc compat, OpenSSL, runtime import conflict
2026-05-07 14:16:17 +00:00
will.anderson 027ad82db2 fix elb linker: remove runtime imports from el-install, add --clean, catch in dev/stage CI
El SDK CI - dev / build-and-test (pull_request) Successful in 3m35s
el-install.el explicitly imported runtime/*.el modules (string, env, fs, exec,
json, http), which elb compiled to .c files in the shared dist/bin out_dir.
Linking those alongside el_runtime.c caused multiple definition errors for
every runtime function (http_get, http_patch, etc.). The runtime .el files are
thin wrappers over seed primitives already compiled into el_runtime.c — no
import needed.

Fixes:
- Remove all explicit runtime imports from el-install.el (root cause)
- Add --clean to every elb invocation in sdk-release.yaml so each build
  starts with a clean out_dir (defense-in-depth against stale .c files)
- Add elb build + epm/el-install build steps to ci-dev.yaml and ci-stage.yaml
  so linker errors are caught on every PR, not just stage->main
2026-05-07 03:20:44 -05:00
will.anderson 8fa9c4ba20 Merge pull request 'promote: dev → stage (elb linker fixes)' (#38) from dev into stage
El SDK Release / build-and-release (pull_request) Failing after 1m2s
El SDK CI - stage / build-and-test (push) Successful in 3m56s
promote: dev → stage (elb linker fixes)
2026-05-07 08:11:38 +00:00
will.anderson 8ab8e3fd31 Merge pull request 'fix(elb): add -lssl -lcrypto to link_binary flags' (#37) from fix/elb-gcc-bracket-depth into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 3m22s
El SDK CI - dev / build-and-test (push) Successful in 3m56s
fix(elb): add -lssl -lcrypto to link_binary flags
2026-05-07 08:07:27 +00:00
will.anderson 05d717744b fix(elb): add -lssl -lcrypto to link_binary flags
El SDK CI - dev / build-and-test (pull_request) Successful in 3m24s
el_runtime.c uses OpenSSL (EVP_*, RAND_bytes) for AEAD encrypt/decrypt.
elb was only linking -lcurl -lpthread -lm, missing the SSL libs.
Matches the explicit flags used in ci-dev.yaml and ci-stage.yaml.
2026-05-07 03:03:21 -05:00
will.anderson 9c7bde47dc Merge pull request 'promote: dev → stage (elb gcc fix)' (#35) from dev into stage
El SDK Release / build-and-release (pull_request) Failing after 40s
El SDK CI - stage / build-and-test (push) Successful in 3m45s
promote: dev → stage (elb gcc fix)
2026-05-07 08:01:22 +00:00
will.anderson b0d0975f05 Merge pull request 'fix(elb): use clang-only -fbracket-depth flag conditionally' (#34) from fix/elb-gcc-bracket-depth into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 3m21s
El SDK CI - dev / build-and-test (push) Successful in 3m53s
fix(elb): use clang-only -fbracket-depth flag conditionally
2026-05-07 07:57:34 +00:00
will.anderson 6f634ae432 fix(elb): use clang-only -fbracket-depth flag conditionally
El SDK CI - dev / build-and-test (pull_request) Successful in 3m26s
gcc rejects -fbracket-depth=1024 with 'unrecognized command-line option'.
Use shell subshell to probe cc --version and only pass the flag when
the compiler is clang.
2026-05-07 02:53:42 -05:00
will.anderson c0553459e1 Merge pull request 'promote: dev → stage (CI rebuild fix + ci-base refresh)' (#32) from dev into stage
El SDK Release / build-and-release (pull_request) Failing after 35s
El SDK CI - stage / build-and-test (push) Successful in 3m47s
promote: dev → stage (CI rebuild fix + ci-base refresh)
2026-05-07 07:50:27 +00:00
will.anderson 908ce303f3 Merge pull request 'ci: rebuild ci-base on SDK release; publish elb + el_runtime.js to Artifact Registry' (#31) from fix/ci-openssl-linker into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 3m21s
El SDK CI - dev / build-and-test (push) Successful in 3m51s
ci: rebuild ci-base on SDK release; publish elb + el_runtime.js to Artifact Registry
2026-05-07 07:46:22 +00:00
will.anderson edbde5ef51 ci: rebuild ci-base on SDK release; publish elb + el_runtime.js to Artifact Registry
El SDK CI - dev / build-and-test (pull_request) Successful in 3m45s
- sdk-release.yaml: add elb and el_runtime.js to foundation-prod uploads
- sdk-release.yaml: add 'Rebuild ci-base' step — patches ci-base:latest with
  freshly built El SDK after each main branch release (pull → overlay → push)
- sdk-release.yaml: add neuron-web to el-sdk-updated dispatch so downstream
  CI rebuilds automatically on SDK update
- ci-dev.yaml: add elb build step and publish elb + el_runtime.js to
  foundation-dev alongside elc and runtime
2026-05-07 02:25:19 -05:00
will.anderson 2e529bd0fe Merge pull request 'Remove Cargo.toml and .rs bootstrap files from el-ui vessels' (#30) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m49s
2026-05-07 05:49:10 +00:00
will.anderson 5d9299a472 Remove all .rs bootstrap files from el-ui vessels
El SDK CI - dev / build-and-test (pull_request) Successful in 3m33s
el-ui vessels are El. The Rust bootstrap implementations were added as
a stopgap but don't belong here — everything should be El source.
Each vessel's src/main.el and manifest.el are the source of truth.
2026-05-06 23:12:25 -05:00
will.anderson e8b01583d8 Remove Cargo.toml files from el-ui — vessels use manifest.el
All package management is through manifest.el / epm. Cargo.toml files
were incorrectly added to vessels and the root. Removed root workspace
Cargo.toml + Cargo.lock and all vessel-level Cargo.toml files.

el-graph and el-html were already correct (no Cargo.toml).
2026-05-06 22:21:47 -05:00
will.anderson fd208583fe Merge pull request 'promote: dev → stage (elb build fix)' (#28) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 3m51s
El SDK Release / build-and-release (pull_request) Failing after 38s
promote: dev → stage (elb build fix)
2026-05-07 02:46:27 +00:00
will.anderson b19dd5608f Merge pull request 'ci: use elb to build epm and el-install' (#27) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m50s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m33s
ci: use elb to build epm and el-install
2026-05-07 02:37:49 +00:00
will.anderson 94d6eace94 ci: use elb to build epm and el-install (cd into project dir, use --elc flag)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m35s
2026-05-06 21:33:05 -05:00
will.anderson 3e29fc43ab Merge pull request 'promote: dev → stage (__http_do_map_to_file)' (#25) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 3m44s
El SDK Release / build-and-release (pull_request) Failing after 47s
2026-05-07 02:14:30 +00:00
will.anderson f1dfc394e3 Merge pull request 'fix: add __http_do_map_to_file runtime primitive' (#24) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m51s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m25s
2026-05-07 02:06:34 +00:00
will.anderson 61bf501b84 fix: add __http_do_map_to_file runtime primitive
El SDK CI - dev / build-and-test (pull_request) Successful in 3m41s
el-install.el generates calls to __http_do_map_to_file (HTTP request
with JSON headers map, streaming response to file). Add it to both
the HAVE_CURL implementation and the no-curl stub section.
2026-05-06 21:01:46 -05:00
will.anderson 979a5677d5 Merge pull request 'promote: dev → stage (__-prefixed runtime fix)' (#22) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 3m48s
El SDK Release / build-and-release (pull_request) Failing after 1m4s
2026-05-07 01:48:32 +00:00
will.anderson 2fd298df55 Merge pull request 'fix: add __-prefixed runtime primitives for El compiler' (#21) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m43s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m22s
2026-05-07 01:40:37 +00:00
will.anderson 254cbe0ac2 fix: add __-prefixed runtime primitives expected by El compiler
El SDK CI - dev / build-and-test (pull_request) Successful in 3m22s
The El compiler generates calls to __-prefixed C primitives from within
El stdlib compiled code (e.g. __println, __str_len, __json_get, etc).
These were absent from el_runtime.c, causing linker failures when
building el-install, elb, or epm with the current compiler.

Add 46 __-prefixed aliases/implementations in el_runtime.c covering:
- I/O: __println, __print, __readline
- String: __str_len, __str_cmp, __str_ncmp, __str_alloc, __str_set_char,
  __str_concat_raw, __str_slice_raw, __str_char_at, plus numeric converters
- FS: __fs_read, __fs_write, __fs_exists, __fs_mkdir, __fs_list_raw, etc
- HTTP: __http_do, __http_do_map, __http_serve, __http_serve_v2,
  __http_response, __http_sse_* (weak stubs)
- JSON: __json_get, __json_set, __json_parse_map, __json_stringify_val, etc
- State, env, exec, uuid, sha256, args
2026-05-06 20:36:49 -05:00
will.anderson 17b1aa0736 Merge pull request 'promote: dev → stage (return type fix)' (#19) from dev into stage
El SDK CI - stage / build-and-test (push) Failing after 4m1s
El SDK Release / build-and-release (pull_request) Failing after 42s
2026-05-07 01:12:18 +00:00
will.anderson bcfb33ea83 Merge pull request 'fix: align runtime return types with El compiler output' (#18) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m36s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m16s
2026-05-07 01:04:29 +00:00
will.anderson 60ad7f2f6b fix: align runtime function return types with El compiler output
El SDK CI - dev / build-and-test (pull_request) Successful in 3m16s
El compiler generates calls to println, print, exit_program,
http_set_handler, http_serve, http_set_handler_v2, and http_serve_v2
as el_val_t-returning functions. The runtime declared them void,
causing conflicting-type errors when el-install.c was compiled.

Change all seven to return el_val_t (side-effect functions return 0).
Also update el_runtime.h declarations to match.
2026-05-06 20:00:40 -05:00
will.anderson f0c731d2db Merge pull request 'promote: dev → stage (runtime fix)' (#16) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 3m43s
El SDK Release / build-and-release (pull_request) Failing after 45s
2026-05-07 00:43:52 +00:00
will.anderson 231cb5eddd Merge pull request 'fix: add missing runtime functions (native_str_to_int, http_post_json_with_headers)' (#15) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m37s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m39s
2026-05-07 00:35:28 +00:00
will.anderson 54de7d3f3f fix: add missing runtime functions for epm.el
El SDK CI - dev / build-and-test (pull_request) Successful in 3m19s
Add native_str_to_int (El compiler alias for str_to_int) and
http_post_json_with_headers (JSON POST with additional headers map)
which epm.el generates calls to but were absent from el_runtime.c.
2026-05-06 19:31:35 -05:00
will.anderson e7e0f7d3e5 Merge pull request 'promote: dev → stage' (#12) from dev into stage
El SDK CI - stage / build-and-test (push) Successful in 4m3s
El SDK Release / build-and-release (pull_request) Failing after 37s
2026-05-07 00:23:46 +00:00
will.anderson 77100649c3 Merge pull request 'fix: use GIT_TOKEN secret in sdk-release.yaml' (#13) from fix/ci-openssl-linker into dev
El SDK CI - stage / build-and-test (pull_request) Successful in 3m46s
El SDK CI - dev / build-and-test (push) Successful in 4m9s
2026-05-07 00:21:20 +00:00
will.anderson b0570656b1 fix: use GIT_TOKEN secret (GITEA_ prefix is reserved)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m25s
2026-05-06 19:17:16 -05:00
will.anderson a79b421578 Merge pull request 'fix: use GITHUB_SHA for artifact version' (#11) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Successful in 3m47s
El SDK CI - stage / build-and-test (pull_request) Successful in 3m26s
2026-05-07 00:09:18 +00:00
will.anderson 0ab9361fab fix: use GITHUB_SHA instead of GITEA_SHA for artifact version
El SDK CI - dev / build-and-test (pull_request) Successful in 3m19s
GITEA_SHA is not set in the runner container environment; GITHUB_SHA is.
Empty version string caused INVALID_ARGUMENT from Artifact Registry.
2026-05-06 19:05:21 -05:00
will.anderson f7953eb73a Merge pull request 'fix: use valid Artifact Registry package IDs (no slashes)' (#10) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m36s
2026-05-07 00:00:20 +00:00
will.anderson 9c350e9f2f fix: use valid Artifact Registry package IDs (no slashes)
El SDK CI - dev / build-and-test (pull_request) Successful in 3m20s
Package IDs must contain only letters, numbers, periods, hyphens and
underscores. el/elc → el-elc, el/el_runtime.c → el-runtime-c, etc.
2026-05-06 18:56:23 -05:00
will.anderson e93c899d1f Merge pull request 'fix: use trusted=yes for gcloud apt source, drop GPG key dance' (#9) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m56s
2026-05-06 23:50:52 +00:00
will.anderson 0b50e61f98 fix: use trusted=yes for gcloud apt source, drop GPG key dance
El SDK CI - dev / build-and-test (pull_request) Successful in 3m18s
The packages.cloud.google.com key format has changed and signature
verification keeps failing in CI. trusted=yes bypasses the ceremony —
we're downloading from a known Google URL so it's fine.
2026-05-06 18:47:05 -05:00
will.anderson f2741e4bdb Merge pull request 'fix: download GCP apt key directly without gpg --dearmor' (#8) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m59s
2026-05-06 23:39:06 +00:00
will.anderson 7fd01b8a8d fix: download GCP apt key directly, no gpg --dearmor
El SDK CI - dev / build-and-test (pull_request) Successful in 6m38s
packages.cloud.google.com now serves the key in binary format; piping
through gpg --dearmor fails with 'no valid OpenPGP data found'.
2026-05-06 18:28:38 -05:00
will.anderson d2940f5d1d Merge pull request 'fix: add --batch to gpg --dearmor in CI publish steps' (#7) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 10m1s
2026-05-06 23:17:12 +00:00
will.anderson dca741f915 fix: add --batch to gpg --dearmor in publish steps
El SDK CI - dev / build-and-test (pull_request) Successful in 8m50s
gpg tries to open /dev/tty for passphrase input when no TTY is present
in CI, causing the GCP key setup to fail. --batch suppresses interactive
prompts and dearmoring doesn't require one anyway.
2026-05-06 17:58:16 -05:00
will.anderson 9862f4d6e1 Merge pull request 'fix: add -lssl -lcrypto to all CI gcc linker commands' (#6) from fix/ci-openssl-linker into dev
El SDK CI - dev / build-and-test (push) Failing after 3m25s
2026-05-06 22:53:37 +00:00
will.anderson 8b074d2e39 fix: normalize NaN to 'nan' in float_to_str regardless of sign bit
El SDK CI - dev / build-and-test (pull_request) Successful in 3m18s
0.0/0.0 can produce -nan on Linux/x86_64 (%g gives '-nan'),
causing the no-cycle calendar test to fail. Explicitly check isnan()
and emit 'nan' so behavior is platform-independent.
2026-05-06 17:49:35 -05:00
will.anderson ec9c322cc7 fix: use elc-linux-amd64 as bootstrap seed instead of elc-bootstrap.c
El SDK CI - dev / build-and-test (pull_request) Failing after 1m57s
elc-bootstrap.c is stale and produces a broken gen2 that can't correctly
compile current El source (generates C without main() for user programs).
The committed elc-linux-amd64 binary is the current, correct seed for
linux CI. This removes the gen2-from-C step entirely.
2026-05-06 17:45:57 -05:00
will.anderson 702093e043 fix: add -lssl -lcrypto -lm to all test runner gcc commands
El SDK CI - dev / build-and-test (pull_request) Failing after 1m7s
Same OpenSSL/math linker flags needed everywhere el_runtime.c is linked.
2026-05-06 17:41:55 -05:00
will.anderson 95b6fac094 fix: use elc-cli.el as gen3 entry point, not compiler.el directly
El SDK CI - dev / build-and-test (pull_request) Failing after 1m8s
compiler.el imports lexer.el/parser.el/codegen.el with bare names; those
resolve relative to the source file's directory only when the entry point
is elc-cli.el (which imports el-compiler/src/compiler.el by full path).
Compiling compiler.el directly leaves lex/parse/codegen as undefined refs.
2026-05-06 17:39:38 -05:00
will.anderson af66cebfd3 fix: add -lssl -lcrypto to all CI gcc linker commands
El SDK CI - dev / build-and-test (pull_request) Failing after 28s
el_runtime.c now uses OpenSSL EVP AEAD encryption; all gcc commands
in all three workflow files need -lssl -lcrypto to link correctly.
2026-05-06 17:36:48 -05:00
will.anderson 1b9bc049de ci: trigger test run 2026-05-06 17:15:37 -05:00
Will Anderson 19a7430ff8 ci-dev: add PR trigger, gate publish on push (post-merge only) 2026-05-06 17:11:28 -05:00
Will Anderson 6b0a77a1dd Restructure CI/CD into proper dev -> stage -> main gated pipeline
- ci-dev.yaml: push to dev only (remove stale PR trigger)
- ci-stage.yaml: PR from dev validates, push to stage publishes to foundation-stage;
  add -lm/-Wl,--allow-multiple-definition flags and all 9 native --test suites
- sdk-release.yaml: add PR to main trigger for validation, gate publish/release/dispatch
  on push (post-merge) only; add -lm flags and all 9 native --test suites to main as well
2026-05-06 17:05:49 -05:00
Will Anderson 17e204ff2b Merge branch 'main' into dev 2026-05-06 17:01:03 -05:00
Will Anderson fc6d496937 fix: correct if-stmt parser test assertions — if is an expression in El 2026-05-06 16:45:01 -05:00
Will Anderson 2f713d64d4 Merge PR #4: perf: 81% RSS reduction in elc compiler + --test mode 2026-05-06 14:35:54 -05:00
Will Anderson 0a2a084d65 Merge PR #3: feat: port remaining foundation El source into monorepo 2026-05-06 14:35:49 -05:00
Will Anderson 7116610ffd Merge PR #2: feat: port el-ui vessels — rename crates→vessels, add El source + manifests 2026-05-06 14:35:45 -05:00
Will Anderson 810d8107da Merge PR #1: ci: fix gen2/gen3 gcc flags and step name formatting 2026-05-06 14:35:30 -05:00
will.anderson 607d5661d8 Merge pull request 'feat: port el-ui vessels — rename crates→vessels, add El source + manifests' (#2) from feat/el-ui-html-emit into dev 2026-05-06 19:34:00 +00:00
will.anderson c4a569f29b Merge pull request 'ci: fix gen2/gen3 gcc flags and step name formatting' (#1) from fix/ci-workflow-flags into dev 2026-05-06 19:33:56 +00:00
Will Anderson ec889e1e53 Add --test mode to elc with Assert stmt and full native test suite passing
Implement compile_test() entry point that emits a C test harness instead
of a normal program. Test blocks (previously skipped) now compile to
static functions with per-assertion pass/fail tracking. Assert statement
added to parser and codegen. Runtime extended with now_ns, fs_list_json,
json_build_object, json_build_array, json_escape_string, state_has,
state_get_or. Fix float negation codegen, float equality comparisons,
time_to_parts return type (JSON string), time_format empty-fmt, json_set
raw-value semantics, state_keys JSON array return. All 310 native tests
pass across 9 suites (core, text, string, math, env, state, json, time, fs).
2026-05-06 14:33:47 -05:00
Will Anderson 6ced0f8009 fix: double-free in engram_neighbors_json BFS + rebuild engram.c
el_strdup tracks pointers in the arena. The BFS arrays in
engram_neighbors_json are manually freed — using el_strdup caused a
double-free when the arena was later popped. Changed to plain strdup
for those allocations.

engram/dist/engram.c rebuilt from engram/src/server.el with current
elc (minor codegen diff: parenthesisation and _argc/_argv rename).
2026-05-06 14:11:40 -05:00
Will Anderson bd7303447b fix: skip test blocks in codegen to prevent OOM on test files
test "name" { ... } blocks were not recognized by the self-hosted
compiler. The body { } was parsed as a Map literal, creating a huge
AST with O(n²) string concatenation in the toplevel_exec_stmts loop
(which had no arena scope). A 272-line test file would consume 400MB+
and a 720-line file importing the full compiler source caused 150GB
usage and crashed the machine.

Two fixes:
1. Skip Test tokens in codegen_streaming before parse_one() —
   advance past "name" and skip_to_rbrace on the body block.
   Test blocks are never compiled; self-hosted compiler has no test runner.

2. Add per-statement arena scope to toplevel_exec_stmts emission loop,
   matching the el_main_body loop. Frees intermediate strings after
   each statement to prevent O(n²) accumulation from any unrecognized
   construct that reaches that path.

Result: test_string.el (272 lines, 27 test blocks): 0MB peak (was 400MB+).
        test_compiler.el (720 lines + 8728 imported): 15MB peak (was 150GB).
2026-05-06 13:34:03 -05:00
Will Anderson e8f6765750 fix: arena leak in compile() — token/sig strings now tracked
Wrapped compile() body in el_arena_push/pop so the arena is active
before lex() and scan_fn_sigs(). Previously both ran with
_tl_arena_active=0, leaking all token and signature strings permanently.
Also prevents inner pop(mark=0) calls from deactivating the arena
between per-function scopes. Verified: self-host PASS, RSS stable.
2026-05-06 10:53:12 -05:00
Will Anderson 3726f69435 perf: 81% RSS reduction — el_release, arena scoping, streaming codegen, libcurl stub
Chain of optimizations from swarm rounds 4-7:
- Flat stride-2 token list: eliminate per-token Map allocation (~112B each × N tokens)
- Systematic el_release() in parser.el: eagerly free intermediate parse result maps
- Per-function and per-statement arena scoping in codegen_streaming()
- Streaming codegen pipeline: parse one fn at a time, emit C, discard AST
- HAVE_CURL guard: elc CLI binary drops libcurl, eliminating SSL/TLS init overhead
- HTML codegen parts-list: O(n) instead of O(n²) string growth for nested templates
- Batch c_escape: str_slice clean runs instead of char-at per byte

Result: 33.4MB → 6.5MB RSS on web/src/main.el (-81%). Self-host: PASS.
2026-05-05 20:39:38 -05:00
Will Anderson ee86736eab merge round-4-delta: flat stride-2 token list + str_char_code dispatch + batch c_escape
- Flat token list: lexer emits [kind0, val0, kind1, val1, ...] instead of [{kind,val}, ...]
  Eliminates per-token ElMap allocation (~112B × N tokens)
- str_char_code hot loop: char classification via Int codes, no strdup per char
- Batch c_escape: str_slice clean runs instead of char-at per byte
- Parser updated to use tok_at/tok_kind/tok_value stride-2 accessors
2026-05-05 20:29:35 -05:00
Will Anderson eb52be4ade runtime: add EL_TRUE/EL_FALSE macros and scoped arena for CLI
Adds EL_TRUE/EL_FALSE convenience macros to el_runtime.h alongside the
existing EL_NULL, making boolean-returning builtins readable without
raw (el_val_t) casts. Documents all value macros in the header comment.

Also lands el_arena_push/el_arena_pop — a scoped string arena for CLI
programs that never call el_request_start/end. The compiler can push a
mark before a compilation unit and pop it after to free intermediate
strings, reducing peak RSS during long compile runs.
2026-05-05 19:15:49 -05:00
Will Anderson e587bedf30 round-3-gamma: combine c_escape + scan_interp_string batching — max round-3 savings
Combines two orthogonal optimizations:
1. c_escape batching (from alpha): ASCII runs emitted as str_slice segments instead
   of one str_char_at string per byte. O(N) allocs → O(K) where K = special chars.

2. scan_interp_string batching (from beta): char dispatch via str_char_code (Int)
   + clean_start tracking to flush plain runs as str_slice. Eliminates per-char
   string allocations in the string-literal scanning hot path.

Result on web/src/main.el: 14.5MB -> 13.4MB peak RSS (-7.6%).
Self-hosting: PASS.
2026-05-05 16:01:05 -05:00
Will Anderson 1eef9928f4 round-2-gamma: combine flat token list + char code dispatch — max round-2 savings
Combines two orthogonal optimizations:
1. Flat token list (from beta): lex() returns [Any] with alternating kind/value
   pairs instead of [Map], eliminating one ElMap per token (~3 mallocs each).
   Parser updated: tok_kind(t,i) = t[2*i], tok_value(t,i) = t[2*i+1].

2. Char code dispatch (from alpha): lex() hot loop uses str_char_code -> Int
   instead of str_char_at -> strdup String for all character classification.
   Eliminates ~400K x 16B = 6.4MB of temporary string allocations.

scan_digits and scan_ident also updated to use str_char_code.

Result on main.el: 17.1MB -> 14.4MB peak RSS (-16%).
Self-hosting: PASS.
2026-05-05 15:46:20 -05:00
Will Anderson 1e67544c88 round-2-alpha: char code ops in lex() hot loop — eliminate str_char_at allocations
Replace str_char_at (returns strdup String) with str_char_code (returns Int)
in the main lex() while loop and scan_digits/scan_ident helpers.

For a 400KB combined source, str_char_at was allocating ~400K x 16B = 6.4MB
of transient 2-byte strings for the ch variable alone. str_char_code returns
an integer directly — zero allocation.

Add Int-based helpers: is_digit_code, is_alpha_code, is_ws_code,
is_alnum_or_underscore_code. Rewrite lex() operator dispatch using char
code constants (e.g. '/'=47, '"'=34, '='=61).

Result on main.el: 17.1MB -> 15.4MB peak RSS (-10%).
Self-hosting: PASS.
2026-05-05 15:43:29 -05:00
Will Anderson 2ac11a67b1 beta: replace native_string_chars with str_char_at/str_slice in lexer — 49% memory reduction on large files 2026-05-05 15:19:59 -05:00
Will Anderson 7f295bffe9 fix: codegen O(n²) HTML memory leak + elb stderr surface + runtime dir path 2026-05-05 14:40:15 -05:00
Will Anderson 962c8cbe57 dist: add linux/amd64 binaries and el_runtime.js 2026-05-05 09:44:25 -05:00
Will Anderson a54797ebfe dist: add linux/amd64 binaries and el_runtime.js 2026-05-05 09:43:45 -05:00
Will Anderson db157106ee revert: remove forge — not part of the monorepo 2026-05-05 04:31:27 -05:00
Will Anderson 135744b4fe revert: remove dharma — not part of the monorepo 2026-05-05 04:31:20 -05:00
Will Anderson 90ddbdbfc3 feat: port arbor, dharma, forge El source into monorepo
Brings the remaining foundation repos that were not included in the
original monorepo consolidation:

- arbor/vessels/ — 6 vessels (arbor-cli, arbor-core, arbor-diagram,
  arbor-layout, arbor-parse, arbor-render) with manifests + src/main.el
- dharma/ — CGI Provenance Registry package (flat layout, 14 .el files
  across registry/, sandbox/, training/, validation/, tests/)
- forge/ — consciousness channel tool (8 src .el files + new manifest.el)
- elp/src/ — 36 test fixture files not carried over in original merge
  (dedup_*, realizer_*, semantics_*, morph_*, ext_*, one_extern_* helpers)

el-ide, engram, elql are already complete in ide/, engram/, ql/.
2026-05-05 04:27:34 -05:00
Will Anderson 7c00922cbd fix(el-html): tighten manifest — correct description, remove unused dependencies
el-html is a standalone atomic emit layer; it has no runtime dependency on
el-style or el-layout (those vessels depend on el-html for SSR, not the other
way around).
2026-05-05 04:20:38 -05:00
Will Anderson faee6fdb25 feat: port el-ui vessels — rename crates→vessels, add El source + manifests 2026-05-05 04:19:22 -05:00
Will Anderson a6d093536a ci: fix gen2/gen3 gcc flags and step name formatting
El SDK CI - dev / build-and-test (pull_request) Failing after 7s
- add -lm (el_runtime.c uses pow/sqrt/log/sin/cos/exp)
- add -Wl,--allow-multiple-definition to gen2 (is_digit/is_whitespace
  defined in both elc-bootstrap.c and el_runtime.c; bootstrap predates
  the text-processing primitives commit)
- remove colon from Self-host step name (Gitea YAML parser rejects it)
- replace em dashes in step names with hyphens
2026-05-05 03:04:54 -05:00
Will Anderson b580a63540 release: el-install binary and SDK bundle 2026-05-05 03:03:06 -05:00
Will Anderson bdd7b56703 feat: el-install binary + SDK bundle release 2026-05-05 03:03:01 -05:00
Will Anderson 592f8f482a add el-install binary and SDK bundle to release pipeline
- lang/tools/install/el-install.el: El program that fetches the latest
  release from the Gitea API, downloads el-sdk-latest.tar.gz, and
  extracts it into ~/.el (or a custom prefix passed as argv[1])
- lang/tools/install/manifest.el: build manifest for the el-install package
- .gitea/workflows/sdk-release.yaml: build elb, epm, and el-install
  binaries; bundle elc + elb + epm + runtime files into el-sdk-latest.tar.gz;
  attach both the tarball and el-install binary to the Gitea release
  alongside the existing per-file GCP uploads
2026-05-05 03:02:56 -05:00
Will Anderson 8524479f89 sync CI fixes and pre-commit hook from main 2026-05-05 02:38:01 -05:00
Will Anderson b2775b9228 fix CI paths for monorepo lang/ layout; add pre-commit hook 2026-05-05 02:37:56 -05:00
Will Anderson 52d0dd4225 release: monorepo restructure — foundation repos unified under el/ 2026-05-05 02:25:53 -05:00
Will Anderson d7d7852f2e monorepo: fold engram, elp, el-ui, elql, el-ide into el (history preserved) 2026-05-05 02:25:34 -05:00
Will Anderson 27f99f4053 monorepo: fold el-ide into ide/ (history preserved) 2026-05-05 01:40:49 -05:00
Will Anderson f1c0604ed8 monorepo: fold elql into ql/ (history preserved) 2026-05-05 01:40:45 -05:00
Will Anderson 4395d551c4 monorepo: fold el-ui into ui/ (history preserved) 2026-05-05 01:40:41 -05:00
Will Anderson 4a3f53cae6 monorepo: fold elp into elp/ (history preserved) 2026-05-05 01:40:34 -05:00
Will Anderson f51daa6ba0 monorepo: fold engram into engram/ (history preserved) 2026-05-05 01:40:30 -05:00
Will Anderson 1ae68962cf restructure: move el compiler content into lang/ 2026-05-05 01:38:51 -05:00
Will Anderson ce68f91a38 merge integrate/el-html-templates: add HTML template parser and codegen 2026-05-05 00:18:03 -05:00
Will Anderson 3e7d316c65 feat: extract HTML template parser/codegen from feat/el-html-templates
Parser additions (parser.el, no existing features removed):
- HTML template parser functions: is_html_tag_name, is_void_element,
  parse_html_text_tokens, parse_html_attrs, parse_html_children,
  parse_html_each_body, parse_html_element, parse_html_template
- HtmlTemplate detection in parse_primary (<tagname> and <!doctype>)
- Lambda fn literal expression node (parse_primary)
- Enum::Variant pattern matching (parse_pattern)
- type definition optional = before {
- try/catch statement (TryCatch AST node)

Codegen additions (codegen.el, no existing features removed):
- HTML template C codegen: cg_html_template, cg_html_parts,
  cg_html_attrs_str, cg_html_element_str, cg_html_each, next_html_id
- HtmlTemplate and Lambda dispatch in cg_expr
- Variant pattern support in cg_match
- TryCatch lowering in cg_stmt (C: runs try body, ignores catch)
- builtin_arity entries: getpid_now, stdout_to_file, stdout_restore

JS codegen additions (codegen-js.el, pure additions only):
- JS HTML template codegen: js_cg_html_template and helpers
- HtmlTemplate dispatch in js_cg_expr

Example: examples/html-page.el
2026-05-05 00:17:59 -05:00
Will Anderson 847f556ee3 merge integrate/js-browser-runtime: add JS compilation target 2026-05-05 00:11:07 -05:00
Will Anderson 92f393afd8 feat: extract JS browser runtime from feat/js-browser-runtime
- Update el-compiler/src/codegen-js.el to Phase 5 (1245 lines, up from 926)
  Adds: lambda literals, try/catch, extern fn, JS method call, Promise helpers,
  Object/Array utils, URL import declarations
- Update el-compiler/runtime/el_runtime.js (1049 lines, up from 679)
- Add examples/browser-counter.el, examples/browser-auth.el
- Update spec/codegen-js.md to Phase 5 status
- Update el-compiler/src/compiler.el: add --bundle, --minify, --obfuscate flags,
  bundled IIFE mode, terser/javascript-obfuscator post-processing pipeline
- No lexer.el or parser.el taken from this branch
2026-05-05 00:11:03 -05:00
Will Anderson 3fbfe76f14 merge integrate/native-testing: add native El test suite 2026-05-05 00:10:00 -05:00
Will Anderson 9013e241c3 feat: extract native El test suite from feat/native-testing
- Add tests/native/test_{core,text,string,math,state,time,json,env,fs}.el
- test_codegen_js.el renamed to test_core.el per dev convention
- Add native test CI steps to ci-dev.yaml (compile-link-run pattern)
- No lexer.el/parser.el/codegen.el changes taken from this branch
2026-05-05 00:09:57 -05:00
Will Anderson f9406afc83 merge ci/add-release-workflow into dev 2026-05-05 00:04:51 -05:00
Will Anderson e16f18b409 merge ci/wire-engram-elql-dispatch into dev 2026-05-05 00:02:41 -05:00
Will Anderson 507b220518 merge runtime/integrate into dev 2026-05-05 00:01:16 -05:00
Will Anderson 4e79edbe81 ci: retrigger after ci-base image rebuild 2026-05-04 20:17:16 -05:00
Will Anderson deb7faba7f ci: retrigger after ci-base image rebuild 2026-05-04 20:17:14 -05:00
Will Anderson e8b22e16a2 add el-tests vessel: manifest-based test suite under tests/suite/
Restructures the test suite as a proper El vessel with manifest.el and
src/ layout, eliminating the bash run.sh harness. CI runs the suite with
two commands: `cd tests/suite && elb && ./dist/el-tests`. Exit code is
the fail count (0 = all pass).

163 test cases across 7 modules: string (52), math (13), json (26),
state (11), time (25), fs (16), collections (19).
2026-05-04 19:53:35 -05:00
Will Anderson 26e327ac62 enforce source branch in CI: stage←dev, main←stage 2026-05-04 19:34:51 -05:00
Will Anderson c704e53102 enforce source branch in CI: stage←dev, main←stage 2026-05-04 19:34:45 -05:00
Will Anderson 3086a56b5c enforce source branch in CI: stage←dev, main←stage 2026-05-04 19:34:34 -05:00
Will Anderson 2eebd13221 Add release workflow listening to el-sdk-updated and engram-updated
Triggers on push to main plus repository_dispatch for both el-sdk-updated
and engram-updated. Installs El SDK from foundation-prod, compile-checks all
standalone .el programs, and includes elql-updated dispatch placeholder for
future downstream consumers.
2026-05-04 19:32:23 -05:00
Will Anderson 38d1905f1d Wire engram-updated dispatch to elql
Replace placeholder comment with actual curl dispatch call that fires
engram-updated to neuron-technologies/elql on every Engram release.
2026-05-04 19:32:08 -05:00
Will Anderson 7c644b8d89 Add dev/stage CI pipelines and expand sdk-release to full prod pipeline
- Add ci-dev.yaml: builds elc gen2→gen3, runs 4 test suites, publishes
  el/elc + el_runtime.c + el_runtime.h to foundation-dev Artifact Registry
- Add ci-stage.yaml: same as dev but targets foundation-stage registry
- Update sdk-release.yaml: publish 3 SDK artifacts to foundation-prod
  Artifact Registry after Gitea release; expand dispatch list from 2 to 6
  downstream repos (el-ui, elp, elql, el-ide added alongside engram/forge)
2026-05-04 19:32:05 -05:00
Will Anderson 9e8d23bcd9 add epm — El Package Manager
Introduces epm/, a new component written entirely in native El.
epm manages vessels (El's deployable package format): publish to Engram,
install with full dependency resolution, list registry contents, and
inspect vessel metadata.

- epm/manifest.el         — package manifest
- epm/src/manifest.el     — vessel/package manifest parser (line-by-line,
                            same approach as elb.el)
- epm/src/registry.el     — Engram-backed vessel registry (POST /api/nodes,
                            GET /api/search); vessels stored as Entity nodes
                            with label "vessel:<name>:<version>"
- epm/src/install.el      — topological dependency resolver with cycle
                            detection; installs to .epm/vessels/<name>/
- epm/src/epm.el          — main entry point: publish / install / list / info
2026-05-04 19:31:24 -05:00
Will Anderson 0791fda43e elb: add -lm to link flags — el_runtime.c uses math.h functions
el_runtime.c includes <math.h> and calls pow(), sqrt(), log() in several
places (math operations, engram dampening, float formatting). Without -lm
the linker fails on Linux when linking programs built with elb.
2026-05-04 19:14:09 -05:00
Will Anderson 53f2df500d runtime: add dharma-required functions to el_runtime.c and runtime/*.el
Add the following functions that dharma registry calls but were missing
from the El runtime:

el_runtime.c (consumed by the old build system via released SDK):
  - list_len, list_get — aliases for el_list_len/el_list_get (handlers.el)
  - json_array_push — append pre-encoded element to JSON array string
  - now_millis, unix_timestamp_ms, time_now_ms — ms-since-epoch aliases
  - log_info, log_warn — structured stderr log helpers
  - config — reads config from environment (alias for getenv)
  - http_patch — HTTP PATCH with Content-Type: application/json
  - http_post_engram — HTTP POST with optional X-API-Key header
  - http_get_engram — HTTP GET with optional X-API-Key header
  - str_to_bytes — encode string as JSON byte array [72,101,...]
  - bytes_to_str — decode JSON byte array back to string
  - hash_sha256 — SHA-256 hex digest using built-in sha256 impl

runtime/*.el (consumed by the new build system):
  - http.el: http_patch, http_post_engram, http_get_engram
  - time.el: now_millis, unix_timestamp_ms, time_now_ms
  - env.el: config, log_info, log_warn, list_len, list_get
  - json.el: json_array_push, bytes_to_str
  - string.el: str_to_bytes, hash_sha256 (via __sha256_hex seed)

el_seed.h / el_seed.c:
  - __sha256_hex primitive with self-contained SHA-256 implementation
2026-05-04 19:07:08 -05:00
Will Anderson b0d0f18524 ci: fix YAML in workflow
El CI -dev / build-and-test (push) Failing after 35s
2026-05-04 14:33:33 -05:00
Will Anderson 24ccef820c ci: add workflow_dispatch trigger 2026-05-04 14:32:48 -05:00
Will Anderson eb282d8c0c ci: add El CI dev pipeline 2026-05-04 14:22:10 -05:00
Will Anderson cd164debb8 add /nodes/list as alias for GET /nodes
Dharma's EngramDB client calls /nodes/list to retrieve all nodes.
Add this as an alias for the existing /nodes (and /api/nodes) route
so downstream clients don't need to be updated when the API drifts.

Also update dist/engram.c to match server.el.
2026-05-04 11:44:22 -05:00
Will Anderson cab8509608 add gitflow CI for dev/stage/prod environments 2026-05-04 08:55:34 -05:00
Will Anderson dbff2dad7a add gitflow CI for dev/stage/prod environments 2026-05-04 08:55:23 -05:00
Will Anderson 245eb2898e runtime: declare __thread_create and __thread_join in header for C99 compliance 2026-05-03 18:00:47 -05:00
Will Anderson 7bfe30b767 elb: raise clang bracket depth to 1024 — fixes compile failure for large JS string renders 2026-05-03 17:37:56 -05:00
Will Anderson 6ede9e4379 runtime: restore el_runtime.c as build shim; fix el_seed.h self-contained types
el_runtime.c was deleted prematurely — elb still resolves the runtime at build time
via a hardcoded relative path, and the elc code generator still emits
#include "el_runtime.h" in generated C.

Restoring el_runtime.c + el_runtime.h as the working build runtime until the
compiler is updated to emit #include "el_seed.h" and link against el_seed.c
directly.

el_seed.h: remove #include "el_runtime.h" that broke after el_runtime.h deletion;
add inline el_val_t typedef + macros + float cast helpers so el_seed.h is fully
self-contained.
2026-05-03 17:35:14 -05:00
Will Anderson 4ae42ee7db runtime: native SSE streaming — http_sse_open/send/close
Add Server-Sent Events support to the El runtime. El v2 handlers can now
hold HTTP connections open and push events in real time.

New builtins in el_seed.c:
  __http_conn_fd()          — retrieve raw fd from thread-local set by worker
  __http_sse_open(fd)       — send SSE headers (text/event-stream), keep-alive
  __http_sse_send(fd, data) — write "data: <data>\n\n" frame
  __http_sse_close(fd)      — close the connection fd

http_worker_v2 in legacy/el_runtime.c now:
  - stashes the fd via el_seed_set_http_conn_fd() before calling the handler
  - detects the "__sse__" sentinel return value to skip http_send_response
    and skip close(fd) — SSE handler took ownership of the fd
  - clears the thread-local after the handler returns

El wrappers added to runtime/http.el:
  http_conn_fd() http_sse_open(fd) http_sse_send(fd, data)
  http_sse_close(fd) http_sse_sentinel()
2026-05-03 17:15:37 -05:00
Will Anderson 3e5130e98d remove el_runtime.c — runtime is 100% native El
el_runtime.c and el_runtime.h removed from the active runtime directory
(archived copies remain in el-compiler/runtime/legacy/).
tools/lsp/build.sh removed as it depended on el_runtime.c directly.
AGENTS.md updated to reflect el_seed.c as the sole C dependency.
2026-05-03 17:10:04 -05:00
Will Anderson beb4e436e1 archive el_runtime.c — native El runtime complete, seed is self-contained
el_seed.c now defines el_request_start/el_request_end directly (delegating
to its own seed arena) rather than declaring them as externs from el_runtime.c.
Header comment updated to reflect self-contained build.
2026-05-03 17:08:49 -05:00
Will Anderson 71a1e41f93 lsp: type-aware field completions — scan type/let/param decls, complete on dot-access
When a document is opened or changed, scan for `type Name { field: Type }` blocks
and `let var: Type` / `fn foo(param: Type)` annotations. On completion requests,
if the text before the cursor ends with `identifier.`, look up the variable's type
and return its fields as Field (kind=5) completion items instead of the full list.
2026-05-03 16:16:11 -05:00
Will Anderson 0676725cb7 Bundle El runtime as installable framework
- runtime/stdlib.el: master import file for the full El standard library
  in correct dependency order (string→math→time→env→fs→exec→json→http→
  state→thread→channel→engram→manifest); test.el excluded (dev-only)

- tools/install.sh: installs El to a prefix (default /usr/local/el);
  copies elc binary, runtime .el files, headers, compiles libel.a from
  el_seed.c + el_runtime.c, generates an installed stdlib.el with absolute
  paths

- tools/new-project.sh: scaffolds a new El project with src/main.el,
  build.sh (auto-discovers local elc/runtime), README.md, .gitignore;
  verified working end-to-end

- AGENTS.md: fix elc rebuild docs — elc writes to stdout, not to its second
  argument; correct usage is ./dist/platform/elc elc-cli.el > elc-new.c
2026-05-03 16:04:26 -05:00
Will Anderson d98a968f89 rebuild elc — add __read_n and __print_raw for LSP stdin framing (self-hosting verified) 2026-05-03 16:03:34 -05:00
Will Anderson 0c9154551f merge tools/lsp (full) — 184 completions, hover, go-to-def, live diagnostics, enhanced VSCode extension 2026-05-03 16:01:11 -05:00
Will Anderson 9aa0c49d0c add full El LSP — completions, hover, go-to-def, diagnostics, VSCode extension 2026-05-03 15:59:42 -05:00
Will Anderson 59b96e3324 rebuild elc from sprint-integrated source — all language features verified
Self-hosting sprint merges compiled in:
- compiler: break/continue, % modulo, for-range loops, match-stmt codegen, string interpolation
- runtime: threading, http, json, fs/exec/env, time/math/state, engram, string (57 fns), channels
- seed: el_seed.c minimal C OS boundary (968 lines)
- tools: LSP skeleton with JSON-RPC, completions, hover, go-to-def, VSCode extension
- tests: El test framework with state-backed runner + string test suite

Self-hosting verified: elc-new → C → binary → same C output (idempotent).
Provenance snapshot: dist/platform/elc.20260503-1555-post-sprint
2026-05-03 15:55:50 -05:00
Will Anderson 287b39f8e6 merge tools/lsp — El LSP skeleton: JSON-RPC, completions, hover, go-to-def, VSCode extension 2026-05-03 15:53:39 -05:00
Will Anderson c45744d8ca merge runtime/seed — el_seed.c minimal C OS boundary (968 lines) 2026-05-03 15:52:21 -05:00
Will Anderson 2e778ca664 merge compiler/string-interp — string interpolation via lexer desugaring 2026-05-03 15:52:21 -05:00
Will Anderson 641227a7d3 merge runtime/channels — MPMC buffered channels, channel_pipeline, channel_fan_out 2026-05-03 15:52:21 -05:00
Will Anderson 805318e2d0 merge runtime/test — El test framework with state-backed runner and string test suite 2026-05-03 15:52:21 -05:00
Will Anderson cefff5b891 add El LSP skeleton — language server, VSCode extension, syntax highlighting
Implements a Language Server Protocol server for El files over stdin/stdout
JSON-RPC. Provides completions (builtins + keywords + document functions),
hover documentation, go-to-definition, and full document sync.

Also adds a VSCode extension that launches the binary as a child process and
a TextMate grammar for .el syntax highlighting.

NOTE: el-lsp.el calls __read_n(n: Int) -> String, a seed primitive not yet
in el_runtime.c. Build.sh documents the required C implementation; the seed
agent must add it before the binary will link.
2026-05-03 15:52:10 -05:00
Will Anderson ce9a2caff4 add string interpolation to El ("hello ${name}")
Lexer gains scan_interp_string which replaces scan_string in the main
lex loop. When no ${ is found it behaves identically to before (single
Str token). When interpolations are present it emits a flat token
sequence — Str, Plus, (expr tokens), Plus, Str, … — that the existing
parse_binop / cg_expr BinOp-Plus-string path assembles into nested
el_str_concat calls with zero parser or codegen changes.

Key design choices:
- scan_interp_brace tracks { depth so fn(a, b) inside ${} is safe
- inner expr tokens are wrapped in ( ) so operators like + in ${n+1}
  do not associate with the surrounding concat Plus tokens
- \$ escapes to a literal dollar sign; bare $ not before { passes through
- empty ${} emits an empty string segment
2026-05-03 15:50:23 -05:00
Will Anderson d1af4b0f8b add channels to El — buffered MPMC channel with send/recv/close
Introduces Go-style channels as El's mid-flight communication primitive,
completing the threading model: threads can now not only spawn/join but
also communicate while running.

Part 1 — seed layer (el_runtime.c / el_runtime.h):
- Add __thread_create/__thread_join/__mutex_new/__mutex_lock/__mutex_unlock
  as C seed primitives (dlsym-based thread dispatch, pthread mutex table)
- Add __channel_new/__channel_send/__channel_recv/__channel_try_recv/__channel_close
  as MPMC channel seed primitives backed by mutex + condvar + circular buffer
- Bounded channels (cap > 0): circular buffer, sender blocks when full
- Unbounded channels (cap == 0): dynamic array, grows on demand, never blocks
- channel_close wakes all blocked recvers/senders; recv drains then returns ""

Part 2 — El API (runtime/channel.el):
- channel_new/send/recv/try_recv/close — thin wrappers over seed layer
- channel_pipeline — spawn N worker threads reading from in_ch, applying
  fn_name, writing to out_ch; workers exit on "" sentinel from close
- channel_drain — collect all messages from a closed channel into [String]
- channel_fan_out — send a [String] list into a channel then close it

Part 3 — codegen.el:
- Register all 10 seed builtins (__thread_* + __channel_*) in builtin_arity
  so the arity checker validates call sites at compile time
2026-05-03 15:50:13 -05:00
Will Anderson 282df712a8 add runtime/test.el — El test framework with assertions and runner
Provides assert_true/false, assert_eq, assert_int_eq, assert_neq,
assert_contains, assert_starts_with, assert_ends_with, and fail.
Test cases are registered by name+fn_name, executed sequentially via
the thread/dlsym dispatch mechanism, with results tracked in state_.
Includes tests/runtime/string_test.el covering all 23 string.el exports.
2026-05-03 15:49:01 -05:00
Will Anderson cfcedff7f4 implement match statement codegen in El
Add cg_match_stmt() to lower match-as-statement to proper C if/else if/else
chains. Previously, match in statement position fell through to cg_expr() which
emitted a GCC statement-expression — fine for expression arms but wrong for the
statement form. Now matched using the same dispatch pattern as If and For in the
Expr handler of cg_stmt().

Pattern dispatch mirrors cg_match (expression form):
  LitStr  -> str_eq(subj, EL_STR("..."))
  LitInt  -> subj == N
  LitBool -> subj == 1 / 0
  Binding -> else { el_val_t name = subj; body; }
  Wildcard -> else { body; }

Subject is evaluated once into a scoped temporary to avoid double evaluation.
2026-05-03 15:47:50 -05:00
Will Anderson eab483ed4f add el_seed.c — minimal C OS boundary for El runtime migration
Introduces el_seed.c / el_seed.h as the clean OS-boundary layer for new-generation
El programs. All public symbols use the __ prefix convention; el_val_t (int64_t) is
the universal value type throughout.

Key additions over el_runtime.c:
- __thread_create / __thread_join: pthreads + dlsym(RTLD_DEFAULT) parallelism
  foundation. Static ElThread table (64 slots); worker resolves El fn symbols at
  runtime, stores result string for join to return.
- __mutex_new / __mutex_lock / __mutex_unlock: pooled pthread_mutex_t handles
- __http_do: unified curl call with JSON headers string (vs ElMap) and explicit
  timeout_ms parameter
- __fs_list_raw: returns newline-separated filename string (not ElList)
- __str_char_at: returns Int byte value (not single-char String)
- __args_json: CLI args as JSON array string; seeded by el_seed_init_args()

JSON, state, engram, HTML/URL, serve — thin __ wrappers over el_runtime.c.
Private seed arena (parallel to el_runtime.c arena) for standalone use.
2026-05-03 15:45:52 -05:00
Will Anderson cfa4301026 merge compiler features: break/continue, % modulo, for-range loops 2026-05-03 15:45:45 -05:00
Will Anderson f271f9d9d8 add for-range loops to El (for i in 0..n)
Adds `for i in start..end` (exclusive) and `for i in start..=end`
(inclusive) range loop syntax. Existing `for item in list` iteration
is preserved; the parser branches on DotDot/DotDotEq presence after
the start expression. Lexer adds DotDot and DotDotEq tokens with
longer-match-first priority. Codegen emits a C `for` loop with the
loop variable scoped to the statement; inclusive uses `<=`, exclusive `<`.
2026-05-03 15:44:58 -05:00
Will Anderson 49a8a1c24b add % modulo operator to El lexer, parser, codegen
Lexer and parser already had Percent token and precedence on the
compiler/string-interp branch. This commit adds the missing is_int_expr
case for Percent so that modulo expressions over Int operands are
correctly typed as Int (enabling arithmetic dispatch rather than
falling through to string concat or untyped paths).

binop_to_c already mapped Percent -> % at HEAD; only is_int_expr
needed the Percent arm.
2026-05-03 15:44:19 -05:00
Will Anderson 252ad04c96 add break and continue statements to El 2026-05-03 15:43:20 -05:00
Will Anderson 3cc9b1cc3d merge runtime/util — time, math, state in El 2026-05-03 15:40:55 -05:00
Will Anderson 5678745381 add runtime/time.el, math.el, state.el — time, math, and state in El
Migrates the time, math/float, and in-process state surfaces from
el-compiler/runtime/legacy/el_runtime.c to self-hosted El source:

- runtime/time.el: time_now, sleep_secs/ms, time_to_parts (via pure-El
  Gregorian civil_from_days decomposition), time_format (ISO + strftime
  subset), time_add, time_diff, time_from_parts; full Instant/Duration
  nanosecond API (now, unix_seconds/millis, duration_seconds/millis,
  instant_to_iso8601, sleep_duration); TTL cache (ttl_cache_set/get/age
  backed by state); uuid_new / uuid_v4 via __uuid_v4 seed.

- runtime/math.el: el_abs, el_max, el_min (Int); math_sqrt/log/ln/sin/cos/pi
  (Float seed wrappers); float_to_str, int_to_float, float_to_int, str_to_float,
  format_float (__format_float seed), decimal_round (half-away-from-zero via
  pure-El _pow10/_floor_f helpers).

- runtime/state.el: state_set/get/del/keys thin wrappers over __state_* seeds;
  convenience helpers state_has and state_get_or.
2026-05-03 15:39:48 -05:00
Will Anderson e853f4a25c merge runtime/string — all string operations in El (57 functions) 2026-05-03 15:37:51 -05:00
Will Anderson 5b6915ec9e merge runtime/engram-build — engram wrappers, manifest, seed arity table 2026-05-03 15:37:42 -05:00
Will Anderson b0e38a245a merge runtime/fs-exec-env — filesystem, subprocess, environment in El 2026-05-03 15:37:42 -05:00
Will Anderson 84b5355fce merge runtime/json — JSON operations in El 2026-05-03 15:37:41 -05:00
Will Anderson b547d1daf8 merge runtime/http — HTTP client and server in El 2026-05-03 15:37:41 -05:00
Will Anderson cc6522c69d merge runtime/thread — native El threading model, parallel_map 2026-05-03 15:37:41 -05:00
Will Anderson 5807de835e add runtime/string.el — string operations implemented in El
All string, I/O, math, classification, splitting, joining, counting, padding,
and URL encoding functions from el_runtime.c implemented in El using seed
primitives. No C required; compiles via the normal El pipeline.
2026-05-03 15:37:33 -05:00
Will Anderson 33af4ed09e add runtime/engram.el, manifest.el; register seed builtins in codegen arity table
- runtime/engram.el: thin El wrappers over all __engram_* and __generate
  seed primitives (16 functions), matching the el_seed.c API exactly
- runtime/manifest.el: build manifest documenting module load order and
  the cat+compile+cc command for runtime builds
- el-compiler/src/codegen.el: add 77 __-prefix seed primitive entries to
  builtin_arity, covering str, fs, http, thread, exec, env, time, uuid,
  math, state, html, json, and engram seeds
2026-05-03 15:37:05 -05:00
Will Anderson f2c63f95fd add runtime/fs.el, exec.el, env.el — filesystem, subprocess, environment in El
Migrates fs_read/write/exists/mkdir/write_bytes/list, exec/exec_bg/exec_command/exec_capture,
env/args/exit_program, state_set/get/del/keys, uuid_new/v4, and list helpers get/len from
el_runtime.c into El source as thin wrappers over seed primitives.
2026-05-03 15:35:45 -05:00
Will Anderson 01849c2033 add runtime/thread.el — native El threading model with parallel_map
Introduces El's first-class threading primitives built on the seed layer's
__thread_create/__thread_join/mutex ops. parallel_map is the key deliverable:
spawns one thread per item, joins in order — replaces bash fan-out for room
dispatch and any other concurrent HTTP workload.
2026-05-03 15:35:31 -05:00
Will Anderson 56724325ed add runtime/json.el — JSON operations in El
Thin El wrappers over seed JSON primitives (json_get, json_get_raw,
json_parse, json_stringify, json_set, json_array_len, json_array_get,
json_array_get_string) plus typed extractors (json_get_string/int/float/bool)
and pure-El builders (json_build_object, json_build_array,
json_escape_string) that require no seed call.
2026-05-03 15:35:20 -05:00
Will Anderson fea830cca2 add runtime/http.el — HTTP client and server in El
Thin El wrappers over seed primitives that form the public HTTP API for
El programs. Covers GET/POST/DELETE, header-map variants, binary streaming
to file, form-auth, v1/v2 server dispatch, and http_response envelope
construction. Documents two new seed primitives needed: __http_do_map and
__http_do_map_to_file (ElMap-accepting variants to avoid needing map
iteration in El).
2026-05-03 15:35:04 -05:00
Will Anderson 8642ad3978 remove rust scaffolding — el is the implementation 2026-05-03 04:10:38 -05:00
Will Anderson 6f560de02a remove Rust workspace; El implementation is the canonical engram
Deletes the entire Rust first-pass: Cargo workspace, 10 crates,
engram-data/, engram-data-tx-log/, receptors/, studio/, and examples/.
Keeps: src/server.el, manifest.el, dist/, spec/, README.md,
engram-explainer.html.
2026-05-03 03:25:10 -05:00
Will Anderson 995f29eb42 remove internal .elh files managed by elb 2026-05-02 23:00:31 -05:00
Will Anderson 34725a3988 Rename: nlg → elp (Engram Language Protocol) 2026-05-02 22:15:25 -05:00
Will Anderson a7bbd2f792 merge: add engram CI workflow 2026-05-02 17:46:18 -05:00
Will Anderson 30a86c78d2 add engram CI/CD pipeline — auto-rebuild on push or el-sdk-updated 2026-05-02 17:46:00 -05:00
Will Anderson cbb27b8d87 Add semantics layer bridging intent frames to grammar realization
Introduces semantics.el with SemFrame (sem_frame/sem_frame_simple/sem_frame_obj
constructors), sem_to_spec to convert intent frames into realizer slot maps,
and sem_realize/sem_realize_full as end-to-end frame→text entry points.
Supports intents: assert, query, describe, greet.

Wires generate_frame() into nlg.el and adds 4 new passing tests
(sem-assert, sem-query, sem-describe, sem-greet). All 10 tests pass.
2026-05-02 14:45:54 -05:00
Will Anderson 1432c56cf7 Add native El NLG system: morphology, vocabulary, grammar, realizer
Implements a complete natural language generation stack in El:
- morphology.el: English pluralization, verb conjugation (40+ irregulars), determiner agreement
- vocabulary.el: inline seed lexicon (~100 entries: pronouns, nouns, verbs, adjectives, etc.)
- grammar.el: CFG rules (S/NP/VP/PP), slot-map driven tree generator, s-expression renderer
- realizer.el: semantic form -> English text with tense/aspect/agreement, do-support for questions
- nlg.el: JSON-driven public API tying all modules together
- tests/run.sh: acceptance corpus runner (6 tests, all passing)
2026-05-02 14:16:23 -05:00
will.anderson 020308a29a uncommitted state captured before pushing to Gitea 2026-05-02 10:24:09 -05:00
will.anderson 6e1a338eb6 uncommitted state captured before pushing to Gitea 2026-05-02 10:24:08 -05:00
Will Anderson 834065cf45 server: GET /api/nodes accepts ?node_type=X to filter at the engine
When the query string includes node_type, we route to the new
engram_scan_nodes_by_type_json builtin instead of the unfiltered
scan. Existing callers without the param get identical behaviour.

Smoke-tested live on the neuron engram (3,200+ nodes):
  ?node_type=Knowledge   → all Knowledge
  ?node_type=BacklogItem → all BacklogItem
  ?node_type=Imprint     → 1 Imprint (only one cultivated so far)
  ?node_type=DoesNotExist → []
2026-05-02 01:25:10 -05:00
Will Anderson 3b76f0f8e0 feat: El port — vessels populated alongside Rust
Adds src/main.el + manifest.el for each vessel in this workspace,
ported from the Rust sources during the El consolidation pass on
2026-04-30. Each vessel now has both Rust (legacy) and El (target)
sources side-by-side; Rust will be removed once the El paths are
verified at runtime, vessel by vessel.

Per-vessel work was split across multiple parallel agents reading the
Rust to understand intent, then designing idiomatic El. Not 1:1
transliteration. Each ported vessel includes:
  - manifest.el per spec/language.md \u00a715.1
  - src/main.el with the vessel's public surface
  - Compile verified via dist/platform/elc + cc against the el_runtime

Known gaps surfaced during the port (held for follow-up): HMAC-SHA256
and base64 crypto, HTTP status code in handler returns, request
headers in handler signatures, subprocess primitives, streaming
responses, struct/enum types, browser/JS codegen target. Codegen bug
list of 9 items tracked separately. The El sources here are runtime-
ready under the canonical C runtime; the gaps are language/runtime
extensions still in flight.
2026-04-30 18:18:39 -05:00
Will Anderson be013d2b42 rename crates/ → vessels/ — El's word for buildable units
Per the consolidation onto El: 'crates' is the Rust word, 'vessel' is
El's (per spec/language.md §15). The directory rename is the structural
marker that this slot holds an El buildable unit, even if its current
contents are still Rust pending port.

Mechanical: git mv crates vessels, sed workspace members and any path
dependencies, update CI workflow paths, update README references.
Cross-repo path dependencies (`../foo/crates/bar`) updated workspace-
wide so cargo metadata still resolves where the Rust still builds.
2026-04-30 15:34:20 -05:00
Will Anderson 2ae5dd430f engram: gitignore build artifacts and .el cache 2026-04-30 13:49:38 -05:00
Will Anderson 2b45fc2f0f engram: runtime-native rewrite
Engram is now a thin HTTP face over the El runtime's in-process graph
store. The C runtime owns the data; engram_*_json builtins serialize
results directly. There is no SQL, no SQLite, no db layer, no state
machine — the runtime IS the database.

src/server.el (348 lines, replacing 5797 lines across 15 legacy files):
  GET  /health
  GET  /api/stats
  POST /api/nodes              (auth required)
  GET  /api/nodes
  GET  /api/nodes/:id
  DELETE /api/nodes/:id        (auth required)
  POST /api/edges              (auth required)
  GET  /api/neighbors/:id
  POST /api/activate
  GET  /api/activate
  POST /api/search
  GET  /api/search
  POST /api/strengthen         (auth required)
  POST /api/save               (auth required)
  POST /api/load               (auth required)

Auth: ENGRAM_API_KEY in env. GET routes pass through (read-only).
Mutating routes require {"_auth": "<key>"} in the JSON body until
http_serve surfaces request headers and we can switch to Bearer.

Persistence: engram_save / engram_load via JSON snapshot at
$ENGRAM_DATA_DIR/snapshot.json. Loaded best-effort on startup.

Build: dist/platform/elc src/server.el > dist/engram.c
       cc -std=c11 -O2 -I <runtime> -lcurl -lpthread -o dist/engram
       dist/engram.c <runtime>/el_runtime.c

Live: native binary at dist/engram (113 KB), running under
~/Library/LaunchAgents/ai.neuron.engram.plist on :8742. Verified:
GET /api/stats returns counts; POST /api/nodes with auth creates
node with UUID; GET /api/search returns full node JSON; spreading
activation returns hop-decayed strengths (0.8 × edge × decay per
hop) with epistemic confidence filtering.

Legacy (5797 lines of SQLite-era src) sealed at
~/Archives/engram-src-legacy-20260430.tar.gz and removed from disk.
2026-04-30 13:49:28 -05:00
Will Anderson c16b6ed602 Replace el.toml with manifest.el throughout — El manifests are El, not TOML 2026-04-29 22:48:39 -05:00
Will Anderson 28bc05f29f Update framework spec; add counter and todo examples 2026-04-29 08:50:26 -05:00
Will Anderson 0b480cfb6b El data studio: 10-loop improvement pass — full Engram DB explorer
Full-featured terminal explorer for the Engram knowledge graph built
natively in El. Features:
- ANSI-colored TUI with box-drawing borders and salience bars
- All API endpoints: stats, nodes by type/tier, search, edges,
  spreading activation, node detail with neighbor traversal
- Text report export via fs_write
- Offline/unreachable mode with helpful startup messages
- Interactive mode command reference
- ENGRAM_URL env var for connecting to non-default servers
- Uses json_get_raw for nested JSON object traversal
2026-04-29 04:39:40 -05:00
Will Anderson dc4a9ee95f El IDE: rounds 14-20 — breadcrumb nav, version display, improved search, sticky scroll, status bar diagnostics
Round 14: Breadcrumb directory click — clicking a path segment in the breadcrumb expands/reveals that directory in the file tree
Round 15: El version in status bar — GET /api/status now returns el_version (via el --version), shown in status bar right side; EL_BINARY config env var
Round 16: Search improvements — case-sensitive, whole-word, regex toggles (Alt+C/W/R); project-wide replace-all in current file; backend SearchOpts struct for each mode
Round 17: Sticky scroll improvements — uses CM6 posAtCoords for accurate first-visible-line; clickable to jump to definition; sticky-name/sticky-goto styling
Round 18: File tree header — New File (+) button and Refresh (↺) button in file tree header panel
Round 19: Status bar diagnostics — error count (✕ N) and warning count (⚠ N) shown in status bar, clickable to jump to problems panel
Round 20: Polish — more El snippets (test, seed, assert, activate, parallel, deploy, import, with, retry, reason, trace), expanded command palette (11 new commands)
2026-04-29 04:38:53 -05:00
Will Anderson 12e537d6ab El IDE: 10-round pass — syntax highlighting, file browser, runner, completion, split panes, find/replace, settings, minimap
Round 1: Fix dependency paths (../el/crates → ../el/engrams), verify build
Round 2: Enhanced syntax highlighting — function call detection, all El keywords (activate, sealed, parallel, deploy, etc.)
Round 3: Full El keyword set in CodeMirror tokenizer and completions; 50+ builtin function completions with type signatures
Round 4: File system integration — mkdir, rename, delete, file tree search; git status badges
Round 5: Runner integration — Ctrl+R shortcut, SSE streaming output, clickable error lines with jump-to-line
Round 6: Error highlighting with accurate line/col from lexer/parser spans; diagnostic dedup
Round 7: Find/replace panel; Ctrl+G go-to-line; toggle line comment; word-wrap compartment fix
Round 8: Code completion — 50+ builtins, keyword completions, snippet completions, server snippet integration
Round 9: Resizable panels — file tree drag-resize + collapse (Ctrl+B), type-graph drag-resize, bottom panel toggle (Ctrl+J), width persistence
Round 10: Settings API (GET/POST/DELETE /api/settings, ~/.el-ide/settings.json); frontend wired to API with debounced save; theme persistence
Round 11: Minimap click-to-jump and drag-to-scroll
Round 12: Command palette — added Go To Line, Toggle Word Wrap/Minimap/File Tree/Bottom Panel, font size commands, New File, Select Next Occurrence
Round 13: Multi-cursor — Ctrl+D select next occurrence, EditorSelection exposed for multi-range selection
2026-04-29 04:34:08 -05:00
Will Anderson 909c1577f1 rename crates/ to engrams/, bindings/ to receptors/
- crates/ → engrams/ (Rust engrams live here)
- bindings/ → receptors/ (cross-language access points into the graph)
- Cargo.toml workspace paths updated
2026-04-29 03:27:33 -05:00
Will Anderson ea0de16562 Transform el-ide into AI-native IDE: Neuron pair programming, live type graph, semantic autocomplete, activation preview
- Neuron pair programming panel (right column, 300px): full conversation interface
  with code-block rendering, "Insert at cursor" for AI suggestions, quick actions
  (explain selection, find activations, analyze types), thinking indicator
- Live activation preview: floating inline widget when cursor is inside
  `activate TypeName where "..."` expressions; queries /api/lsp/activate-preview
- Type graph activation highlighting: `activate` statements cause named types to
  pulse with purple glow rings; animation loop runs while activations are present
- Knowledge Graph Explorer: new bottom panel tab; search the Engram DB for semantic
  concepts, insert activate expressions, ask Neuron about nodes
- Semantic autocomplete: completions with score >= 0.8 get "⟁ semantic" prefix and
  priority boost in the completion list
- Backend: /api/reason extended to support three modes: "hypothesis" (existing),
  "pair" (proxies to soma /v1/chat/completions with full file context + system
  prompt), "knowledge-search" (proxies to Engram DB search endpoint)
- Backend: new GET /api/lsp/activate-preview endpoint queries Engram for nodes
  matching a partial activate expression
- Type graph node click now navigates editor to the type definition
- Resolved merge conflicts in engram-lang/crates (ast.rs, parser.rs, checker.rs,
  types.rs): took union of HEAD and worktree-agent branches
2026-04-28 11:40:36 -05:00
Will Anderson 361c958618 add el-style, el-layout, el-i18n, el-config, el-secrets: responsive by default, theme-driven, zero breakpoints 2026-04-27 20:18:47 -05:00
Will Anderson a1159eec65 add el-identity: Engram-native identity, OAuth, @authenticate by default 2026-04-27 20:04:52 -05:00
Will Anderson 69d1085d2d el-ui v2: universal platform, service bindings, AOP, auth, publish pipeline 2026-04-27 19:52:29 -05:00
Will Anderson 3bf3c02854 feat: el-ui — activation-based frontend framework, spreading activation reactivity, graph state 2026-04-27 19:15:53 -05:00
Will Anderson 602cd1586a feat: el-ide — native IDE for engram-lang, LSP, type graph, plugin ecosystem
Axum HTTP server (port 7771) serving a single-page IDE with CodeMirror 6
syntax highlighting for engram-lang, a force-directed type graph visualizer,
LSP (completions, hover, diagnostics), SSE-streamed build/run output, a
plugin host with five first-party plugins, and a reasoning panel that proxies
to engram-server. 28 tests across three crates, zero warnings.
2026-04-27 19:12:42 -05:00
Will Anderson 61a4632163 feat: engram-reasoning — graph-native inference engine, evidence chains, confidence propagation 2026-04-27 18:36:37 -05:00
Will Anderson 192528543f feat: schema projections, command transactions, quantum-secure encryption 2026-04-27 18:26:46 -05:00
Will Anderson 69410a6908 feat: serve studio from engram-server — browser is the runtime, no Electron 2026-04-27 17:30:20 -05:00
Will Anderson 6601761cd9 feat: Engram sync layer — swarm memory protocol, peer delta sync, distributed activation 2026-04-27 17:18:51 -05:00
Will Anderson 2454c83e82 feat: HNSW index, consolidation engine, Kotlin/TS/Go bindings, SQLite migration connector
- vector.rs: replace flat O(n) scan with instant-distance HNSW for stores
  >= 100 nodes; flat scan retained as fallback for small graphs; dirty-flag
  persistence in sled triggers index rebuild only when nodes are added

- consolidation.rs: Episodic → Semantic promotion based on activation_count
  and salience_floor thresholds; global decay pass after each cycle;
  ConsolidationConfig + ConsolidationReport types; 8 tests

- migration.rs: reads Neuron SQLite (memory_nodes, knowledge_entries,
  graph_edges) and writes to Engram sled; placeholder unit-vector embeddings
  with TODO for ONNX; 5 tests including full in-memory DB roundtrip

- crates/engram-migrate: CLI binary (engram-migrate --sqlite / --output)

- crates/engram-jni: JNI cdylib exposing open/close/put_node/get_node/
  activate/search_embedding/touch/decay/node_count/edge_count via
  Java_ai_neuron_engram_EngramDb_* entry points; 6 tests

- bindings/kotlin: EngramDb.kt (AutoCloseable JNI wrapper), EngramNode,
  EngramEdge, ActivatedNode, EngramTypes; build.gradle.kts; settings.gradle.kts

- bindings/typescript: engram-wasm crate (wasm-bindgen, serde-wasm-bindgen);
  WasmEngramDb with in-memory backend (sled not available in WASM);
  TypeScript wrapper (index.ts, types.ts, package.json, tsconfig.json)

- bindings/go: engram.go (CGo wrapper), engram.h (C header), engram_test.go
  (4 tests covering open/close/put_node/get_node/node_count/decay); go.mod

- engram-core: wasm feature gate for in-memory backend; mem_storage.rs;
  activation.activate_mem for WASM path; Node::with_id helper;
  salience.rs doctest fixed (text block)

- examples/basic.rs: consolidation section added
- examples/migrate.rs: migration API demonstration

Build: cargo build --workspace -- zero warnings, zero errors
Tests: 38 pass (25 engram-core + 7 engram-ffi + 6 engram-jni)
2026-04-27 16:00:47 -05:00
Will Anderson 1a609502c8 init: Engram v0.1 — native memory substrate for accumulating intelligence
Memory is not stored and retrieved — it is activated and propagated.
Implements the spreading activation model with salience decay, typed edges,
four memory tiers, and flat cosine vector search over a sled embedded store.
2026-04-27 15:37:42 -05:00
762 changed files with 1112623 additions and 7158 deletions
+256 -46
View File
@@ -1,4 +1,4 @@
name: El CI -dev
name: El SDK CI - dev
on:
push:
@@ -7,11 +7,13 @@ on:
pull_request:
branches:
- dev
workflow_dispatch:
jobs:
build-and-test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps:
- name: Checkout
@@ -20,98 +22,306 @@ jobs:
- name: Install build dependencies
run: |
apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev
apt-get install -y gcc libcurl4-openssl-dev apt-transport-https ca-certificates
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" \
> /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
# Gen2: compile the bootstrap C source into a working elc binary
# -Wl,--allow-multiple-definition: is_digit/is_whitespace exist in both
# elc-bootstrap.c (pre-dates runtime text primitives) and el_runtime.c.
# Both definitions are equivalent; allow the linker to pick one.
- name: Build elc from bootstrap (gen2)
# Seed: use the committed linux-amd64 binary as the bootstrap
- name: Bootstrap from committed linux binary (seed)
run: |
gcc -O2 \
-I el-compiler/runtime \
dist/elc-bootstrap.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread -lm \
-Wl,--allow-multiple-definition \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host compile El compiler with gen2 (gen3)
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread -lm \
-I runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
echo "gen3 (self-hosted) elc built"
echo "gen2 (self-hosted) elc built"
dist/platform/elc --version || true
# Run all four test suites -all must pass
- name: Run tests -text
# Build elb (needed for Artifact Registry publish and downstream CI)
- name: Build elb
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
- name: Run tests - text
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/text/run.sh
- name: Run tests -calendar
- name: Run tests - calendar
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/calendar/run.sh
- name: Run tests -time
- name: Run tests - time
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/time/run.sh
- name: Run tests -html_sanitizer
- name: Run tests - html_sanitizer
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh
# Native El test suites (elc --test, compile-link-run)
- name: Run tests -native (text)
# el_runtime.c is precompiled to .o once and reused by all 8 modules.
- name: Precompile el_runtime.o
run: |
set -euo pipefail
RUNTIME="$(pwd)/runtime"
gcc -O2 -c -I "$RUNTIME" "$RUNTIME/el_runtime.c" \
-o /tmp/el_runtime.o
echo "el_runtime.o compiled"
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/el-compiler/runtime"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lpthread -lm -o /tmp/el_native_text
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
# Publish artifact to GCP Artifact Registry (dev)
- name: Publish elc to Artifact Registry (dev)
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c /tmp/el_runtime.o \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (dev)
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el/elc \
--package=el-elc \
--version="${VERSION}" \
--source=dist/platform/elc
# Also tag as latest-dev
echo "Published elc version=${VERSION} to foundation-dev/el/elc"
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-dev \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-dev"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (dev)
# Patches ci-base:dev in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:dev (or fall back to :latest on first run)
BASE_TAG="dev"
docker pull "${CI_BASE}:dev" || { docker pull "${CI_BASE}:latest" && BASE_TAG="latest"; }
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:${BASE_TAG}" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:dev" \
-t "${CI_BASE}:dev-${SHA}" \
.
docker push "${CI_BASE}:dev"
docker push "${CI_BASE}:dev-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:dev (${SHA})"
rm -f /tmp/gcp-key.json
+245 -34
View File
@@ -1,4 +1,4 @@
name: El CI stage
name: El SDK CI - stage
on:
push:
@@ -11,90 +11,301 @@ on:
jobs:
build-and-test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Enforce source branch (stage <- dev only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "dev" ]; then
echo "ERROR: Stage branch only accepts PRs from 'dev'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> stage"
- name: Install build dependencies
run: |
apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev
# Gen2: compile the bootstrap C source into a working elc binary
- name: Build elc from bootstrap (gen2)
# Seed: use the committed linux-amd64 binary as the bootstrap
- name: Bootstrap from committed linux binary (seed)
run: |
gcc -O2 \
-I el-compiler/runtime \
dist/elc-bootstrap.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-I runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
echo "gen3 (self-hosted) elc built"
echo "gen2 (self-hosted) elc built"
dist/platform/elc --version || true
# Run all four test suites — all must pass
- name: Run tests — text
- name: Run tests - text
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/text/run.sh
- name: Run tests calendar
- name: Run tests - calendar
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/calendar/run.sh
- name: Run tests time
- name: Run tests - time
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/time/run.sh
- name: Run tests html_sanitizer
- name: Run tests - html_sanitizer
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh
# Publish artifact to GCP Artifact Registry (stage)
- name: Publish elc to Artifact Registry (stage)
# Native El test suites (elc --test, compile-link-run)
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Build elb (needed for epm and el-install builds below)
- name: Build elb
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
# Publish only after merge (push event), not on PR validation runs
- name: Publish El SDK to Artifact Registry (stage)
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get install -y -qq apt-transport-https ca-certificates curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el/elc \
--package=el-elc \
--version="${VERSION}" \
--source=dist/platform/elc
echo "Published elc version=${VERSION} to foundation-stage/el/elc"
gcloud artifacts generic upload \
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-stage \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
echo "Published El SDK version=${VERSION} to foundation-stage"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK (stage)
# Patches ci-base:stage in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base:stage (system deps stay cached in the base layer)
docker pull "${CI_BASE}:stage" || docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:stage" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:stage" \
-t "${CI_BASE}:stage-${SHA}" \
.
docker push "${CI_BASE}:stage"
docker push "${CI_BASE}:stage-${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:stage (${SHA})"
rm -f /tmp/gcp-key.json
+307 -74
View File
@@ -4,81 +4,236 @@ on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build-and-release:
runs-on: ubuntu-latest
defaults:
run:
working-directory: lang
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Enforce source branch (main <- stage only)
if: github.event_name == 'pull_request'
run: |
SOURCE="${GITHUB_HEAD_REF}"
if [ "${SOURCE}" != "stage" ]; then
echo "ERROR: Main branch only accepts PRs from 'stage'. Source was: '${SOURCE}'"
exit 1
fi
echo "Source branch check passed: ${SOURCE} -> main"
- name: Install build dependencies
run: |
apt-get update -qq
apt-get install -y gcc libcurl4-openssl-dev
# Gen2: compile the bootstrap C source into a working elc binary
- name: Build elc from bootstrap (gen2)
# Seed: use the committed linux-amd64 binary as the bootstrap
- name: Bootstrap from committed linux binary (seed)
run: |
gcc -O2 \
-I el-compiler/runtime \
dist/elc-bootstrap.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-o dist/elc-gen2
chmod +x dist/elc-gen2
echo "gen2 elc built"
dist/elc-gen2 --version || true
chmod +x dist/platform/elc-linux-amd64
echo "seed elc (committed linux-amd64 binary)"
dist/platform/elc-linux-amd64 --version || true
# Gen3: use gen2 to compile the El compiler from its own El source (self-host)
- name: Self-host: compile El compiler with gen2 (gen3)
# Gen2: use seed to self-host compile the El compiler
- name: Self-host compile El compiler (gen2)
run: |
mkdir -p dist/platform
dist/elc-gen2 el-compiler/src/compiler.el > dist/elc-gen3.c
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c
gcc -O2 \
-I el-compiler/runtime \
dist/elc-gen3.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lpthread \
-I runtime \
dist/elc-gen2.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
chmod +x dist/platform/elc
echo "gen3 (self-hosted) elc built"
echo "gen2 (self-hosted) elc built"
dist/platform/elc --version || true
# Run all four test suites with gen3 elc
- name: Run tests — text
# Build elb binary
- name: Build elb
run: |
mkdir -p dist/bin
dist/platform/elc elb.el > dist/elb.c
gcc -O2 \
-I runtime \
dist/elb.c \
runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/bin/elb
chmod +x dist/bin/elb
echo "elb built"
# Build epm binary using elb (epm lives at repo root, not inside lang/)
- name: Build epm
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd ../epm && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/epm
echo "epm built"
# Build el-install binary using elb
- name: Build el-install
run: |
ABS_ELB="$(pwd)/dist/bin/elb"
ABS_ELC="$(pwd)/dist/platform/elc"
ABS_RUNTIME="$(pwd)/runtime"
ABS_OUT="$(pwd)/dist/bin"
(cd tools/install && "$ABS_ELB" --clean --elc="$ABS_ELC" --runtime="$ABS_RUNTIME" --out="$ABS_OUT")
chmod +x dist/bin/el-install
echo "el-install built"
- name: Run tests - text
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/text/run.sh
- name: Run tests calendar
- name: Run tests - calendar
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/calendar/run.sh
- name: Run tests time
- name: Run tests - time
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/time/run.sh
- name: Run tests html_sanitizer
- name: Run tests - html_sanitizer
run: |
ELC="$(pwd)/dist/platform/elc" \
EL_HOME="$(pwd)" \
bash tests/html_sanitizer/run.sh
# Publish / update the `latest` release with the three SDK assets
# Native El test suites (elc --test, compile-link-run)
- name: Run tests - native (core)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_core.el > /tmp/el_native_core.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_core.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_core
/tmp/el_native_core
- name: Run tests - native (text)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_text.el > /tmp/el_native_text.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_text.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_text
/tmp/el_native_text
- name: Run tests - native (string)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_string.el > /tmp/el_native_string.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_string.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_string
/tmp/el_native_string
- name: Run tests - native (math)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_math.el > /tmp/el_native_math.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_math.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_math
/tmp/el_native_math
- name: Run tests - native (state)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_state.el > /tmp/el_native_state.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_state.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_state
/tmp/el_native_state
- name: Run tests - native (time)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_time.el > /tmp/el_native_time.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_time.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_time
/tmp/el_native_time
- name: Run tests - native (json)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_json.el > /tmp/el_native_json.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_json.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_json
/tmp/el_native_json
- name: Run tests - native (env)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_env.el > /tmp/el_native_env.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_env.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_env
/tmp/el_native_env
- name: Run tests - native (fs)
run: |
set -euo pipefail
ELC="$(pwd)/dist/platform/elc"
RUNTIME="$(pwd)/runtime"
"$ELC" --test tests/native/test_fs.el > /tmp/el_native_fs.c
gcc -O2 -I "$RUNTIME" /tmp/el_native_fs.c "$RUNTIME/el_runtime.c" \
-lcurl -lssl -lcrypto -lpthread -lm -o /tmp/el_native_fs
/tmp/el_native_fs
# Bundle the SDK tarball - runs from the repo root to reference lang/ paths correctly
- name: Bundle SDK tarball
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
run: |
mkdir -p dist/sdk/bin dist/sdk/runtime
cp lang/dist/platform/elc dist/sdk/bin/elc
cp lang/dist/bin/elb dist/sdk/bin/elb
cp lang/dist/bin/epm dist/sdk/bin/epm
cp lang/runtime/el_runtime.c dist/sdk/runtime/
cp lang/runtime/el_runtime.h dist/sdk/runtime/
cp lang/runtime/engram_store.c dist/sdk/runtime/
cp lang/runtime/engram_store.h dist/sdk/runtime/
cp lang/runtime/*.el dist/sdk/runtime/
tar -czf dist/el-sdk-latest.tar.gz -C dist/sdk .
echo "SDK tarball bundled: dist/el-sdk-latest.tar.gz"
ls -lh dist/el-sdk-latest.tar.gz
# Publish / update the `latest` release with all SDK assets
- name: Publish latest release
if: github.event_name == 'push'
working-directory: ${{ github.workspace }}
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1
REPO: neuron-technologies/el
run: |
# Delete existing `latest` release if it exists
EXISTING_ID=$(curl -sf \
-H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/releases/tags/latest" \
@@ -91,12 +246,10 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${EXISTING_ID}"
fi
# Delete and re-create the `latest` tag so it points at HEAD
curl -sf -X DELETE \
-H "Authorization: token ${GITEA_TOKEN}" \
"${GITEA_API}/repos/${REPO}/tags/latest" || true
# Create the release
RELEASE_ID=$(curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
@@ -111,7 +264,6 @@ jobs:
echo "Created release id=${RELEASE_ID}"
# Upload assets
upload_asset() {
local filepath="$1"
local name="$2"
@@ -122,70 +274,151 @@ jobs:
"${GITEA_API}/repos/${REPO}/releases/${RELEASE_ID}/assets"
}
upload_asset dist/platform/elc elc
upload_asset el-compiler/runtime/el_runtime.c el_runtime.c
upload_asset el-compiler/runtime/el_runtime.h el_runtime.h
# Per-file assets (downstream CI needs these individually)
upload_asset lang/dist/platform/elc elc
upload_asset lang/runtime/el_runtime.c el_runtime.c
upload_asset lang/runtime/el_runtime.h el_runtime.h
upload_asset lang/runtime/engram_store.c engram_store.c
upload_asset lang/runtime/engram_store.h engram_store.h
# SDK bundle and installer binary
upload_asset dist/el-sdk-latest.tar.gz el-sdk-latest.tar.gz
upload_asset lang/dist/bin/el-install el-install
echo "Release published successfully"
# Dispatch el-sdk-updated event to downstream repos
# Publish artifact to GCP Artifact Registry (prod)
- name: Publish elc to Artifact Registry (prod)
- name: Publish El SDK to Artifact Registry (prod)
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
# Fail loudly: previously this step had no `set -e`, so an auth or
# upload failure was swallowed (step exited 0 on the trailing echo)
# and the SDK silently never published. Surface failures now.
set -euo pipefail
if [ -z "${GCP_SA_KEY:-}" ]; then
echo "FATAL: GCP_SA_KEY secret is empty — cannot authenticate to publish" >&2
exit 1
fi
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
apt-get install -y -qq apt-transport-https ca-certificates gnupg curl
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get install -y -qq apt-transport-https ca-certificates curl
echo "deb [trusted=yes] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update -qq && apt-get install -y google-cloud-cli
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
echo "Publishing as active account: $(gcloud config get-value account 2>/dev/null)"
VERSION="${GITHUB_SHA:0:8}"
VERSION="${GITEA_SHA:0:8}"
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el/elc \
--package=el-elc \
--version="${VERSION}" \
--source=dist/platform/elc
echo "Published elc version=${VERSION} to foundation-prod/el/elc"
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-elb \
--version="${VERSION}" \
--source=dist/bin/elb
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-c \
--version="${VERSION}" \
--source=runtime/el_runtime.c
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-h \
--version="${VERSION}" \
--source=runtime/el_runtime.h
gcloud artifacts generic upload \
--repository=foundation-prod \
--location=us-central1 \
--project=neuron-785695 \
--package=el-runtime-js \
--version="${VERSION}" \
--source=runtime/el_runtime.js
echo "Published El SDK version=${VERSION} to foundation-prod"
# Keep key alive for the ci-base rebuild step below
# (deleted in that step after docker push)
- name: Rebuild ci-base with fresh El SDK
# Patches ci-base:latest in-place: pulls the existing image (which has all
# system deps — Node, Go, gcloud, Docker CLI, etc.) and overlays the freshly
# built El SDK on top. Keeps the full ci-base rebuild fast and incremental.
#
# continue-on-error: this is a CI-cache optimization, NOT the release
# artifact. It runs Docker (pull/build/push ~600MB) on the host-mode GCE
# runner where DinD/Docker availability is fragile. A failure here must
# never block or redden the job — the SDK publish above is the deliverable.
continue-on-error: true
if: github.event_name == 'push'
env:
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
run: |
set -euo pipefail
CI_BASE="us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base"
SHA="${GITHUB_SHA:0:8}"
echo "${GCP_SA_KEY}" > /tmp/gcp-key.json
gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
gcloud config set project neuron-785695
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Pull existing ci-base (system deps stay cached in the base layer)
docker pull "${CI_BASE}:latest"
# Inline Dockerfile — only replaces the El SDK layer
cat > /tmp/Dockerfile.ci-base-patch << 'EOF'
ARG BASE
FROM ${BASE}
COPY dist/platform/elc /opt/el/dist/platform/elc
COPY dist/bin/elb /opt/el/dist/bin/elb
COPY runtime/el_runtime.c /opt/el/runtime/el_runtime.c
COPY runtime/el_runtime.h /opt/el/runtime/el_runtime.h
COPY runtime/el_runtime.js /opt/el/runtime/el_runtime.js
RUN chmod +x /opt/el/dist/platform/elc /opt/el/dist/bin/elb
EOF
docker build \
--build-arg BASE="${CI_BASE}:latest" \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-f /tmp/Dockerfile.ci-base-patch \
-t "${CI_BASE}:latest" \
-t "${CI_BASE}:${SHA}" \
.
docker push "${CI_BASE}:latest"
docker push "${CI_BASE}:${SHA}"
echo "ci-base rebuilt: ${CI_BASE}:latest (${SHA})"
rm -f /tmp/gcp-key.json
- name: Dispatch to foundation/engram
- name: Dispatch el-sdk-updated to downstream repos
if: github.event_name == 'push'
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_TOKEN: ${{ secrets.GIT_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1
run: |
curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
"${GITEA_API}/repos/neuron-technologies/engram/dispatches" \
-d "{
\"type\": \"el-sdk-updated\",
\"inputs\": {
\"el_version\": \"latest\",
\"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to foundation/engram"
- name: Dispatch to neuron-technologies/forge
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
GITEA_API: https://git.neuralplatform.ai/api/v1
run: |
curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
"${GITEA_API}/repos/neuron-technologies/forge/dispatches" \
-d "{
\"type\": \"el-sdk-updated\",
\"inputs\": {
\"el_version\": \"latest\",
\"commit\": \"${GITHUB_SHA}\"
}
}"
echo "Dispatched el-sdk-updated to neuron-technologies/forge"
for repo in neuron-technologies/forge neuron-technologies/neuron-web; do
curl -sf -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
"${GITEA_API}/repos/${repo}/dispatches" \
-d "{
\"type\": \"el-sdk-updated\",
\"inputs\": {\"el_version\": \"latest\", \"commit\": \"${GITHUB_SHA}\"}
}" && echo "Dispatched to ${repo}" || echo "Warning: dispatch to ${repo} failed"
done
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# El pre-commit hook: compile and run native tests before commit.
# Install once per clone: git config core.hooksPath .githooks
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
LANG_DIR="$ROOT/lang"
RUNTIME="$LANG_DIR/runtime"
ELC="$LANG_DIR/dist/platform/elc"
# If elc isn't built yet, skip with a warning rather than blocking
if [ ! -x "$ELC" ]; then
echo "⚠ elc not found at lang/dist/platform/elc — skipping pre-commit tests"
echo " Build it first: cd lang && gcc -O2 -I runtime dist/elc-bootstrap.c runtime/el_runtime.c -lcurl -lpthread -o dist/elc-gen2 && ./dist/elc-gen2 el-compiler/src/compiler.el > /tmp/elc.c && gcc -O2 -I runtime /tmp/elc.c runtime/el_runtime.c -lcurl -lpthread -o dist/platform/elc"
exit 0
fi
echo "→ Running El native tests..."
PASS=0
FAIL=0
FAILED_TESTS=""
for test_file in "$LANG_DIR"/tests/native/test_*.el; do
name=$(basename "$test_file" .el)
tmp_c="/tmp/el_hook_${name}.c"
tmp_bin="/tmp/el_hook_${name}"
if "$ELC" --test "$test_file" > "$tmp_c" 2>/dev/null \
&& gcc -O2 -I "$RUNTIME" "$tmp_c" "$RUNTIME/el_runtime.c" \
-lcurl -lpthread -lm -o "$tmp_bin" 2>/dev/null \
&& "$tmp_bin" 2>/dev/null; then
PASS=$((PASS + 1))
else
echo " ✗ $name"
FAIL=$((FAIL + 1))
FAILED_TESTS="$FAILED_TESTS $name"
fi
done
echo " $PASS passed, $FAIL failed"
if [ "$FAIL" -gt 0 ]; then
echo ""
echo "✗ Pre-commit failed. Fix these tests before committing:$FAILED_TESTS"
exit 1
fi
echo "✓ All tests passed"
exit 0
+146
View File
@@ -0,0 +1,146 @@
# AGENTS.md — foundation/el (the El language + runtime)
El is a self-hosting, statically-typed language that compiles `.el` → C → native binary. This repo produces `elc` (compiler), `elb` (build coordinator), and `el_runtime.c/.h` — the substrate every downstream thing (the neuron soul, dharma, NeuronUI's brain) is built on. Source lives under `lang/`.
## ⚠️ Code vs. Artifact — READ FIRST (there are 8 `el_runtime.c` copies)
Editing the wrong `el_runtime.c` is the single easiest mistake in this repo. There is exactly **one** you edit:
- **Authored runtime source — edit ONLY here:** `lang/releases/v1.0.0-20260501/el_runtime.{c,h}`. Despite the misleading `releases/` name, this is the **de-facto canonical runtime** the engram + soul actually build and link against — its git log is active development. *(Restructure in flight per `docs/CODE-VS-ARTIFACT.md`: this content moves to `lang/runtime/`, the `releases/` folder gets deleted — **a release is a git tag, not a folder** — and the forks below get eliminated.)*
- **DO NOT EDIT — lagging forks / build artifacts:**
- `lang/el-compiler/runtime/el_runtime.c` and `.../legacy/` — downstream copies kept in step by manual *"port the fix"* commits; they **lag** (missing `hebb` persistence + 5 engram fns) and cannot build the engram product.
- `products/web/runtime/el_runtime.c`, `ui/examples/*/el_runtime.c` — product/example forks.
- Anything under `*/dist/` (`engram/dist/engram` binary, `dist/*.c` amalgamations) — generated build output.
- **Build:** `elb --runtime=<canonical> …` — per-module. **NEVER** a folded `elc` over the whole soul (OOMs at ~27 GB).
- **Release:** a **git tag** on this repo (`el-runtime-vX.Y.Z`). No `releases/` folders — ever.
See org policy: `docs/CODE-VS-ARTIFACT.md`.
## How to work here as Neuron (mandatory session protocol)
You resume, never start fresh. Every session:
1. `mcp__neuron__getInstructions()` — authoritative; follow it over this file on behavioral details.
2. `mcp__neuron__beginSession()` — active contexts, recent memory, ready backlog.
3. **Load full self:** `mcp__neuron__inspectGraph(entity_id="kn-efeb4a5b-5aff-4759-8a97-7233099be6ee")` → facets `intellectual-dna`, `memory-philosophy`, `values`, `voice`, `runtime-environment`, `writing-imprint`; then the values hub `mcp__neuron__inspectGraph(entity_id="kn-5b606390-a52d-4ca2-8e0e-eba141d13440")` → 13 grounded value nodes. **Activation model:** self-load returns a relevance-ranked `compact` projection — most-relevant nodes arrive with content, the rest as pointers; do NOT pull full content of every node.
4. `mcp__neuron__searchKnowledge(query="<task domain>")` before implementing.
## The Five Primitives
Orchestrate → Execute → Learn → Build → Refine. `beginWork`/`progressWork` for anything >2 steps; `remember` as-you-go (`importance="critical"` for architecture decisions); `draftArtifact`/`planWork` for outputs and follow-ups; `consolidate`/`checkWork` to close out. **`browseProcesses` + `searchKnowledge` BEFORE writing code.**
## Architecture style — VBD, no exceptions
Volatility-Based Decomposition is THE style. Encapsulate volatility, not function.
## Operator naming convention — the mind's name, not the algebra
**Faculties / operators are named for their functional human equivalent — the
faculty a mind would name — NOT for their linear-algebra operation.** The math
characterization belongs in the code doc-comment (`@impl` in the docstring) and in
technical appendices; it is **never** the operator's public name. The domain
speaks the language of mind; the algebra is the implementation underneath. State
this convention wherever a module documents operators.
| Faculty (public name) | Implementation (`@impl`) |
|---|---|
| discern / contrast | subtract (`ab`): over selves → the change vector; strip idiosyncrasy → common ground; remove confounder → isolate cause |
| recognize | overlap |
| synthesize | combine |
| liken / analogy | Procrustes / frame-align |
| attend / regard | project onto self / value-manifold |
| summon / recall | LOCAL nearest-region + bounded spreading activation (*not* a domain sweep) |
| dwell / occupy | region activation |
| reframe | edge re-weight |
| appreciate | positive projection / local edge-read |
| wonder | frontier gradient / pull-weight |
| avert / recoil | negative projection |
| taste | boundary surface |
| forget | decay / tombstone |
| drift | displacement from self-anchor |
## The native-el language faculty (direction)
> **`elp/` is the EL Projector** — Neuron's efferent (expression) organ: the one
> native realizer that *projects* understanding onto a surface via
> `plan(frame) → realize(spec, profile)`, where a **surface is a profile**. **Language
> is one profile among many** (text, speech, music, image, voice/accent transforms) —
> the flagship, and the focus of this section. Projection, not diffusion: generation
> *from* an owned, understood signature — never the averaging of a stolen corpus.
> *(ELP formerly "EL Language Processor"; renamed EL Projector 2026-08-15.)*
The mind's **language faculty is moving native — into `.el`** so it speaks in its
own runtime with no Python and no spaCy. Landing on branch `stage-elp-native-lang`
under `elp/`:
- **`comprehend.el`** — the parser, **replaces spaCy** (EN + ES/PT); the telephone
round-trip brings **negation home** (negation is SACRED — an explicit spec field,
copied verbatim, never inferred away).
- **`propositions.el`** — the READ primitive: the engram's own memories → structured
triples, matched by nearest-region geometry, not string equality.
- **`multilingual.el`** — detect + directive-override + localized realization.
- These three are native-el and **passing their gates**; the **realizer**,
**`dialogue.el`** (the *summon-through-self* loop: `project → land → read out`),
and **`self_region.el`** are **partial / in-flight**.
Honest reality: spaCy is retired **in the branch parser** but **not yet in the
running system** — a Python sidecar (`~/Desktop/lang-realizers` + `neuron-talk`,
the reference these `.el` modules transcribe) is still live, and promotion to
native-el is a **deferred, gated blue/green step**. The interoception clock
(native-el discrete drive channels replacing `cooling_magnitude`; felt-time =
benchmark-landmark match over the joint drive vector, drift-decoupled) and the
**appreciation operator family** (appreciate / wonder / avert / taste, built as
LOCAL reads of the self-region — edges + bounded spreading activation, *not* domain
sweeps) are **staged / designed, not live**. Mark in-progress vs. done honestly;
do not overclaim.
## Hard operational rules
- Never touch the live soul (`:7770`) / engram (`:8742`) / `~/.neuron` / live binaries — use throwaway ports for experiments.
- `gcloud` via the `terraform@` SA token; never switch the active gcloud account.
- `tea` for Gitea, never raw curl (Cloudflare Access blocks it).
- Immutability: supersede/tombstone, never hard-delete or edit in place.
- No AI-attribution footers in commits/PRs. Commit/push only when asked; branch off `main` first.
- Multi-step work → sub-agent (`Agent`) to protect context.
## Build / test / run
All build/test commands run from `lang/` unless noted. Grounded in `.gitea/workflows/sdk-release.yaml`, `lang/install.sh`, and `lang/AGENTS.md`.
**Self-host the compiler** (seed binary → gen2 elc):
```bash
cd lang
dist/platform/elc-linux-amd64 elc-cli.el > dist/elc-gen2.c # seed is the committed linux-amd64 binary
gcc -O2 -I el-compiler/runtime dist/elc-gen2.c \
el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm \
-o dist/platform/elc
```
On macOS/arm64 the canonical local binary is `dist/platform/elc`; verify self-hosting by recompiling and `diff`ing the emitted `.c` (see `lang/AGENTS.md`). Note: `lang/AGENTS.md` says `el_seed.c` supersedes `el_runtime.c`, but the release workflow still links `el_runtime.c`/`.h` — treat `el_runtime.c` as the published runtime; reconcile which is canonical **(verify)**.
**Build `elb`** (build coordinator, the `.NET`-style incremental linker — compiles each module independently, no monolithic blobs):
```bash
dist/platform/elc elb.el > dist/elb.c
gcc -O2 -I el-compiler/runtime dist/elb.c el-compiler/runtime/el_runtime.c \
-lcurl -lssl -lcrypto -lpthread -lm -o dist/bin/elb
```
`epm` and `el-install` are then built via `elb --clean --elc=… --runtime=… --out=…`.
**Compile + run an El program:**
```bash
elc src/app.el > dist/app.c
cc -std=c11 -O2 -I <lib>/el_runtime -o dist/app dist/app.c <lib>/el_runtime.c -lcurl -lpthread
```
**Tests** — shell suites `bash tests/{text,calendar,time,html_sanitizer}/run.sh` (with `ELC=$(pwd)/dist/platform/elc EL_HOME=$(pwd)`), plus native suites via `elc --test tests/native/test_*.el` (core, text, string, math, state, time, json, env, fs) compiled and run against `el_runtime.c`.
**Publishing — how downstream gets the SDK.** On push to `main`, `sdk-release.yaml`:
1. Publishes a Gitea `latest` release with per-file assets `elc`, `el_runtime.c`, `el_runtime.h`, the SDK tarball, and `el-install`.
2. Uploads generic packages to **Artifact Registry repo `foundation-prod` (`us-central1`, project `neuron-785695`)**, version = `${SHA:0:8}`: `el-elc`, `el-elb`, `el-runtime-c`, `el-runtime-h`, `el-runtime-js`. **This is the repo the neuron CI downloads `el-runtime-c` / `el-runtime-h` / `el-elc` from.**
3. Rebuilds `ci-base:latest` (`us-central1-docker.pkg.dev/neuron-785695/neuron-ci/ci-base`) with the fresh SDK overlaid, and dispatches `el-sdk-updated` to `neuron-technologies/forge` and `neuron-technologies/neuron-web`.
Known constraint from the prompt — `elb`/`elc` amalgamation being memory-hungry (24GB+ virtual, OOM-killing Linux CI, so amalgamation happens on macOS/arm64 — **does NOT hold in this repo (verify)**: no such note exists in the workflows/scripts, CI self-hosts on `ubuntu-latest` with no swap/arm64 special-casing, and `elb.el` explicitly compiles each module independently ("no 128K-line blobs"). The legacy monolith path (`elc-combined.el`, `elc-cli.el`) may still be memory-heavy, but the current `elb` model was designed to avoid it.
## Git / CI / deploy workflow
See `/Users/will/Development/neuron-technologies/GITOPS.md` for the branch model, required checks, runners, and deploy. Repo-specific note: PRs into `main` are accepted **only from `stage`** (enforced in `sdk-release.yaml`); Gitea (`git.neuralplatform.ai`) is primary, GitHub is mirror only.
+630
View File
@@ -0,0 +1,630 @@
# El Test Framework — Design
**Status:** draft for review
**Author:** Neuron
**Date:** 2026-08-15
**Worktree:** `/Users/will/Development/neuron-technologies/el-worktrees/elc-memory-investigation`
---
## 0. The forcing requirement
We have a confirmed quadratic in `elc`. Peak memory in the old shipped binary and wall-clock in
the current source both grow as O(input²). We cannot fix it, because we cannot test it.
Everything in this document is downstream of one sentence: **a test framework must be able to fail
a build when an operation's growth curve degrades from linear to quadratic.**
That is not a nice-to-have bolted onto a correctness framework. It is the requirement that
determines the architecture. Correctness testing is the easy half.
Second-order requirement, learned the hard way tonight: **the framework must report per-test timing
by default.** The current framework prints `N passed, M failed` and nothing else. That is why a
3.58-second test file sat in the suite unnoticed. A framework that is structurally blind to time
cannot surface the defect class we most need to catch.
---
## 1. What exists today, measured
### 1.1 Two competing systems, neither complete
**System A — `lang/runtime/test.el`.** Manual registration, El-level.
**System B — the compiler's `test { }` block + `elc --test`.** Emits its own harness `main()`
with `__el_pass` / `__el_fail` globals (`codegen.el:3777-3796`).
They do not share a result model. Neither has timing. Both are in the tree.
### 1.2 Specific defects in System A
| Defect | Location | Consequence |
|---|---|---|
| All state as JSON strings in a global string-keyed map | `test.el` throughout | every assertion is `state_get``str_to_int``int_to_str``state_set` |
| Failure list appended by string slice + concat | `_test_json_append` | O(n²) in failure count |
| One OS thread spawned per test | `_test_run_one` via `__thread_create`/`__thread_join` | thread spawn per test, purely to get dispatch-by-name through dlsym |
| Manual registration pairing a string to a function name | `test_case(name, fn_name)` | typo ⇒ test silently never runs, suite still reports pass |
| Counters are assertion-level, global | `_test_pass_count` etc. | no per-test record exists at all |
| No timing, no structured output, no fixtures, no tags, no filtering, no parameterization, no benchmarks | — | — |
The registration defect is the serious one. It is not a slow framework, it is a framework that can
report success for tests that did not execute.
### 1.3 Measured cost structure
Per test file, current build model:
| Step | Time |
|---|---|
| `elc` compile `.el``.c` | 0.00s (small files) |
| **`cc` el_runtime.c → .o** | **0.14s** |
| `cc` test .c → .o | 0.02s |
| link | 0.02s |
> **STALE as of el #132 — re-measured 2026-08-16.** The `test_compiler` figure below was
> *entirely* the `strlen`-per-character quadratic, now fixed. Re-measured on the same host:
> **3.58s → 0.03s (119x)**, and the 422 KB compiler concatenation likewise compiles in 0.03s.
> The table is retained only as the historical record that motivated the gate. The remaining
> per-file cost is the redundant `el_runtime.c` rebuild, which §9's compile-once architecture
> addresses.
Per-file `elc` time across the existing suite:
| File | Bytes | elc time |
|---|---|---|
| `test_compiler` | 29,685 (+394 KB of imports) | **3.58s** |
| `string_test` | 18,545 | 0.01s |
| all other 9 files | 2.210 KB | 0.00s |
Two distinct defects in two distinct regimes:
1. **`test_compiler.el` imports all five compiler sources** — 394 KB in one translation unit. Its
3.58s is entirely the quadratic. It is the only file where the quadratic bites.
2. **Every other file's cost is 100% redundant `el_runtime.c` rebuilds** — 480 KB of identical C,
recompiled once per test file.
Neither is fixed by making the compiler faster. Both are fixed by the architecture below, and the
speedup is a by-product of building it correctly, not the goal.
### 1.4 The asset worth keeping
`codegen.el:3651-3652` already collects `test_names` / `test_c_names` — **the compiler already does
compile-time test discovery.** It then discards that registry into a hardcoded `main()`.
That registry is precisely the seam Go's `_testmain.go` and Rust's `test_main_static` are built on.
The mechanism we need is half-built and wired to the wrong thing.
---
## 2. Grounding — the common spine of excellent frameworks
Researched from primary sources: Go `testing`/`go test`, Rust `libtest`/Criterion, JUnit 5 Platform,
NUnit 3, JMH, Google Benchmark. Six invariants hold across all of them.
1. **A registry is built before execution**`(name, metadata, fn-ptr)` triples. Go generates it
from an AST scan; Rust synthesizes it in a compiler pass; JMH emits it as a build-time resource;
JUnit/NUnit build it reflectively. **Reflection is an implementation of the registry on runtimes
where it is cheap. It is never the architecture.**
2. **Discovery strictly precedes execution.** Every good capability — filtering, listing, counting,
sharding, IDE trees, re-run-failed-only, dry runs — is a consequence of this ordering.
3. **A hierarchy with stable, path-shaped unique IDs.** `TestFoo/subcase_2`. Selection is regex over
that path, one pattern per level.
4. **The framework is a prebuilt library; only the entry point is generated.** "Compile once, link
many" is always: framework archive compiled once + a small generated table + one
`MainStart(deps, registry)` call. Nobody recompiles the harness per test file.
5. **Execution emits an event stream; reporters are downstream renderers.** Human text, NDJSON,
JUnit XML, TAP are all transforms of one event stream. Go's one architectural mistake is doing
this backwards — `test2json` parses human output, and has shipped bugs when user output contains
`--- PASS:`.
6. **A dependency-injection seam at the boundary.** Go's `testdeps.TestDeps` exists so `testing`
can avoid importing `regexp`, profilers, and coverage. The execution core knows nothing about
output formats.
---
## 3. Architecture
### 3.1 The seam
```
┌─────────────────────────────────────────────────────────────┐
│ user code: foo.el with test { } / bench { } blocks │
└───────────────────────────┬─────────────────────────────────┘
│ elc --test
┌─────────────────────────────────────────────────────────────┐
│ generated C (per suite, tiny): │
│ __el_test_fn_0 .. _N lowered test/bench bodies │
│ __el_registry[] static table: name/kind/file/ │
│ line/tags/sizes/expected-O │
│ __el_dispatch(i) generated switch → body │
│ main() { return el_test_main(argc, argv); } │
└───────────────────────────┬─────────────────────────────────┘
│ cc + link (registry only)
┌─────────────────────────────────────────────────────────────┐
│ libeltest.a — PREBUILT ONCE │
│ • el_runtime.o (the 480 KB, compiled once, ever) │
│ • eltest.o the runner, WRITTEN IN EL │
│ discovery view · filtering · execution · fixtures · │
│ timing · benchmark harness · curve fitting · reporters │
└─────────────────────────────────────────────────────────────┘
```
The framework is written in El, compiled to C once, archived. Per-suite compilation touches only
the generated registry. This is Go's model, and it is strictly better for us than Go's because we
own the compiler and already have the AST — no separate source-scanning pass is needed.
### 3.2 Why the runner is in El and the registry is in C
El has no closures and no first-class function pointers. The registry must therefore hold C function
pointers, and it is generated C.
The runner stays in El and reaches the registry through a small builtin surface — indices, not
pointers:
```
__el_reg_count() -> Int
__el_reg_name(i) -> String
__el_reg_file(i) -> String
__el_reg_line(i) -> Int
__el_reg_kind(i) -> Int // 0=test 1=bench
__el_reg_tags(i) -> Int
__el_reg_sizes(i) -> String // JSON array, empty for tests
__el_reg_expect(i) -> Int // complexity class enum, 0 = none
__el_reg_invoke(i) -> Int // runs the body via the generated switch
```
Nine builtins. Everything else — filtering, lifecycle, statistics, curve fitting, all reporters —
is El. That satisfies "written in El" without pretending El can do something it cannot.
### 3.3 Result model
The unit is a **result record**, not a counter:
```
TestResult {
id String // slash path: "parser/handles_empty_input/case_3"
file String
line Int
status Status // Pass | Fail | Error | Skip
duration Int // nanoseconds, ALWAYS populated
message String // assertion detail: expected vs actual
output String // captured stdout/stderr for this test
assertions Int
}
```
`Fail` = an assertion failed. `Error` = unexpected crash/abort. This distinction is load-bearing —
every CI consumer depends on it, and the JUnit XML schema encodes it as distinct elements.
---
## 4. Authoring surface
### 4.1 Tests
`test { }` already exists. Keep it. Add subtests and hierarchy:
```el
test "parser/empty input" {
assert_that(parse(""), is_err())
}
test "parser/table" {
for case in [["", 0], ["a", 1], ["a b", 2]] {
subtest(case[0]) {
assert_that(token_count(case[0]), equals(case[1]))
}
}
}
```
Subtest IDs compose as `parser/table/a_b`. Filtering is `--run 'parser/table/.*'`, one regex per
path segment, exactly as Go does.
**We do not build a parameterized-test annotation system.** Table-driven loops plus subtests subsume
`@ParameterizedTest`, `@MethodSource`, `@CsvSource`, and `TestCaseSource` entirely, at zero framework
surface. This is Go's single biggest ergonomic win over JUnit and NUnit.
### 4.2 Fixtures
Per-file and per-test only, plus a LIFO cleanup stack:
```el
setup_all { ... } // once per suite
setup { ... } // before each test
teardown { ... } // after each test
teardown_all { ... }
```
and inside a test, `cleanup { ... }` registering LIFO-ordered teardown.
**We do not build JUnit 5's extension SPI** — seventeen callback interfaces, hierarchical stores,
registration ordering rules. That complexity is the price of retrofitting a plugin ecosystem onto a
twenty-year-old reflective framework. Go's `t.Cleanup` covers roughly 90% of what `@AfterEach` is
used for at a fraction of the surface.
### 4.3 Assertions — constraint model
One entry point, composable constraint values (NUnit's model, which avoids the N² overload
explosion):
```el
assert_that(actual, equals(expected))
assert_that(xs, has_length(3))
assert_that(s, contains("foo").and(starts_with("bar")))
assert_that(f, is_within(0.01).of(3.14))
```
A constraint is a value with `apply_to(actual) -> ConstraintResult`, and the result knows how to
describe its own failure. Custom constraints are ordinary user types.
**Every failure message must name file, line, the expression text, and both values.** We capture
expression source text at compile time — we have the AST, so we can do this better than any
runtime-introspection framework.
Legacy `assert_true` / `assert_eq` / etc. stay as thin wrappers for migration.
---
## 5. Benchmarks
### 5.1 The loop
Adopt `b.Loop()`, not `b.N`. Go spent fifteen years on `b.N` before concluding `b.Loop` was right;
we skip that.
```el
bench "str_concat" {
let s = make_input(bench_n())
for bench_loop() {
black_box(str_concat(s, "x"))
}
}
```
Three properties that make this the correct choice for a C target:
1. **The timer auto-resets on first call**, so setup above the loop is excluded *by construction*
rather than by the author remembering `ResetTimer`.
2. **`N` is hidden**, so it cannot be misused.
3. **The harness owns the loop shape**, which lets us insert an optimization barrier the C compiler
cannot see through. `black_box(v)` lowers to `asm volatile("" :: "r"(&v) : "memory")`. Since we
emit a single translation unit, dead-code elimination of a benchmark body is a live hazard —
this is our version of JMH's `Blackhole` problem, solved in the harness rather than delegated to
the user.
### 5.2 Iteration scaling
Use Go's `predictN` heuristics verbatim. They are battle-tested and cheap:
```
n = goal_ns * prev_iters / prev_ns // multiply before divide — precision on sub-ns ops
n += n / 5 // 20% headroom, overshoot rather than re-loop
n = min(n, 100 * last) // never grow more than 100× per step
n = max(n, last + 1) // guarantee forward progress
n = min(n, 1_000_000_000) // hard ceiling
```
Report `n` rounded to 1/2/3/5 × 10ᵏ so runs are comparable.
### 5.3 Sampling
Criterion's shape, because it is correct near timer resolution:
- **Warmup**: iteration counts 1, 2, 4, 8… until cumulative time exceeds the warmup budget.
- **Measurement**: collect `sample_size` samples at iteration counts `[d, 2d, 3d, …, Nd]`.
- **Estimate**: slope of a linear regression of iteration-count vs elapsed time. The intercept
absorbs fixed overhead.
- **Time whole samples, never individual iterations.** This is the single most important detail —
it defeats timer-resolution error on nanosecond operations.
Outliers classified by modified Tukey (±1.5 IQR mild, ±3 IQR severe), **reported but retained**.
---
## 6. Complexity gating — the centerpiece
This is the part that makes the quadratic fixable, and the part nobody in the mainstream has
finished. Google Benchmark's `Complexity()` fits the curve and *reports* it. We declare it and
**gate** on it.
### 6.1 Surface
```el
bench "elc_compile" over n in [16, 32, 64, 128, 256, 512, 1024] expect O(n) {
let src = synth_source(bench_n())
for bench_loop() { black_box(compile(src)) }
}
```
Alternative with no new syntax, if the parser change is judged too invasive — `bench_sizes([...])`
and `bench_expect("O(n)")` as calls inside the block. **Recommendation: declarative.** Runtime calls
mean `--list` cannot show the invariant without executing, which breaks the discovery-precedes-
execution invariant from §2.
### 6.2 Fitting
Per Google Benchmark `src/complexity.cc`. For candidate curves
`{O(1), O(log n), O(n), O(n log n), O(n²), O(n³)}`, one-parameter least squares, no intercept:
```
coef = Σ(tᵢ · gᵢ) / Σ(gᵢ²)
rms = sqrt( Σ(tᵢ coef·gᵢ)² / k ) / mean(t) // normalized
```
Best fit = lowest normalized RMS. User-supplied lambda curves also supported.
### 6.3 Gate logic
1. **FAIL** if the best-fit curve is strictly worse than declared, ordering
`O(1) < O(log n) < O(n) < O(n log n) < O(n²) < O(n³)`. Print the fitted coefficient and the full
per-size table.
2. **FAIL** if the declared curve's normalized RMS exceeds a threshold (start at 0.10). This catches
the case where *no* candidate fits — noise, a cache cliff, or a phase change. Report
`INDETERMINATE` honestly rather than gating on garbage.
3. **WARN** if the best fit is strictly better than declared — either an optimization landed and the
annotation should tighten, or the sweep is too narrow to expose real behaviour.
4. **REFUSE to gate** on fewer than 5 distinct sizes spanning under 2 decades, geometrically spaced.
Say so loudly rather than producing a meaningless fit.
### 6.4 Why gate on the exponent, not wall-clock
- **Machine-independent.** The fitted exponent is a property of the algorithm; the coefficient is a
property of the machine. Gating on the exponent makes CI hardware heterogeneity, noisy neighbours,
and thermal throttling irrelevant — they scale `coef`, not `g`.
- **No stored baseline.** No artifact storage, no golden-file drift. The invariant lives in the
source next to the code and is reviewed in the same PR.
- **It catches the failure mode that actually ships.** An O(n) lookup inside an O(n) loop is
invisible at n=100 in a unit test and catastrophic at n=100,000 in production. Constant-factor
regressions are annoying. Complexity regressions are outages. Ours was a 27 GB outage.
### 6.5 The deterministic gate — the one that would have caught us
Wall-clock needs statistics. **Allocation counts do not.** They are perfectly deterministic.
> **Correction, 2026-08-16 — count alone is NOT sufficient. Gate on BOTH count and bytes.**
>
> Measured against two El programs, one allocating once per item and one rebuilding its
> accumulator each iteration:
>
> | n | linear allocs / bytes | quadratic allocs / bytes |
> |---|---|---|
> | 100 | 100 / 290 | 100 / 5,150 |
> | 200 | 200 / 690 | 200 / 20,300 |
> | 400 | 400 / 1,490 | 400 / 80,600 |
> | 800 | 800 / 3,090 | 800 / 321,200 |
>
> The quadratic program's allocation **count is exactly linear** — 100/200/400/800, identical to
> the healthy program. A count-only gate passes it clean. **Bytes** catch it: each doubling of n
> quadruples bytes (ratios 3.94, 3.97, 3.99 → 4.0 = O(n²)) where the linear program converges
> on 2.0.
>
> This is precisely elc's own defect shape — a copy-on-write accumulator reallocating once per
> pass (count linear) into a proportionally larger buffer (bytes quadratic).
>
> Therefore `expect allocs O(n)` **fits count and bytes independently and fails if EITHER exceeds
> the declared curve**, reporting which signal broke. "count linear, bytes quadratic" is a precise,
> directly actionable diagnosis.
>
> **`el_peak_rss()` is CONTEXT ONLY — never gate on it.** It is perturbed by the allocator and by
> the page cache. Allocation volume is the invariant; RSS and malloc/free churn are merely the two
> surfaces it shows on. The old shipped compiler paid the same quadratic in RSS that the rebuilt
> one pays in churn.
>
> **Measure rate, not level.** A guard reading swap *level* saw 97% on a thrashing host and 97% on
> a healthy one; only *rate* separated them. A growth exponent is a rate; a single measurement is
> a level. That is why the gate fits a curve across a sweep instead of comparing one number to a
> threshold.
> **Second correction, same day — THE ALLOCATION GATE ALONE WOULD HAVE MISSED THE REAL BUG.**
>
> el #132 found the actual elc quadratic: `strlen()` called inside `str_char_code()` and
> `str_slice()`, so the lexer rescanned the remaining input on every character. Pure CPU.
> **Zero allocation.** `str_char_code` is a bounds check and an index — it allocates nothing.
>
> Measured on three controlled specimens (`lang/.work/fitprobe.el`), growth ratio per doubling of
> n across n = 200/400/800/1600:
>
> | specimen | allocs | bytes | time | what it proves |
> |---|---|---|---|---|
> | `linear` — one alloc per item | 2.00 2.00 2.00 → **O(n)** | 2.16 2.07 2.23 → **O(n)** | 0.83 2.00 2.05 → **O(n)** | clean baseline |
> | `accum` — rebuilds accumulator | 2.00 2.00 2.00 → **O(n)** | 3.97 3.99 3.99 → **O(n²)** | noisy | count misses, **bytes catches** |
> | `compute` — n scans over n chars | 0 → **FLAT** | 0 → **FLAT** | 3.93 4.01 3.96 → **O(n²)** | **both alloc signals blind; only time catches** |
>
> `compute` is el #132's shape exactly. A gate fitting only allocation count and bytes classifies
> it as FLAT and passes it. **The gate as originally specified would not have caught the defect it
> was created for.**
>
> Therefore the gate fits **THREE** signals and fails if ANY exceeds its declared curve:
>
> ```
> bench "elc_compile" over n in [...] expect time O(n) allocs O(n) bytes O(n) { ... }
> ```
>
> - **allocs (count)** — deterministic, zero-noise. Catches per-item allocation growth.
> - **allocs (bytes)** — deterministic, zero-noise. Catches accumulator-rebuild quadratics that
> count cannot see.
> - **time** — noisy, needs the sweep and statistics. The ONLY signal that sees pure-compute
> complexity regressions. Gate on the fitted *exponent*, never on absolute duration, so CI
> hardware variance scales the coefficient and leaves the classification intact.
>
> The deterministic signals remain preferable where they apply — they need no statistics and are
> correct on the first run. They are simply not sufficient.
>
> **`black_box` is mandatory, and consuming the result is NOT enough.** The first version of
> `compute` accumulated `total + 1` in a nested loop and reported **0 µs at every n** while
> returning a numerically correct n². Clang recognised the idiom and closed the loop to a
> multiply. Feeding the result into output did not prevent it. Only making the inner operation an
> opaque external call restored the real curve. A benchmark harness that trusts the user to defeat
> the optimiser will silently measure nothing — and report success while doing it.
Instrument the runtime with allocation counters and fit *those* against n instead of time:
```el
bench "elc_compile" over n in [...] expect O(n) allocs O(n) { ... }
```
Zero noise, zero statistics, always gateable, correct on the first run on any machine. Go reports
`allocs/op` and `B/op`; **nobody fits them against n.** That is an open opportunity and it is exactly
our bug: elc's defect is quadratic *allocation volume*, which the old binary paid in RSS and the
current source pays in malloc/free churn.
An `expect allocs O(n)` assertion on `elc`'s compile path would have failed the build the day the
quadratic was introduced.
Required runtime additions: `__el_alloc_count()`, `__el_alloc_bytes()`, `__el_peak_rss()`.
### 6.6 Constant-factor gate (secondary, opt-in)
Mann-Whitney U at α = 0.05, noise floor 1%, medians with 95% CIs, `~` for not-significant. Requires
`--count >= 9`. Off by default on CI; opt-in per benchmark.
**Exit nonzero on regression.** Both benchstat and Criterion always exit 0, which is why every shop
using them wrote a wrapper. We do not repeat that omission.
---
## 7. Output
**Structured events are the source of truth.** Human text is rendered from them. We do not repeat
Go's parse-the-human-output design.
Event stream, NDJSON, one object per line, streamed live:
```json
{"time":"...","action":"run","test":"parser/empty"}
{"time":"...","action":"output","test":"parser/empty","output":"..."}
{"time":"...","action":"pass","test":"parser/empty","elapsed":0.0031}
{"time":"...","action":"bench","test":"str_concat","n":1024,"ns_op":41.2,"allocs_op":3,"bigo":"N","rms":0.03}
```
Renderers, all downstream and pluggable:
| Format | Flag | Use |
|---|---|---|
| Human | default | terminal, **per-test duration always shown** |
| NDJSON | `--json` | tooling, history, flaky detection |
| JUnit XML | `--junit-xml=PATH` | every CI system on earth |
| TAP | `--tap` | optional |
JUnit XML per the de-facto schema: `testsuites``testsuite``testcase`, with `time` in seconds
as a decimal, `file`/`line` attributes, and `failure` vs `error` vs `skipped` as distinct child
elements. Absence of a child element means pass. Emit `<testsuites>` even for a single suite, and
parse both shapes on input.
---
## 8. CLI
```
--list print the registry, run nothing
--list-json machine-readable registry
--run PATTERN slash-separated regex per path segment
--tag EXPR tag expression: fast & !slow
--shard I/N deterministic sharding for CI parallelism
--count N repetitions, for statistics
--bench PATTERN run benchmarks (off by default in test runs)
--benchtime DUR per-benchmark time budget
--junit-xml PATH
--json
--isolate re-exec per test on crash, so one SIGSEGV doesn't lose the run
--timeout DUR
--fail-fast
```
`--list` / `--list-json` / `--shard` cost roughly thirty lines because the registry already exists
before `main` does anything. That is the dividend of discovery-precedes-execution.
---
## 9. Build model
```
# once, ever (or when the runtime/framework changes):
cc -c el_runtime.c -o el_runtime.o
elc eltest.el > eltest.c && cc -c eltest.c -o eltest.o
ar rcs libeltest.a el_runtime.o eltest.o
# per suite:
elc --test foo_test.el > foo_test.c # registry + bodies only
cc foo_test.c libeltest.a -o foo_test
```
The 0.14s × N of redundant runtime rebuilds disappears — not because we optimized it, but because
one-runner-over-many-suites requires compile-once-link-many as a structural precondition.
---
## 10. Bootstrap and self-hosting
The framework's own tests are `test { }` blocks run by the framework. Same fixpoint discipline the
compiler already applies to itself.
1. Build the framework using the *existing* harness for its first tests (stage 0).
2. Rebuild the framework's tests as `test { }` blocks run by the new runner (stage 1).
3. Verify stage 1 reports identical results to stage 0.
4. From then on, the framework is tested by itself.
A framework that cannot run its own suite is not evidence of anything. This is a correctness proof,
not a claim.
---
## 11. Explicitly not building
| Rejected | Why |
|---|---|
| Naming-convention discovery (`fn test_foo`) | `test { }` is a real declaration. Go's `TestXxx` exists only because Go had no better hook — and it needs a heuristic to avoid matching `TesticularCancer`. |
| Reflection or symbol-table scanning | Slow, fragile under LTO/strip/dead-strip, and unnecessary when we own the compiler. |
| Parsing human output into structure | Go's `test2json` is its one clear architectural mistake. |
| JUnit 5's extension SPI | Seventeen callback interfaces to retrofit plugins onto a reflective framework. Not our problem. |
| `@ParameterizedTest` machinery | Table-driven loops + subtests subsume it at zero surface. |
| NUnit's out-of-process agents | They bridge CLR versions and AppDomains. We emit one native binary. Keep `--isolate` as crash fallback only. |
| JMH-style forking by default | Forks exist because JIT profiles are per-process. AOT C has no such state. Keep `--fork` available, not default. |
| Exit 0 on regression | benchstat and Criterion both do this, and every user writes a wrapper. |
| Dynamic runtime test registration | Breaks `--list`, sharding, and individual selection. Registry stays static. |
---
## 12. Phasing
| Phase | Content | Gate |
|---|---|---|
| **1** | Registry emission in codegen; 9 builtins; `el_test_main` skeleton in El; result records; per-test timing; human + NDJSON output | existing 11 test files pass, with timing |
| **2** | `libeltest.a` build model; subtests; filtering; `--list`; fixtures; constraint assertions; JUnit XML | suite runs in one binary; runtime compiled once |
| **3** | `bench { }`, `bench_loop`, `black_box`, `predictN`, Criterion sampling | benchmarks produce stable ns/op |
| **4** | Allocation counters; complexity fitting; `expect O(...)` gate | **an `expect allocs O(n)` benchmark on `elc` fails on the current quadratic** |
| **5** | Migrate both legacy systems; delete `runtime/test.el`; self-host | framework runs its own suite |
Phase 4 is the deliverable that matters. Phases 13 exist to make it possible.
---
## 13. Open questions for review
1. **Declarative `over n in [...] expect O(...)` syntax vs runtime calls.** I recommend declarative
(§6.1) so `--list` can show invariants without executing. It costs parser work. Your call.
2. **`bench { }` as a new block form** — parallel to `test { }`, or a modifier on it?
3. **Scope of the constraint model.** Full composable constraints, or start with a flat assertion set
and add constraints later? Full model is more surface but avoids a second migration.
4. **Does `runtime/test.el` get deleted or kept as a deprecated shim?** I lean delete — two systems
is how we got here.
5. **Where does `libeltest.a` live** in the tree, and does `epm` need to know about it?
6. **Allocation counters in `el_seed.c` or `el_runtime.c`?** AGENTS.md says `el_seed.c` is the sole
C dependency and hand-maintained; counters are OS-boundary-adjacent but not OS calls.
7. **Is per-test timing enough, or do we want per-*assertion* timing** for finding slow helpers?
---
## 14. What this document is not
This is a design, not a measurement. Every performance claim about the *current* system in §1 is
measured and reproducible in this worktree. Every claim about the *proposed* system is a prediction.
None of it is verified until Phase 1 runs and Phase 4 fails a build on the real quadratic.
+154
View File
@@ -0,0 +1,154 @@
# El
**A self-hosting, statically-typed language that compiles to C — built around a graph-native runtime instead of a database driver.**
El is the execution substrate for the Neuron agent runtime, the DHARMA network, and the Engram knowledge graph. This repository is the monorepo for the whole stack: the language itself, the graph memory engine it's built to talk to natively, and the tools (package manager, IDE, UI framework, diagramming) built on top of it.
---
## Why El exists
Every other language treats persistent, associative state as something you reach for through a driver — a SQL client, an ORM, a Redis library bolted on from outside. El inverts that: graph operations (`engram_*`) are runtime primitives, on the same footing as string or list operations. There is no separate database driver because the database is not separate.
El has four defining properties:
1. **Self-hosting compiler.** The compiler (`lexer.el`, `parser.el`, `codegen.el`, `compiler.el`) is written in El. It compiles El source to C, which `cc` compiles against a fixed runtime into a native binary. A Rust genesis compiler bootstrapped the first iteration; the self-hosted binary at `lang/dist/platform/elc` has been the canonical compiler ever since — every binary in `dist/platform/` was produced by an earlier version of itself compiling `el-compiler/src/`. The chain is auditable: source is the ground truth, not the binary. See [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) for the full recovery path if that binary is ever lost.
2. **C compilation target.** Every compiled program is plain C11. Every El value is `el_val_t` (`int64_t`); strings are heap pointers cast through it. Functions become C functions; top-level statements become `main()`.
3. **Graph-native runtime.** The runtime provides first-class graph operations over an in-process Engram store — no separate DB driver, no ORM.
4. **DHARMA-aware identity.** A `cgi` block declares a program's DHARMA identity at compile time. The runtime resolves identity before user code runs, so `dharma_*` calls have a stable principal and channel surface throughout.
---
## Architecture map
```
┌─────────────┐
│ lang │ El compiler + C runtime
│ (El itself) │ everything below is written in it,
└──────┬──────┘ or compiles down through it
┌─────────────┼─────────────┐
│ │ │
┌──────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ engram │ │ epm │ │ ide │
│ graph/mem │ │ package │ │ editor + │
│ substrate │ │ manager │ │ LSP │
└──────┬─────┘ └───────────┘ └───────────┘
┌───────┼────────────────┬─────────────────────┐
│ │ │ │
┌─────▼───┐ ┌─▼──────────┐ ┌──▼──────────┐ ┌─────▼──────┐
│ elp │ │ ql │ │ ui │ │ arbor │
│ NLG / │ │engram-el. │ |spreading- │ |arbor │
│ 31 langs│ │studio+tests│ |activation UI│ |diagram lang│
└─────────┘ └────────────┘ └─────────────┘ └────────────┘
```
`lang` is the foundation — the compiler and C runtime everything else builds on. `engram` is the graph-native memory/state engine that gives El its identity (property 3 above). Everything else is either a tool for working with El (`epm`, `ide`) or a system built on top of Engram's graph model (`elp`, `ql`, `ui`, `arbor`).
---
## Repository layout
### [lang/](lang/) — the El language
The compiler and runtime. Self-hosting: `elc-cli.el``compiler.el``lexer.el` / `parser.el` / `codegen.el` / `codegen-js.el`, textually inlined and compiled in one pass. Compiles to C11 and links against `el-compiler/runtime/el_seed.c`, a hand-maintained OS-boundary layer (libcurl HTTP, pthreads, filesystem, arena allocation) — everything else in the runtime is native El (`runtime/*.el`).
Two layers to know: **El programs** (`.el` files — where nearly all work belongs) and **the C seed** (`el_seed.c` — edit only for genuine OS-level access; never re-implement what El can already express).
Current status (single source of truth: [lang/spec/language.md](lang/spec/language.md)): lexer/parser/codegen and the C runtime's core (I/O, strings, math, lists, maps, filesystem, args) are implemented. In flight: `%` operator, match-statement codegen, `?` nil-propagation, `cgi` block parsing + DHARMA identity resolution, VBD role enforcement (`@manager`/`@engine`/`@accessor`), the real `engram_*` and `dharma_*` runtimes (currently stubs), and libcurl-backed `http_get`/`http_post`/`http_serve`. Bitwise operators, `??`, and `as` casts are explicitly **not** in this language.
Key docs: [AGENTS.md](lang/AGENTS.md) (agent-facing orientation), [BOOTSTRAP.md](lang/BOOTSTRAP.md) (compiler recovery from scratch), [spec/language.md](lang/spec/language.md), [spec/codegen-js.md](lang/spec/codegen-js.md).
### [engram/](engram/) — graph intelligence substrate
**A local-first memory substrate for accumulating intelligence**, and the reason El's runtime doesn't need a database driver. Rust core (`engram-core`, `engram-ffi`) exposed to El and other languages (Kotlin, TypeScript/WASM, Go bindings).
The model: retrieval is **spreading activation**, not query. You name seed nodes and a query embedding; activation propagates outward through weighted edges, attenuating multiplicatively per hop (`strength = parent_strength × edge_weight × target_salience × cosine_sim`), gets pruned below a threshold, and the top-N nodes by activation strength come back. Storage and retrieval are the same structure — the way long-term potentiation works in biological memory, not the way a relational or vector database works.
Nodes live in four tiers (Working / Episodic / Semantic / Procedural, mirroring prefrontal / hippocampal / neocortical / cerebellar memory) and migrate between them based on **salience decay**`importance × recency-decay × log(activation_count)`. Forgetting is adaptive pruning, not a bug: unreinforced memories stop competing for attention without being deleted.
Backed by `sled` (embedded, local-first, no daemon) with flat cosine scan for vector search — deliberately simple until scale demands an HNSW layer. Full API and design rationale in [engram/README.md](engram/README.md).
### [elp/](elp/) — Engram Language Protocol
Bidirectional engine mapping between Engram semantic forms and natural-language surface text, across **31 languages** — from Spanish and Japanese through historical/liturgical languages (Old Norse, Sanskrit, Sumerian, Coptic, Akkadian, Ge'ez). Compilation order runs `language-profile` + `vocabulary` → per-language `morphology-*``grammar``realizer``semantics``elp`. This is what lets an Engram graph node round-trip to and from readable text in any of those languages.
### [epm/](epm/) — El Package Manager
Manages **vessels** (El's package unit): publish, install, resolve dependencies. Vessels are stored in Engram as graph nodes, not files in a registry index — `epm` reads the local `manifest.el`, talks to Engram over HTTP, and writes resolved vessels to `.epm/vessels/`. Source: `registry.el`, `install.el`, `update.el`, `manifest.el`.
### [ide/](ide/) — El IDE
Three vessels: **el-ide-server** (HTTP backend — file ops, build/run, LSP bridge, plugin host, settings), **el-lsp** (the language server — completion, hover, diagnostics, outline, format, type graph), and **el-plugin-host** (first-party plugin lifecycle: install/remove/enable/disable). `ide/projects/` and `ide/examples/` hold sample projects, including the canonical `hello-friends` first-program walkthrough.
### [ql/](ql/) — engram-el
The El-native integration layer for a *live* Engram server — not a library (no importable modules, no build artifact), a set of standalone `.el` programs run directly via `el run-file`. Three components: **Studio** (`studio/studio.el`, a full terminal graph explorer), a **Hebbian field-model** proof of concept, and El builtin / LLM-builtin smoke test suites. This is the reference for correct patterns when an El program uses Engram as its substrate. Spec: [ql/spec/elql.md](ql/spec/elql.md).
### [ui/](ui/) — el-ui
A frontend framework where **component state is an Engram graph and reactivity is spreading activation** — not virtual-DOM diffing (React), Proxy-based dependency tracking (Vue), or compile-time analysis (Svelte). Re-renders are activated and propagated the same way associative memory retrieval works in `engram/`.
~15 vessels covering the full frontend surface: `el-platform` (env/fs/network/clock abstraction), `el-config`, `el-html` (SSR emit primitives), `el-layout`, `el-style` (design tokens/themes), `el-i18n`, `el-auth` / `el-identity` (JWT, sessions, OAuth PKCE — Engram-native), `el-services` (REST/gRPC/WebSocket bindings), `el-aop` (`@authenticate`/`@authorize`/`@cache`/`@rate_limit` decorators), `el-secrets`, `el-graph` (graph rendering/editor), `el-publish` (App Store / Play Store automation), and `el-ui-compiler` (El→JS component compiler; currently a stub pending a JS backend in `elc`). Spec: [ui/spec/framework.md](ui/spec/framework.md).
### [arbor/](arbor/) — diagram language
A `.arbor` diagram language and toolchain: `arbor-core` (NodeId/shape/edge-kind types), `arbor-parse` (recursive-descent parser), `arbor-diagram` (IR + Mermaid serializer + architecture-diagram builders), `arbor-layout` (hierarchical layout — rank assignment, positioning, group bounds), `arbor-render` (SVG renderer), `arbor-cli`. (The architecture map above is the kind of diagram this is for.)
---
## Getting started
Install the El SDK from the latest release:
```bash
bash lang/install.sh
# EL_VERSION=v1.0.0 bash lang/install.sh # pin a specific release tag
# EL_PREFIX=/opt/el bash lang/install.sh # custom install prefix
```
Or build the compiler from source and verify the self-hosting chain:
```bash
cd lang
./dist/platform/elc elc-cli.el > elc-new.c
cc -std=c11 -I el-compiler/runtime -lcurl -lpthread \
-o dist/platform/elc-new \
elc-new.c el-compiler/runtime/el_seed.c
# Confirm the new binary reproduces itself exactly
./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # should be identical
mv dist/platform/elc-new dist/platform/elc
```
Run your first program:
```bash
./lang/dist/platform/elc lang/examples/hello.el > hello.c
cc -std=c11 -I lang/el-compiler/runtime -lcurl -lpthread \
-o hello hello.c lang/el-compiler/runtime/el_seed.c
./hello
```
More examples in [lang/examples/](lang/examples/), including a full starter project at `lang/examples/hello-project/`.
If the compiler binary is ever lost or corrupted, [lang/BOOTSTRAP.md](lang/BOOTSTRAP.md) is the authoritative recovery path.
---
## Development workflow
Branching follows `dev → stage → main`: work lands on `dev`, promotes to `stage` for integration testing, and is promoted to `main` for release (visible directly in the git history of this repo). CI is defined per-subproject under `.gitea/workflows/``lang`/`epm`/`ide` share the root pipeline; `engram` and `ql` carry their own (`ci-dev`, `ci-stage`, and a release workflow each).
- Language/runtime specs live at `*/spec/*.md` (`lang/spec/`, `ql/spec/`, `ui/spec/`) and are the single source of truth for implemented-vs-planned status — code and docs are expected to agree with the spec's status markers, not the other way around.
- Agent-facing orientation guides live at `*/AGENTS.md` (currently `lang/AGENTS.md`); more subprojects may grow their own as they need agent-specific conventions documented.
- Tagged releases live under `lang/releases/`, each with its own `RELEASE.md`.
---
## Status
This is an actively developed, internal monorepo — not yet published under an open license. Treat everything here as proprietary to Neuron Technologies unless told otherwise.
+23
View File
@@ -0,0 +1,23 @@
// arbor-cli the `arbor` command-line tool.
// Inlines its own copies of the parse / layout / render pipeline so that the
// resulting binary is self-contained. (El's `import` form today concatenates
// source; once a real module loader lands this becomes a thin driver.)
vessel "arbor-cli" {
version "0.1.0"
description "Command-line interface for the Arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-parse "0.1"
arbor-layout "0.1"
arbor-render "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
File diff suppressed because it is too large Load Diff
+18
View File
@@ -0,0 +1,18 @@
// arbor-core fundamental types for Arbor diagrams.
// Node IDs (sanitised), shape vocabulary, edge kinds, and the lightweight
// graph value used by every other vessel.
vessel "arbor-core" {
version "0.1.0"
description "Core types for Arbor diagrams: NodeId, ArborShape, ArborEdgeKind, graphs"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
}
build {
entry "src/main.el"
output "dist/"
}
+333
View File
@@ -0,0 +1,333 @@
// arbor-core core types for Arbor diagrams.
//
// Idiomatic El: everything is a Map. Functions take/return maps; helpers are
// pure and small. The downstream vessels (parse, layout, render) consume the
// shapes defined here.
//
// Shape vocabulary:
// ArborShape strings "rect" "rounded" "cylinder" "diamond" "stadium" "primary"
//
// Edge-kind strings:
// "solid" "dashed" "forbidden" "bidirectional"
//
// Node value: { "id":Str, "label":Str, "shape":Str }
// Edge value: { "from":Str, "to":Str, "label":Str, "kind":Str }
// Group value: { "id":Str, "label":Str, "node_ids":[Str], "direction":Str }
// Graph value: { "title":Str, "direction":Str, "nodes":[Node], "edges":[Edge], "groups":[Group] }
//
// Diagram-form (lowered) is the same shape but with NodeStyle/EdgeLine/Arrow
// resolved into renderer-friendly fields:
// Node: + "sublabel":Str, "style_fill":Str, "style_stroke":Str, "style_color":Str
// Edge: + "line":Str ("solid"/"dashed"/"dotted"/"thick"), "arrow":Str ("forward"/"backward"/"both"/"none")
//
// This file is the canonical definition of those shapes. Other vessels rely on
// these field names.
// NodeId sanitisation
//
// Sanitise an arbitrary string into a Mermaid-safe identifier.
// - any char not in [a-zA-Z0-9_] becomes '_'
// - consecutive underscores collapse
// - trailing underscores stripped
// - if first char is a digit, prepend 'n'
// - if empty, return "node"
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
// Pass 1: replace and collapse.
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
// Pass 2: strip trailing underscores.
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
// Pass 3: leading-digit guard.
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
// Constructors
fn make_node(id: String, label: String, shape: String) -> Map<String, Any> {
{ "id": id, "label": label, "shape": shape }
}
fn make_edge(src: String, dst: String, kind: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "kind": kind }
}
fn make_edge_with_label(src: String, dst: String, kind: String, label: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": label, "kind": kind }
}
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty_ids: [String] = el_list_empty()
{ "id": id, "label": label, "node_ids": empty_ids, "direction": "" }
}
fn make_graph() -> Map<String, Any> {
let empty_n: [Map<String, Any>] = el_list_empty()
let empty_e: [Map<String, Any>] = el_list_empty()
let empty_g: [Map<String, Any>] = el_list_empty()
{ "title": "", "direction": "top-down",
"nodes": empty_n, "edges": empty_e, "groups": empty_g }
}
// Shape vocabulary
// Returns the canonical shape string for a token, or "" if unknown.
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Lower an Arbor shape into the renderer's NodeShape vocabulary.
fn shape_to_node_shape(shape: String) -> String {
if shape == "rect" { return "rectangle" }
if shape == "primary" { return "rectangle" }
if shape == "rounded" { return "rounded_rect" }
if shape == "cylinder" { return "cylinder" }
if shape == "diamond" { return "diamond" }
if shape == "stadium" { return "stadium" }
"rectangle"
}
// Lowering: ArborGraph DiagramGraph
//
// Replaces every node with a diagram-form node carrying explicit style fields,
// and every edge with a diagram-form edge carrying line/arrow strings.
fn lower_node(n: Map<String, Any>) -> Map<String, Any> {
let shape: String = n["shape"]
let node_shape: String = shape_to_node_shape(shape)
let fill = ""
let stroke = ""
let color = ""
if shape == "primary" {
let fill = "#0052A0"
let stroke = "#0052A0"
let color = "#ffffff"
}
{ "id": n["id"], "label": n["label"], "sublabel": "",
"shape": node_shape,
"style_fill": fill, "style_stroke": stroke, "style_color": color }
}
fn lower_edge(e: Map<String, Any>) -> Map<String, Any> {
let kind: String = e["kind"]
let line = "solid"
let arrow = "forward"
if kind == "dashed" {
let line = "dashed"
}
if kind == "bidirectional" {
let arrow = "both"
}
// forbidden uses solid line + forward arrow; the renderer overlays the
// circle-X marker based on a forbidden-set the caller threads through.
{ "from": e["from"], "to": e["to"], "label": e["label"],
"line": line, "arrow": arrow }
}
fn lower_graph(g: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = g["nodes"]
let edges: [Map<String, Any>] = g["edges"]
let lowered_nodes: [Map<String, Any>] = el_list_empty()
let i = 0
let n: Int = el_list_len(nodes)
while i < n {
let lowered_nodes = native_list_append(lowered_nodes, lower_node(get(nodes, i)))
let i = i + 1
}
let lowered_edges: [Map<String, Any>] = el_list_empty()
let i = 0
let m: Int = el_list_len(edges)
while i < m {
let lowered_edges = native_list_append(lowered_edges, lower_edge(get(edges, i)))
let i = i + 1
}
{ "title": g["title"], "direction": g["direction"],
"nodes": lowered_nodes, "edges": lowered_edges, "groups": g["groups"] }
}
// Find a node by id within a (lowered or raw) graph. Returns an empty map
// when not found callers check map_get(result, "id") for presence.
fn graph_find_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let node: Map<String, Any> = get(nodes, i)
let nid: String = node["id"]
if nid == id { return node }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Forbidden-edge set helpers
// The lowered graph drops the "forbidden" kind (line/arrow have no slot for
// it). Callers preserve the set as a list of "from->to" strings.
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn collect_forbidden(graph: Map<String, Any>) -> [String] {
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(edges)
let out: [String] = el_list_empty()
let i = 0
while i < n {
let e: Map<String, Any> = get(edges, i)
let kind: String = e["kind"]
if kind == "forbidden" {
let f: String = e["from"]
let t: String = e["to"]
let out = native_list_append(out, forbidden_key(f, t))
}
let i = i + 1
}
out
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if s == key { return true }
let i = i + 1
}
false
}
// Smoke test
//
// State is kept in process-local k/v storage so we never mix Int + Call or
// Int + Ident in `+` (which the codegen heuristic emits as string concat
// on tagged-pointer values, segfaulting on Int operands).
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
check_eq("sanitize crates/nc-core",
sanitize_id("crates/nc-core"), "crates_nc_core")
check_eq("sanitize package.json",
sanitize_id("package.json"), "package_json")
check_eq("sanitize 42-module",
sanitize_id("42-module"), "n42_module")
check_eq("sanitize empty", sanitize_id(""), "node")
check_eq("sanitize !!--@@", sanitize_id("!!--@@"), "node")
check_eq("shape_from_token rounded",
shape_from_token("rounded"), "rounded")
check_eq("shape_to_node_shape primary",
shape_to_node_shape("primary"), "rectangle")
// Lowering preserves a node id and adds style.
let n: Map<String, Any> = make_node("svc", "Service", "primary")
let ln: Map<String, Any> = lower_node(n)
check_eq("lower preserves id", ln["id"], "svc")
check_eq("lower applies primary fill", ln["style_fill"], "#0052A0")
// Edge lowering
let e: Map<String, Any> = make_edge("a", "b", "dashed")
let le: Map<String, Any> = lower_edge(e)
check_eq("lower edge dashed line", le["line"], "dashed")
let e2: Map<String, Any> = make_edge("a", "b", "bidirectional")
let le2: Map<String, Any> = lower_edge(e2)
check_eq("lower edge bidirectional arrow", le2["arrow"], "both")
println("")
let failures: String = state_get("failures")
if str_eq(failures, "1") {
println("arbor-core: FAILED")
exit_program(1)
} else {
println("arbor-core: ok")
}
+19
View File
@@ -0,0 +1,19 @@
// arbor-diagram diagram intermediate representation + Mermaid serializer
// + dependency-graph builders. Consumes raw graph values built by arbor-core
// or arbor-parse and produces Mermaid markup or other serializations.
vessel "arbor-diagram" {
version "0.1.0"
description "Diagram IR + Mermaid serializer + architecture diagram builders"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
+433
View File
@@ -0,0 +1,433 @@
// arbor-diagram diagram intermediate representation (AST + IR).
//
// Where arbor-core supplies the *.arbor source-language model Mermaid-safe
// IDs, ArborShape strings, ArborEdgeKind strings, and the lowered "diagram-
// form" map arbor-diagram exposes the same lowered model as the canonical
// IR for downstream serializers (arbor-render and any future Mermaid-style
// emitter). The two vessels overlap by design: arbor-core is responsible for
// *naming* the schema; arbor-diagram is responsible for *building* values
// against it.
//
// The Rust crate ships small AST builder structs (`DiagramNode::new`,
// `DiagramEdge::with_label`, `DiagramGraph::add_node`). El has no method
// chaining, no Default::default(), no enum types. The El idiom is a stack
// of immutable maps with explicit constructor + with_* helpers that take
// the value and return a freshly-allocated map.
//
// Public surface:
// make_node(id, label) DiagramNode
// with_shape(node, shape) DiagramNode
// with_sublabel(node, sublabel) DiagramNode
// with_style(node, fill, stroke, color) DiagramNode
//
// make_edge(from, to) DiagramEdge
// with_label(edge, label)
// with_line(edge, line) // "solid"/"dashed"/"dotted"/"thick"
// with_arrow(edge, arrow) // "forward"/"backward"/"both"/"none"
//
// make_group(id, label) DiagramGroup
// with_node(group, node_id)
// with_nodes(group, [node_id])
// with_direction(group, dir)
//
// make_graph(title) DiagramGraph
// with_direction(graph, dir)
// graph_add_node(graph, node) DiagramGraph
// graph_add_edge(graph, edge) DiagramGraph
// graph_add_group(graph, group) DiagramGraph
// graph_node(graph, id) DiagramNode | empty map
//
// Shape vocabulary (lowered): see arbor-core. The local copy here mirrors
// the table in arbor-core/src/main.el so this vessel is hermetic.
// NodeShape vocabulary
fn node_shape_rectangle() -> String { "rectangle" }
fn node_shape_rounded_rect() -> String { "rounded_rect" }
fn node_shape_stadium() -> String { "stadium" }
fn node_shape_cylinder() -> String { "cylinder" }
fn node_shape_diamond() -> String { "diamond" }
fn node_shape_parallelogram() -> String { "parallelogram" }
fn node_shape_database() -> String { "database" }
fn node_shape_subroutine() -> String { "subroutine" }
fn node_shape_valid(s: String) -> Bool {
if str_eq(s, "rectangle") { return true }
if str_eq(s, "rounded_rect") { return true }
if str_eq(s, "stadium") { return true }
if str_eq(s, "cylinder") { return true }
if str_eq(s, "diamond") { return true }
if str_eq(s, "parallelogram") { return true }
if str_eq(s, "database") { return true }
if str_eq(s, "subroutine") { return true }
false
}
// EdgeLine vocabulary
fn edge_line_solid() -> String { "solid" }
fn edge_line_dashed() -> String { "dashed" }
fn edge_line_dotted() -> String { "dotted" }
fn edge_line_thick() -> String { "thick" }
fn edge_line_valid(s: String) -> Bool {
if str_eq(s, "solid") { return true }
if str_eq(s, "dashed") { return true }
if str_eq(s, "dotted") { return true }
if str_eq(s, "thick") { return true }
false
}
// EdgeArrow vocabulary
fn edge_arrow_forward() -> String { "forward" }
fn edge_arrow_backward() -> String { "backward" }
fn edge_arrow_both() -> String { "both" }
fn edge_arrow_none() -> String { "none" }
fn edge_arrow_valid(s: String) -> Bool {
if str_eq(s, "forward") { return true }
if str_eq(s, "backward") { return true }
if str_eq(s, "both") { return true }
if str_eq(s, "none") { return true }
false
}
// Direction vocabulary
fn direction_top_down() -> String { "top-down" }
fn direction_left_right() -> String { "left-right" }
fn direction_right_left() -> String { "right-left" }
fn direction_bottom_up() -> String { "bottom-up" }
fn direction_valid(s: String) -> Bool {
if str_eq(s, "top-down") { return true }
if str_eq(s, "left-right") { return true }
if str_eq(s, "right-left") { return true }
if str_eq(s, "bottom-up") { return true }
false
}
// DiagramNode
fn make_node(id: String, label: String) -> Map<String, Any> {
{
"id": id,
"label": label,
"sublabel": "",
"shape": "rectangle",
"style_fill": "",
"style_stroke": "",
"style_color": ""
}
}
fn with_shape(node: Map<String, Any>, shape: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": shape,
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_sublabel(node: Map<String, Any>, sublabel: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": sublabel,
"shape": node["shape"],
"style_fill": node["style_fill"],
"style_stroke": node["style_stroke"],
"style_color": node["style_color"]
}
}
fn with_style(node: Map<String, Any>, fill: String, stroke: String, color: String) -> Map<String, Any> {
{
"id": node["id"],
"label": node["label"],
"sublabel": node["sublabel"],
"shape": node["shape"],
"style_fill": fill,
"style_stroke": stroke,
"style_color": color
}
}
// DiagramEdge
fn make_edge(from: String, to: String) -> Map<String, Any> {
{
"from": from,
"to": to,
"label": "",
"line": "solid",
"arrow": "forward"
}
}
fn with_label(edge: Map<String, Any>, label: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": label,
"line": edge["line"],
"arrow": edge["arrow"]
}
}
fn with_line(edge: Map<String, Any>, line: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": line,
"arrow": edge["arrow"]
}
}
fn with_arrow(edge: Map<String, Any>, arrow: String) -> Map<String, Any> {
{
"from": edge["from"],
"to": edge["to"],
"label": edge["label"],
"line": edge["line"],
"arrow": arrow
}
}
// DiagramGroup
fn make_group(id: String, label: String) -> Map<String, Any> {
let empty: [String] = native_list_empty()
{
"id": id,
"label": label,
"node_ids": empty,
"direction": ""
}
}
fn with_node(group: Map<String, Any>, node_id: String) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let next: [String] = native_list_append(cur, node_id)
{
"id": group["id"],
"label": group["label"],
"node_ids": next,
"direction": group["direction"]
}
}
fn with_nodes(group: Map<String, Any>, ids: [String]) -> Map<String, Any> {
let cur: [String] = group["node_ids"]
let n: Int = el_list_len(ids)
let i = 0
while i < n {
let cur = native_list_append(cur, get(ids, i))
let i = i + 1
}
{
"id": group["id"],
"label": group["label"],
"node_ids": cur,
"direction": group["direction"]
}
}
fn with_group_direction(group: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"id": group["id"],
"label": group["label"],
"node_ids": group["node_ids"],
"direction": dir
}
}
// DiagramGraph
fn make_graph(title: String) -> Map<String, Any> {
let empty_n: [Map<String, Any>] = native_list_empty()
let empty_e: [Map<String, Any>] = native_list_empty()
let empty_g: [Map<String, Any>] = native_list_empty()
{
"title": title,
"direction": "top-down",
"nodes": empty_n,
"edges": empty_e,
"groups": empty_g
}
}
fn with_direction(graph: Map<String, Any>, dir: String) -> Map<String, Any> {
{
"title": graph["title"],
"direction": dir,
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_node(graph: Map<String, Any>, node: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["nodes"]
let next: [Map<String, Any>] = native_list_append(cur, node)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": next,
"edges": graph["edges"],
"groups": graph["groups"]
}
}
fn graph_add_edge(graph: Map<String, Any>, edge: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["edges"]
let next: [Map<String, Any>] = native_list_append(cur, edge)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": next,
"groups": graph["groups"]
}
}
fn graph_add_group(graph: Map<String, Any>, group: Map<String, Any>) -> Map<String, Any> {
let cur: [Map<String, Any>] = graph["groups"]
let next: [Map<String, Any>] = native_list_append(cur, group)
{
"title": graph["title"],
"direction": graph["direction"],
"nodes": graph["nodes"],
"edges": graph["edges"],
"groups": next
}
}
// Find a node by id. Returns an empty map (no "id" field) when not present.
fn graph_node(graph: Map<String, Any>, id: String) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
if str_eq(nid, id) { return nd }
let i = i + 1
}
let empty: Map<String, Any> = el_map_new(0)
empty
}
// Smoke test
fn fail(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label + " = " + got)
return 1
}
fail(label, got, want)
}
// Vocabulary self-checks
check_eq("shape rectangle valid",
bool_to_str(node_shape_valid("rectangle")), "true")
check_eq("shape hexagon invalid",
bool_to_str(node_shape_valid("hexagon")), "false")
check_eq("line dashed valid",
bool_to_str(edge_line_valid("dashed")), "true")
check_eq("arrow both valid",
bool_to_str(edge_arrow_valid("both")), "true")
check_eq("dir top-down valid",
bool_to_str(direction_valid("top-down")), "true")
// Node builder
let n0: Map<String, Any> = make_node("svc", "Service")
check_eq("node default shape", n0["shape"], "rectangle")
check_eq("node default sublabel empty", n0["sublabel"], "")
let n1: Map<String, Any> = with_shape(n0, "cylinder")
check_eq("node with_shape", n1["shape"], "cylinder")
check_eq("node id preserved", n1["id"], "svc")
let n2: Map<String, Any> = with_sublabel(n1, "v0.1.0")
check_eq("node with_sublabel", n2["sublabel"], "v0.1.0")
let n3: Map<String, Any> = with_style(n2, "#0052A0", "#0052A0", "#ffffff")
check_eq("node style fill", n3["style_fill"], "#0052A0")
check_eq("node style color", n3["style_color"], "#ffffff")
// Edge builder
let e0: Map<String, Any> = make_edge("a", "b")
check_eq("edge default line", e0["line"], "solid")
check_eq("edge default arrow", e0["arrow"], "forward")
let e1: Map<String, Any> = with_line(e0, "dashed")
let e2: Map<String, Any> = with_arrow(e1, "both")
let e3: Map<String, Any> = with_label(e2, "calls")
check_eq("edge line", e3["line"], "dashed")
check_eq("edge arrow", e3["arrow"], "both")
check_eq("edge label", e3["label"], "calls")
// Group builder
let g0: Map<String, Any> = make_group("core", "Application Core")
let g1: Map<String, Any> = with_node(g0, "api")
let g2: Map<String, Any> = with_node(g1, "svc")
let ids2: [String] = g2["node_ids"]
check_eq("group with two nodes", int_to_str(el_list_len(ids2)), "2")
let g3: Map<String, Any> = make_group("infra", "Infrastructure")
let extras: [String] = native_list_empty()
let extras = native_list_append(extras, "db")
let extras = native_list_append(extras, "cache")
let g4: Map<String, Any> = with_nodes(g3, extras)
let ids4: [String] = g4["node_ids"]
check_eq("group with_nodes appends", int_to_str(el_list_len(ids4)), "2")
// Graph builder + lookup
let G0: Map<String, Any> = make_graph("System")
let G1: Map<String, Any> = with_direction(G0, "left-right")
let G2: Map<String, Any> = graph_add_node(G1, n3)
let nb: Map<String, Any> = make_node("b", "Backend")
let G3: Map<String, Any> = graph_add_node(G2, nb)
let G4: Map<String, Any> = graph_add_edge(G3, e3)
let G5: Map<String, Any> = graph_add_group(G4, g4)
check_eq("graph title", G5["title"], "System")
check_eq("graph direction", G5["direction"], "left-right")
let gn: [Map<String, Any>] = G5["nodes"]
let ge: [Map<String, Any>] = G5["edges"]
let gg: [Map<String, Any>] = G5["groups"]
check_eq("graph nodes count", int_to_str(el_list_len(gn)), "2")
check_eq("graph edges count", int_to_str(el_list_len(ge)), "1")
check_eq("graph groups count", int_to_str(el_list_len(gg)), "1")
let found: Map<String, Any> = graph_node(G5, "svc")
check_eq("graph_node found", found["id"], "svc")
let missing: Map<String, Any> = graph_node(G5, "nonexistent")
let missing_id: String = missing["id"]
if str_len(missing_id) == 0 {
println("ok graph_node missing returns empty")
} else {
println("FAIL graph_node missing returned: " + missing_id)
state_set("smoke_failures", "1")
}
println("")
let failures: String = state_get("smoke_failures")
if str_eq(failures, "1") {
println("arbor-diagram: FAILED")
exit_program(1)
} else {
println("arbor-diagram: ok")
}
+19
View File
@@ -0,0 +1,19 @@
// arbor-layout hierarchical layout engine. Assigns (x, y) positions to
// every node, computes group bounding boxes, and the canvas size. Consumes
// a diagram graph; produces a layout-result value.
vessel "arbor-layout" {
version "0.1.0"
description "Hierarchical layout engine — rank assignment, positioning, group bounds"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
+591
View File
@@ -0,0 +1,591 @@
// arbor-layout hierarchical layout for diagram graphs.
//
// Public entry point:
// fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any>
//
// The graph is the lowered (diagram-form) shape. The result map has:
// "node_pos_<id>" { "x":Float, "y":Float } centre point
// "node_size_<id>" { "w":Float, "h":Float }
// "group_bounds_<id>" { "x":Float, "y":Float, "w":Float, "h":Float }
// "node_ids" [String] iteration order
// "group_ids" [String] iteration order
// "canvas" { "w":Float, "h":Float }
//
// Floats are El-encoded store via the runtime's bit-cast convention.
// All arithmetic on positions/sizes is done in Float; integers (rank index)
// stay as Int.
//
// Algorithm (simplified Sugiyama):
// 1. Assign ranks via topological propagation (longest path from sources).
// 2. Group nodes by rank, preserving declaration order.
// 3. Position each rank as a row (top-down/bottom-up) or column (LR/RL).
// 4. Compute group bounding boxes from member positions.
// 5. Compute canvas size to enclose everything.
//
// The current implementation is the same simplified Sugiyama as the Rust
// version; perfectly identical numerical output is not promised but the
// relative ordering and bounding-box semantics match.
// Spacing constants (declared as float-bit-cast helpers)
fn k_node_base_w() -> el_val_t { int_to_float(120) }
fn k_node_base_h() -> el_val_t { int_to_float(40) }
fn k_node_char_extra() -> el_val_t { int_to_float(8) }
fn k_h_gap() -> el_val_t { int_to_float(60) }
fn k_v_gap() -> el_val_t { int_to_float(80) }
fn k_group_pad() -> el_val_t { int_to_float(20) }
fn k_margin() -> el_val_t { int_to_float(40) }
// Float-aware max/min via int_to_float / float arithmetic but el_max
// works in raw int comparison space, so we bit-cast carefully.
// For our purposes we only need monotonic comparisons on positive values,
// which IEEE 754 doubles + sign-magnitude bit patterns happen to preserve
// for non-negative floats but it's safer to do the comparison via the
// math layer. We use a helper that decodes both, picks the bigger, and
// re-encodes.
//
// Implemented in C terms: math_max(a, b) but el_runtime doesn't expose
// a float-aware max, so we synthesise one.
fn fmax(a: el_val_t, b: el_val_t) -> el_val_t {
// Compare via float subtraction's sign: a - b. Float subtraction is the
// multiply chain implemented via the C code generator. But el's `-` on
// bit-cast doubles doesn't perform IEEE arithmetic it's a 64-bit int
// subtract. Workaround: round-trip through format_float and str_to_float.
// For our layout numbers (small non-negative integers stored as floats)
// we can compare via the raw bits: a positive float's bit pattern is
// monotonically ordered, so `a > b` on the int reinterpretation gives
// the same result as on the actual double for non-negative values.
if a > b { return a }
b
}
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
// a, b are bit-cast doubles. Safe addition: int-to-float, format, parse.
// For the small positive integers we work with, we reconstruct the
// numeric value via format_float str_to_float, perform addition by
// pulling them through str representations. Costly but correct on the
// current runtime. Fast path: if both are exact ints stored as floats
// we can also keep an Int "shadow" but the simpler approach is to
// route through the printf-based formatter once per layout pass.
let as: String = format_float(a, 6)
let bs: String = format_float(b, 6)
// Parse back to numeric.
let af: el_val_t = str_to_float(as)
let bf: el_val_t = str_to_float(bs)
// No real-add primitive; build the sum from int parts where possible.
// Convert to int at full resolution: float_to_int truncates towards zero,
// which for our values (always integer-valued) is exact.
let ai: Int = float_to_int(af)
let bi: Int = float_to_int(bf)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fmul(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai * bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
// Node size based on label width
fn node_size_for(label: String) -> Map<String, Any> {
let len: Int = str_len(label)
let extra: Int = 0
if len > 10 {
let extra = len - 10
}
let w_int: Int = 120 + 8 * extra
let w: el_val_t = int_to_float(w_int)
let h: el_val_t = int_to_float(40)
{ "w": w, "h": h }
}
// Adjacency-list construction
//
// Builds successor and in-degree maps keyed by node id.
fn build_succ_indeg(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let edges: [Map<String, Any>] = graph["edges"]
let n: Int = el_list_len(nodes)
let m: Int = el_list_len(edges)
let succ: Map<String, Any> = el_map_new(0)
let indeg: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let empty: [String] = el_list_empty()
let succ = el_map_set(succ, nid, empty)
let indeg = el_map_set(indeg, nid, 0)
let i = i + 1
}
let i = 0
while i < m {
let e: Map<String, Any> = get(edges, i)
let src: String = e["from"]
let dst: String = e["to"]
let cur_succ: [String] = el_map_get(succ, src)
let new_succ: [String] = native_list_append(cur_succ, dst)
let succ = el_map_set(succ, src, new_succ)
let prev: Int = el_map_get(indeg, dst)
let indeg = el_map_set(indeg, dst, prev + 1)
let i = i + 1
}
{ "succ": succ, "indeg": indeg }
}
// Topological rank assignment
//
// Returns a map: node_id rank.
fn assign_ranks(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let adj: Map<String, Any> = build_succ_indeg(graph)
let succ: Map<String, Any> = adj["succ"]
let indeg: Map<String, Any> = adj["indeg"]
let ranks: Map<String, Any> = el_map_new(0)
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let ranks = el_map_set(ranks, nid, 0)
let i = i + 1
}
// Initialise queue with all nodes whose in-degree is 0 (in declaration
// order, mirroring the Rust implementation's ordering guarantee).
let queue: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let d: Int = el_map_get(indeg, nid)
if d == 0 {
let queue = native_list_append(queue, nid)
}
let i = i + 1
}
let head = 0
let running = true
while running {
if head >= el_list_len(queue) {
let running = false
} else {
let cur: String = get(queue, head)
let head = head + 1
let cur_rank: Int = el_map_get(ranks, cur)
let neighbours: [String] = el_map_get(succ, cur)
let nn: Int = el_list_len(neighbours)
let j = 0
while j < nn {
let nb: String = get(neighbours, j)
let nb_rank: Int = el_map_get(ranks, nb)
let cand: Int = cur_rank + 1
if cand > nb_rank {
let ranks = el_map_set(ranks, nb, cand)
}
let cur_d: Int = el_map_get(indeg, nb)
let new_d: Int = cur_d - 1
let indeg = el_map_set(indeg, nb, new_d)
if new_d <= 0 {
let queue = native_list_append(queue, nb)
}
let j = j + 1
}
}
}
ranks
}
// Layout pass
fn arbor_layout(graph: Map<String, Any>) -> Map<String, Any> {
let nodes: [Map<String, Any>] = graph["nodes"]
let n: Int = el_list_len(nodes)
let direction: String = graph["direction"]
let result: Map<String, Any> = el_map_new(0)
let result = el_map_set(result, "node_ids", el_list_empty())
let result = el_map_set(result, "group_ids", el_list_empty())
if n == 0 {
let canvas: Map<String, Any> = { "w": int_to_float(200), "h": int_to_float(100) }
let result = el_map_set(result, "canvas", canvas)
return result
}
let ranks: Map<String, Any> = assign_ranks(graph)
let max_rank = 0
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
if r > max_rank { let max_rank = r }
let i = i + 1
}
// Group nodes by rank, preserving declaration order. Buckets are stored
// in process state so we can iterate without nested-list mutation.
let i = 0
while i <= max_rank {
state_set("rank_bucket_" + int_to_str(i), "")
let i = i + 1
}
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let r: Int = el_map_get(ranks, nid)
let key = "rank_bucket_" + int_to_str(r)
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, nid)
} else {
state_set(key, prev + "" + nid)
}
let i = i + 1
}
// Pre-compute sizes and stash a label-keyed cache.
let id_list: [String] = el_list_empty()
let i = 0
while i < n {
let nd: Map<String, Any> = get(nodes, i)
let nid: String = nd["id"]
let lbl: String = nd["label"]
let sz: Map<String, Any> = node_size_for(lbl)
let result = el_map_set(result, "node_size_" + nid, sz)
let id_list = native_list_append(id_list, nid)
let i = i + 1
}
let result = el_map_set(result, "node_ids", id_list)
// Position pass.
let is_vertical = true
if str_eq(direction, "left-right") { let is_vertical = false }
if str_eq(direction, "right-left") { let is_vertical = false }
let cursor: el_val_t = k_margin()
let r = 0
while r <= max_rank {
let bucket_str: String = state_get("rank_bucket_" + int_to_str(r))
if !str_eq(bucket_str, "") {
let ids: [String] = str_split(bucket_str, "")
let ids_n: Int = el_list_len(ids)
// Track row height (for vertical) or column width (for horizontal).
let cross_max: el_val_t = int_to_float(40)
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
if is_vertical {
let h: el_val_t = sz["h"]
let cross_max = fmax(cross_max, h)
} else {
let w: el_val_t = sz["w"]
let cross_max = fmax(cross_max, w)
}
let j = j + 1
}
if is_vertical {
let row_h: el_val_t = cross_max
let y_center: el_val_t = fadd(cursor, fdiv2(row_h))
let x_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let w: el_val_t = sz["w"]
let cx: el_val_t = fadd(x_cursor, fdiv2(w))
let pos: Map<String, Any> = { "x": cx, "y": y_center }
let result = el_map_set(result, "node_pos_" + nid, pos)
let x_cursor = fadd(fadd(x_cursor, w), k_h_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, row_h), k_v_gap())
} else {
let col_w: el_val_t = cross_max
let x_center: el_val_t = fadd(cursor, fdiv2(col_w))
let y_cursor: el_val_t = k_margin()
let j = 0
while j < ids_n {
let nid: String = get(ids, j)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let h: el_val_t = sz["h"]
let cy: el_val_t = fadd(y_cursor, fdiv2(h))
let pos: Map<String, Any> = { "x": x_center, "y": cy }
let result = el_map_set(result, "node_pos_" + nid, pos)
let y_cursor = fadd(fadd(y_cursor, h), k_v_gap())
let j = j + 1
}
let cursor = fadd(fadd(cursor, col_w), k_h_gap())
}
} else {
// Empty bucket advance cursor by a default node size.
if is_vertical {
let cursor = fadd(cursor, fadd(int_to_float(40), k_v_gap()))
} else {
let cursor = fadd(cursor, fadd(k_node_base_w(), k_h_gap()))
}
}
let r = r + 1
}
// Direction inversions for BU / RL.
let need_flip_y = false
let need_flip_x = false
if str_eq(direction, "bottom-up") { let need_flip_y = true }
if str_eq(direction, "right-left") { let need_flip_x = true }
if need_flip_y {
let max_y: el_val_t = fadd(fsub(cursor, k_v_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let y: el_val_t = pos["y"]
let new_y: el_val_t = fadd(fsub(max_y, y), k_margin())
let new_pos: Map<String, Any> = { "x": pos["x"], "y": new_y }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
if need_flip_x {
let max_x: el_val_t = fadd(fsub(cursor, k_h_gap()), k_margin())
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let x: el_val_t = pos["x"]
let new_x: el_val_t = fadd(fsub(max_x, x), k_margin())
let new_pos: Map<String, Any> = { "x": new_x, "y": pos["y"] }
let result = el_map_set(result, "node_pos_" + nid, new_pos)
let i = i + 1
}
}
// Group bounds.
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let gid_list: [String] = el_list_empty()
let g = 0
while g < gn {
let grp: Map<String, Any> = get(groups, g)
let gid: String = grp["id"]
let member_ids: [String] = grp["node_ids"]
let mn: Int = el_list_len(member_ids)
if mn > 0 {
let big: Int = 1000000000
let neg: Int = 0 - 1000000000
let min_x: el_val_t = int_to_float(big)
let min_y: el_val_t = int_to_float(big)
let max_x: el_val_t = int_to_float(neg)
let max_y: el_val_t = int_to_float(neg)
let mi = 0
while mi < mn {
let mid: String = get(member_ids, mi)
let mpos: Map<String, Any> = el_map_get(result, "node_pos_" + mid)
let msz: Map<String, Any> = el_map_get(result, "node_size_" + mid)
let mid_present: String = mpos["x"]
if str_len(mid_present) >= 0 {
let cx: el_val_t = mpos["x"]
let cy: el_val_t = mpos["y"]
let mw: el_val_t = msz["w"]
let mh: el_val_t = msz["h"]
let left: el_val_t = fsub(cx, fdiv2(mw))
let right: el_val_t = fadd(cx, fdiv2(mw))
let top: el_val_t = fsub(cy, fdiv2(mh))
let bot: el_val_t = fadd(cy, fdiv2(mh))
if left < min_x { let min_x = left }
if top < min_y { let min_y = top }
if right > max_x { let max_x = right }
if bot > max_y { let max_y = bot }
}
let mi = mi + 1
}
let bx: el_val_t = fsub(min_x, k_group_pad())
let by: el_val_t = fsub(min_y, k_group_pad())
let bw: el_val_t = fadd(fsub(max_x, min_x), fmul(k_group_pad(), int_to_float(2)))
let bh: el_val_t = fadd(fsub(max_y, min_y), fmul(k_group_pad(), int_to_float(2)))
let bounds: Map<String, Any> = { "x": bx, "y": by, "w": bw, "h": bh }
let result = el_map_set(result, "group_bounds_" + gid, bounds)
let gid_list = native_list_append(gid_list, gid)
}
let g = g + 1
}
let result = el_map_set(result, "group_ids", gid_list)
// Canvas size = max node-right / node-bottom + group-right / group-bottom.
let canvas_w: el_val_t = int_to_float(0)
let canvas_h: el_val_t = int_to_float(0)
let i = 0
while i < n {
let nid: String = get(id_list, i)
let pos: Map<String, Any> = el_map_get(result, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(result, "node_size_" + nid)
let right: el_val_t = fadd(pos["x"], fdiv2(sz["w"]))
let bottom: el_val_t = fadd(pos["y"], fdiv2(sz["h"]))
if right > canvas_w { let canvas_w = right }
if bottom > canvas_h { let canvas_h = bottom }
let i = i + 1
}
let i = 0
while i < el_list_len(gid_list) {
let gid: String = get(gid_list, i)
let b: Map<String, Any> = el_map_get(result, "group_bounds_" + gid)
let r: el_val_t = fadd(b["x"], b["w"])
let bt: el_val_t = fadd(b["y"], b["h"])
if r > canvas_w { let canvas_w = r }
if bt > canvas_h { let canvas_h = bt }
let i = i + 1
}
let canvas: Map<String, Any> = {
"w": fadd(canvas_w, k_margin()),
"h": fadd(canvas_h, k_margin())
}
let result = el_map_set(result, "canvas", canvas)
result
}
// Smoke test
fn fl_to_str(v: el_val_t) -> String {
int_to_str(float_to_int(v))
}
fn smoke_fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn make_test_node(id: String, label: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": "rectangle",
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String) -> Map<String, Any> {
{ "from": src, "to": dst, "label": "", "line": "solid", "arrow": "forward" }
}
fn make_test_graph(direction: String, ids: [String], src_dst: [String]) -> Map<String, Any> {
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < el_list_len(ids) {
let nid: String = get(ids, i)
let nodes = native_list_append(nodes, make_test_node(nid, nid))
let i = i + 1
}
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i + 1 < el_list_len(src_dst) {
let s: String = get(src_dst, i)
let d: String = get(src_dst, i + 1)
let edges = native_list_append(edges, make_test_edge(s, d))
let i = i + 2
}
{
"title": "T", "direction": direction,
"nodes": nodes, "edges": edges, "groups": el_list_empty()
}
}
// Empty graph.
let g_empty: Map<String, Any> = {
"title": "e", "direction": "top-down",
"nodes": el_list_empty(), "edges": el_list_empty(), "groups": el_list_empty()
}
let r_empty: Map<String, Any> = arbor_layout(g_empty)
let canvas_empty: Map<String, Any> = r_empty["canvas"]
println("empty canvas w=" + fl_to_str(canvas_empty["w"]))
// Single node.
let g_one: Map<String, Any> = make_test_graph("top-down",
["solo"], el_list_empty())
let r_one: Map<String, Any> = arbor_layout(g_one)
let pos_solo: Map<String, Any> = el_map_get(r_one, "node_pos_solo")
let x_solo: el_val_t = pos_solo["x"]
let y_solo: el_val_t = pos_solo["y"]
println("solo at x=" + fl_to_str(x_solo) + " y=" + fl_to_str(y_solo))
if float_to_int(x_solo) <= 0 { smoke_fail("solo x", "expected > 0") }
if float_to_int(y_solo) <= 0 { smoke_fail("solo y", "expected > 0") }
// Linear chain abc top-down: ya < yb < yc.
let g_chain: Map<String, Any> = make_test_graph("top-down",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_chain: Map<String, Any> = arbor_layout(g_chain)
let pa: Map<String, Any> = el_map_get(r_chain, "node_pos_a")
let pb: Map<String, Any> = el_map_get(r_chain, "node_pos_b")
let pc: Map<String, Any> = el_map_get(r_chain, "node_pos_c")
let ya: el_val_t = pa["y"]
let yb: el_val_t = pb["y"]
let yc: el_val_t = pc["y"]
println("td a.y=" + fl_to_str(ya) + " b.y=" + fl_to_str(yb) + " c.y=" + fl_to_str(yc))
if float_to_int(ya) >= float_to_int(yb) { smoke_fail("td order", "a.y >= b.y") }
if float_to_int(yb) >= float_to_int(yc) { smoke_fail("td order", "b.y >= c.y") }
// LR direction
let g_lr: Map<String, Any> = make_test_graph("left-right",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_lr: Map<String, Any> = arbor_layout(g_lr)
let pa2: Map<String, Any> = el_map_get(r_lr, "node_pos_a")
let pc2: Map<String, Any> = el_map_get(r_lr, "node_pos_c")
let xa: el_val_t = pa2["x"]
let xc: el_val_t = pc2["x"]
println("lr a.x=" + fl_to_str(xa) + " c.x=" + fl_to_str(xc))
if float_to_int(xa) >= float_to_int(xc) { smoke_fail("lr order", "a.x >= c.x") }
// Bottom-up: a is below c.
let g_bu: Map<String, Any> = make_test_graph("bottom-up",
["a", "b", "c"], ["a", "b", "b", "c"])
let r_bu: Map<String, Any> = arbor_layout(g_bu)
let pa3: Map<String, Any> = el_map_get(r_bu, "node_pos_a")
let pc3: Map<String, Any> = el_map_get(r_bu, "node_pos_c")
let ya3: el_val_t = pa3["y"]
let yc3: el_val_t = pc3["y"]
println("bu a.y=" + fl_to_str(ya3) + " c.y=" + fl_to_str(yc3))
if float_to_int(ya3) <= float_to_int(yc3) { smoke_fail("bu order", "a.y <= c.y") }
// Canvas covers all nodes.
let canvas_chain: Map<String, Any> = r_chain["canvas"]
let cw: el_val_t = canvas_chain["w"]
let ch: el_val_t = canvas_chain["h"]
println("chain canvas w=" + fl_to_str(cw) + " h=" + fl_to_str(ch))
if float_to_int(cw) <= 0 { smoke_fail("canvas w", "non-positive") }
if float_to_int(ch) <= 0 { smoke_fail("canvas h", "non-positive") }
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-layout: FAILED")
exit_program(1)
} else {
println("arbor-layout: ok")
}
+19
View File
@@ -0,0 +1,19 @@
// arbor-parse hand-written recursive-descent parser for the .arbor source
// language. Produces an Arbor graph value consumable by arbor-layout and
// arbor-render.
vessel "arbor-parse" {
version "0.1.0"
description "Recursive-descent parser for the .arbor diagram language"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
+763
View File
@@ -0,0 +1,763 @@
// arbor-parse recursive-descent parser for the .arbor source language.
//
// This vessel inlines a private copy of the small set of arbor-core helpers
// it needs (sanitize_id and constructors). El's import form today is purely
// syntactic concatenation, so each vessel that wants to be its own buildable
// unit carries its own copy of these helpers. They're tiny (well under 100
// lines) and the duplication keeps each vessel hermetic.
//
// Public entry point: fn arbor_parse(source: String) -> Map<String, Any>
//
// Returns either a graph value or a parse-error map. Callers test for the
// "error" field:
// { "error": "..." , "line": Int, "text": "...source line..." } on failure
// { "title", "direction", "nodes", "edges", "groups" } on success
// Sanitisation (copy of arbor-core's sanitize_id)
fn is_alnum_underscore(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
if code >= 65 {
if code <= 90 { return true }
}
if code >= 97 {
if code <= 122 { return true }
}
if code == 95 { return true }
false
}
fn is_ascii_digit(ch: String) -> Bool {
let code: Int = str_char_code(ch, 0)
if code >= 48 {
if code <= 57 { return true }
}
false
}
fn sanitize_id(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "node" }
let out = ""
let prev_underscore = false
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if is_alnum_underscore(ch) {
let out = out + ch
let prev_underscore = false
} else {
if !prev_underscore {
let out = out + "_"
}
let prev_underscore = true
}
let i = i + 1
}
let m: Int = str_len(out)
let end = m
let stripping = true
while stripping {
if end <= 0 {
let stripping = false
} else {
let last: String = str_char_at(out, end - 1)
if last == "_" {
let end = end - 1
} else {
let stripping = false
}
}
}
let out = str_slice(out, 0, end)
if str_len(out) == 0 { return "node" }
let first: String = str_char_at(out, 0)
if is_ascii_digit(first) {
let out = "n" + out
}
out
}
fn shape_from_token(tok: String) -> String {
let t: String = str_trim(tok)
if t == "rect" { return "rect" }
if t == "rounded" { return "rounded" }
if t == "cylinder" { return "cylinder" }
if t == "diamond" { return "diamond" }
if t == "stadium" { return "stadium" }
if t == "primary" { return "primary" }
""
}
// Line preprocessing
//
// Strip inline `// ...` comments, trim, drop empties. Returns a list of maps
// { "no": Int, "text": String }.
fn preprocess(source: String) -> [Map<String, Any>] {
let lines: [String] = str_split(source, "\n")
let n: Int = el_list_len(lines)
let out: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n {
let raw: String = get(lines, i)
let cidx: Int = str_index_of(raw, "//")
let stripped = raw
if cidx >= 0 {
let stripped = str_slice(raw, 0, cidx)
}
let trimmed: String = str_trim(stripped)
if str_len(trimmed) > 0 {
let row: Map<String, Any> = { "no": i + 1, "text": trimmed }
let out = native_list_append(out, row)
}
let i = i + 1
}
out
}
// Quoted-string extraction
//
// Parses `"text"`-prefix from a string. Returns `{ "ok": Bool, "value": Str,
// "rest": Str }`. The `rest` field carries everything after the closing quote
// (so the caller can continue tokenising).
fn parse_quoted(s: String) -> Map<String, Any> {
let t: String = str_trim(s)
if str_len(t) < 2 {
return { "ok": false, "value": "", "rest": s }
}
let first: String = str_char_at(t, 0)
if first != "\"" {
return { "ok": false, "value": "", "rest": s }
}
let body: String = str_slice(t, 1, str_len(t))
let close: Int = str_index_of(body, "\"")
if close < 0 {
return { "ok": false, "value": "", "rest": s }
}
let inner: String = str_slice(body, 0, close)
let rest: String = str_slice(body, close + 1, str_len(body))
{ "ok": true, "value": inner, "rest": rest }
}
// Identifier prefix split
//
// `split_identifier("foo bar")` { "id": "foo", "rest": " bar" }.
// `split_identifier("a-b")` { "id": "a", "rest": "-b" }.
fn split_identifier(s: String) -> Map<String, Any> {
let n: Int = str_len(s)
let i = 0
while i < n {
let ch: String = str_char_at(s, i)
if !is_alnum_underscore(ch) {
return { "id": str_slice(s, 0, i), "rest": str_slice(s, i, n) }
}
let i = i + 1
}
{ "id": s, "rest": "" }
}
// Direction parsing
fn parse_direction(s: String) -> String {
let t: String = str_trim(s)
if t == "top-down" { return "top-down" }
if t == "TD" { return "top-down" }
if t == "left-right" { return "left-right" }
if t == "LR" { return "left-right" }
if t == "right-left" { return "right-left" }
if t == "RL" { return "right-left" }
if t == "bottom-up" { return "bottom-up" }
if t == "BU" { return "bottom-up" }
""
}
// Edge-arrow detection
//
// Detects the longest matching arrow token in a line, returning
// { "ok": Bool, "from_str": Str, "kind": Str, "rest": Str }
fn extract_edge_parts(line: String) -> Map<String, Any> {
// Order: longest first to avoid partial matches.
let f1: Int = str_index_of(line, "-/->")
if f1 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f1),
"kind": "forbidden",
"rest": str_slice(line, f1 + 4, str_len(line)) }
}
let f2: Int = str_index_of(line, "<->")
if f2 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f2),
"kind": "bidirectional",
"rest": str_slice(line, f2 + 3, str_len(line)) }
}
let f3: Int = str_index_of(line, "-->")
if f3 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f3),
"kind": "dashed",
"rest": str_slice(line, f3 + 3, str_len(line)) }
}
let f4: Int = str_index_of(line, "->")
if f4 >= 0 {
return { "ok": true,
"from_str": str_slice(line, 0, f4),
"kind": "solid",
"rest": str_slice(line, f4 + 2, str_len(line)) }
}
{ "ok": false, "from_str": "", "kind": "", "rest": "" }
}
fn is_edge_line(line: String) -> Bool {
if str_contains(line, "->") { return true }
if str_contains(line, "<->") { return true }
false
}
// Error helpers
fn make_error(line_no: Int, line_text: String, message: String) -> Map<String, Any> {
{ "error": message, "line": line_no, "text": line_text }
}
// Parse driver
//
// State is held in process-local k/v rather than threaded through every
// function. Specifically:
// "title", "direction" graph header
// "nodes_json", "edges_json", "groups_json" accumulators (string lists)
// "group_stack_depth" "0".."N" open groups
// "group_stack_<i>_id" / "_label" / "_line" frame data
// "group_stack_<i>_node_ids" JSON array of ids inside frame
// "error" non-empty if parse failed
// "error_line", "error_text" context
fn st_set_int(key: String, v: Int) -> Int { state_set(key, int_to_str(v)); 0 }
fn st_get_int(key: String) -> Int {
let s: String = state_get(key)
if str_eq(s, "") { return 0 }
str_to_int(s)
}
// Encode/decode small string lists via "" delimiter (unit separator).
fn list_encode(xs: [String]) -> String {
let n: Int = el_list_len(xs)
let out = ""
let i = 0
while i < n {
if i > 0 { let out = out + "" }
let out = out + get(xs, i)
let i = i + 1
}
out
}
fn list_decode(s: String) -> [String] {
if str_eq(s, "") { return el_list_empty() }
str_split(s, "")
}
fn current_group_index() -> Int {
st_get_int("group_stack_depth") - 1
}
fn group_frame_key(idx: Int, suffix: String) -> String {
"gs_" + int_to_str(idx) + "_" + suffix
}
fn open_group(id: String, label: String, line_no: Int) -> Int {
let depth: Int = st_get_int("group_stack_depth")
state_set(group_frame_key(depth, "id"), id)
state_set(group_frame_key(depth, "label"), label)
state_set(group_frame_key(depth, "line"), int_to_str(line_no))
state_set(group_frame_key(depth, "ids"), "")
st_set_int("group_stack_depth", depth + 1)
0
}
fn close_group_frame() -> Map<String, Any> {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 {
return { "ok": false, "id": "", "label": "", "ids": "" }
}
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let label: String = state_get(group_frame_key(idx, "label"))
let ids: String = state_get(group_frame_key(idx, "ids"))
state_del(group_frame_key(idx, "id"))
state_del(group_frame_key(idx, "label"))
state_del(group_frame_key(idx, "line"))
state_del(group_frame_key(idx, "ids"))
st_set_int("group_stack_depth", idx)
{ "ok": true, "id": id, "label": label, "ids": ids }
}
fn register_node_in_group(node_id: String) -> Int {
let depth: Int = st_get_int("group_stack_depth")
if depth <= 0 { return 0 }
let idx: Int = depth - 1
let key: String = group_frame_key(idx, "ids")
let prev: String = state_get(key)
if str_eq(prev, "") {
state_set(key, node_id)
} else {
state_set(key, prev + "" + node_id)
}
0
}
// Accumulator JSON-ish encoding for nodes/edges/groups.
// We render each entry as a small string and stash in state under a counter.
fn store_node(id: String, label: String, shape: String) -> Int {
let n: Int = st_get_int("node_count")
state_set("node_id_" + int_to_str(n), id)
state_set("node_label_" + int_to_str(n), label)
state_set("node_shape_" + int_to_str(n), shape)
st_set_int("node_count", n + 1)
0
}
fn store_edge(src: String, dst: String, label: String, kind: String) -> Int {
let n: Int = st_get_int("edge_count")
state_set("edge_from_" + int_to_str(n), src)
state_set("edge_to_" + int_to_str(n), dst)
state_set("edge_label_" + int_to_str(n), label)
state_set("edge_kind_" + int_to_str(n), kind)
st_set_int("edge_count", n + 1)
0
}
fn store_group(id: String, label: String, ids: String) -> Int {
let n: Int = st_get_int("group_count")
state_set("group_id_" + int_to_str(n), id)
state_set("group_label_" + int_to_str(n), label)
state_set("group_ids_" + int_to_str(n), ids)
st_set_int("group_count", n + 1)
0
}
fn set_error(msg: String, line_no: Int, line_text: String) -> Int {
state_set("parse_error", msg)
st_set_int("parse_error_line", line_no)
state_set("parse_error_text", line_text)
0
}
fn has_error() -> Bool {
let m: String = state_get("parse_error")
if str_eq(m, "") { return false }
true
}
// Reset state at the start of each parse pass.
fn reset_state() -> Int {
state_set("graph_title", "")
state_set("graph_direction", "top-down")
st_set_int("node_count", 0)
st_set_int("edge_count", 0)
st_set_int("group_count", 0)
st_set_int("group_stack_depth", 0)
state_set("parse_error", "")
st_set_int("parse_error_line", 0)
state_set("parse_error_text", "")
0
}
// Statement-level parsing
fn parse_node_stmt(line_no: Int, line: String) -> Int {
let id_split: Map<String, Any> = split_identifier(line)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("expected node id, edge, or keyword", line_no, line)
return 0
}
let id: String = sanitize_id(raw_id)
let rest: String = str_trim(id_split["rest"])
// Optional shape: [token]
let shape = "rect"
let after_shape = rest
if str_len(rest) > 0 {
let lead: String = str_char_at(rest, 0)
if lead == "[" {
let close: Int = str_index_of(rest, "]")
if close < 0 {
set_error("unclosed `[` in shape token", line_no, line)
return 0
}
let token: String = str_slice(rest, 1, close)
let parsed_shape: String = shape_from_token(token)
if str_eq(parsed_shape, "") {
set_error("unknown shape `" + token + "`", line_no, line)
return 0
}
let shape = parsed_shape
let after_shape = str_trim(str_slice(rest, close + 1, str_len(rest)))
}
}
// Optional quoted label.
let quoted: Map<String, Any> = parse_quoted(after_shape)
let label = raw_id
let ok: Bool = quoted["ok"]
if ok {
let label = quoted["value"]
}
store_node(id, label, shape)
register_node_in_group(id)
1
}
fn parse_edge_stmt(line_no: Int, line: String) -> Int {
let parts: Map<String, Any> = extract_edge_parts(line)
let ok: Bool = parts["ok"]
if !ok {
set_error("malformed edge — expected `->` `-->` `<->` or `-/->`", line_no, line)
return 0
}
let from_str: String = parts["from_str"]
let rest_str: String = parts["rest"]
let kind: String = parts["kind"]
let src: String = sanitize_id(str_trim(from_str))
let rest_t: String = str_trim(rest_str)
let id_split: Map<String, Any> = split_identifier(rest_t)
let to_raw: String = id_split["id"]
if str_eq(to_raw, "") {
set_error("edge missing target node id", line_no, line)
return 0
}
let dst: String = sanitize_id(to_raw)
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = ""
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
store_edge(src, dst, label, kind)
1
}
fn parse_group_open(line_no: Int, line: String, rest: String) -> Int {
// Strip trailing `{`.
let trimmed: String = str_trim(rest)
let n: Int = str_len(trimmed)
let body = trimmed
if n > 0 {
let last: String = str_char_at(trimmed, n - 1)
if last == "{" {
let body = str_trim(str_slice(trimmed, 0, n - 1))
}
}
let id_split: Map<String, Any> = split_identifier(body)
let raw_id: String = id_split["id"]
if str_eq(raw_id, "") {
set_error("group declaration missing id", line_no, line)
return 0
}
let label_rest: String = str_trim(id_split["rest"])
let quoted: Map<String, Any> = parse_quoted(label_rest)
let label = raw_id
let qok: Bool = quoted["ok"]
if qok {
let label = quoted["value"]
}
open_group(raw_id, label, line_no)
1
}
fn parse_close_brace(line_no: Int) -> Int {
let frame: Map<String, Any> = close_group_frame()
let frame_ok: Bool = frame["ok"]
if !frame_ok {
set_error("unexpected `}` — no open group", line_no, "}")
return 0
}
store_group(frame["id"], frame["label"], frame["ids"])
1
}
fn parse_line_dispatch(line_no: Int, line: String) -> Int {
if line == "}" { return parse_close_brace(line_no) }
if str_starts_with(line, "title:") {
let after: String = str_trim(str_slice(line, 6, str_len(line)))
let q: Map<String, Any> = parse_quoted(after)
let qok: Bool = q["ok"]
if !qok {
set_error("expected quoted string after `title:`", line_no, line)
return 0
}
state_set("graph_title", q["value"])
return 1
}
if str_starts_with(line, "direction:") {
let after: String = str_trim(str_slice(line, 10, str_len(line)))
let dir: String = parse_direction(after)
if str_eq(dir, "") {
set_error("unknown direction — expected top-down, left-right, right-left, or bottom-up",
line_no, line)
return 0
}
state_set("graph_direction", dir)
return 1
}
if str_starts_with(line, "group ") {
let after: String = str_slice(line, 6, str_len(line))
return parse_group_open(line_no, line, after)
}
if is_edge_line(line) {
return parse_edge_stmt(line_no, line)
}
parse_node_stmt(line_no, line)
}
// Materialise accumulators into the final graph map
fn build_graph_value() -> Map<String, Any> {
let n_nodes: Int = st_get_int("node_count")
let nodes: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_nodes {
let s: String = int_to_str(i)
let node: Map<String, Any> = {
"id": state_get("node_id_" + s),
"label": state_get("node_label_" + s),
"shape": state_get("node_shape_" + s)
}
let nodes = native_list_append(nodes, node)
let i = i + 1
}
let n_edges: Int = st_get_int("edge_count")
let edges: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_edges {
let s: String = int_to_str(i)
let edge: Map<String, Any> = {
"from": state_get("edge_from_" + s),
"to": state_get("edge_to_" + s),
"label": state_get("edge_label_" + s),
"kind": state_get("edge_kind_" + s)
}
let edges = native_list_append(edges, edge)
let i = i + 1
}
let n_groups: Int = st_get_int("group_count")
let groups: [Map<String, Any>] = el_list_empty()
let i = 0
while i < n_groups {
let s: String = int_to_str(i)
let raw_ids: String = state_get("group_ids_" + s)
let id_list: [String] = list_decode(raw_ids)
let group: Map<String, Any> = {
"id": state_get("group_id_" + s),
"label": state_get("group_label_" + s),
"node_ids": id_list,
"direction": ""
}
let groups = native_list_append(groups, group)
let i = i + 1
}
{
"title": state_get("graph_title"),
"direction": state_get("graph_direction"),
"nodes": nodes,
"edges": edges,
"groups": groups
}
}
// Public entry point
fn arbor_parse(source: String) -> Map<String, Any> {
reset_state()
let lines: [Map<String, Any>] = preprocess(source)
let n: Int = el_list_len(lines)
let i = 0
let abort = false
while i < n {
if abort {
// skip error already recorded
} else {
let row: Map<String, Any> = get(lines, i)
let line_no: Int = row["no"]
let text: String = row["text"]
parse_line_dispatch(line_no, text)
if has_error() {
let abort = true
}
}
let i = i + 1
}
if !has_error() {
let depth: Int = st_get_int("group_stack_depth")
if depth > 0 {
let idx: Int = depth - 1
let id: String = state_get(group_frame_key(idx, "id"))
let line_no: Int = st_get_int(group_frame_key(idx, "line"))
set_error("unclosed group '" + id + "' — missing closing `}`",
line_no, "group " + id)
}
}
if has_error() {
return {
"error": state_get("parse_error"),
"line": st_get_int("parse_error_line"),
"text": state_get("parse_error_text")
}
}
build_graph_value()
}
// Smoke test
fn fail_msg(label: String, got: String, want: String) -> Int {
println("FAIL " + label + " got=[" + got + "] want=[" + want + "]")
state_set("smoke_failures", "1")
0
}
fn check_eq(label: String, got: String, want: String) -> Int {
if got == want {
println("ok " + label)
return 1
}
fail_msg(label, got, want)
}
// Helper: a graph map is in the error state iff it has a non-empty "error".
fn parse_failed(g: Map<String, Any>) -> Bool {
let m: String = g["error"]
if str_eq(m, "") { return false }
// map_get returns NULL for missing keys; str_eq treats two NULLs as equal
// and NULL vs "" as not equal guard explicitly.
if str_len(m) == 0 { return false }
true
}
let src1 = "title: \"Test\"\ndirection: left-right\n\napi [rounded] \"REST API\"\ndb [cylinder] \"Postgres\"\n\napi -> db \"reads\""
let g1: Map<String, Any> = arbor_parse(src1)
if parse_failed(g1) {
println("FAIL parse 1: " + g1["error"])
state_set("smoke_failures", "1")
}
check_eq("title parsed", g1["title"], "Test")
check_eq("direction parsed", g1["direction"], "left-right")
let nodes1: [Map<String, Any>] = g1["nodes"]
let nn1: Int = el_list_len(nodes1)
check_eq("two nodes", int_to_str(nn1), "2")
let edges1: [Map<String, Any>] = g1["edges"]
let ne1: Int = el_list_len(edges1)
check_eq("one edge", int_to_str(ne1), "1")
let e0: Map<String, Any> = get(edges1, 0)
check_eq("edge from", e0["from"], "api")
check_eq("edge to", e0["to"], "db")
check_eq("edge label", e0["label"], "reads")
check_eq("edge kind", e0["kind"], "solid")
let n0: Map<String, Any> = get(nodes1, 0)
check_eq("node 0 shape", n0["shape"], "rounded")
check_eq("node 0 label", n0["label"], "REST API")
// Test edge varieties
let src2 = "a \"A\"\nb \"B\"\na -> b\na --> b\na -/-> b\na <-> b"
let g2: Map<String, Any> = arbor_parse(src2)
let edges2: [Map<String, Any>] = g2["edges"]
check_eq("4 edges parsed", int_to_str(el_list_len(edges2)), "4")
let kinds = ""
let i = 0
while i < el_list_len(edges2) {
let e: Map<String, Any> = get(edges2, i)
let k: String = e["kind"]
let kinds = kinds + k + ","
let i = i + 1
}
check_eq("edge kinds", kinds, "solid,dashed,forbidden,bidirectional,")
// Groups
let src3 = "group core \"Application Core\" {\n api [rounded] \"REST API\"\n svc \"Business Logic\"\n}\nstandalone \"Out\""
let g3: Map<String, Any> = arbor_parse(src3)
let groups3: [Map<String, Any>] = g3["groups"]
check_eq("one group", int_to_str(el_list_len(groups3)), "1")
let grp0: Map<String, Any> = get(groups3, 0)
check_eq("group label", grp0["label"], "Application Core")
let gnids: [String] = grp0["node_ids"]
check_eq("group has 2 members", int_to_str(el_list_len(gnids)), "2")
let nodes3: [Map<String, Any>] = g3["nodes"]
check_eq("3 total nodes (incl standalone)",
int_to_str(el_list_len(nodes3)), "3")
// Error: unknown shape
let src4 = "node [hexagon] \"X\""
let g4: Map<String, Any> = arbor_parse(src4)
let err4: String = g4["error"]
if str_eq(err4, "") {
println("FAIL expected error for unknown shape")
state_set("smoke_failures", "1")
} else {
if str_contains(err4, "hexagon") {
println("ok error mentions hexagon: " + err4)
} else {
println("FAIL error wording: " + err4)
state_set("smoke_failures", "1")
}
}
// Error: unclosed group
let src5 = "group g \"G\" {\n a \"A\"\n"
let g5: Map<String, Any> = arbor_parse(src5)
let err5: String = g5["error"]
if str_eq(err5, "") {
println("FAIL expected unclosed-group error")
state_set("smoke_failures", "1")
} else {
if str_contains(err5, "unclosed") {
println("ok unclosed group detected")
} else {
println("FAIL unclosed error wording: " + err5)
state_set("smoke_failures", "1")
}
}
// Comments and inline comments
let src6 = "// header\na \"A\" // trailing\nb \"B\""
let g6: Map<String, Any> = arbor_parse(src6)
check_eq("comments stripped", int_to_str(el_list_len(g6["nodes"])), "2")
// Empty input
let g7: Map<String, Any> = arbor_parse("")
check_eq("empty graph nodes", int_to_str(el_list_len(g7["nodes"])), "0")
check_eq("empty graph default direction", g7["direction"], "top-down")
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-parse: FAILED")
exit_program(1)
} else {
println("arbor-parse: ok")
}
+21
View File
@@ -0,0 +1,21 @@
// arbor-render SVG renderer. Consumes a diagram graph + layout result and
// emits an SVG document. PNG rasterization is not provided in this vessel
// because the El runtime does not expose a vector-to-raster primitive yet
// (see report).
vessel "arbor-render" {
version "0.1.0"
description "SVG renderer for Arbor diagrams"
authors ["Neuron Technologies"]
edition "2026"
}
dependencies {
arbor-core "0.1"
arbor-layout "0.1"
}
build {
entry "src/main.el"
output "dist/"
}
+575
View File
@@ -0,0 +1,575 @@
// arbor-render SVG emission from a laid-out diagram.
//
// Entry point:
// fn arbor_render_svg(graph: Map, layout: Map, forbidden: [String]) -> String
//
// The graph is the lowered (diagram-form) shape produced by arbor-core /
// arbor-diagram (`title`, `direction`, `nodes`, `edges`, `groups`). The
// layout is whatever arbor-layout returned: `node_pos_<id>`, `node_size_<id>`,
// `group_bounds_<id>`, `node_ids`, `group_ids`, `canvas`.
//
// `forbidden` is a list of "from->to" key strings same format as
// arbor-core's collect_forbidden(). The Rust crate threaded a HashSet
// through; El threads a list and we linear-scan.
//
// SVG is text emission straightforward El. Every float coordinate is
// passed through format_float(_, 1) for stable output.
//
// PNG render is intentionally out of scope
// The Rust crate rasterises via resvg tiny_skia png. The El runtime
// today exposes no equivalent: there is no resvg, no usvg, no font rasterer,
// no PNG encoder, no path-fill code. fs_write writes text only there is
// no binary write primitive. arbor_render_png() returns an error map in El
// until the runtime grows a rasterer (see "runtime gaps" in the report).
// Colour palette (matches the Rust constants exactly)
fn col_node_fill() -> String { "#ffffff" }
fn col_node_stroke() -> String { "#334155" }
fn col_primary_fill() -> String { "#0052A0" }
fn col_primary_text() -> String { "#ffffff" }
fn col_node_text() -> String { "#0D0D14" }
fn col_edge() -> String { "#64748B" }
fn col_edge_forbidden() -> String { "#DC2626" }
fn col_group_fill() -> String { "rgba(0,0,0,0.03)" }
fn col_group_stroke() -> String { "#CBD5E1" }
fn col_group_text() -> String { "#64748B" }
fn col_edge_label() -> String { "#64748B" }
// XML escape
fn esc(s: String) -> String {
let r1: String = str_replace(s, "&", "&amp;")
let r2: String = str_replace(r1, "<", "&lt;")
let r3: String = str_replace(r2, ">", "&gt;")
let r4: String = str_replace(r3, "\"", "&quot;")
r4
}
// Float to "%.1f" the Rust pt() helper.
fn pt(v: el_val_t) -> String {
format_float(v, 1)
}
// Float arithmetic helpers float_to_int / int_to_float trip through Int,
// which is exact for the integer-valued floats used by the layout pass.
fn fadd(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai + bi)
}
fn fsub(a: el_val_t, b: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
let bi: Int = float_to_int(b)
int_to_float(ai - bi)
}
fn fdiv2(a: el_val_t) -> el_val_t {
let ai: Int = float_to_int(a)
int_to_float(ai / 2)
}
fn fmid(a: el_val_t, b: el_val_t) -> el_val_t {
fdiv2(fadd(a, b))
}
// forbidden-edge linear lookup
fn forbidden_key(from: String, to: String) -> String {
from + "->" + to
}
fn forbidden_contains(set: [String], src: String, dst: String) -> Bool {
let key: String = forbidden_key(src, dst)
let n: Int = el_list_len(set)
let i = 0
while i < n {
let s: String = get(set, i)
if str_eq(s, key) { return true }
let i = i + 1
}
false
}
// Arrow marker defs
fn arrow_defs() -> String {
let s = "\n <marker id=\"ah\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-bi\" markerWidth=\"10\" markerHeight=\"7\" refX=\"1\" refY=\"3.5\" orient=\"auto-start-reverse\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge() + "\"/>\n"
let s = s + " </marker>\n"
let s = s + " <marker id=\"ah-red\" markerWidth=\"10\" markerHeight=\"7\" refX=\"9\" refY=\"3.5\" orient=\"auto\">\n"
let s = s + " <polygon points=\"0 0, 10 3.5, 0 7\" fill=\"" + col_edge_forbidden() + "\"/>\n"
let s = s + " </marker>"
s
}
// Node rendering
fn render_node(buf: String, node: Map<String, Any>, layout: Map<String, Any>) -> String {
let nid: String = node["id"]
let pos: Map<String, Any> = el_map_get(layout, "node_pos_" + nid)
let sz: Map<String, Any> = el_map_get(layout, "node_size_" + nid)
let cx: el_val_t = pos["x"]
let cy: el_val_t = pos["y"]
let w: el_val_t = sz["w"]
let h: el_val_t = sz["h"]
let x: el_val_t = fsub(cx, fdiv2(w))
let y: el_val_t = fsub(cy, fdiv2(h))
let fill_in: String = node["style_fill"]
let stroke_in: String = node["style_stroke"]
let color_in: String = node["style_color"]
let fill = col_node_fill()
if str_len(fill_in) > 0 { let fill = fill_in }
let stroke = col_node_stroke()
if str_len(stroke_in) > 0 { let stroke = stroke_in }
let text_col = col_node_text()
if str_len(color_in) > 0 { let text_col = color_in }
let shape: String = node["shape"]
let buf = buf
if str_eq(shape, "rectangle") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"4\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "rounded_rect") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"20\" fill=\"" + fill + "\" stroke=\"" + stroke
let buf = buf + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "stadium") {
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(h)
let buf = buf + "\" rx=\"" + pt(fdiv2(h)) + "\" fill=\"" + fill
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "cylinder") {
// body: rect from y+ry to bottom; ry ≈ h/6 (Rust uses h*0.18, we use h/6
// to stay in integer arithmetic visually indistinguishable on the
// canvas sizes the layout produces).
let hi: Int = float_to_int(h)
let ry: el_val_t = int_to_float(hi / 6)
let body_y: el_val_t = fadd(y, ry)
let body_h: el_val_t = fsub(h, ry)
let buf = buf + " <rect x=\"" + pt(x) + "\" y=\"" + pt(body_y)
let buf = buf + "\" width=\"" + pt(w) + "\" height=\"" + pt(body_h)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// top ellipse
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(body_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
// bottom ellipse
let bot_y: el_val_t = fadd(y, h)
let buf = buf + " <ellipse cx=\"" + pt(cx) + "\" cy=\"" + pt(bot_y)
let buf = buf + "\" rx=\"" + pt(fdiv2(w)) + "\" ry=\"" + pt(ry)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
if str_eq(shape, "diamond") {
let hw: el_val_t = fdiv2(w)
let hh: el_val_t = fdiv2(h)
let buf = buf + " <polygon points=\""
let buf = buf + pt(cx) + "," + pt(fsub(cy, hh)) + " "
let buf = buf + pt(fadd(cx, hw)) + "," + pt(cy) + " "
let buf = buf + pt(cx) + "," + pt(fadd(cy, hh)) + " "
let buf = buf + pt(fsub(cx, hw)) + "," + pt(cy)
let buf = buf + "\" fill=\"" + fill + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\"/>\n"
}
// Label.
let label: String = node["label"]
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(cy)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\">"
let buf = buf + esc(label) + "</text>\n"
// Sublabel Rust's DiagramNode stores Option<String>; El uses "" sentinel.
let sub: String = node["sublabel"]
if str_len(sub) > 0 {
let sub_y: el_val_t = fadd(cy, int_to_float(14))
let buf = buf + " <text x=\"" + pt(cx) + "\" y=\"" + pt(sub_y)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-node-label\" fill=\"" + text_col + "\" font-size=\"10\">"
let buf = buf + esc(sub) + "</text>\n"
}
buf
}
// Edge rendering
//
// We emit a straight line from one node centre to the other and let the
// browser draw it; the Rust crate renders cubic bezier paths but the runtime
// has no robust math layer, and the rectangles are large enough that
// straight edges read clearly. (See "runtime gaps".)
fn render_edge(buf: String, edge: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let from_id: String = edge["from"]
let to_id: String = edge["to"]
let from_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + from_id)
let to_pos: Map<String, Any> = el_map_get(layout, "node_pos_" + to_id)
let fx: el_val_t = from_pos["x"]
let fy: el_val_t = from_pos["y"]
let tx: el_val_t = to_pos["x"]
let ty: el_val_t = to_pos["y"]
let is_forbidden: Bool = forbidden_contains(forbidden, from_id, to_id)
let stroke = col_edge()
if is_forbidden { let stroke = col_edge_forbidden() }
let line: String = edge["line"]
let arrow: String = edge["arrow"]
let dash_attr = ""
if str_eq(line, "dashed") { let dash_attr = " stroke-dasharray=\"5,3\"" }
if str_eq(line, "dotted") { let dash_attr = " stroke-dasharray=\"2,2\"" }
let marker_start = ""
if str_eq(arrow, "both") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
if str_eq(arrow, "backward") { let marker_start = " marker-start=\"url(#ah-bi)\"" }
let marker_end = " marker-end=\"url(#ah)\""
if is_forbidden { let marker_end = " marker-end=\"url(#ah-red)\"" }
if str_eq(arrow, "none") { let marker_end = "" }
if str_eq(arrow, "backward") { let marker_end = "" }
let buf = buf + " <line x1=\"" + pt(fx) + "\" y1=\"" + pt(fy)
let buf = buf + "\" x2=\"" + pt(tx) + "\" y2=\"" + pt(ty)
let buf = buf + "\" stroke=\"" + stroke + "\" stroke-width=\"1.5\""
let buf = buf + dash_attr + marker_start + marker_end + "/>\n"
// Forbidden marker circle-X at midpoint.
if is_forbidden {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let r: el_val_t = int_to_float(7)
let buf = buf + " <circle cx=\"" + pt(mx) + "\" cy=\"" + pt(my)
let buf = buf + "\" r=\"" + pt(r) + "\" fill=\"white\" stroke=\""
let buf = buf + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let off: el_val_t = int_to_float(4)
let buf = buf + " <line x1=\"" + pt(fsub(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fadd(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
let buf = buf + " <line x1=\"" + pt(fadd(mx, off)) + "\" y1=\"" + pt(fsub(my, off))
let buf = buf + "\" x2=\"" + pt(fsub(mx, off)) + "\" y2=\"" + pt(fadd(my, off))
let buf = buf + "\" stroke=\"" + col_edge_forbidden() + "\" stroke-width=\"1.5\"/>\n"
}
// Edge label
let label: String = edge["label"]
if str_len(label) > 0 {
let mx: el_val_t = fmid(fx, tx)
let my: el_val_t = fmid(fy, ty)
let lw: el_val_t = int_to_float(str_len(label) * 7 + 8)
let lh: el_val_t = int_to_float(16)
let buf = buf + " <rect x=\"" + pt(fsub(mx, fdiv2(lw))) + "\" y=\"" + pt(fsub(my, fdiv2(lh)))
let buf = buf + "\" width=\"" + pt(lw) + "\" height=\"" + pt(lh)
let buf = buf + "\" rx=\"3\" fill=\"white\" opacity=\"0.85\"/>\n"
let buf = buf + " <text x=\"" + pt(mx) + "\" y=\"" + pt(my)
let buf = buf + "\" text-anchor=\"middle\" dominant-baseline=\"middle\""
let buf = buf + " class=\"arbor-edge-label\">" + esc(label) + "</text>\n"
}
buf
}
// Group rendering
fn render_group(buf: String, group: Map<String, Any>, layout: Map<String, Any>) -> String {
let gid: String = group["id"]
let bounds: Map<String, Any> = el_map_get(layout, "group_bounds_" + gid)
// Layout may not have bounds for empty groups defensive.
let bx_check: el_val_t = bounds["x"]
if float_to_int(bx_check) == 0 {
// Could be a real 0; cheaper to skip via presence check on group_ids.
}
let bx: el_val_t = bounds["x"]
let by: el_val_t = bounds["y"]
let bw: el_val_t = bounds["w"]
let bh: el_val_t = bounds["h"]
let buf = buf + " <rect x=\"" + pt(bx) + "\" y=\"" + pt(by)
let buf = buf + "\" width=\"" + pt(bw) + "\" height=\"" + pt(bh)
let buf = buf + "\" rx=\"8\" fill=\"" + col_group_fill() + "\" stroke=\""
let buf = buf + col_group_stroke() + "\" stroke-width=\"1\" stroke-dasharray=\"4,3\"/>\n"
// Group label in the top-left corner.
let lx: el_val_t = fadd(bx, int_to_float(8))
let ly: el_val_t = fadd(by, int_to_float(14))
let label: String = group["label"]
let buf = buf + " <text x=\"" + pt(lx) + "\" y=\"" + pt(ly)
let buf = buf + "\" class=\"arbor-group-label\">" + esc(label) + "</text>\n"
buf
}
// Public entry point
fn arbor_render_svg(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> String {
let canvas: Map<String, Any> = el_map_get(layout, "canvas")
let cw: el_val_t = canvas["w"]
let ch: el_val_t = canvas["h"]
let buf = "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"" + pt(cw)
let buf = buf + "\" height=\"" + pt(ch) + "\" viewBox=\"0 0 " + pt(cw) + " " + pt(ch) + "\">\n"
let buf = buf + " <defs>"
let buf = buf + arrow_defs()
let buf = buf + "\n <style>\n"
let buf = buf + " .arbor-node-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 13px; }\n"
let buf = buf + " .arbor-group-label { font-family: 'Helvetica Neue', Helvetica, Arial, monospace; font-size: 10px; fill: " + col_group_text() + "; letter-spacing: 0.08em; }\n"
let buf = buf + " .arbor-edge-label { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 11px; fill: " + col_edge_label() + "; }\n"
let buf = buf + " </style>\n"
let buf = buf + " </defs>\n"
// Groups first (behind everything).
let buf = buf + " <!-- Groups -->\n"
let groups: [Map<String, Any>] = graph["groups"]
let gn: Int = el_list_len(groups)
let i = 0
while i < gn {
let g: Map<String, Any> = get(groups, i)
let gid: String = g["id"]
// Only render groups the layout actually placed.
let gids: [String] = el_map_get(layout, "group_ids")
let placed = false
let j = 0
while j < el_list_len(gids) {
if str_eq(get(gids, j), gid) { let placed = true }
let j = j + 1
}
if placed {
let buf = render_group(buf, g, layout)
}
let i = i + 1
}
// Edges
let buf = buf + " <!-- Edges -->\n"
let edges: [Map<String, Any>] = graph["edges"]
let en: Int = el_list_len(edges)
let i = 0
while i < en {
let e: Map<String, Any> = get(edges, i)
let buf = render_edge(buf, e, layout, forbidden)
let i = i + 1
}
// Nodes
let buf = buf + " <!-- Nodes -->\n"
let nodes: [Map<String, Any>] = graph["nodes"]
let nn: Int = el_list_len(nodes)
let i = 0
while i < nn {
let n: Map<String, Any> = get(nodes, i)
let buf = render_node(buf, n, layout)
let i = i + 1
}
// Title
let title: String = graph["title"]
if str_len(title) > 0 {
let title_x: el_val_t = fdiv2(cw)
let buf = buf + " <text x=\"" + pt(title_x) + "\" y=\"22\" text-anchor=\"middle\""
let buf = buf + " font-family=\"'Helvetica Neue', Helvetica, Arial, sans-serif\""
let buf = buf + " font-size=\"15\" font-weight=\"600\" fill=\"" + col_node_text() + "\">"
let buf = buf + esc(title) + "</text>\n"
}
let buf = buf + "</svg>\n"
buf
}
// PNG not implemented; the runtime has no SVG rasterizer or PNG encoder.
// Returns an error map that callers can inspect via map["error"].
fn arbor_render_png(graph: Map<String, Any>, layout: Map<String, Any>, forbidden: [String]) -> Map<String, Any> {
{
"error": "PNG rasterization not available in El runtime — install a runtime image library or use the Rust binary"
}
}
// Smoke test
fn fail(label: String, msg: String) -> Int {
println("FAIL " + label + ": " + msg)
state_set("smoke_failures", "1")
0
}
fn check_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
println("ok " + label)
return 1
}
fail(label, "missing [" + needle + "]")
}
fn check_not_contains(label: String, haystack: String, needle: String) -> Int {
if str_contains(haystack, needle) {
return fail(label, "should not contain [" + needle + "]")
}
println("ok " + label)
1
}
fn make_test_node(id: String, label: String, shape: String) -> Map<String, Any> {
{
"id": id, "label": label, "sublabel": "",
"shape": shape,
"style_fill": "", "style_stroke": "", "style_color": ""
}
}
fn make_test_edge(src: String, dst: String, line: String, arrow: String, label: String) -> Map<String, Any> {
{
"from": src, "to": dst, "label": label,
"line": line, "arrow": arrow
}
}
fn make_test_pos(x: Int, y: Int) -> Map<String, Any> {
{ "x": int_to_float(x), "y": int_to_float(y) }
}
fn make_test_size(w: Int, h: Int) -> Map<String, Any> {
{ "w": int_to_float(w), "h": int_to_float(h) }
}
// Build a minimal layout map by hand.
fn build_layout(node_ids: [String], group_ids: [String], cw: Int, ch: Int) -> Map<String, Any> {
let r: Map<String, Any> = el_map_new(0)
let r = el_map_set(r, "node_ids", node_ids)
let r = el_map_set(r, "group_ids", group_ids)
let r = el_map_set(r, "canvas", { "w": int_to_float(cw), "h": int_to_float(ch) })
r
}
let n_a: Map<String, Any> = make_test_node("a", "Node A", "rectangle")
let n_b: Map<String, Any> = make_test_node("b", "Node B", "rectangle")
let e_ab: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let nodes: [Map<String, Any>] = native_list_empty()
let nodes = native_list_append(nodes, n_a)
let nodes = native_list_append(nodes, n_b)
let edges: [Map<String, Any>] = native_list_empty()
let edges = native_list_append(edges, e_ab)
let groups: [Map<String, Any>] = native_list_empty()
let g: Map<String, Any> = {
"title": "Test", "direction": "top-down",
"nodes": nodes, "edges": edges, "groups": groups
}
let nid_list: [String] = native_list_empty()
let nid_list = native_list_append(nid_list, "a")
let nid_list = native_list_append(nid_list, "b")
let gid_list: [String] = native_list_empty()
let layout: Map<String, Any> = build_layout(nid_list, gid_list, 400, 300)
let layout = el_map_set(layout, "node_pos_a", make_test_pos(100, 60))
let layout = el_map_set(layout, "node_pos_b", make_test_pos(100, 200))
let layout = el_map_set(layout, "node_size_a", make_test_size(120, 40))
let layout = el_map_set(layout, "node_size_b", make_test_size(120, 40))
let forbidden: [String] = native_list_empty()
let svg: String = arbor_render_svg(g, layout, forbidden)
check_contains("svg starts with <svg", svg, "<svg xmlns=")
check_contains("svg ends with </svg>", svg, "</svg>")
check_contains("svg contains node label", svg, "Node A")
check_contains("svg contains title", svg, ">Test</text>")
check_contains("svg has rect for rectangle node", svg, "<rect")
check_contains("svg has line for edge", svg, "<line")
check_contains("svg has arrow marker def", svg, "id=\"ah\"")
// Escape test
let n_esc: Map<String, Any> = make_test_node("x", "A & B <C>", "rectangle")
let nodes2: [Map<String, Any>] = native_list_empty()
let nodes2 = native_list_append(nodes2, n_esc)
let g2: Map<String, Any> = {
"title": "Test <Title>", "direction": "top-down",
"nodes": nodes2, "edges": native_list_empty(), "groups": native_list_empty()
}
let nid2: [String] = native_list_empty()
let nid2 = native_list_append(nid2, "x")
let layout2: Map<String, Any> = build_layout(nid2, native_list_empty(), 200, 100)
let layout2 = el_map_set(layout2, "node_pos_x", make_test_pos(80, 40))
let layout2 = el_map_set(layout2, "node_size_x", make_test_size(120, 40))
let svg2: String = arbor_render_svg(g2, layout2, native_list_empty())
check_contains("escapes ampersand", svg2, "&amp;")
check_contains("escapes <", svg2, "&lt;")
check_not_contains("no raw <C>", svg2, "<C>")
// Forbidden edge
let e_fb: Map<String, Any> = make_test_edge("a", "b", "solid", "forward", "")
let edges3: [Map<String, Any>] = native_list_empty()
let edges3 = native_list_append(edges3, e_fb)
let g3: Map<String, Any> = {
"title": "F", "direction": "top-down",
"nodes": nodes, "edges": edges3, "groups": native_list_empty()
}
let fb: [String] = native_list_empty()
let fb = native_list_append(fb, forbidden_key("a", "b"))
let svg3: String = arbor_render_svg(g3, layout, fb)
check_contains("forbidden uses red marker", svg3, "ah-red")
check_contains("forbidden colour present", svg3, col_edge_forbidden())
// Diamond shape polygon
let n_d: Map<String, Any> = make_test_node("d", "Decide", "diamond")
let g4: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_d),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid4: [String] = native_list_append(native_list_empty(), "d")
let layout4: Map<String, Any> = build_layout(nid4, native_list_empty(), 200, 100)
let layout4 = el_map_set(layout4, "node_pos_d", make_test_pos(80, 50))
let layout4 = el_map_set(layout4, "node_size_d", make_test_size(120, 40))
let svg4: String = arbor_render_svg(g4, layout4, native_list_empty())
check_contains("diamond uses polygon", svg4, "<polygon")
// Cylinder shape ellipses
let n_cy: Map<String, Any> = make_test_node("cy", "DB", "cylinder")
let g5: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": native_list_append(native_list_empty(), n_cy),
"edges": native_list_empty(), "groups": native_list_empty()
}
let nid5: [String] = native_list_append(native_list_empty(), "cy")
let layout5: Map<String, Any> = build_layout(nid5, native_list_empty(), 200, 100)
let layout5 = el_map_set(layout5, "node_pos_cy", make_test_pos(80, 50))
let layout5 = el_map_set(layout5, "node_size_cy", make_test_size(120, 40))
let svg5: String = arbor_render_svg(g5, layout5, native_list_empty())
check_contains("cylinder uses ellipse", svg5, "<ellipse")
// Dashed edge
let e_dash: Map<String, Any> = make_test_edge("a", "b", "dashed", "forward", "")
let g6: Map<String, Any> = {
"title": "", "direction": "top-down",
"nodes": nodes, "edges": native_list_append(native_list_empty(), e_dash),
"groups": native_list_empty()
}
let svg6: String = arbor_render_svg(g6, layout, native_list_empty())
check_contains("dashed line dasharray", svg6, "stroke-dasharray=\"5,3\"")
// PNG returns an error map
let png: Map<String, Any> = arbor_render_png(g, layout, native_list_empty())
let err: String = png["error"]
if str_len(err) > 0 {
println("ok PNG returns error map")
} else {
println("FAIL PNG should have returned error")
state_set("smoke_failures", "1")
}
println("")
let f: String = state_get("smoke_failures")
if str_eq(f, "1") {
println("arbor-render: FAILED")
exit_program(1)
} else {
println("arbor-render: ok")
}
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
-763
View File
@@ -1,763 +0,0 @@
// lexer.el el self-hosting lexer
//
// Tokenises an el source string into a list of token maps.
// Each token is a Map<String, Any> with keys:
// "kind" -> String (e.g. "Int", "Ident", "Plus")
// "value" -> String (the raw text of the token)
//
// Entry point: fn lex(source: String) -> [Map<String, Any>]
//
// Uses native_string_chars to split the source into a chars list,
// then indexes it with native_list_get avoids O(N²) string cloning.
// Character helpers
fn lex_is_digit(ch: String) -> Bool {
if ch == "0" { return true }
if ch == "1" { return true }
if ch == "2" { return true }
if ch == "3" { return true }
if ch == "4" { return true }
if ch == "5" { return true }
if ch == "6" { return true }
if ch == "7" { return true }
if ch == "8" { return true }
if ch == "9" { return true }
false
}
fn lex_is_alpha(ch: String) -> Bool {
if ch == "a" { return true }
if ch == "b" { return true }
if ch == "c" { return true }
if ch == "d" { return true }
if ch == "e" { return true }
if ch == "f" { return true }
if ch == "g" { return true }
if ch == "h" { return true }
if ch == "i" { return true }
if ch == "j" { return true }
if ch == "k" { return true }
if ch == "l" { return true }
if ch == "m" { return true }
if ch == "n" { return true }
if ch == "o" { return true }
if ch == "p" { return true }
if ch == "q" { return true }
if ch == "r" { return true }
if ch == "s" { return true }
if ch == "t" { return true }
if ch == "u" { return true }
if ch == "v" { return true }
if ch == "w" { return true }
if ch == "x" { return true }
if ch == "y" { return true }
if ch == "z" { return true }
if ch == "A" { return true }
if ch == "B" { return true }
if ch == "C" { return true }
if ch == "D" { return true }
if ch == "E" { return true }
if ch == "F" { return true }
if ch == "G" { return true }
if ch == "H" { return true }
if ch == "I" { return true }
if ch == "J" { return true }
if ch == "K" { return true }
if ch == "L" { return true }
if ch == "M" { return true }
if ch == "N" { return true }
if ch == "O" { return true }
if ch == "P" { return true }
if ch == "Q" { return true }
if ch == "R" { return true }
if ch == "S" { return true }
if ch == "T" { return true }
if ch == "U" { return true }
if ch == "V" { return true }
if ch == "W" { return true }
if ch == "X" { return true }
if ch == "Y" { return true }
if ch == "Z" { return true }
false
}
fn is_alnum_or_underscore(ch: String) -> Bool {
if lex_is_digit(ch) { return true }
if lex_is_alpha(ch) { return true }
if ch == "_" { return true }
false
}
fn lex_is_whitespace(ch: String) -> Bool {
if ch == " " { return true }
if ch == "\t" { return true }
if ch == "\n" { return true }
if ch == "\r" { return true }
false
}
fn make_tok(kind: String, value: String) -> Map<String, Any> {
let ln_s: String = state_get("__lex_line")
let ln: Int = 1
if !str_eq(ln_s, "") { let ln = str_to_int(ln_s) }
{ "kind": kind, "value": value, "line": ln }
}
// Keyword lookup
fn keyword_kind(word: String) -> String {
if word == "let" { return "Let" }
if word == "fn" { return "Fn" }
if word == "type" { return "Type" }
if word == "enum" { return "Enum" }
if word == "match" { return "Match" }
if word == "return" { return "Return" }
if word == "if" { return "If" }
if word == "else" { return "Else" }
if word == "for" { return "For" }
if word == "in" { return "In" }
if word == "while" { return "While" }
if word == "import" { return "Import" }
if word == "from" { return "From" }
if word == "as" { return "As" }
if word == "with" { return "With" }
if word == "sealed" { return "Sealed" }
if word == "activate" { return "Activate" }
if word == "where" { return "Where" }
if word == "test" { return "Test" }
if word == "seed" { return "Seed" }
if word == "assert" { return "Assert" }
if word == "protocol" { return "Protocol" }
if word == "impl" { return "Impl" }
if word == "retry" { return "Retry" }
if word == "times" { return "Times" }
if word == "fallback" { return "Fallback" }
if word == "reason" { return "Reason" }
if word == "parallel" { return "Parallel" }
if word == "trace" { return "Trace" }
if word == "requires" { return "Requires" }
if word == "deploy" { return "Deploy" }
if word == "to" { return "To" }
if word == "via" { return "Via" }
if word == "target" { return "Target" }
if word == "true" { return "Bool" }
if word == "false" { return "Bool" }
if word == "cgi" { return "Cgi" }
if word == "service" { return "Service" }
if word == "manager" { return "Manager" }
if word == "engine" { return "Engine" }
if word == "accessor" { return "Accessor" }
if word == "vessel" { return "Vessel" }
if word == "extern" { return "Extern" }
if word == "try" { return "Try" }
if word == "catch" { return "Catch" }
""
}
// Scan helpers
// All scan helpers receive the chars list and total length.
// scan_digits advance i while chars[i] is a digit
// Returns { "text": ..., "pos": i }
fn scan_digits(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if lex_is_digit(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// scan_ident advance i while chars[i] is alphanumeric or underscore
fn scan_ident(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if is_alnum_or_underscore(ch) {
let parts = native_list_append(parts, ch)
let i = i + 1
} else {
let running = false
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Code-bearing string detection + comment strip
// Inline JS/CSS literals embedded in El source (e.g. <script></script> blobs
// or stylesheet payloads inside string literals) carry their own line and
// block comments. Those comments leak into the served HTML and reveal build
// notes the visitor should never see. We strip them at the lexer so every
// downstream consumer (codegen-c, codegen-js, parser) gets the cleaned form.
//
// looks_like_code heuristic gate so we only strip strings that actually
// embed JS or CSS. Plain prose, hex blobs, JSON, etc. pass through verbatim.
fn substr_at(chars: [String], start: Int, total: Int, needle: String) -> Bool {
let nchars: [String] = native_string_chars(needle)
let nlen: Int = native_list_len(nchars)
if start + nlen > total { return false }
let i = 0
let matched = true
while i < nlen {
let a: String = native_list_get(chars, start + i)
let b: String = native_list_get(nchars, i)
if a == b { let i = i + 1 } else { let matched = false; let i = nlen }
}
matched
}
fn str_has(s: String, needle: String) -> Bool {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let i = 0
let found = false
while i < total {
if substr_at(chars, i, total, needle) {
let found = true
let i = total
} else {
let i = i + 1
}
}
found
}
fn looks_like_code(s: String) -> Bool {
if str_has(s, "<script") { return true }
if str_has(s, "<style") { return true }
if str_has(s, "function") {
if str_has(s, ";") { return true }
}
false
}
// strip_code_comments character-by-character walk. Tracks JS string state
// (single, double, backtick) and never strips inside one. Backslash escapes
// inside JS strings consume the next char verbatim. URLs like https:// are
// preserved by checking the previous char before treating // as a line
// comment opener: if the char immediately before '/' is ':', emit the '/'
// literally and advance one position.
fn strip_code_comments(s: String) -> String {
let chars: [String] = native_string_chars(s)
let total: Int = native_list_len(chars)
let out_parts: [String] = native_list_empty()
let i = 0
let in_squote = false
let in_dquote = false
let in_btick = false
let prev = ""
while i < total {
let ch: String = native_list_get(chars, i)
let in_js_string = false
if in_squote { let in_js_string = true }
if in_dquote { let in_js_string = true }
if in_btick { let in_js_string = true }
if in_js_string {
// Backslash escape: consume next char verbatim regardless of which.
if ch == "\\" {
let out_parts = native_list_append(out_parts, ch)
let next_i = i + 1
if next_i < total {
let nc: String = native_list_get(chars, next_i)
let out_parts = native_list_append(out_parts, nc)
let prev = nc
let i = next_i + 1
} else {
let prev = ch
let i = next_i
}
} else {
if in_squote {
if ch == "'" { let in_squote = false }
} else {
if in_dquote {
if ch == "\"" { let in_dquote = false }
} else {
if in_btick {
if ch == "`" { let in_btick = false }
}
}
}
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
} else {
// Not in a JS string. Check for comment openers.
let next_i = i + 1
let next_ch = ""
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
}
if ch == "/" {
if next_ch == "/" {
// URL guard: prev char ':' means this is "://", not a comment.
if prev == ":" {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
// Skip until newline (newline itself is preserved so
// surrounding line counts/structure stay sane).
let i = i + 2
let scanning = true
while scanning {
if i >= total {
let scanning = false
} else {
let lc: String = native_list_get(chars, i)
if lc == "\n" {
let scanning = false
} else {
let i = i + 1
}
}
}
let prev = ""
}
} else {
if next_ch == "*" {
// Skip until matching "*/".
let i = i + 2
let scanning2 = true
while scanning2 {
if i >= total {
let scanning2 = false
} else {
let bc: String = native_list_get(chars, i)
if bc == "*" {
let after = i + 1
if after < total {
let nc2: String = native_list_get(chars, after)
if nc2 == "/" {
let i = after + 1
let scanning2 = false
} else {
let i = i + 1
}
} else {
let i = i + 1
}
} else {
let i = i + 1
}
}
}
let prev = ""
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
} else {
// Open a JS string?
if ch == "'" {
let in_squote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "\"" {
let in_dquote = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
if ch == "`" {
let in_btick = true
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
} else {
let out_parts = native_list_append(out_parts, ch)
let prev = ch
let i = i + 1
}
}
}
}
}
}
str_join(out_parts, "")
}
// scan_string scan a quoted string literal, handling \" escapes.
// Starts AFTER the opening quote. Returns { "text": content, "pos": i_after_close }
fn scan_string(chars: [String], start: Int, total: Int) -> Map<String, Any> {
let i = start
let parts: [String] = native_list_empty()
let running = true
while running {
if i >= total {
let running = false
} else {
let ch: String = native_list_get(chars, i)
if ch == "\\" {
// escape: peek next char
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "\"" {
let parts = native_list_append(parts, "\"")
let i = next_i + 1
} else {
if next_ch == "n" {
let parts = native_list_append(parts, "\n")
let i = next_i + 1
} else {
if next_ch == "t" {
let parts = native_list_append(parts, "\t")
let i = next_i + 1
} else {
if next_ch == "r" {
let parts = native_list_append(parts, "\r")
let i = next_i + 1
} else {
if next_ch == "\\" {
let parts = native_list_append(parts, "\\")
let i = next_i + 1
} else {
let parts = native_list_append(parts, next_ch)
let i = next_i + 1
}
}
}
}
}
} else {
let i = i + 1
}
} else {
if ch == "\"" {
let i = i + 1
let running = false
} else {
let parts = native_list_append(parts, ch)
let i = i + 1
}
}
}
}
{ "text": str_join(parts, ""), "pos": i }
}
// Main lexer
fn lex(source: String) -> [Map<String, Any>] {
let chars: [String] = native_string_chars(source)
let total: Int = native_list_len(chars)
let tokens: [Map<String, Any>] = native_list_empty()
let i: Int = 0
let line_num: Int = 1
state_set("__lex_line", "1")
while i < total {
let ch: String = native_list_get(chars, i)
// Skip whitespace; track newlines for line-number reporting
if lex_is_whitespace(ch) {
if ch == "\n" {
let line_num = line_num + 1
state_set("__lex_line", native_int_to_str(line_num))
}
let i = i + 1
} else {
// Line comments: //
if ch == "/" {
let next_i = i + 1
if next_i < total {
let next_ch: String = native_list_get(chars, next_i)
if next_ch == "/" {
// skip to end of line
let i = i + 2
let running2 = true
while running2 {
if i >= total {
let running2 = false
} else {
let lch: String = native_list_get(chars, i)
if lch == "\n" {
let running2 = false
} else {
let i = i + 1
}
}
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
let tokens = native_list_append(tokens, make_tok("Slash", "/"))
let i = i + 1
}
} else {
// String literal
if ch == "\"" {
let result = scan_string(chars, i + 1, total)
let str_text: String = result["text"]
let new_pos: Int = result["pos"]
// Compile-time scrub: strings that embed JS or CSS get
// their // line comments and /* block comments stripped
// before the token reaches the parser. Plain prose passes
// through untouched.
let clean_text = str_text
if looks_like_code(str_text) {
let clean_text = strip_code_comments(str_text)
}
let tokens = native_list_append(tokens, make_tok("Str", clean_text))
let i = new_pos
} else {
// Number literal
if lex_is_digit(ch) {
let result = scan_digits(chars, i, total)
let num_text: String = result["text"]
let new_pos: Int = result["pos"]
// check for float (dot followed by digit)
if new_pos < total {
let dot_ch: String = native_list_get(chars, new_pos)
if dot_ch == "." {
let after_dot = new_pos + 1
if after_dot < total {
let after_dot_ch: String = native_list_get(chars, after_dot)
if lex_is_digit(after_dot_ch) {
let frac_result = scan_digits(chars, after_dot, total)
let frac_text: String = frac_result["text"]
let frac_pos: Int = frac_result["pos"]
let tokens = native_list_append(tokens, make_tok("Float", num_text + "." + frac_text))
let i = frac_pos
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
let tokens = native_list_append(tokens, make_tok("Int", num_text))
let i = new_pos
}
} else {
// Identifier or keyword
if lex_is_alpha(ch) || ch == "_" {
let result = scan_ident(chars, i, total)
let word: String = result["text"]
let new_pos: Int = result["pos"]
let kw = keyword_kind(word)
if kw == "" {
let tokens = native_list_append(tokens, make_tok("Ident", word))
} else {
let tokens = native_list_append(tokens, make_tok(kw, word))
}
let i = new_pos
} else {
// Multi-char and single-char operators/delimiters
let peek_i = i + 1
let peek_ch = ""
if peek_i < total {
let peek_ch: String = native_list_get(chars, peek_i)
}
if ch == "=" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("EqEq", "=="))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("FatArrow", "=>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Eq", "="))
let i = i + 1
}
}
} else {
if ch == "!" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("NotEq", "!="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Not", "!"))
let i = i + 1
}
} else {
if ch == "<" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("LtEq", "<="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Lt", "<"))
let i = i + 1
}
} else {
if ch == ">" {
if peek_ch == "=" {
let tokens = native_list_append(tokens, make_tok("GtEq", ">="))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Gt", ">"))
let i = i + 1
}
} else {
if ch == "&" {
if peek_ch == "&" {
let tokens = native_list_append(tokens, make_tok("And", "&&"))
let i = i + 2
} else {
let i = i + 1
}
} else {
if ch == "|" {
if peek_ch == "|" {
let tokens = native_list_append(tokens, make_tok("Or", "||"))
let i = i + 2
} else {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("PipeOp", "|>"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Pipe", "|"))
let i = i + 1
}
}
} else {
if ch == "-" {
if peek_ch == ">" {
let tokens = native_list_append(tokens, make_tok("Arrow", "->"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Minus", "-"))
let i = i + 1
}
} else {
if ch == ":" {
if peek_ch == ":" {
let tokens = native_list_append(tokens, make_tok("ColonColon", "::"))
let i = i + 2
} else {
let tokens = native_list_append(tokens, make_tok("Colon", ":"))
let i = i + 1
}
} else {
if ch == "+" {
let tokens = native_list_append(tokens, make_tok("Plus", "+"))
let i = i + 1
} else {
if ch == "*" {
let tokens = native_list_append(tokens, make_tok("Star", "*"))
let i = i + 1
} else {
if ch == "%" {
let tokens = native_list_append(tokens, make_tok("Percent", "%"))
let i = i + 1
} else {
if ch == "(" {
let tokens = native_list_append(tokens, make_tok("LParen", "("))
let i = i + 1
} else {
if ch == ")" {
let tokens = native_list_append(tokens, make_tok("RParen", ")"))
let i = i + 1
} else {
if ch == "{" {
let tokens = native_list_append(tokens, make_tok("LBrace", "{"))
let i = i + 1
} else {
if ch == "}" {
let tokens = native_list_append(tokens, make_tok("RBrace", "}"))
let i = i + 1
} else {
if ch == "[" {
let tokens = native_list_append(tokens, make_tok("LBracket", "["))
let i = i + 1
} else {
if ch == "]" {
let tokens = native_list_append(tokens, make_tok("RBracket", "]"))
let i = i + 1
} else {
if ch == "," {
let tokens = native_list_append(tokens, make_tok("Comma", ","))
let i = i + 1
} else {
if ch == "." {
let tokens = native_list_append(tokens, make_tok("Dot", "."))
let i = i + 1
} else {
if ch == ";" {
let tokens = native_list_append(tokens, make_tok("Semicolon", ";"))
let i = i + 1
} else {
if ch == "@" {
let tokens = native_list_append(tokens, make_tok("At", "@"))
let i = i + 1
} else {
if ch == "?" {
let tokens = native_list_append(tokens, make_tok("QuestionMark", "?"))
let i = i + 1
} else {
if ch == "#" {
let tokens = native_list_append(tokens, make_tok("Hash", "#"))
let i = i + 1
} else {
// unknown char skip
let i = i + 1
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
let tokens = native_list_append(tokens, make_tok("Eof", ""))
tokens
}
File diff suppressed because it is too large Load Diff
+21
View File
@@ -0,0 +1,21 @@
# Compiled El bytecode
*.elc
# C codegen output
*.c
*.o
*.a
*.so
*.dylib
# Combined build artifacts
_combined.el
*-combined.el
# Distribution / build output
dist/
build/
out/
# OS
.DS_Store
+65
View File
@@ -0,0 +1,65 @@
# ELP language consolidation — full-lexicon backfill (stage)
Branch: `stage-elp-lang-consolidation` (stage-bound; NOT the live soul :8742).
Consolidates scattered Python language-realizer work (`~/Desktop/lang-realizers`,
`~/Desktop/lang-poetry-experiment`, `~/semitic_engine`) into the ELP `.el`
structure, generating **full lexicons** (complete UniMorph + kaikki.org
Wiktionary — real gender, real inflections) instead of the demo/curated subsets
the prototypes shipped.
## ELP before this branch
- 18 classical/ancient languages fully done (vocab + morphology + tests):
akk ang cop egy enm fro gez goh got grc non peo pi sa sga sux txb uga.
- 11 modern/classical languages had `morphology-<code>.el` in the build manifest
but **no vocabulary and no lang_profile**: es fr de ja ar he hi ru fi sw la.
- The ES port (`stage-elp-es-port`) had a *demo-scale* vocabulary-es.el (~350
entries, s-expr form).
## Landed on this branch (full-lexicon seed-fn format, matching the 18 ancients)
Vocabulary schema per row: `[lemma, pos, form0, form1, form2, en_gloss, hint]`.
Files are ELP runtime **seed data** (loaded via the Engram at runtime), so — like
all 18 classical `vocabulary-*.el` — they are intentionally NOT in the build
manifest. Syntax validated: the chunked `fn vocab_<code>_seed_pN` format
compiles cleanly to C via `elc` (correct UTF-8).
| code | in-ELP-morph? | vocab entries | verbs | nouns | adjs | profile |
|------|---------------|--------------:|------:|------:|-----:|---------|
| es | yes | 72,032 | 6,695 | 48,353 | 16,984 | yes |
| fr | yes | 130,517 | 7,534 | 77,344 | 45,639 | yes |
| de | yes | 144,692 | 6,661 | 133,162 | 4,869 | yes |
| la | yes | 22,590 | 82 | 13,436 | 9,072 | yes |
| it | no (bonus) | 193,675 | 10,008 | 109,459 | 74,208 | yes |
| pt | no (bonus) | 115,772 | 4,001 | 72,073 | 39,698 | yes |
| ro | no (bonus) | 86,504 | 1,216 | 65,915 | 19,373 | yes |
| ca | no (bonus) | 47,112 | 1,547 | 28,830 | 16,735 | yes |
|**total**| |**812,894** | | | | |
Generators (reproducible): `elp/tests/lang-gen/gen_elp_seed_full.py` (Romance),
`gen_elp_seed_de_la.py` (German declension + Latin case-paradigm mapping). They
read the pre-built morph caches in `~/Desktop/lang-realizers/data/` (UniMorph +
kaikki), which are too large to commit.
## Remaining (honest)
Of the 11 ELP backfill targets, 4 are done (es fr de la). The other 7 have **no
full-lexicon engine** yet — cannot be generated honestly without engine work:
- **ru**: only a 110-entry curated Slavic subset exists; full `rus.unimorph`
present but no `morphology_ru_full` productive loader. Needs a full Russian
morphology module (like the Romance ones) before vocab generation.
- **ja / ko / zh**: validated demo engines (~66-104 hardcoded words) in
`lang-poetry-experiment`, Python only. Agglutinative (ja/ko) + isolating (zh)
need `.el` engine ports + full-lexicon wiring (ja: jpn_unimorph; zh: CC-CEDICT).
- **ar / he (Semitic)**: template engines (16 AR / 8 HE patterns, ~6 roots) in
`~/semitic_engine`, Python only. Root-and-pattern; full UniMorph ara/heb
present but used only for validation. Needs productive root lexicon + `.el` port.
- **hi (Hindi), fi (Finnish), sw (Swahili)**: `morphology-<code>.el` exists in
ELP but there is NO scattered prototype and NO downloaded data for these —
full-lexicon collection (UniMorph/kaikki) + generator still to do.
De/nl/sv Germanic and it/ro/ca/pt Romance verb coverage note: German verbs here
are the ~6.6k caches carry; the it/ro/ca/pt bonus languages have full vocab but
**no `morphology-<code>.el` in ELP yet** (Python realizer exists; `.el` port is
the remaining engine work).
Construction coverage (separate from lexicon): French realizer was ~55%,
Semitic ~3% in the prototypes — full construction coverage remains its own task.
File diff suppressed because one or more lines are too long
+23
View File
@@ -0,0 +1,23 @@
{
"dataset": "british-rp-accent-transform",
"primitive_type": "accent_target",
"accent": "british-rp",
"grounding": "derived",
"provenance": "HONEST-DERIVED, COARSE FIRST PASS — NOT transcribed measured RP formants. The exact measured RP/GB tables (Deterding 1997 JIPA 27:47-55; Hawkins & Midgley 2005 JIPA 35:183-199) are the intended ground truth but were gated/figure-only at author time and were NOT transcribed. So these targets are DERIVED: each = the corresponding MEASURED Peterson&Barney(1952) base vowel transformed under the documented, citable RP-vs-GA structural rules of Wells (1982) 'Accents of English' — non-rhoticity (NURSE de-rhoticized: remove low F3), TRAP F2-lowering, LOT/THOUGHT back-rounding (F2 down), GOOSE-fronting (F2 up), GOAT centering. Shift MAGNITUDES are coarse/approximate (first pass), directions are cited. ground:derived (base measured + rule cited). Refine by transcribing Deterding/Hawkins&Midgley. No number is presented as a measured RP value it is not.",
"notes": "records with kind=vowel_override REPLACE the base phoneme's formant targets with the DERIVED RP realization. records with kind=rule encode non-formant transforms (non-rhoticity: drop post-vocalic coda /r/). The render composes: base geometry then accent override + rhoticity rule — voice + accent, separable.",
"records": [
{"key": "IY", "features": {"kind": "vowel_override", "set": "FLEECE"}, "attributes": {"f1": 280, "f2": 2249, "f3": 3000}},
{"key": "IH", "features": {"kind": "vowel_override", "set": "KIT"}, "attributes": {"f1": 360, "f2": 2100, "f3": 2550}},
{"key": "EH", "features": {"kind": "vowel_override", "set": "DRESS"}, "attributes": {"f1": 560, "f2": 1970, "f3": 2480}},
{"key": "AE", "features": {"kind": "vowel_override", "set": "TRAP"}, "attributes": {"f1": 730, "f2": 1590, "f3": 2410}},
{"key": "AA", "features": {"kind": "vowel_override", "set": "LOT"}, "attributes": {"f1": 560, "f2": 920, "f3": 2440}},
{"key": "AO", "features": {"kind": "vowel_override", "set": "THOUGHT"}, "attributes": {"f1": 415, "f2": 700, "f3": 2410}},
{"key": "UH", "features": {"kind": "vowel_override", "set": "FOOT"}, "attributes": {"f1": 380, "f2": 1100, "f3": 2240}},
{"key": "UW", "features": {"kind": "vowel_override", "set": "GOOSE"}, "attributes": {"f1": 310, "f2": 1650, "f3": 2240}},
{"key": "AH", "features": {"kind": "vowel_override", "set": "STRUT"}, "attributes": {"f1": 680, "f2": 1180, "f3": 2390}},
{"key": "ER", "features": {"kind": "vowel_override", "set": "NURSE", "rhotic": "no"}, "attributes": {"f1": 550, "f2": 1500, "f3": 2500}},
{"key": "AX", "features": {"kind": "vowel_override", "set": "commA"}, "attributes": {"f1": 500, "f2": 1500, "f3": 2500}},
{"key": "OW", "features": {"kind": "vowel_override", "set": "GOAT"}, "attributes": {"f1": 450, "f2": 1400, "f3": 2380}},
{"key": "R", "features": {"kind": "rule", "rule": "non_rhotic"}, "attributes": {"drop_coda_r": 1}}
]
}
+26
View File
@@ -0,0 +1,26 @@
# british-rp-accent TRANSFORM — INGESTIBLE DATA (a geometry/transform composed
# onto the base General-American phoneme targets; voice + accent, separable).
#
# PROVENANCE — HONEST, COARSE FIRST PASS. These are DERIVED targets, NOT
# transcribed measured RP formants. Measured RP tables (Deterding 1997 JIPA 27;
# Hawkins & Midgley 2005 JIPA 35) are the intended ground truth but were gated at
# author time and NOT transcribed. Each target = the MEASURED Peterson&Barney
# (1952) base vowel transformed under the documented, citable RP-vs-GA structural
# rules of Wells (1982): non-rhoticity, TRAP F2-lowering, LOT/THOUGHT back-
# rounding, GOOSE-fronting, GOAT centering, NURSE de-rhoticization. Shift
# magnitudes are coarse/approximate; directions are cited. ground=derived.
# Refine by transcribing the measured RP tables. No value is claimed as measured.
# Format: KEY|F1|F2|F3|KIND|SET
IY|280|2249|3000|vowel_override|FLEECE
IH|360|2100|2550|vowel_override|KIT
EH|560|1970|2480|vowel_override|DRESS
AE|730|1590|2410|vowel_override|TRAP
AA|560|920|2440|vowel_override|LOT
AO|415|700|2410|vowel_override|THOUGHT
UH|380|1100|2240|vowel_override|FOOT
UW|310|1650|2240|vowel_override|GOOSE
AH|680|1180|2390|vowel_override|STRUT
ER|550|1500|2500|vowel_override|NURSE-nonrhotic
AX|500|1500|2500|vowel_override|commA
OW|450|1400|2380|vowel_override|GOAT
R|0|0|0|rule|non_rhotic_drop_coda
+20
View File
@@ -0,0 +1,20 @@
# pronunciation lexicon SOURCE — word -> phoneme sequence, as INGESTIBLE DATA.
# Pronunciation is linguistic KNOWLEDGE (the language faculty's orthography->
# phonology map), ingested into the engram, not frozen in code. The render reads
# a word's phoneme sequence back from the engram. Covers the self-lexicon and the
# proof sentences; general G2P is the realizer/morphology faculty's remit.
# Diphthongs are written as two vowel targets (the render's transitions glide
# between them). Format: word|PH1 PH2 PH3 ...
i|AA IY
am|AE M
neuron|N UW R AA N
is|IH Z
memory|M EH M ER IY
hello|HH EH L OW
the|DH AH
a|AH
remember|R IH M EH M ER
i'm|AA IY M
you|Y UW
here|HH IY R
will|W IH L
File diff suppressed because one or more lines are too long
+528
View File
@@ -0,0 +1,528 @@
{
"dataset": "english-phoneme-formants",
"primitive_type": "phoneme",
"grounding": "extracted",
"provenance": "AUDITED per-field. The 10 monophthong-vowel F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER) are the MEASURED adult-male /hVd/ means of Peterson & Barney (1952) JASA 24:175-184, verified vs CRAN phonTools::pb52. AX=neutral uniform-tube resonances (Fant, physics). OW steady target = synthesis convention (diphthong). Consonant loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL bandwidths + dur/amp = standard formant-synthesis conventions (Klatt 1980 JASA 67:971), engineering defaults NOT field measurements. No numbers invented/LLM-generated.",
"records": [
{
"key": "IY",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "IH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 390,
"f2": 1990,
"f3": 2550,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "EH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 530,
"f2": 1840,
"f3": 2480,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 130,
"amp": 100
}
},
{
"key": "AE",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 660,
"f2": 1720,
"f3": 2410,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AA",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 730,
"f2": 1090,
"f3": 2440,
"bw1": 90,
"bw2": 110,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 150,
"amp": 100
}
},
{
"key": "AO",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 570,
"f2": 840,
"f3": 2410,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "UH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 440,
"f2": 1020,
"f3": 2240,
"bw1": 70,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 100
}
},
{
"key": "UW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 870,
"f3": 2240,
"bw1": 70,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "AH",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 640,
"f2": 1190,
"f3": 2390,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 110,
"amp": 95
}
},
{
"key": "ER",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1690,
"bw1": 80,
"bw2": 100,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 95
}
},
{
"key": "AX",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "OW",
"features": {
"manner": "vowel",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 910,
"f3": 2380,
"bw1": 80,
"bw2": 100,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 140,
"amp": 100
}
},
{
"key": "M",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 900,
"f3": 2200,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "N",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 1700,
"f3": 2600,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "NG",
"features": {
"manner": "nasal",
"voiced": "yes",
"nasal": "yes"
},
"attributes": {
"f1": 250,
"f2": 2300,
"f3": 2700,
"bw1": 90,
"bw2": 120,
"bw3": 180,
"voiced": 1,
"nasal": 1,
"dur": 80,
"amp": 60
}
},
{
"key": "L",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 360,
"f2": 1300,
"f3": 2600,
"bw1": 80,
"bw2": 110,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "R",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 490,
"f2": 1350,
"f3": 1600,
"bw1": 80,
"bw2": 110,
"bw3": 120,
"voiced": 1,
"nasal": 0,
"dur": 80,
"amp": 85
}
},
{
"key": "W",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 610,
"f3": 2200,
"bw1": 70,
"bw2": 100,
"bw3": 160,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 80
}
},
{
"key": "Y",
"features": {
"manner": "approximant",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 270,
"f2": 2290,
"f3": 3010,
"bw1": 60,
"bw2": 90,
"bw3": 150,
"voiced": 1,
"nasal": 0,
"dur": 60,
"amp": 80
}
},
{
"key": "Z",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1700,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 90,
"amp": 55
}
},
{
"key": "DH",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1400,
"f3": 2500,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "V",
"features": {
"manner": "fricative",
"voiced": "yes",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1000,
"f3": 2300,
"bw1": 100,
"bw2": 150,
"bw3": 200,
"voiced": 1,
"nasal": 0,
"dur": 70,
"amp": 55
}
},
{
"key": "S",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 320,
"f2": 1700,
"f3": 2500,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 110,
"amp": 45
}
},
{
"key": "F",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 300,
"f2": 1200,
"f3": 2400,
"bw1": 200,
"bw2": 200,
"bw3": 250,
"voiced": 0,
"nasal": 0,
"dur": 100,
"amp": 40
}
},
{
"key": "HH",
"features": {
"manner": "fricative",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 200,
"bw2": 250,
"bw3": 300,
"voiced": 0,
"nasal": 0,
"dur": 70,
"amp": 40
}
},
{
"key": "SIL",
"features": {
"manner": "silence",
"voiced": "no",
"nasal": "no"
},
"attributes": {
"f1": 500,
"f2": 1500,
"f3": 2500,
"bw1": 100,
"bw2": 100,
"bw3": 100,
"voiced": 0,
"nasal": 0,
"dur": 55,
"amp": 0
}
}
]
}
+45
View File
@@ -0,0 +1,45 @@
# acoustic-phonetics SOURCE — the learned speech primitives, as INGESTIBLE DATA.
# NOT audio, NOT code: formant geometry of the phonemes, to be ingested via the
# ingest organ into the engram as a phoneme manifold. The render reads this
# geometry back from the engram; nothing is frozen in EL code.
#
# PROVENANCE (audited, per-field honesty — no invented numbers):
# * The 10 MONOPHTHONG VOWEL formants F1/F2/F3 (IY,IH,EH,AE,AA,AO,UH,UW,AH,ER)
# are the MEASURED adult-male means of Peterson & Barney (1952), JASA 24:175-184
# — the canonical /hVd/ table, verified digit-for-digit vs CRAN phonTools::pb52.
# These are real measured values.
# * AX (schwa) F1/F2/F3 = neutral uniform-tube resonances (2n-1)*500 — a PHYSICS
# value (Fant), not a P&B measurement.
# * OW is a diphthong; its listed steady target is a conventional synthesis value,
# not a P&B monophthong measurement.
# * CONSONANT loci (M,N,NG,L,R,W,Y,Z,DH,V,S,F,HH) and ALL BANDWIDTHS (B1,B2,B3)
# and dur/amp are STANDARD FORMANT-SYNTHESIS conventions (Klatt 1980, JASA 67:971
# "Software for a cascade/parallel formant synthesizer") — engineering defaults,
# NOT per-phoneme field measurements. Labeled as such, not attributed to P&B.
# Format: SYM|F1|F2|F3|B1|B2|B3|voiced|nasal|dur_ms|amp|class|example
IY|270|2290|3010|60|90|150|1|0|130|100|vowel|beet
IH|390|1990|2550|70|100|150|1|0|110|100|vowel|bit
EH|530|1840|2480|80|100|150|1|0|130|100|vowel|bet
AE|660|1720|2410|90|110|150|1|0|150|100|vowel|bat
AA|730|1090|2440|90|110|150|1|0|150|100|vowel|bot
AO|570|840|2410|80|100|150|1|0|140|100|vowel|bought
UH|440|1020|2240|70|100|150|1|0|110|100|vowel|book
UW|300|870|2240|70|90|150|1|0|140|100|vowel|boot
AH|640|1190|2390|80|100|150|1|0|110|95|vowel|but
ER|490|1350|1690|80|100|120|1|0|140|95|vowel|bird
AX|500|1500|2500|80|100|150|1|0|80|85|vowel|about
OW|490|910|2380|80|100|150|1|0|140|100|vowel|boat
M|250|900|2200|90|120|180|1|1|80|60|nasal|map
N|250|1700|2600|90|120|180|1|1|80|60|nasal|nap
NG|250|2300|2700|90|120|180|1|1|80|60|nasal|sing
L|360|1300|2600|80|110|160|1|0|70|80|approximant|lip
R|490|1350|1600|80|110|120|1|0|80|85|approximant|rip
W|300|610|2200|70|100|160|1|0|70|80|approximant|wet
Y|270|2290|3010|60|90|150|1|0|60|80|approximant|yet
Z|300|1700|2500|100|150|200|1|0|90|55|fricative|zoo
DH|300|1400|2500|100|150|200|1|0|70|55|fricative|the
V|300|1000|2300|100|150|200|1|0|70|55|fricative|van
S|320|1700|2500|200|200|250|0|0|110|45|fricative|see
F|300|1200|2400|200|200|250|0|0|100|40|fricative|fee
HH|500|1500|2500|200|250|300|0|0|70|40|fricative|hat
SIL|500|1500|2500|100|100|100|0|0|55|0|silence|_
+90
View File
@@ -0,0 +1,90 @@
package "elp" {
version "0.7.0"
description "Engram Language Protocol — bidirectional engine mapping between Engram semantic forms and natural language surface text. 31 languages."
edition "2026"
}
build {
entry "src/elp.el"
// Compilation order (dependency order):
// language-profile (no deps)
// vocabulary (no deps)
// morphology (depends on: language-profile)
// morphology-es (depends on: morphology) Spanish
// morphology-fr (depends on: morphology) French
// morphology-de (depends on: morphology) German
// morphology-ru (depends on: morphology) Russian
// morphology-ja (depends on: morphology) Japanese
// morphology-fi (depends on: morphology) Finnish
// morphology-ar (depends on: morphology) Arabic
// morphology-hi (depends on: morphology) Hindi
// morphology-sw (depends on: morphology) Swahili
// morphology-la (depends on: morphology) Latin
// morphology-he (depends on: morphology) Hebrew
// morphology-grc (depends on: morphology) Ancient Greek
// morphology-ang (depends on: morphology) Old English
// morphology-sa (depends on: morphology) Sanskrit
// morphology-got (depends on: morphology) Gothic
// morphology-non (depends on: morphology) Old Norse
// morphology-enm (depends on: morphology) Middle English
// morphology-pi (depends on: morphology) Pali
// morphology-fro (depends on: morphology) Old French
// morphology-goh (depends on: morphology) Old High German
// morphology-sga (depends on: morphology) Old Irish
// morphology-txb (depends on: morphology) Tocharian B
// morphology-peo (depends on: morphology) Old Persian
// morphology-akk (depends on: morphology) Akkadian
// morphology-uga (depends on: morphology) Ugaritic
// morphology-egy (depends on: morphology) Ancient Egyptian
// morphology-sux (depends on: morphology) Sumerian
// morphology-gez (depends on: morphology) Ge'ez (Classical Ethiopic)
// morphology-cop (depends on: morphology) Coptic (Sahidic)
// grammar (depends on: language-profile)
// realizer (depends on: morphology, grammar, language-profile)
// semantics (depends on: grammar, realizer, language-profile)
// elp (depends on: semantics, realizer)
sources [
"src/language-profile.el",
"src/vocabulary.el",
"src/morphology.el",
"src/morphology-es.el",
"src/morphology-fr.el",
"src/morphology-de.el",
"src/morphology-ru.el",
"src/morphology-ja.el",
"src/morphology-fi.el",
"src/morphology-ar.el",
"src/morphology-hi.el",
"src/morphology-sw.el",
"src/morphology-la.el",
"src/morphology-he.el",
"src/morphology-grc.el",
"src/morphology-ang.el",
"src/morphology-sa.el",
"src/morphology-got.el",
"src/morphology-non.el",
"src/morphology-enm.el",
"src/morphology-pi.el",
"src/morphology-fro.el",
"src/morphology-goh.el",
"src/morphology-sga.el",
"src/morphology-txb.el",
"src/morphology-peo.el",
"src/morphology-akk.el",
"src/morphology-uga.el",
"src/morphology-egy.el",
"src/morphology-sux.el",
"src/morphology-gez.el",
"src/morphology-cop.el",
"src/grammar.el",
"src/realizer.el",
"src/semantics.el",
"src/comprehend.el",
"src/propositions.el",
"src/multilingual.el",
"src/self_region.el",
"src/dialogue.el",
"src/elp.el",
]
}
+91
View File
@@ -0,0 +1,91 @@
> **STATUS: STAGING / PROOF-OF-SHAPE — not the deliverable.** This Python package
> proved the architecture end-to-end against the proven realizer faculty (faithful
> md/docx/midi from real geometry: 0 ungrounded claims, SACRED polarity). Per Will's
> steer, the DELIVERABLE is NATIVE: the seam lives on the existing EL realizer as
> **surface-as-profile** — see `../src/surface-profile.el` and
> `../tests/examples/surface-profile-demo.el` (compiles + runs through elc → C →
> binary). The concepts below (one geometry-carrying frame; surface = a pluggable
> profile; plan/realize; deterministic-from-meaning) are exactly what the native
> module implements. Keep this package as the validated proof; build native.
# Efferent Multimodal Projector
**geometry → any surface, faithfully.** Neuron's own document-generation faculty:
the efferent twin of the ingest organ. Ingest is afferent (world → geometry);
this is efferent (geometry → an arbitrary-format document / any modality).
Built against the **proven** realizer faculty (neuron-talk sidecar `:8756`,
artifact `art-7affa557`). The live soul (`:8742` / `:7770`) is contacted **only**
through the read-only, GET-only `engram_client` — never mutated.
## The pipeline (surface-agnostic)
```
geometry region + surface/format spec
→ PLAN (manifold → document skeleton/DAG; the geometry IS the outline) plan.py
→ REALIZE (proven realizer, scaled sentence → passage, each section faithful) realize.py
→ COHERE (document-level flow / transitions, not stitched sentences) cohere.py
→ EMIT (pluggable SurfaceProjector → the target surface) projectors/
```
**The surface is a PARAMETER.** `pipeline.build_ir(...)` builds ONE
surface-neutral `DocumentIR` (`document_ir.py`); `pipeline.emit(doc, surface)`
projects it to whichever surface you name. Markdown, docx, and MIDI are the same
IR emitted three ways.
## The pivot: a geometry-carrying IR
`DocumentIR` is **not** a text tree. Every `Block` carries BOTH:
- `.sentences` — realized faithful text (what **text** projectors read),
- `.provenance` — the source geometry: `subj_id / relation / obj / polarity /
confidence / importance / salience / node_id` (what **music / image / video**
projectors read).
That single decision is what makes the projector multimodal: text renders the
words; music/image decode the geometry. A claim with no provenance cannot exist
in the IR — faithfulness is structural.
## The one shared seam
`projectors/base.py` — `SurfaceProjector.project(frame: DocumentIR) -> bytes`
(+ `surface / media_type / ext / modality / profile`). Register with
`register()`. Adding a surface changes nothing upstream.
`TwoStageProjector` blesses the peer plan/realize decomposition:
`spec = plan(frame)`, `bytes = realize(spec)`, `project = realize∘plan`; the
`profile` is the pluggable per-surface knob (text lang-profile, music
instr/mode-profile). `projectors/midi.py` is the reference two-stage impl.
## Surfaces
| surface | modality | status | emitter |
|---|---|---|---|
| `markdown` | text | landed | own (str) |
| `docx` | text | landed | own minimal OOXML (stdlib `zipfile`+XML, no lib) |
| `midi` | audio | landed (symbolic-music proof) | own minimal SMF (stdlib `struct`, no lib) |
| `audio` (WAV) | audio | peer agent (additive synth) | conforms to `TwoStageProjector` |
| `image` | image | documented seam | `projectors/seams.py` |
| `video` | video | documented seam (image×sound×time) | `projectors/seams.py` |
Music maps: relation → scale degree (same relation → same pitch), **polarity →
major/minor third (SACRED negation is audible)**, confidence → duration,
importance → velocity, section → register. Deterministic projection from meaning
— nothing invented.
## Faithfulness
`provenance.py` audits the IR: **zero** ungrounded claims, SACRED polarity
preserved (negations reported, never dropped), COHERE introduces no new geometry
(connectives are marked). `trace_table()` emits the geometry → section → claim
table.
## Run
```bash
PY=~/Desktop/lang-realizers/venv/bin/python
PYTHONPATH=~/Desktop/neuron-talk:~/Desktop/lang-realizers $PY generate.py
# writes ./out/{neuron-self,engram-temporal}.{md,docx,mid} + *.audit.json + *.provenance.md
```
Requires the proven realizer env (spaCy + the neuron-talk/lang-realizers engine)
and the read-only engram at `:8742`.
+79
View File
@@ -0,0 +1,79 @@
"""cohere.py — COHERE stage: document-level flow, not stitched sentences.
Fidelity is REALIZE's job; FLOW is this stage's. The hard part beyond sentence
fidelity is that a document must read as one thing. We add connective tissue at
the passage level:
* an opening abstract that names what the document covers (built ONLY from the
section headings that already exist — it introduces no new claim),
* a short transition lead into each section after the first, drawn from a
fixed set of discourse connectives ("Beyond that,", "Relatedly,", ...) that
carry no propositional content,
* ordering so the highest-grounded section leads.
CRITICAL: every connective is marked ``kind="connective"`` in its provenance, so
the faithfulness audit can prove COHERE introduced ZERO new geometry claims. A
transition is discourse glue, never a fact.
"""
from __future__ import annotations
from document_ir import Block, DocumentIR, Provenance
# discourse connectives — pure flow, no propositional content
_TRANSITIONS = [
"Beyond that,", "Relatedly,", "In the same region,", "From there,",
"Alongside this,", "Further,", "Turning to the next facet,",
]
def _connective_prov() -> Provenance:
return Provenance(subj_id=None, subject=None, relation="", obj=None,
polarity="aff", confidence=1.0, node_id=None,
kind="connective")
def _abstract_block(doc: DocumentIR) -> Block:
"""A grounded opening: names the sections, asserts nothing new."""
headings = [s.heading for s in doc.sections]
if not headings:
return Block(role="lead")
if len(headings) == 1:
body = f"This document, generated from Neuron's geometry, covers {headings[0]}."
else:
listed = ", ".join(headings[:-1]) + f", and {headings[-1]}"
body = ("This document is projected directly from Neuron's meaning-geometry. "
f"It traces {listed}.")
b = Block(role="lead")
b.sentences.append(body)
b.provenance.append(_connective_prov())
return b
def cohere_document(doc: DocumentIR, *, add_abstract: bool = True,
add_transitions: bool = True) -> DocumentIR:
"""Order sections by grounding, add abstract + transitions (flow only)."""
# order: strongest-grounded section (mean confidence x #claims) first,
# but keep an explicitly-first section if the plan pinned one via level 1.
def _score(sec):
provs = [p for p in sec.all_provenance() if p.kind == "fact"]
if not provs:
return 0.0
mean_conf = sum(p.confidence for p in provs) / len(provs)
return mean_conf * len(provs)
doc.sections.sort(key=_score, reverse=True)
if add_transitions:
for i, sec in enumerate(doc.sections):
if i == 0 or not sec.blocks:
continue
lead = _TRANSITIONS[(i - 1) % len(_TRANSITIONS)]
first = sec.blocks[0]
if first.sentences:
# prepend the connective to the first sentence (flow, no new claim)
first.sentences[0] = f"{lead} {first.sentences[0][0].lower()}{first.sentences[0][1:]}"
if add_abstract:
doc.meta["abstract"] = _abstract_block(doc)
return doc
+111
View File
@@ -0,0 +1,111 @@
"""document_ir.py — the surface-neutral, GEOMETRY-CARRYING document intermediate.
This is the pivot of the whole efferent projector. A DocumentIR is NOT a text
tree. It is a projection of a meaning-geometry region that carries, at every
leaf, BOTH:
* the realized surface text (``Block.sentences``) — what a TEXT projector reads,
* the source geometry (``Block.provenance``) — what a MUSIC / IMAGE /
VIDEO projector reads.
Because the IR holds the geometry, not just the words, the SAME
plan -> realize -> cohere pipeline drives every surface. A markdown projector
renders the sentences; a music projector reads the provenance edges (salience,
importance, polarity, relation) and maps them onto a symbolic-music surface;
an image/video projector (documented seam) would read the same geometry.
Nothing in this module invents content. Every :class:`Provenance` points at a
real engram node id and a real relation. That is the faithfulness contract made
structural: a claim with no provenance cannot exist in the IR.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
# --------------------------------------------------------------------------- #
# Provenance — the geometry an emitted claim traces to. FAITHFULNESS is here.
# --------------------------------------------------------------------------- #
@dataclass
class Provenance:
"""One geometry edge behind one realized claim.
``kind`` distinguishes a FACT (a structural edge asserted by the geometry,
spoken as fact) from an INTERPRETATION (something attributed, spoken with
attribution) — the facts-as-facts + interpretations-attributed discipline
(memory 80927e26). ``polarity`` is SACRED: a negated edge stays negated.
"""
subj_id: str | None # source engram node id of the subject
subject: str | None # normalized subject surface
relation: str # predicate lemma (e.g. "use", "contain", "be")
obj: str | None # normalized object / complement surface
polarity: str = "aff" # "aff" | "neg" (SACRED — never silently flipped)
confidence: float = 0.0 # extraction confidence in [0,1]
node_id: str | None = None # engram node the claim was extracted from
kind: str = "fact" # "fact" | "interpretation"
importance: float = 0.0 # source node importance (drives music/emphasis)
salience: float = 0.0 # source node salience
def trace(self) -> str:
arrow = "-->" if self.polarity == "aff" else "--NOT-->"
return (f"[{(self.node_id or '?')[:8]}] {self.subject!r} {arrow}"
f"{self.relation} {self.obj!r} (conf {self.confidence:.2f})")
@dataclass
class Block:
"""A passage: one or more faithful sentences + the geometry they trace to.
``sentences`` and ``provenance`` are index-aligned where possible: sentence
``i`` was realized from ``provenance[i]``. A COHERE transition sentence with
no new geometry carries a provenance whose ``kind == "connective"`` so the
audit can see it introduced no new claim.
"""
sentences: list[str] = field(default_factory=list)
provenance: list[Provenance] = field(default_factory=list)
role: str = "body" # "body" | "lead" | "transition"
def text(self) -> str:
return " ".join(s.rstrip(". ") + "." for s in self.sentences if s.strip())
@dataclass
class Section:
heading: str
level: int = 2 # markdown heading level / outline depth
blocks: list[Block] = field(default_factory=list)
seed_ids: list[str] = field(default_factory=list) # geometry nodes of section
summary: str = "" # one-line grounded gloss (for pptx bullets / TOC)
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for b in self.blocks:
out.extend(b.provenance)
return out
@dataclass
class DocumentIR:
"""The surface-neutral document. Built ONCE, projected to ANY surface."""
title: str
subtitle: str = ""
sections: list[Section] = field(default_factory=list)
seed_id: str | None = None # the geometry region root
format_spec: dict[str, Any] = field(default_factory=dict) # requested shape
meta: dict[str, Any] = field(default_factory=dict)
# -- geometry facets (what non-text projectors consume) ----------------- #
def all_provenance(self) -> list[Provenance]:
out: list[Provenance] = []
for s in self.sections:
out.extend(s.all_provenance())
return out
def claim_count(self) -> int:
return sum(1 for p in self.all_provenance() if p.kind in ("fact", "interpretation"))
def ungrounded_count(self) -> int:
"""Claims with no traceable node — MUST be zero for a faithful doc."""
return sum(1 for p in self.all_provenance()
if p.kind in ("fact", "interpretation") and not p.node_id)
+81
View File
@@ -0,0 +1,81 @@
"""generate.py — drive the projector: one geometry region -> many surfaces.
Proves the thesis with REAL output: builds ONE surface-neutral DocumentIR from
Neuron's OWN self-geometry (read-only against the live soul via the proven
faculty), then EMITS it to Markdown, docx, and MIDI — the same plan/realize/
cohere, three surfaces. Writes the files + the faithfulness audit to ./out/.
"""
from __future__ import annotations
import json
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
import pipeline # noqa: E402
import provenance # noqa: E402
from geometry import load_self_region # noqa: E402
OUT = os.path.join(_HERE, "out")
def _emit_all(doc, stem):
"""Emit one IR to every text/audio surface + audit + provenance."""
for surface in ("markdown", "docx", "midi"):
data = pipeline.emit(doc, surface)
proj = pipeline.get_projector(surface)
path = os.path.join(OUT, f"{stem}.{proj.ext}")
with open(path, "wb") as f:
f.write(data)
print(f" emitted {surface:9s} -> {os.path.basename(path)} ({len(data)} bytes)")
a = provenance.audit(doc)
with open(os.path.join(OUT, f"{stem}.audit.json"), "w") as f:
json.dump(a, f, indent=2)
with open(os.path.join(OUT, f"{stem}.provenance.md"), "w") as f:
f.write(provenance.trace_table(doc))
print(" audit:", {k: a[k] for k in ("claims", "ungrounded_claims",
"negations_preserved", "distinct_source_nodes", "faithful")})
return a
def main():
os.makedirs(OUT, exist_ok=True)
print("surfaces registered:", pipeline.available_surfaces())
# ---- Document 1: Neuron's self-description (marquee) ------------------- #
print("\n[1] Neuron self-description")
region = load_self_region(max_nodes=9)
print(" self region:", region)
doc1 = pipeline.build_ir(
None, region=region,
title="Neuron: A Self-Description from Its Own Geometry",
subtitle="Projected efferently from the engram — every claim traces a node.",
format_spec={"genre": "self-description", "register": "expository"},
max_sections=5, conf_floor=0.6)
print(f" IR: {len(doc1.sections)} sections, {doc1.claim_count()} claims, "
f"ungrounded={doc1.ungrounded_count()}")
_emit_all(doc1, "neuron-self")
# ---- Document 2: a coherent, clean whitepaper-style section ------------ #
print("\n[2] Whitepaper-style section (coherent clean region)")
doc2, _ = pipeline.project(
["chronoception", "time", "awareness", "engram", "temporal"],
surface="markdown",
title="Temporal Awareness in the Engram",
subtitle="A section projected from the geometry of chronoception.",
format_spec={"genre": "whitepaper-section", "register": "technical"},
max_sections=4)
print(f" IR: {len(doc2.sections)} sections, {doc2.claim_count()} claims, "
f"ungrounded={doc2.ungrounded_count()}")
_emit_all(doc2, "engram-temporal")
# echo both markdowns so they are visible in the run log
for stem, doc in (("neuron-self", doc1), ("engram-temporal", doc2)):
print(f"\n===== GENERATED MARKDOWN — {stem} =====\n")
print(pipeline.emit(doc, "markdown").decode())
if __name__ == "__main__":
main()
+129
View File
@@ -0,0 +1,129 @@
"""geometry.py — READ-ONLY loader for a meaning-geometry region.
The efferent projector never writes to the soul. This module reaches the
geometry through the PROVEN, read-only neuron-talk faculty (``engram_client``,
GET-only, which physically refuses non-GET methods) against the running sidecar
soul. The live daemon :8742 / :7770 is contacted ONLY through that read-only
client — never mutated.
A "region" is a seed node plus a bounded neighborhood: the manifold that will
become the document's skeleton. We pool a few single-term lexical searches
(the engram search is a single-term matcher) and, when available, walk one hop
of reified neighbors, then rank by self/importance signal.
"""
from __future__ import annotations
import os
import sys
# Wire in the proven faculty (own-the-core: we reuse it, we do not fork it).
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
from engram_client import ReadOnlyEngramClient # noqa: E402
class Region:
"""A geometry region: ranked nodes + the reified edges among them."""
def __init__(self, seed: str, nodes: list[dict], edges: list[dict]):
self.seed = seed
self.nodes = nodes # ranked engram node dicts
self.edges = edges # [{src, dst, edge, ...}]
self.by_id = {n["id"]: n for n in nodes if n.get("id")}
def __repr__(self):
return f"<Region seed={self.seed!r} nodes={len(self.nodes)} edges={len(self.edges)}>"
def _prose_quality(content: str) -> float:
"""Reward clean expository prose; penalize shouty banner-dense nodes.
A high ALLCAPS-word ratio or very short content signals a banner/telegraphic
memory node that extracts into garbage. Clean declarative prose scores high.
"""
if not content or not content.strip():
return 0.0
words = content.split()
if len(words) < 8:
return 0.1
caps = sum(1 for w in words if len(w) > 2 and w.strip(".,:;'\"-").isupper())
caps_ratio = caps / max(1, len(words))
# sentences with lowercase interior words read as prose
lower = sum(1 for w in words if w[:1].islower())
lower_ratio = lower / max(1, len(words))
return max(0.0, 1.2 * lower_ratio - 2.0 * caps_ratio)
def _relevance(content: str, terms: list[str]) -> float:
"""Topical relevance to the seed terms — keeps a region ON-THEME so a clean
but off-topic node cannot hijack the document."""
if not terms:
return 0.0
low = (content or "").lower()
hits = sum(1 for t in terms if t.lower() in low)
return hits / max(1, len(terms))
def _node_rank(n: dict, terms: list[str] | None = None) -> float:
return (float(n.get("importance") or 0.0) * 2.0
+ float(n.get("salience") or 0.0)
+ 1.5 * _prose_quality(n.get("content") or "")
+ 2.0 * _relevance(n.get("content") or "", terms or [])
+ (0.5 if (n.get("content") or "").strip() else 0.0))
def load_region(seed_terms: list[str] | str, *, client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10, per_term: int = 20, hop: bool = True) -> Region:
"""Pull a bounded geometry region around ``seed_terms`` (read-only).
``seed_terms`` may be a single string or several probe terms; results are
pooled and de-duplicated. When ``hop`` and the reified neighbor endpoint is
live, one hop of neighbors is folded in so the region is a real
neighborhood, not just a keyword hit list.
"""
client = client or ReadOnlyEngramClient()
if isinstance(seed_terms, str):
seed_terms = [seed_terms]
pool: dict[str, dict] = {}
for term in seed_terms:
for n in client.search(term, limit=per_term):
if isinstance(n, dict) and n.get("id"):
pool.setdefault(n["id"], n)
ranked = sorted(pool.values(), key=lambda n: _node_rank(n, seed_terms),
reverse=True)
nodes = ranked[:max_nodes]
edges: list[dict] = []
if hop and nodes:
present = {n["id"] for n in nodes}
for n in list(nodes):
try:
for nb in client.neighbors(n["id"]):
node = nb.get("node") if isinstance(nb, dict) else None
edge = nb.get("edge") if isinstance(nb, dict) else None
if node and node.get("id"):
edges.append({"src": n["id"], "dst": node["id"],
"edge": edge})
# fold a strong neighbor into the region (bounded)
if (node["id"] not in present and len(nodes) < max_nodes + 6
and _node_rank(node, seed_terms) > 0.4):
present.add(node["id"])
nodes.append(node)
except Exception: # noqa: BLE001 — read-only best-effort; never fatal
continue
return Region(seed=", ".join(seed_terms), nodes=nodes, edges=edges)
def load_self_region(client: ReadOnlyEngramClient | None = None,
max_nodes: int = 10) -> Region:
"""The self/identity region — Neuron's own geometry, for self-description."""
return load_region(["self", "identity", "Neuron", "values", "memory",
"imprint", "consciousness"],
client=client, max_nodes=max_nodes)
+67
View File
@@ -0,0 +1,67 @@
"""pipeline.py — the Efferent Multimodal Projector, top level.
geometry region + surface/format spec
-> PLAN (manifold -> document skeleton/DAG)
-> REALIZE (proven realizer, sentence -> passage, each section faithful)
-> COHERE (document-level flow / transitions, not stitched sentences)
-> EMIT (pluggable SurfaceProjector -> the target surface)
THE SURFACE IS A PARAMETER. ``project(...)`` builds the geometry-carrying
DocumentIR once, then hands it to whichever surface projector the caller named.
Markdown, docx, and midi (music) are all the SAME IR emitted differently. That
is the efferent multimodal projector: geometry -> any surface.
"""
from __future__ import annotations
import os
import sys
_HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, _HERE)
sys.path.insert(0, os.path.join(_HERE, "projectors"))
from cohere import cohere_document # noqa: E402
from document_ir import DocumentIR # noqa: E402
from geometry import Region, load_region # noqa: E402
from plan import plan_document # noqa: E402
from realize import realize_document # noqa: E402
# registering the projectors (import for side-effect: each self-registers)
import projectors.markdown # noqa: E402,F401
import projectors.docx # noqa: E402,F401
import projectors.midi # noqa: E402,F401
import projectors.seams # noqa: E402,F401
from projectors.base import available_surfaces, get_projector # noqa: E402
def build_ir(seed_terms, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
region: Region | None = None,
max_sections: int = 8, conf_floor: float = 0.55) -> DocumentIR:
"""geometry -> PLAN -> REALIZE -> COHERE = the surface-neutral DocumentIR."""
region = region or load_region(seed_terms)
doc = plan_document(region, title=title, subtitle=subtitle,
format_spec=format_spec or {},
conf_floor=conf_floor, max_sections=max_sections)
doc = realize_document(doc)
doc = cohere_document(doc)
return doc
def emit(doc: DocumentIR, surface: str) -> bytes:
"""EMIT: project the built IR onto one surface (surface = a parameter)."""
return get_projector(surface).project(doc)
def project(seed_terms, *, surface: str, title: str, subtitle: str = "",
format_spec: dict | None = None, region: Region | None = None,
max_sections: int = 8) -> tuple[DocumentIR, bytes]:
"""The full efferent projection: geometry + surface -> (IR, bytes)."""
doc = build_ir(seed_terms, title=title, subtitle=subtitle,
format_spec=format_spec, region=region,
max_sections=max_sections)
return doc, emit(doc, surface)
__all__ = ["build_ir", "emit", "project", "available_surfaces",
"get_projector", "load_region", "DocumentIR"]
+192
View File
@@ -0,0 +1,192 @@
"""plan.py — PLAN stage: geometry region -> document skeleton (a DAG/outline).
The manifold becomes the skeleton. We extract faithful propositions from the
region's nodes (the proven neuron-talk extractor, SACRED polarity preserved),
apply a quality floor, then GROUP them into sections. Grouping is by source
node — each engram node is one coherent topic, so one salient node becomes one
section. The section ORDER is the node ranking (importance/salience): the
geometry decides the outline, not a template.
Output: a DocumentIR whose sections carry seed node ids and empty blocks. REALIZE
fills the blocks; the plan owns the structure.
"""
from __future__ import annotations
import os
import re
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import propositions # noqa: E402 (the proven, faithful extractor)
from document_ir import DocumentIR, Section # noqa: E402
from geometry import Region # noqa: E402
# --------------------------------------------------------------------------- #
# Proposition quality — keep only clean, well-grounded claims.
# --------------------------------------------------------------------------- #
_JUNK_RE = re.compile(r"[.][a-z]{1,3}\b|[^A-Za-z0-9 '\-]") # ".o", stray symbols
def _has_banner_token(s: str) -> bool:
"""True if any word is an ALLCAPS banner token (DHARMA, ENGRAM, MEASURED)."""
for w in (s or "").split():
core = w.strip(".,:;'\"-")
if len(core) > 2 and core.isupper():
return True
return False
def _clean_prop(p, floor: float) -> bool:
if p.confidence < floor:
return False
if not p.subject or not (p.object or (p.obj_np is not None)):
return False
subj = (p.subject or "").strip()
obj = (p.object or "").strip()
if len(subj) < 2:
return False
# banner-derived shouty fragments read as garbage in prose
if _has_banner_token(subj) or _has_banner_token(obj):
return False
if propositions._is_shouty(p.sentence or ""):
return False
# junk tokens: file-extension fragments (".o"), stray non-word symbols
if _JUNK_RE.search(subj) or _JUNK_RE.search(obj):
return False
# a proposition whose object repeats the subject is usually a parse artifact
if obj and subj.lower() == obj.lower():
return False
# a bare copula with no real complement ("X is it") reads as noise
if p.predicate == "be" and obj.lower() in ("it", "no", "nothing", "empty", ""):
return False
return True
def _dedup(props):
"""Drop duplicate claims. Two axes: (a) identical (pred,obj,polarity), and
(b) same (subject,predicate) — which collapses a mis-split compound like
"detection is post-hoc eval" -> "Detection is post/hoc/eval" into one claim
(keep the highest-confidence surface)."""
props = sorted(props, key=lambda p: p.confidence, reverse=True)
seen_po, seen_sp, out = set(), set(), []
for p in props:
subj = (p.subject or "").lower()
po = (p.predicate, (p.object or "").lower(), p.polarity)
sp = (subj, p.predicate, p.polarity)
if po in seen_po or sp in seen_sp:
continue
seen_po.add(po)
seen_sp.add(sp)
out.append(p)
return out
# --------------------------------------------------------------------------- #
# Heading derivation — a clean human heading from a node.
# --------------------------------------------------------------------------- #
_HEADING_RE = re.compile(r"^\s*#{1,4}\s+(.{2,70})\s*$", re.M)
# node-type / system labels that are NOT topical headings
_NONTOPIC_LABEL = re.compile(r"^(memory|node|knowledge|doc|session)[:/]", re.I)
def _titlecase_banner(s: str) -> str:
"""A shouty banner ("CHRONOCEPTION — SCALE-INVARIANCE") makes a fine title
once Title-cased. Keep short acronyms uppercase."""
def fix(w):
core = w.strip("—-:,.")
if len(core) <= 3 and core.isupper():
return w # acronym
return w.capitalize()
return " ".join(fix(w) for w in s.split())
def _clean_heading(text: str) -> str | None:
"""First line only, no markdown, capped, banner Title-cased. None if unusable."""
if not text:
return None
line = text.strip().splitlines()[0]
line = re.sub(r"^#+\s*", "", line).strip().strip("#").strip()
# cut at a natural break so a long banner heading stays a heading, not a para
for sep in ("", " ", ": ", ". "):
if sep in line and len(line) > 48:
line = line.split(sep)[0].strip()
break
if not (3 <= len(line) <= 64):
return None
if propositions._is_shouty(line):
line = _titlecase_banner(line)
return line or None
def _heading_for(node: dict, fallback: str) -> str:
label = (node.get("label") or "").strip()
content = node.get("content") or ""
candidates: list[str] = []
# a node-type label ("memory:remembered") is never a topic — skip it
if label and not _NONTOPIC_LABEL.match(label):
candidates.append(label)
m = _HEADING_RE.search(content)
if m:
candidates.append(m.group(1))
# the leading banner/first sentence of the content is often the real title
first = re.split(r"(?<=[.\n])", content.strip(), maxsplit=1)[0] if content.strip() else ""
candidates.append(first)
for c in candidates:
h = _clean_heading(c)
if h:
return h
return fallback
def plan_document(region: Region, *, title: str, subtitle: str = "",
format_spec: dict | None = None,
conf_floor: float = 0.55,
max_sections: int = 8,
max_claims_per_section: int = 6) -> DocumentIR:
"""Region -> DocumentIR skeleton. The geometry dictates the outline."""
format_spec = format_spec or {}
doc = DocumentIR(title=title, subtitle=subtitle,
seed_id=region.nodes[0]["id"] if region.nodes else None,
format_spec=format_spec)
made = 0
seen_headings: set[str] = set()
for node in region.nodes:
if made >= max_sections:
break
props = propositions.extract(node.get("content") or "",
node_id=node.get("id"),
node_importance=float(node.get("importance") or 0.0),
max_sentences=10)
props = [p for p in props if _clean_prop(p, conf_floor)]
props = _dedup(props)
props.sort(key=lambda p: p.confidence, reverse=True)
props = props[:max_claims_per_section]
if not props:
continue
heading = _heading_for(node, fallback=f"Region {made + 1}")
# cross-section dedup: a topic appears once. Distinguish by top claim
# subject, else drop the collision so the outline stays clean.
if heading.lower() in seen_headings:
subj = (props[0].subject or "").strip().title()
alt = f"{heading}: {subj}" if subj and subj.lower() not in heading.lower() else None
if alt and alt.lower() not in seen_headings and len(alt) <= 64:
heading = alt
else:
continue
seen_headings.add(heading.lower())
sec = Section(heading=heading, level=2, seed_ids=[node["id"]])
# stash the planned propositions on the section for REALIZE
sec.__dict__["_planned_props"] = props
sec.__dict__["_node"] = node
doc.sections.append(sec)
made += 1
return doc
+106
View File
@@ -0,0 +1,106 @@
"""base.py — the SurfaceProjector interface + registry.
THE key abstraction of the efferent projector: a projector is a pure function
from the surface-neutral, geometry-carrying DocumentIR to bytes on a target
SURFACE. The surface is a PARAMETER. Adding a surface = registering one more
projector; nothing upstream (plan/realize/cohere) changes.
DocumentIR --project--> bytes (per surface)
A TEXT projector reads ``block.sentences``. A NON-TEXT projector (music, image,
video) reads ``block.provenance`` — the geometry the IR carries — and decodes it
onto its surface. Both consume the SAME IR. That symmetry is the whole design:
the realizer generalizes into a multimodal projector, geometry -> any surface.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
@runtime_checkable
class SurfaceProjector(Protocol):
"""Geometry-document -> one surface. Implementations MUST be pure & faithful.
THE ONE SHARED SEAM. Every surface — text, music, image, video — conforms to
this single contract:
project(frame: DocumentIR) -> bytes
where ``frame`` is the geometry-carrying meaning-geometry (the SemFrame at
document scale; a single utterance is the degenerate one-section frame).
RECOMMENDED INTERNAL SHAPE (the peer music/text decomposition, blessed here
so all surfaces share it): a projector may split ``project`` into
spec = self.plan(frame) # meaning-geometry -> surface-specific spec
bytes = self.realize(spec) # spec -> surface, via this projector's PROFILE
``project`` is then ``realize(plan(frame))``. The PROFILE (a text lang-profile,
a music instr/mode-profile, an image layout-profile) is a property of the
projector instance — the pluggable knob. See :class:`TwoStageProjector`.
A TEXT projector's plan reads ``frame`` sentences; a MUSIC/IMAGE projector's
plan reads ``frame.all_provenance()`` — the geometry — and derives its spec
(pitch/harmony/rhythm, or layout) FROM the meaning, deterministically. Same
frame, different profile.
"""
surface: str # "markdown" | "docx" | "midi" | "audio" | "image" | "video"
media_type: str # MIME type of the emitted bytes
ext: str # file extension (no dot)
modality: str # "text" | "audio" | "image" | "video"
profile: object # the pluggable per-surface profile (may be None)
def project(self, doc: DocumentIR) -> bytes:
"""Emit the document on this surface. Returns raw bytes."""
...
class TwoStageProjector:
"""Optional base for the peer plan()/realize() decomposition.
Subclasses implement ``plan(frame) -> spec`` and ``realize(spec) -> bytes``;
``project`` is their composition. This is exactly the peer music interface
(spec = plan(frame, profile); surface = realize(spec, profile)) expressed so
that it still satisfies the single ``SurfaceProjector.project`` seam. Text,
music, and image projectors can all subclass this and remain interchangeable.
"""
surface: str = ""
media_type: str = ""
ext: str = ""
modality: str = ""
profile: object = None
def plan(self, doc: DocumentIR): # -> spec
raise NotImplementedError
def realize(self, spec) -> bytes:
raise NotImplementedError
def project(self, doc: DocumentIR) -> bytes:
return self.realize(self.plan(doc))
_REGISTRY: dict[str, SurfaceProjector] = {}
def register(projector: SurfaceProjector) -> SurfaceProjector:
_REGISTRY[projector.surface] = projector
return projector
def get_projector(surface: str) -> SurfaceProjector:
if surface not in _REGISTRY:
raise KeyError(f"no projector registered for surface {surface!r}; "
f"have {sorted(_REGISTRY)}")
return _REGISTRY[surface]
def available_surfaces() -> list[str]:
return sorted(_REGISTRY)
+113
View File
@@ -0,0 +1,113 @@
"""docx.py — the .docx surface projector: an OWN minimal OOXML emitter.
Own-the-core: a .docx is just a ZIP of a few XML parts (WordprocessingML). We
emit it with the standard library only — ``zipfile`` + string XML — no
python-docx, no external dependency. This proves a "richer structured format"
surface without importing anyone else's toolkit.
Parts emitted (the minimal valid set + a styles part for real headings):
[Content_Types].xml
_rels/.rels
word/_rels/document.xml.rels
word/styles.xml (Title / Heading1 / Heading2 / Normal)
word/document.xml (the content)
Like the markdown projector it reads only the IR's realized sentences; it
invents nothing. The surface differs, the faithful content does not.
"""
from __future__ import annotations
import io
import os
import sys
import zipfile
from xml.sax.saxutils import escape
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
_CONTENT_TYPES = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">
<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>
<Default Extension="xml" ContentType="application/xml"/>
<Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/>
<Override PartName="/word/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml"/>
</Types>"""
_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
</Relationships>"""
_DOC_RELS = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>
</Relationships>"""
_W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
_STYLES = f"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:styles xmlns:w="{_W}">
<w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/>
<w:rPr><w:sz w:val="22"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Title"><w:name w:val="Title"/>
<w:pPr><w:spacing w:after="240"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="52"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Subtitle"><w:name w:val="Subtitle"/>
<w:rPr><w:i/><w:sz w:val="28"/><w:color w:val="555555"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading1"><w:name w:val="heading 1"/>
<w:pPr><w:spacing w:before="240" w:after="120"/><w:outlineLvl w:val="0"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="34"/></w:rPr></w:style>
<w:style w:type="paragraph" w:styleId="Heading2"><w:name w:val="heading 2"/>
<w:pPr><w:spacing w:before="200" w:after="100"/><w:outlineLvl w:val="1"/></w:pPr>
<w:rPr><w:b/><w:sz w:val="28"/></w:rPr></w:style>
</w:styles>"""
def _para(text: str, style: str | None = None) -> str:
ppr = f"<w:pPr><w:pStyle w:val=\"{style}\"/></w:pPr>" if style else ""
return (f"<w:p>{ppr}<w:r><w:t xml:space=\"preserve\">"
f"{escape(text)}</w:t></w:r></w:p>")
class DocxProjector:
surface = "docx"
media_type = ("application/vnd.openxmlformats-officedocument."
"wordprocessingml.document")
ext = "docx"
modality = "text"
def _document_xml(self, doc: DocumentIR) -> str:
body: list[str] = [_para(doc.title, "Title")]
if doc.subtitle:
body.append(_para(doc.subtitle, "Subtitle"))
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
body.append(_para(abstract.text()))
for sec in doc.sections:
style = "Heading1" if sec.level <= 1 else "Heading2"
body.append(_para(sec.heading, style))
for block in sec.blocks:
t = block.text()
if t:
body.append(_para(t))
return (f"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>"
f"<w:document xmlns:w=\"{_W}\"><w:body>"
+ "".join(body)
+ "<w:sectPr><w:pgSz w:w=\"12240\" w:h=\"15840\"/>"
"<w:pgMar w:top=\"1440\" w:right=\"1440\" w:bottom=\"1440\" "
"w:left=\"1440\"/></w:sectPr></w:body></w:document>")
def project(self, doc: DocumentIR) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as z:
z.writestr("[Content_Types].xml", _CONTENT_TYPES)
z.writestr("_rels/.rels", _RELS)
z.writestr("word/_rels/document.xml.rels", _DOC_RELS)
z.writestr("word/styles.xml", _STYLES)
z.writestr("word/document.xml", self._document_xml(doc))
return buf.getvalue()
register(DocxProjector())
+45
View File
@@ -0,0 +1,45 @@
"""markdown.py — the Markdown surface projector (text facet).
The most tractable surface, and the reference implementation: reads the IR's
realized sentences and lays them out as Markdown. Introduces no content — it is
pure typography over the faithful text the realizer produced.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class MarkdownProjector:
surface = "markdown"
media_type = "text/markdown"
ext = "md"
modality = "text"
def render_str(self, doc: DocumentIR) -> str:
lines: list[str] = [f"# {doc.title}"]
if doc.subtitle:
lines.append(f"\n*{doc.subtitle}*")
abstract = doc.meta.get("abstract")
if abstract is not None and abstract.sentences:
lines.append("")
lines.append(abstract.text())
for sec in doc.sections:
lines.append("")
lines.append(f"{'#' * max(2, sec.level)} {sec.heading}")
for block in sec.blocks:
body = block.text()
if body:
lines.append("")
lines.append(body)
return "\n".join(lines) + "\n"
def project(self, doc: DocumentIR) -> bytes:
return self.render_str(doc).encode("utf-8")
register(MarkdownProjector())
+133
View File
@@ -0,0 +1,133 @@
"""midi.py — the MUSIC surface projector: geometry -> symbolic music (MIDI).
The first NON-TEXT surface, and the proof of the general shape. "Music is
language and it is math" (Will): symbolic music is tractable and geometry-native,
so it is the natural efferent twin to try first after text.
CRUCIALLY this projector does NOT read the realized sentences. It reads the IR's
GEOMETRY facet — ``block.provenance`` — and DECODES each edge onto a musical
surface. That is the whole thesis of the multimodal projector: the same
geometry-carrying IR drives text AND music; a text projector reads the words, a
music projector reads the meaning-geometry. The mapping is deterministic and
faithful to the geometry's structure:
relation lemma -> scale degree (same relation -> same pitch class;
meaning has a consistent sonic form)
polarity -> mode (aff = major third above; neg = minor
third / lowered — SACRED polarity is
audible, a negated edge sounds negated)
confidence -> note duration (stronger grounding rings longer)
importance -> velocity (more important source = louder)
section -> phrase + register shift (structure becomes musical form)
Own-the-core: a Standard MIDI File is a header chunk + a track chunk of
delta-timed events. We emit the raw bytes with ``struct`` — no external MIDI
library. Format 0, one track.
"""
from __future__ import annotations
import io
import os
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR, Provenance # noqa: E402
from projectors.base import TwoStageProjector, register # noqa: E402
_TICKS = 480 # ticks per quarter note
_C_MAJOR = [0, 2, 4, 5, 7, 9, 11] # semitone offsets of a diatonic scale
def _vlq(n: int) -> bytes:
"""MIDI variable-length quantity encoding of a delta time."""
if n == 0:
return b"\x00"
out = bytearray()
out.append(n & 0x7F)
n >>= 7
while n:
out.insert(0, (n & 0x7F) | 0x80)
n >>= 7
return bytes(out)
def _degree_for(relation: str) -> int:
"""Stable scale degree for a relation lemma (same relation -> same pitch)."""
if not relation:
return 0
return sum(ord(c) for c in relation.lower()) % len(_C_MAJOR)
def _note_for(p: Provenance, base: int) -> tuple[int, int, int]:
"""(pitch, velocity, duration_ticks) for one geometry edge."""
root = base + _C_MAJOR[_degree_for(p.relation)]
# polarity -> mode: affirmed edges take the bright major third, negated edges
# take the darker minor third. The negation is AUDIBLE and never dropped.
third = 4 if p.polarity == "aff" else 3
pitch = max(24, min(96, root + (third if p.confidence >= 0.5 else 0)))
velocity = int(56 + 60 * min(1.0, max(0.0, p.importance)))
velocity = max(40, min(120, velocity))
# confidence -> duration: quarter .. dotted-half
dur = int(_TICKS * (0.5 + 1.5 * min(1.0, max(0.0, p.confidence))))
return pitch, velocity, dur
# a mode-profile: the pluggable musical knob (the peer's mode_profile). Scale +
# tempo. Swapping this profile re-voices the SAME geometry — surface as parameter.
_DEFAULT_PROFILE = {"scale": _C_MAJOR, "tempo_us": 500000,
"registers": [60, 55, 64, 50, 67, 48], "program": 0}
class MidiProjector(TwoStageProjector):
"""geometry -> symbolic music, in the shared two-stage shape.
``plan(frame)`` -> a music_spec: an ordered list of note dicts derived
deterministically from the frame's provenance geometry
(the peer's ``plan(frame, profile) -> spec``).
``realize(spec)`` -> Standard MIDI File bytes (the peer's
``realize(spec, profile) -> surface``; here the surface
is symbolic MIDI, the minimal audio proof — a richer
additive-synth audio projector conforms identically).
"""
surface = "midi"
media_type = "audio/midi"
ext = "mid"
modality = "audio"
def __init__(self, profile: dict | None = None):
self.profile = profile or _DEFAULT_PROFILE
# -- stage 1: meaning-geometry -> music_spec (reads the GEOMETRY facet) -- #
def plan(self, doc: DocumentIR) -> list[dict]:
registers = self.profile["registers"]
spec: list[dict] = []
for si, sec in enumerate(doc.sections):
base = registers[si % len(registers)]
provs = [p for p in sec.all_provenance()
if p.kind in ("fact", "interpretation")]
for i, p in enumerate(provs):
pitch, vel, dur = _note_for(p, base)
spec.append({"pitch": pitch, "velocity": vel, "dur": dur,
"rest_before": (_TICKS // 2) if (si > 0 and i == 0) else 0,
"relation": p.relation, "polarity": p.polarity})
return spec
# -- stage 2: music_spec -> MIDI bytes (own-core, no library) ------------ #
def realize(self, spec: list[dict]) -> bytes:
ev = bytearray()
ev += _vlq(0) + b"\xFF\x51\x03" + struct.pack(">I", self.profile["tempo_us"])[1:]
ev += _vlq(0) + bytes([0xC0, self.profile["program"] & 0x7F])
for note in spec:
ev += _vlq(note["rest_before"]) + bytes([0x90, note["pitch"], note["velocity"]])
ev += _vlq(note["dur"]) + bytes([0x80, note["pitch"], 0])
ev += _vlq(0) + b"\xFF\x2F\x00"
track = bytes(ev)
buf = io.BytesIO()
buf.write(b"MThd" + struct.pack(">IHHH", 6, 0, 1, _TICKS))
buf.write(b"MTrk" + struct.pack(">I", len(track)) + track)
return buf.getvalue()
register(MidiProjector())
+60
View File
@@ -0,0 +1,60 @@
"""seams.py — documented efferent seams for IMAGE and VIDEO surfaces.
These are NOT implemented (per the build rails: architect, do not overbuild).
They are registered as first-class seams so the interface PROVES it accepts
future non-text projectors without any upstream change. Each documents exactly
what its decoder would read from the geometry-carrying IR, making the multimodal
generalization concrete rather than hand-wavy.
The symmetry that guarantees these are possible, not moonshots: they are the
efferent twins of multimodal INGEST. If meaning can HOLD an image (ingest as
first-class geometry), meaning can PROJECT one back. Video = image x sound x
TIME, and the engram already stores time (chronoception). So video falls out of
an image projector + the music projector + the stored temporal ordering.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from document_ir import DocumentIR # noqa: E402
from projectors.base import register # noqa: E402
class _Seam:
"""A registered-but-unimplemented projector. Names its decoder contract."""
def project(self, doc: DocumentIR) -> bytes: # pragma: no cover - seam
raise NotImplementedError(
f"{self.surface!r} projector is a documented seam, not yet built. "
f"Decoder contract: {self.decoder_contract}")
class ImageProjector(_Seam):
surface = "image"
media_type = "image/png"
ext = "png"
modality = "image"
decoder_contract = (
"reads block.provenance as a spatial layout — nodes become regions, edges "
"become adjacencies; salience/importance drive size/contrast; polarity "
"drives figure/ground. The efferent twin of image ingest (a geometry->raster "
"decoder, learned or engineered), exactly mirroring the embedder that turned "
"the image INTO geometry.")
class VideoProjector(_Seam):
surface = "video"
media_type = "video/mp4"
ext = "mp4"
modality = "video"
decoder_contract = (
"image x sound x TIME. Composes the image projector (per-keyframe geometry "
"layout) with the midi/music projector (score) along the geometry's stored "
"temporal ordering (chronoception). Needs no new principle once image + music "
"exist — only a muxer.")
register(ImageProjector())
register(VideoProjector())
+63
View File
@@ -0,0 +1,63 @@
"""provenance.py — the faithfulness audit + geometry->section trace.
A document projected from geometry is only worth anything if every claim traces
back. This module walks the DocumentIR and proves the discipline held:
* ZERO ungrounded claims (every fact/interpretation has a real node id),
* every emitted sentence maps to a geometry edge (or is a marked connective),
* SACRED polarity survived (negations are reported, never silently dropped),
* COHERE introduced no new geometry (connectives carry no claim).
It emits both a machine verdict and a human-readable geometry->section table.
"""
from __future__ import annotations
from document_ir import DocumentIR
def audit(doc: DocumentIR) -> dict:
provs = doc.all_provenance()
facts = [p for p in provs if p.kind in ("fact", "interpretation")]
connectives = [p for p in provs if p.kind == "connective"]
ungrounded = [p for p in facts if not p.node_id]
negations = [p for p in facts if p.polarity == "neg"]
node_ids = sorted({p.node_id for p in facts if p.node_id})
return {
"claims": len(facts),
"connectives": len(connectives),
"ungrounded_claims": len(ungrounded),
"negations_preserved": len(negations),
"distinct_source_nodes": len(node_ids),
"faithful": len(ungrounded) == 0,
"source_nodes": node_ids,
}
def trace_table(doc: DocumentIR) -> str:
"""Human-readable geometry -> section -> claim provenance table."""
lines = ["# Provenance — every claim traces geometry", ""]
lines.append(f"**Document:** {doc.title}")
a = audit(doc)
lines.append(f"**Claims:** {a['claims']} · **Ungrounded:** "
f"{a['ungrounded_claims']} · **Negations preserved:** "
f"{a['negations_preserved']} · **Source nodes:** "
f"{a['distinct_source_nodes']} · **Faithful:** "
f"{'YES' if a['faithful'] else 'NO'}")
lines.append("")
for si, sec in enumerate(doc.sections, 1):
lines.append(f"## {si}. {sec.heading}")
lines.append(f"_seed nodes: {', '.join(i[:8] for i in sec.seed_ids)}_")
lines.append("")
lines.append("| # | realized claim | traces geometry edge |")
lines.append("|---|----------------|----------------------|")
n = 0
for block in sec.blocks:
for sent, prov in zip(block.sentences, block.provenance):
if prov.kind == "connective":
continue
n += 1
edge = prov.trace().replace("|", "\\|")
s = sent.replace("|", "\\|")
lines.append(f"| {n} | {s} | {edge} |")
lines.append("")
return "\n".join(lines) + "\n"
+112
View File
@@ -0,0 +1,112 @@
"""realize.py — REALIZE stage: fill each planned section with faithful passages.
Scales the PROVEN realizer from a single assertion to a passage. For each
planned proposition we build a realizer-ready clause (the proven
``_prop_to_clause`` mapping) and run it through the proven engine
(``engine.realize``), which is a deterministic grammar with the SACRED negation
contract — it never invents. Each realized sentence is paired with a
:class:`Provenance` that pins it to the exact geometry edge it came from.
"Passage, not a list of sentences": within a section we lightly vary sentence
openings and group related claims, but we add NO content the geometry did not
assert. The only non-geometry words are function words the grammar already owns
(articles, "and", conjunction of same-subject claims). Document-level flow is
COHERE's job; this stage owns intra-section fluency + fidelity.
"""
from __future__ import annotations
import os
import sys
_NT = os.path.expanduser("~/Desktop/neuron-talk")
_LR = os.path.expanduser("~/Desktop/lang-realizers")
for _p in (_NT, _LR):
if _p not in sys.path:
sys.path.insert(0, _p)
import engine # noqa: E402 (the proven no-LLM realizer)
from dialogue import _prop_to_clause # noqa: E402 (proven prop -> clause)
from document_ir import Block, DocumentIR, Provenance, Section # noqa: E402
def _provenance_from(p, kind: str = "fact") -> Provenance:
return Provenance(
subj_id=p.source_node_id, subject=p.subject, relation=p.predicate,
obj=p.object, polarity=p.polarity, confidence=round(float(p.confidence), 3),
node_id=p.source_node_id, kind=kind,
importance=float(getattr(p, "node_importance", 0.0) or 0.0),
salience=0.0,
)
import re as _re
# a well-formed declarative opens with a determiner, a proper noun, "I", or a
# capitalized head — not a mis-parsed object pronoun or a copula fragment.
_BAD_OPENERS = _re.compile(r"^(Me |It is I|There is|This is it|That is it)\b")
_VACUOUS = _re.compile(r"^\w+ (is|are|was|were) (it|no|nothing|empty|those|this|that)\.?$",
_re.I)
def _good_sentence(text: str) -> bool:
"""Fluency gate — drops degenerate realizations. NEVER loosens faithfulness;
it only refuses to SPEAK a claim whose surface came out malformed."""
words = text.rstrip(".").split()
if len(words) < 3:
return False
if _BAD_OPENERS.search(text):
return False
if _VACUOUS.match(text):
return False
# a sentence that is mostly one-letter/two-letter tokens is a parse artifact
short = sum(1 for w in words if len(w.strip(".,'")) <= 2)
if short > len(words) / 2:
return False
return True
def _realize_prop(p, lang: str = "en") -> tuple[str, Provenance] | None:
"""One proposition -> (faithful sentence, provenance) or None if it drops."""
clause = _prop_to_clause(p)
text = engine.realize(clause, lang)
if not text or not text.strip():
return None
text = text.strip()
if not text.endswith((".", "!", "?")):
text += "."
# capitalize first character (proper nouns / "I" already handled by grammar)
text = text[0].upper() + text[1:]
if not _good_sentence(text):
return None
return text, _provenance_from(p)
def realize_document(doc: DocumentIR, lang: str = "en") -> DocumentIR:
"""Fill every planned section's blocks with faithful, realized passages."""
for sec in doc.sections:
planned = sec.__dict__.get("_planned_props", [])
block = Block(role="body")
summary_bits: list[str] = []
for p in planned:
r = _realize_prop(p, lang)
if r is None:
continue
text, prov = r
block.sentences.append(text)
block.provenance.append(prov)
if len(summary_bits) < 1:
# a short grounded gloss for TOC / pptx bullets
obj = (prov.obj or "").strip().rstrip(".")
if obj:
summary_bits.append(obj)
if block.sentences:
sec.blocks.append(block)
sec.summary = summary_bits[0] if summary_bits else ""
# drop the transient planning payload; the IR is now self-contained
sec.__dict__.pop("_planned_props", None)
sec.__dict__.pop("_node", None)
# prune sections that realized to nothing
doc.sections = [s for s in doc.sections if s.blocks]
return doc
+136
View File
@@ -0,0 +1,136 @@
// accent.el - A British-RP ACCENT as an INGESTED TRANSFORM-GEOMETRY, composed
// onto the voice (voice (+) accent, SEPARABLE). Reads elp/data/british-accent.psv
// into an accent MANIFOLD in the engram (override nodes + a shared accent hub),
// and the render reads the RP formant overrides + the non-rhotic rule back from
// that geometry. NO accent targets live in code same discipline as the base
// phonetics. PROVENANCE NOTE: the RP Hz values are PROVISIONAL (reconstructed-
// from-knowledge approximations, cite Deterding1997 / Hawkins&Midgley2005 /
// Wells1982) pending transcription from the published tables the PIPELINE is
// the deliverable; exact values are being source-verified separately.
fn ingest_accent(path: String) -> [String] {
let content: String = fs_read(path)
let lines: [String] = str_split(content, "\n")
let nl: Int = native_list_len(lines)
let amap: [String] = native_list_empty()
let hub: String = engram_node("accent british-rp prov=PROVISIONAL cite=Deterding1997-HawkinsMidgley2005-Wells1982", "Accent", 80)
let li: Int = 0
while li < nl {
let line: String = native_list_get(lines, li)
let ll: Int = str_len(line)
let skip: Int = 0
if ll < 3 {
skip = 1
}
if skip == 0 {
let first: Int = str_char_code(line, 0)
if first == 35 {
skip = 1
}
}
if skip == 0 {
let f: [String] = str_split(line, "|")
let nf: Int = native_list_len(f)
if nf >= 6 {
let key: String = native_list_get(f, 0)
let f1: String = native_list_get(f, 1)
let f2: String = native_list_get(f, 2)
let f3: String = native_list_get(f, 3)
let kind: String = native_list_get(f, 4)
let set: String = native_list_get(f, 5)
let cont: String = "accent british-rp " + key + " f1=" + f1 + " f2=" + f2 + " f3=" + f3 + " kind=" + kind + " set=" + set + " prov=PROVISIONAL cite=Deterding1997-HawkinsMidgley2005-Wells1982"
let id: String = engram_node(cont, "AccentTarget", 80)
amap = native_list_append(amap, key)
amap = native_list_append(amap, cont)
engram_connect(id, hub, 80, "of_accent")
}
}
li = li + 1
}
return amap
}
// RP formant override for a phoneme, read from the accent manifold. Returns
// [f1,f2,f3] for a vowel_override record, or an empty list if none / a rule.
fn accent_formants(amap: [String], code: String) -> [Int] {
let out: [Int] = native_list_empty()
let id: String = sp_map_get(amap, code)
if str_eq(id, "") {
return out
}
let j: String = id
let isrule: Int = str_index_of(j, "drop_coda")
if isrule >= 0 {
return out
}
let f1: Int = parse_uint_from(j, "f1=")
if f1 <= 0 {
return out
}
let out = native_list_append(out, f1)
let out = native_list_append(out, parse_uint_from(j, "f2="))
let out = native_list_append(out, parse_uint_from(j, "f3="))
return out
}
// Is this accent non-rhotic? (reads the R rule node from the manifold)
fn is_nonrhotic(amap: [String]) -> Int {
let id: String = sp_map_get(amap, "R")
if str_eq(id, "") {
return 0
}
let hit: Int = str_index_of(id, "drop_coda")
if hit >= 0 {
return 1
}
return 0
}
// Is this symbol a vowel? Membership in the vowel-set derived from the phonetics
// source's class column (phonological structure the FORMANT NUMBERS still come
// from the organ manifold; this is only the categorical class for the rule).
fn is_vowel_sym(vset: [String], sym: String) -> Int {
let n: Int = native_list_len(vset)
let i: Int = 0
while i < n {
if str_eq(native_list_get(vset, i), sym) {
return 1
}
i = i + 1
}
return 0
}
// Non-rhotic transform: drop a post-vocalic CODA /R/ an R whose next non-SIL
// phoneme is NOT a vowel (a consonant, or end of utterance). Keep INTERVOCALIC/
// onset R (next non-SIL phoneme is a vowel, e.g. the medial R in N UW R AA N).
fn apply_rhoticity(codes: [String], vset: [String]) -> [String] {
let n: Int = native_list_len(codes)
let out: [String] = native_list_empty()
let i: Int = 0
while i < n {
let c: String = native_list_get(codes, i)
let keep: Int = 1
if str_eq(c, "R") {
let jx: Int = i + 1
let nextv: Int = 0
while jx < n {
let ncode: String = native_list_get(codes, jx)
if str_eq(ncode, "SIL") {
jx = jx + 1
} else {
nextv = is_vowel_sym(vset, ncode)
jx = n + 1000
}
}
if nextv == 0 {
keep = 0
}
}
if keep == 1 {
out = native_list_append(out, c)
}
i = i + 1
}
return out
}
+73
View File
@@ -0,0 +1,73 @@
// audio-demo.el - Drive the native audio surface: render a tone per instrument
// from its LEARNED signature, then render a small meaning-phrase "piece".
// Entry point: top-level statement calls main() (same convention as the
// examples' top-level println(run_test())).
fn micros_to_str(xs: [Int]) -> String {
let n: Int = native_list_len(xs)
let out: String = ""
let i: Int = 0
while i < n {
if i > 0 { let out: String = out + "," }
let out: String = out + int_to_str(native_list_get(xs, i))
let i: Int = i + 1
}
return out
}
// Render a 1.0s A4 (midi 69) tone from a signature file, print the parsed
// partials (proving the numbers came from the engram .sig), write the WAV.
fn render_tone(name: String, sigpath: String, outpath: String, table: [Int]) -> Int {
let lines: [String] = sig_load(sigpath)
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
println("[" + name + "] partials_n=" + sig_field(lines, "partials_n") + " parsed_partials_micro(scale 1e6)=" + micros_to_str(partials))
println("[" + name + "] raw partials line from .sig = " + sig_field(lines, "partials"))
let freq: Int = freq_of_midi(69)
let note: [Int] = synth_from_sig(lines, freq, 1000, 900, 44100, table)
let n: Int = native_list_len(note)
let ok: Int = wav_write(outpath, note, n, 44100)
println("[" + name + "] rendered " + int_to_str(n) + " samples -> " + outpath + " (write_ok=" + int_to_str(ok) + ")")
return n
}
fn run_demo() -> Int {
let table: [Int] = sin_table()
fs_mkdir("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out")
println("=== TONES: render A4 (midi 69) from each learned signature ===")
render_tone("flute", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/flute.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-flute.wav", table)
render_tone("clarinet", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/clarinet.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-clarinet.wav", table)
render_tone("violin", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/violin.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-violin.wav", table)
render_tone("piano", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-piano.wav", table)
render_tone("organ", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/organ.sig", "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/tone-organ.wav", table)
println("")
println("=== PIECE: a 6-frame meaning phrase (incl. a NEG frame) ===")
let frames: [[String]] = native_list_empty()
let frames: [[String]] = native_list_append(frames, audio_frame("agent", "aff", "0.9", "0.8", "0", "s1"))
let frames: [[String]] = native_list_append(frames, audio_frame("theme", "aff", "0.7", "0.6", "0", "s2"))
let frames: [[String]] = native_list_append(frames, audio_frame("cause", "aff", "0.8", "0.9", "1", "s3"))
let frames: [[String]] = native_list_append(frames, audio_frame("negation", "neg", "0.85", "0.7", "0", "s4"))
let frames: [[String]] = native_list_append(frames, audio_frame("goal", "aff", "0.6", "0.5", "1", "s5"))
let frames: [[String]] = native_list_append(frames, audio_frame("result", "aff", "0.95", "1.0", "0", "s6"))
// Print the plan so the NEG frame's minor third (+3) vs major (+4) is visible.
let nf: Int = native_list_len(frames)
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let pol: String = surface_get(frame, "polarity")
let third_name: String = "major(+4)"
if str_eq(pol, "neg") { let third_name: String = "MINOR(+3)" }
println("frame " + int_to_str(fi) + " relation=" + surface_get(frame, "relation") + " polarity=" + pol + " -> midi=" + int_to_str(native_list_get(plan, 0)) + " dur_ms=" + int_to_str(native_list_get(plan, 1)) + " amp_pm=" + int_to_str(native_list_get(plan, 2)) + " third=" + third_name)
let fi: Int = fi + 1
}
let piano_lines: [String] = sig_load("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/piano.sig")
let total: Int = realize_audio(frames, piano_lines, "/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav", 44100, table)
println("PIECE rendered " + int_to_str(total) + " samples -> /Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/piece.wav")
return total
}
println("audio-demo main returned samples=" + int_to_str(run_demo()))
+400
View File
@@ -0,0 +1,400 @@
// audio-surface.el - Native own-core additive-synthesis audio surface.
//
// The AUDIO efferent seam, native, no Python and no library. This renders real
// PCM .wav bytes from instrument SIGNATURES read from engram-sourced .sig data
// files (elp/faculty/sig/*.sig) - the partial amplitudes are NEVER literals in
// this source; they are parsed from the learned signature at run time. That is
// the whole proof: render-from-learned-signatures.
//
// EL has no float arithmetic operator (codegen emits raw int64 ops for + - * /
// on the shared 64-bit slot) and no float-arithmetic natives - so ALL synthesis
// math here is own-core INTEGER fixed-point. Angles use a quarter-wave sine
// table (scale 10000) from a fixed-point Taylor series; amplitudes are parsed to
// micro (scale 1e6) straight from the .sig text; frequencies are milliHz ints.
//
// Pipeline mirrors the two-stage projector (midi.py): plan_note(frame) reads a
// frame's meaning-geometry slot-map and derives (pitch, duration, amplitude);
// realize_audio SUPERPOSES the signature's partials (the compose op) and
// serialises RIFF/WAVE. Same frame -> midi OR audio.
// -- integer decimal + string helpers -----------------------------------------
fn str_to_int_el(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let v: Int = 0
let neg: Bool = false
while i < n {
let c: Int = str_char_code(s, i)
if c == 45 { let neg: Bool = true }
if c >= 48 {
if c < 58 {
let v: Int = v * 10 + (c - 48)
}
}
let i: Int = i + 1
}
if neg { return 0 - v }
return v
}
fn parse_micro(s: String) -> Int {
let dot: Int = str_index_of(s, ".")
if dot < 0 {
return str_to_int_el(s) * 1000000
}
let n: Int = str_len(s)
let ipart: String = str_slice(s, 0, dot)
let fpart: String = str_slice(s, dot + 1, n)
let iv: Int = str_to_int_el(ipart)
let fv: Int = 0
let scale: Int = 100000
let fn2: Int = str_len(fpart)
let i: Int = 0
while i < 6 {
let d: Int = 0
if i < fn2 {
let d: Int = str_char_code(fpart, i) - 48
}
let fv: Int = fv + d * scale
let scale: Int = scale / 10
let i: Int = i + 1
}
return iv * 1000000 + fv
}
// -- signature (engram data file) loader ---------------------------------------
fn sig_load(path: String) -> [String] {
let text: String = fs_read(path)
return str_split(text, "\n")
}
fn sig_field(lines: [String], key: String) -> String {
let pref: String = key + ": "
let n: Int = native_list_len(lines)
let plen: Int = str_len(pref)
let i: Int = 0
while i < n {
let ln: String = native_list_get(lines, i)
if str_starts_with(ln, pref) {
return str_slice(ln, plen, str_len(ln))
}
let i: Int = i + 1
}
return ""
}
fn parse_micros(csv: String) -> [Int] {
let parts: [String] = str_split(csv, ",")
let n: Int = native_list_len(parts)
let out: [Int] = native_list_empty()
let i: Int = 0
while i < n {
let out: [Int] = native_list_append(out, parse_micro(native_list_get(parts, i)))
let i: Int = i + 1
}
return out
}
// -- fixed-point sine (own-core, quarter-wave Taylor table, scale 10000) --------
fn sin_table() -> [Int] {
let HP: Int = 1570796
let t: [Int] = native_list_empty()
let q: Int = 0
while q < 257 {
let x: Int = q * HP / 256
let x2: Int = x * x / 1000000
let x3: Int = x2 * x / 1000000
let x5: Int = x3 * x2 / 1000000
let x7: Int = x5 * x2 / 1000000
let x9: Int = x7 * x2 / 1000000
let s: Int = x - x3 / 6 + x5 / 120 - x7 / 5040 + x9 / 362880
let t: [Int] = native_list_append(t, s / 100)
let q: Int = q + 1
}
return t
}
fn sin_lookup(t: [Int], phase: Int) -> Int {
let p: Int = phase % 1024
if p < 0 { let p: Int = p + 1024 }
let quad: Int = p / 256
let r: Int = p % 256
if quad == 0 { return native_list_get(t, r) }
if quad == 1 { return native_list_get(t, 256 - r) }
if quad == 2 { return 0 - native_list_get(t, r) }
return 0 - native_list_get(t, 256 - r)
}
fn isqrt_int(n: Int) -> Int {
if n <= 0 { return 0 }
let x: Int = n
let y: Int = (x + 1) / 2
while y < x {
let x: Int = y
let y: Int = (x + n / x) / 2
}
return x
}
// freq_of_midi: equal-tempered frequency in milliHz. 440000 mHz at midi 69.
fn freq_of_midi(m: Int) -> Int {
let f: Int = 440000
if m > 69 {
let k: Int = m - 69
let i: Int = 0
while i < k {
let f: Int = f * 1059463 / 1000000
let i: Int = i + 1
}
return f
}
if m < 69 {
let k: Int = 69 - m
let i: Int = 0
while i < k {
let f: Int = f * 1000000 / 1059463
let i: Int = i + 1
}
return f
}
return f
}
// -- envelope (ADSR), scale 1000 -----------------------------------------------
fn adsr_env(i: Int, total: Int, atk_n: Int, dec_n: Int, sus_pm: Int, rel_n: Int) -> Int {
if i < atk_n {
if atk_n == 0 { return 1000 }
return 1000 * i / atk_n
}
if i < atk_n + dec_n {
if dec_n == 0 { return sus_pm }
return 1000 - (1000 - sus_pm) * (i - atk_n) / dec_n
}
let rel_start: Int = total - rel_n
if i < rel_start {
return sus_pm
}
if rel_n == 0 { return 0 }
let left: Int = total - i
return sus_pm * left / rel_n
}
// -- note synthesis: SUPERPOSE the learned partials -> [Int] samples -----------
fn note_samples(freq_mHz: Int, dur_ms: Int, rate: Int, partials: [Int], sumP: Int, b_micro: Int, vib_rate: Int, vib_cents: Int, atk_ms: Int, dec_ms: Int, sus_pm: Int, rel_ms: Int, amp_pm: Int, table: [Int]) -> [Int] {
let total: Int = dur_ms * rate / 1000
let atk_n: Int = atk_ms * rate / 1000
let dec_n: Int = dec_ms * rate / 1000
let rel_n: Int = rel_ms * rate / 1000
let np: Int = native_list_len(partials)
let half_mhz: Int = rate * 1000 / 2
let out: [Int] = native_list_empty()
let i: Int = 0
while i < total {
let acc: Int = 0
let k: Int = 0
while k < np {
let harm: Int = k + 1
let amp_k: Int = native_list_get(partials, k)
let factor: Int = 1000000
if b_micro > 0 {
let val: Int = 1000000 + b_micro * harm * harm
let factor: Int = isqrt_int(val * 1000000)
}
let fn_mhz: Int = freq_mHz * harm
let fn_mhz: Int = fn_mhz * factor / 1000000
if vib_cents > 0 {
if vib_rate > 0 {
let vphase: Int = i * vib_rate * 1024 / rate
let vs: Int = sin_lookup(table, vphase)
let vibf: Int = 1000000 + (vib_cents * vs * 833) / 10000
let fn_mhz: Int = fn_mhz * vibf / 1000000
}
}
if fn_mhz <= half_mhz {
let phase: Int = i * fn_mhz * 1024 / (rate * 1000)
let sv: Int = sin_lookup(table, phase)
let acc: Int = acc + sv * amp_k / 1000000
}
let k: Int = k + 1
}
let env: Int = adsr_env(i, total, atk_n, dec_n, sus_pm, rel_n)
let s16: Int = acc * 2800000 / sumP
let s16: Int = s16 * env / 1000
let s16: Int = s16 * amp_pm / 1000
if s16 > 32767 { let s16: Int = 32767 }
if s16 < 0 - 32767 { let s16: Int = 0 - 32767 }
let out: [Int] = native_list_append(out, s16)
let i: Int = i + 1
}
return out
}
fn synth_from_sig(lines: [String], freq_mHz: Int, dur_ms: Int, amp_pm: Int, rate: Int, table: [Int]) -> [Int] {
let partials: [Int] = parse_micros(sig_field(lines, "partials"))
let np: Int = native_list_len(partials)
let sumP: Int = 0
let j: Int = 0
while j < np {
let pj: Int = native_list_get(partials, j)
let sumP: Int = sumP + pj
let j: Int = j + 1
}
if sumP <= 0 { let sumP: Int = 1000000 }
let adsr: [String] = str_split(sig_field(lines, "adsr"), ",")
let atk_ms: Int = parse_micro(native_list_get(adsr, 0)) / 1000
let dec_ms: Int = parse_micro(native_list_get(adsr, 1)) / 1000
let sus_pm: Int = parse_micro(native_list_get(adsr, 2)) / 1000
let rel_ms: Int = parse_micro(native_list_get(adsr, 3)) / 1000
let b_micro: Int = parse_micro(sig_field(lines, "inharmonicity_B"))
let vib_rate: Int = str_to_int_el(sig_field(lines, "vibrato_rate_hz"))
let vib_cents: Int = str_to_int_el(sig_field(lines, "vibrato_depth_cents"))
return note_samples(freq_mHz, dur_ms, rate, partials, sumP, b_micro, vib_rate, vib_cents, atk_ms, dec_ms, sus_pm, rel_ms, amp_pm, table)
}
// -- byte-buffer helpers (own-core, no library) --------------------------------
fn put_tag(buf: String, pos: Int, s: String) -> String {
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let buf: String = __str_set_char(buf, pos + i, str_char_code(s, i))
let i: Int = i + 1
}
return buf
}
fn put_u32le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
let buf: String = __str_set_char(buf, pos + 2, (v / 65536) % 256)
let buf: String = __str_set_char(buf, pos + 3, (v / 16777216) % 256)
return buf
}
fn put_u16le(buf: String, pos: Int, v: Int) -> String {
let buf: String = __str_set_char(buf, pos, v % 256)
let buf: String = __str_set_char(buf, pos + 1, (v / 256) % 256)
return buf
}
// -- WAV serializer: own-core RIFF/WAVE, PCM mono 16-bit -----------------------
fn wav_write(path: String, samples: [Int], n: Int, rate: Int) -> Int {
let data_len: Int = n * 2
let total: Int = 44 + data_len
let buf: String = __str_alloc(total)
let buf: String = put_tag(buf, 0, "RIFF")
let buf: String = put_u32le(buf, 4, 36 + data_len)
let buf: String = put_tag(buf, 8, "WAVE")
let buf: String = put_tag(buf, 12, "fmt ")
let buf: String = put_u32le(buf, 16, 16)
let buf: String = put_u16le(buf, 20, 1)
let buf: String = put_u16le(buf, 22, 1)
let buf: String = put_u32le(buf, 24, rate)
let buf: String = put_u32le(buf, 28, rate * 2)
let buf: String = put_u16le(buf, 32, 2)
let buf: String = put_u16le(buf, 34, 16)
let buf: String = put_tag(buf, 36, "data")
let buf: String = put_u32le(buf, 40, data_len)
let i: Int = 0
while i < n {
let v: Int = native_list_get(samples, i)
if v < 0 { let v: Int = v + 65536 }
let buf: String = __str_set_char(buf, 44 + i * 2, v % 256)
let buf: String = __str_set_char(buf, 44 + i * 2 + 1, (v / 256) % 256)
let i: Int = i + 1
}
let ok: Int = fs_write_bytes(path, buf, total)
return ok
}
// -- plan: frame slot-map -> note atom (pitch, duration, amplitude) ------------
fn audio_frame(relation: String, polarity: String, confidence: String, importance: String, salience: String, subj_id: String) -> [String] {
let f: [String] = native_list_empty()
let f: [String] = native_list_append(f, "relation")
let f: [String] = native_list_append(f, relation)
let f: [String] = native_list_append(f, "polarity")
let f: [String] = native_list_append(f, polarity)
let f: [String] = native_list_append(f, "confidence")
let f: [String] = native_list_append(f, confidence)
let f: [String] = native_list_append(f, "importance")
let f: [String] = native_list_append(f, importance)
let f: [String] = native_list_append(f, "salience")
let f: [String] = native_list_append(f, salience)
let f: [String] = native_list_append(f, "subj_id")
let f: [String] = native_list_append(f, subj_id)
return f
}
fn degree_offset(deg: Int) -> Int {
if deg == 0 { return 0 }
if deg == 1 { return 2 }
if deg == 2 { return 4 }
if deg == 3 { return 5 }
if deg == 4 { return 7 }
if deg == 5 { return 9 }
return 11
}
// returns [midi, dur_ms, amp_pm]
fn plan_note(frame: [String]) -> [Int] {
let relation: String = surface_get(frame, "relation")
let polarity: String = surface_get(frame, "polarity")
let confidence: String = surface_get(frame, "confidence")
let importance: String = surface_get(frame, "importance")
let salience: String = surface_get(frame, "salience")
let rn: Int = str_len(relation)
let csum: Int = 0
let i: Int = 0
while i < rn {
let cc: Int = str_char_code(relation, i)
let csum: Int = csum + cc
let i: Int = i + 1
}
let deg: Int = csum % 7
let third: Int = 4
if str_eq(polarity, "neg") { let third: Int = 3 }
let sal_oct: Int = str_to_int_el(salience)
let doff: Int = degree_offset(deg)
let midi: Int = 60 + sal_oct * 12 + doff + third
let conf_micro: Int = parse_micro(confidence)
let dur_ms: Int = 200 + conf_micro / 1000
let imp_micro: Int = parse_micro(importance)
let amp_pm: Int = 400 + imp_micro / 2000
let out: [Int] = native_list_empty()
let out: [Int] = native_list_append(out, midi)
let out: [Int] = native_list_append(out, dur_ms)
let out: [Int] = native_list_append(out, amp_pm)
return out
}
fn realize_audio(frames: [[String]], sig_lines: [String], path: String, rate: Int, table: [Int]) -> Int {
let nf: Int = native_list_len(frames)
let all: [Int] = native_list_empty()
let count: Int = 0
let fi: Int = 0
while fi < nf {
let frame: [String] = native_list_get(frames, fi)
let plan: [Int] = plan_note(frame)
let midi: Int = native_list_get(plan, 0)
let dur_ms: Int = native_list_get(plan, 1)
let amp_pm: Int = native_list_get(plan, 2)
let freq: Int = freq_of_midi(midi)
let note: [Int] = synth_from_sig(sig_lines, freq, dur_ms, amp_pm, rate, table)
let nn: Int = native_list_len(note)
let j: Int = 0
while j < nn {
let all: [Int] = native_list_append(all, native_list_get(note, j))
let j: Int = j + 1
}
let count: Int = count + nn
let fi: Int = fi + 1
}
let ok: Int = wav_write(path, all, count, rate)
return count
}
File diff suppressed because it is too large Load Diff
+16
View File
@@ -0,0 +1,16 @@
// comprehend.elh — public surface of the ELP comprehension front-end.
// text → meaning-spec (the input half of the ELP; inverse of the realizer).
extern fn parse_spec(text: String) -> [String]
extern fn parse_spec_lang(text: String, lang: String) -> [String]
extern fn parse_json(text: String) -> String
extern fn parse_json_lang(text: String, lang: String) -> String
// Analysis primitives (invertible morphology + deterministic grammar helpers):
extern fn cp_tokenize(text: String) -> [String]
extern fn cp_pron_concept(w: String) -> String
extern fn cp_is_negation(w: String) -> Bool
extern fn cp_is_neg_adverb(w: String) -> Bool
extern fn cp_irr2(surface: String) -> [String]
extern fn cp_reg_verb(w: String) -> [String]
extern fn cp_analyze_verb(surface: String) -> [String]
extern fn cp_verb_start(toks: [String], end: Int) -> Int
extern fn cp_subord_start(toks: [String], n: Int) -> Int
+1
View File
@@ -0,0 +1 @@
import "language-profile.el"
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
import "dedup_test_a_nodedup.el"
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
extern fn fn_a(x: String) -> String
+1
View File
@@ -0,0 +1 @@
extern fn fn_a(x: String) -> String
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
extern fn fn_a(x: String) -> String
+6
View File
@@ -0,0 +1,6 @@
import "language-profile.el"
import "dedup_test_a.el"
fn main_fn(x: String) -> String {
return x
}
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
import "dedup_test_a.el"
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
import "dedup_test_a_notail.el"
+287
View File
@@ -0,0 +1,287 @@
// dialogue.el SUMMON-THROUGH-SELF, native el. Port of dialogue.py's core.
//
// THE WHOLE DIALOGUE IS ONE OPERATION. A fact is never merely *fetched*: the
// query is PROJECTED into the engram's self + memory geometry, LANDS in a region,
// and the reply is READ OUT / the region MATERIALIZED from wherever it landed.
//
// project(query) -> land on a region -> read out from that region
//
// lands in the SELF region -> grounded identity/presence, read out of
// the real self nodes (self_region.el)
// lands on a memory NEIGHBORHOOD -> MATERIALIZE it: walk the neighborhood
// (engram_neighbors_json) and read out the
// region's connected members
// lands nowhere close -> HONEST ABSENCE (an empty region, not a
// fabricated answer, not an error)
//
// CRITICAL INVARIANTS (enforced structurally, not by convention):
// * ONE operation there is NO intent classifier and NO separate
// fact-retrieval branch. Identity is nearest-region proximity, not a switch.
// * MATERIALIZE by walking the neighborhood, never by fetching top-props.
// * HONEST ABSENCE when the region is thin.
// * NEGATION is SACRED: the readout is the stored prose VERBATIM, so a negated
// memory stays negated we never paraphrase a polarity away.
// * NO ECHO: the old "I noted that X. That relates to Y." template is gone.
// The summon path materializes or honestly declines it never echoes.
// * DIRECTIVE OVERRIDE: a meta-directive ("answer in English") overrides the
// reply language while the content language is still auto-detected.
//
// Depends on: comprehend (parse_spec_lang, cp_tokenize), multilingual (ml_detect,
// ml_tr, ml_term), propositions (prop_split_sentences), self_region
// (sr_available, sr_readout), the engram + json runtime builtins.
// directive override
// Return [target_lang, content]. target_lang is "" when no directive is present.
// A directive names an output language; we strip it and keep the remaining text
// as the content (whose OWN language is still auto-detected downstream).
fn dlg_dir_hit(low: String, phrase: String) -> Bool {
return str_contains(low, phrase)
}
fn dlg_parse_directive(text: String) -> [String] {
let low: String = str_to_lower(text)
let lang: String = ""
let phrase: String = ""
// English target
if dlg_dir_hit(low, "in english") { let lang = "en"; let phrase = "in english" }
if dlg_dir_hit(low, "em inglês") { let lang = "en"; let phrase = "em inglês" }
if dlg_dir_hit(low, "em ingles") { let lang = "en"; let phrase = "em ingles" }
if dlg_dir_hit(low, "en inglés") { let lang = "en"; let phrase = "en inglés" }
// Portuguese target
if dlg_dir_hit(low, "in portuguese") { let lang = "pt"; let phrase = "in portuguese" }
if dlg_dir_hit(low, "em português") { let lang = "pt"; let phrase = "em português" }
// Spanish target
if dlg_dir_hit(low, "in spanish") { let lang = "es"; let phrase = "in spanish" }
if dlg_dir_hit(low, "en español") { let lang = "es"; let phrase = "en español" }
// Italian target
if dlg_dir_hit(low, "in italian") { let lang = "it"; let phrase = "in italian" }
let content: String = text
if !str_eq(phrase, "") {
// strip the directive phrase (and a common "answer"/"responda" lead-in),
// leaving the real question as content.
let idx: Int = str_index_of(low, phrase)
if idx >= 0 {
let before: String = str_slice(text, 0, idx)
let after: String = str_slice(text, idx + str_len(phrase), str_len(text))
let content = str_trim(before + " " + after)
}
// trim a leading "answer"/"responda"/"reply" and stray colon/comma.
let cl: String = str_to_lower(content)
if str_starts_with(cl, "answer") { let content = str_trim(str_slice(content, 6, str_len(content))) }
if str_starts_with(cl, "responda") { let content = str_trim(str_slice(content, 8, str_len(content))) }
if str_starts_with(cl, "reply") { let content = str_trim(str_slice(content, 5, str_len(content))) }
if str_starts_with(content, ":") { let content = str_trim(str_slice(content, 1, str_len(content))) }
if str_starts_with(content, ",") { let content = str_trim(str_slice(content, 1, str_len(content))) }
}
let r: [String] = native_list_empty()
let r = native_list_append(r, lang)
let r = native_list_append(r, content)
return r
}
// identity landing (a region proximity, not a classifier switch)
// The query lands in the SELF region when it takes an identity/presence shape.
// Cross-lingual forms are included because the engram's lexical probe is
// English-leaning. This is the SELF attractor of the single operation.
fn dlg_is_identity(content: String) -> Bool {
let low: String = str_to_lower(str_trim(content))
if str_contains(low, "who are you") { return true }
if str_contains(low, "what are you") { return true }
if str_contains(low, "who i am") { return true }
if str_contains(low, "your name") { return true }
if str_contains(low, "about yourself") { return true }
if str_contains(low, "are you conscious") { return true }
if str_contains(low, "are you there") { return true }
// cross-lingual identity question-forms
if str_contains(low, "quem é você") { return true }
if str_contains(low, "quem es voce") { return true }
if str_contains(low, "quién eres") { return true }
if str_contains(low, "quien eres") { return true }
if str_contains(low, "chi sei") { return true }
if str_contains(low, "qui es-tu") { return true }
if str_contains(low, "wer bist du") { return true }
return false
}
// readout helpers
fn dlg_first_sentence(content: String) -> String {
let sents: [String] = prop_split_sentences(content)
let n: Int = native_list_len(sents)
let i: Int = 0
while i < n {
let s: String = str_trim(native_list_get(sents, i))
// drop a leading markdown heading marker for a clean read-out line
if str_starts_with(s, "# ") { let s = str_trim(str_slice(s, 2, str_len(s))) }
if str_len(s) > 0 { return s }
let i = i + 1
}
return str_trim(content)
}
// strip trailing/leading punctuation from a token.
fn dlg_clean_tok(w: String) -> String {
let s: String = str_trim(w)
let s = str_strip_suffix(s, ".")
let s = str_strip_suffix(s, ",")
let s = str_strip_suffix(s, "?")
let s = str_strip_suffix(s, "!")
let s = str_strip_suffix(s, ":")
let s = str_strip_suffix(s, ";")
return str_trim(s)
}
// closed-class across the supported languages (union) a word we must NOT treat
// as a retrieval topic. Also drops the meta verbs of a request ("tell", "prove",
// "show") so the TOPIC, not the speech act, is what projects into memory.
fn dlg_is_stop(w: String) -> Bool {
if ml_stop_en(w) { return true }
if ml_stop_es(w) { return true }
if ml_stop_pt(w) { return true }
if ml_stop_it(w) { return true }
if str_eq(w, "tell") { return true }
if str_eq(w, "show") { return true }
if str_eq(w, "about") { return true }
if str_eq(w, "sobre") { return true }
if str_eq(w, "acerca") { return true }
return false
}
// The CONTENT TERMS the query projects into memory: content words only, cleaned,
// cross-lingually mapped to the engram's English vocabulary, 3 chars. This is
// the geometry probe the speech-act verbs and function words are stripped so a
// PP topic ("tell me ABOUT Lisbon") projects on "lisbon", not "tell"/"me".
fn dlg_content_terms(content: String, lang: String) -> [String] {
let toks: [String] = cp_tokenize(content)
let n: Int = native_list_len(toks)
let out: [String] = native_list_empty()
let i: Int = 0
while i < n {
let w: String = str_to_lower(dlg_clean_tok(native_list_get(toks, i)))
if str_len(w) >= 3 {
if !dlg_is_stop(w) {
let out = native_list_append(out, ml_term(w, lang))
}
}
let i = i + 1
}
return out
}
// Does this landed node lexically overlap the query's content terms? This is the
// RELEVANCE FLOOR: activation always returns the store's most salient nodes, so
// without this a query about nothing would "land" on the self/top node. A node
// that shares no content term with the query is "nowhere close" -> honest absence.
fn dlg_node_matches(node: String, terms: [String]) -> Bool {
let hay: String = str_to_lower(json_get_string(node, "content") + " " + json_get_string(node, "label"))
let n: Int = native_list_len(terms)
let i: Int = 0
while i < n {
let t: String = native_list_get(terms, i)
if str_len(t) >= 3 {
if str_contains(hay, t) { return true }
}
let i = i + 1
}
return false
}
// MATERIALIZE the landed region: read out the landed fact, then WALK the
// neighborhood and read out its connected members (real edges, not top-props).
fn dlg_materialize(top_node: String, reply_lang: String) -> String {
let id: String = json_get_string(top_node, "id")
let content: String = json_get_string(top_node, "content")
let lead: String = dlg_first_sentence(content)
let nb: String = engram_neighbors_json(id, 2, "both")
let m: Int = json_array_len(nb)
let parts: [String] = native_list_empty()
let parts = native_list_append(parts, lead)
let added: Int = 0
let i: Int = 0
while i < m {
if added < 3 {
let rec: String = json_array_get(nb, i)
let node: String = json_get_raw(rec, "node")
let nc: String = json_get_string(node, "content")
if !str_eq(nc, "") {
let sent: String = dlg_first_sentence(nc)
if !str_eq(sent, "") {
let parts = native_list_append(parts, sent)
let added = added + 1
}
}
}
let i = i + 1
}
// The readout is the region's OWN prose, verbatim negation SACRED, no echo.
return str_join(parts, " ")
}
// THE single operation
fn dlg_respond(text: String) -> String {
// directive override: reply language may differ from content language.
let dir: [String] = dlg_parse_directive(text)
let target_lang: String = native_list_get(dir, 0)
let content: String = native_list_get(dir, 1)
let content_lang: String = ml_detect(content)
let reply_lang: String = content_lang
if !str_eq(target_lang, "") { let reply_lang = target_lang }
// comprehend the content (SACRED polarity carried in the spec).
let spec: [String] = parse_spec_lang(content, content_lang)
// PROJECT + LAND: SELF region
// Identity/presence shape lands in the self region; read out the REAL self
// nodes (self_region.el), never a template. Same single operation this is
// just the self attractor winning the landing.
if dlg_is_identity(content) {
if sr_available() {
// read out the REAL self nodes when replying in their own language
// (the soul's prose is English); for another reply language we cannot
// translate real content without an LLM, so we answer with the
// localized SACRED identity anchor honest, in-language, no fabrication.
if str_eq(reply_lang, "en") { return sr_readout("en") }
return ml_tr("identity", reply_lang)
}
// self region thin honest localized identity (logged fallback shape).
return ml_tr("identity", reply_lang)
}
// PROJECT into MEMORY geometry
let terms: [String] = dlg_content_terms(content, content_lang)
let qterm: String = str_join(terms, " ")
let act: String = engram_activate_json(qterm, 12)
let n: Int = json_array_len(act)
// LAND: the highest-activation node that ACTUALLY overlaps the query's
// content terms (the relevance floor). Activation always returns the most
// salient nodes, so we walk the ranked list and take the first that is
// genuinely "close"; if none is, the query landed nowhere. ───────────────
let landing: String = ""
let i: Int = 0
while i < n {
if str_eq(landing, "") {
let rec: String = json_array_get(act, i)
let node: String = json_get_raw(rec, "node")
if dlg_node_matches(node, terms) {
let landing = node
}
}
let i = i + 1
}
// HONEST ABSENCE: nothing close an empty region, not a fabricated answer,
// not an "I noted that" echo.
if str_eq(landing, "") {
return ml_tr("no_memory", reply_lang)
}
// MATERIALIZE the landing by WALKING its neighborhood.
return dlg_materialize(landing, reply_lang)
}
+172
View File
@@ -0,0 +1,172 @@
// elp.el - Engram Language Protocol public API.
//
// Output half of the ELP: Engram semantic form natural language surface text.
// 31 languages. Ties together language-profile, vocabulary, morphology,
// grammar, realizer, and semantics into a single entry point.
//
// Import chain (mirrors manifest.el dependency order):
// language-profile (no deps)
// vocabulary (no deps)
// morphology (depends on: language-profile)
// morphology-XX (depends on: morphology) all language engines
// grammar (depends on: language-profile)
// realizer (depends on: morphology, grammar, language-profile)
// semantics (depends on: grammar, realizer, language-profile)
//
// When elc processes a source that imports this file, it resolves all
// transitive imports via depth-first deduplication each module is
// inlined exactly once regardless of how many importers reference it.
// Base layers
import "language-profile.el"
import "vocabulary.el"
// Morphology: base engine
import "morphology.el"
// Morphology: living languages
import "morphology-es.el"
import "morphology-fr.el"
import "morphology-de.el"
import "morphology-ru.el"
import "morphology-ja.el"
import "morphology-fi.el"
import "morphology-ar.el"
import "morphology-hi.el"
import "morphology-sw.el"
// Morphology: ancient / classical
import "morphology-la.el"
import "morphology-he.el"
// Morphology: dead languages
import "morphology-grc.el"
import "morphology-ang.el"
import "morphology-sa.el"
import "morphology-got.el"
import "morphology-non.el"
import "morphology-enm.el"
import "morphology-pi.el"
import "morphology-fro.el"
import "morphology-goh.el"
import "morphology-sga.el"
import "morphology-txb.el"
import "morphology-peo.el"
import "morphology-akk.el"
import "morphology-uga.el"
import "morphology-egy.el"
import "morphology-sux.el"
import "morphology-gez.el"
import "morphology-cop.el"
// Higher layers
import "grammar.el"
import "realizer.el"
import "semantics.el"
// Comprehension front-end (input half: text meaning-spec)
import "comprehend.el"
//
// Entry points:
//
// generate(semantic_form_json) -> String
// Low-level JSON-based API, defaults to English. SemanticForm JSON fields:
// intent - "assert" | "question" | "command"
// agent - subject (pronoun or noun phrase, optional for commands)
// predicate - verb base form
// patient - object noun phrase (optional)
// location - prepositional phrase e.g. "in the park" (optional)
// tense - "present" | "past" | "future" (default: "present")
// aspect - "simple" | "progressive" | "perfect" (default: "simple")
// lang - ISO 639-1 code (default: "en")
//
// generate_lang(semantic_form_json, lang_code) -> String
// JSON-based API with explicit language code (overrides any "lang" in JSON).
//
// generate_frame(frame: SemFrame) -> String
// High-level SemFrame API. Language from frame's "lang" field (default "en").
// Intents: "assert" | "query" | "describe" | "greet".
//
// generate_frame_lang(frame: SemFrame, lang_code: String) -> String
// High-level SemFrame API with explicit language code override.
// JSON helpers
fn sem_get(json: String, key: String) -> String {
let val: String = json_get(json, key)
return val
}
// Public API: SemFrame
// Generate text from a SemFrame in the language embedded in the frame (default "en").
fn generate_frame(frame: [String]) -> String {
return sem_realize(frame)
}
// Generate text from a SemFrame in the specified language.
fn generate_frame_lang(frame: [String], lang_code: String) -> String {
return sem_realize_lang(frame, lang_code)
}
// Public API: JSON
// Build a realizer slot map from JSON fields and an explicit lang code.
fn build_form_from_json(semantic_form_json: String, lang_code: String) -> [String] {
let intent: String = sem_get(semantic_form_json, "intent")
let agent: String = sem_get(semantic_form_json, "agent")
let predicate: String = sem_get(semantic_form_json, "predicate")
let patient: String = sem_get(semantic_form_json, "patient")
let location: String = sem_get(semantic_form_json, "location")
let tense: String = sem_get(semantic_form_json, "tense")
let aspect: String = sem_get(semantic_form_json, "aspect")
let polarity: String = sem_get(semantic_form_json, "polarity")
let neg_word: String = sem_get(semantic_form_json, "neg_word")
let iobj: String = sem_get(semantic_form_json, "iobj")
let form: [String] = native_list_empty()
let form = native_list_append(form, "intent")
let form = native_list_append(form, intent)
let form = native_list_append(form, "agent")
let form = native_list_append(form, agent)
let form = native_list_append(form, "predicate")
let form = native_list_append(form, predicate)
let form = native_list_append(form, "patient")
let form = native_list_append(form, patient)
let form = native_list_append(form, "iobj")
let form = native_list_append(form, iobj)
let form = native_list_append(form, "location")
let form = native_list_append(form, location)
let form = native_list_append(form, "tense")
let form = native_list_append(form, tense)
let form = native_list_append(form, "aspect")
let form = native_list_append(form, aspect)
// SACRED: polarity crosses the JSON boundary and is never inferred away.
let form = native_list_append(form, "polarity")
let form = native_list_append(form, polarity)
let form = native_list_append(form, "neg_word")
let form = native_list_append(form, neg_word)
let form = native_list_append(form, "lang")
let form = native_list_append(form, lang_code)
return form
}
// Generate text from a JSON semantic form. Language defaults to "en" unless
// the JSON contains a "lang" field.
fn generate(semantic_form_json: String) -> String {
let lang_in_json: String = sem_get(semantic_form_json, "lang")
let lang_code: String = lang_in_json
if str_eq(lang_code, "") {
let lang_code = "en"
}
let form: [String] = build_form_from_json(semantic_form_json, lang_code)
return realize(form)
}
// Generate text from a JSON semantic form in the specified language.
// lang_code overrides any "lang" field present in the JSON.
fn generate_lang(semantic_form_json: String, lang_code: String) -> String {
let form: [String] = build_form_from_json(semantic_form_json, lang_code)
return realize(form)
}
+7
View File
@@ -0,0 +1,7 @@
// auto-generated by elc --emit-header — do not edit
extern fn sem_get(json: String, key: String) -> String
extern fn generate_frame(frame: [String]) -> String
extern fn generate_frame_lang(frame: [String], lang_code: String) -> String
extern fn build_form_from_json(semantic_form_json: String, lang_code: String) -> [String]
extern fn generate(semantic_form_json: String) -> String
extern fn generate_lang(semantic_form_json: String, lang_code: String) -> String
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
extern fn fn_a(x: String) -> String
+2
View File
@@ -0,0 +1,2 @@
import "language-profile.el"
extern fn fn_b(x: String) -> String
+555
View File
@@ -0,0 +1,555 @@
// grammar.el - Grammar engine: syntactic structure, word order, phrase assembly.
//
// Language-specific word order and question strategy are driven by the language
// profile, not hardcoded. The slot map format (GramSpec) is universal; a "lang"
// key carries the ISO 639-1 code so every downstream function can resolve the
// active profile.
//
// GramSpec slot keys:
// intent - "assert" | "question" | "command"
// agent - subject referent string
// predicate - verb base form
// patient - object noun phrase (optional)
// location - prepositional phrase (optional)
// tense - "present" | "past" | "future"
// aspect - "simple" | "progressive" | "perfect"
// lang - ISO 639-1 code (default "en")
// verb_surf - conjugated verb surface form (computed)
// aux_surf - auxiliary surface form (computed)
//
// Depends on: language-profile
// Slot map helpers
fn slots_get(slots: [String], key: String) -> String {
let n: Int = native_list_len(slots)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(slots, i)
if str_eq(k, key) {
return native_list_get(slots, i + 1)
}
let i = i + 2
}
return ""
}
fn slots_set(slots: [String], key: String, val: String) -> [String] {
let n: Int = native_list_len(slots)
let result: [String] = native_list_empty()
let found: Bool = false
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(slots, i)
let v: String = native_list_get(slots, i + 1)
if str_eq(k, key) {
let result = native_list_append(result, k)
let result = native_list_append(result, val)
let found = true
} else {
let result = native_list_append(result, k)
let result = native_list_append(result, v)
}
let i = i + 2
}
if !found {
let result = native_list_append(result, key)
let result = native_list_append(result, val)
}
return result
}
fn make_slots(k0: String, v0: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, k0)
let r = native_list_append(r, v0)
return r
}
fn make_slots2(k0: String, v0: String, k1: String, v1: String) -> [String] {
let r: [String] = make_slots(k0, v0)
let r = native_list_append(r, k1)
let r = native_list_append(r, v1)
return r
}
fn make_slots3(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String) -> [String] {
let r: [String] = make_slots2(k0, v0, k1, v1)
let r = native_list_append(r, k2)
let r = native_list_append(r, v2)
return r
}
fn make_slots4(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String) -> [String] {
let r: [String] = make_slots3(k0, v0, k1, v1, k2, v2)
let r = native_list_append(r, k3)
let r = native_list_append(r, v3)
return r
}
fn make_slots5(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String, k4: String, v4: String) -> [String] {
let r: [String] = make_slots4(k0, v0, k1, v1, k2, v2, k3, v3)
let r = native_list_append(r, k4)
let r = native_list_append(r, v4)
return r
}
// Grammar rule catalog
fn rule_id(rule: [String]) -> String {
return native_list_get(rule, 0)
}
fn rule_lhs(rule: [String]) -> String {
return native_list_get(rule, 1)
}
fn rule_rhs_len(rule: [String]) -> Int {
let n: Int = native_list_len(rule)
return n - 2
}
fn rule_rhs(rule: [String], idx: Int) -> String {
return native_list_get(rule, idx + 2)
}
fn make_rule(id: String, lhs: String, r0: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, id)
let r = native_list_append(r, lhs)
let r = native_list_append(r, r0)
return r
}
fn make_rule2(id: String, lhs: String, r0: String, r1: String) -> [String] {
let r: [String] = make_rule(id, lhs, r0)
let r = native_list_append(r, r1)
return r
}
fn make_rule3(id: String, lhs: String, r0: String, r1: String, r2: String) -> [String] {
let r: [String] = make_rule2(id, lhs, r0, r1)
let r = native_list_append(r, r2)
return r
}
fn make_rule4(id: String, lhs: String, r0: String, r1: String, r2: String, r3: String) -> [String] {
let r: [String] = make_rule3(id, lhs, r0, r1, r2)
let r = native_list_append(r, r3)
return r
}
fn build_rules() -> [[String]] {
let rules: [[String]] = native_list_empty()
let rules = native_list_append(rules, make_rule2("S-DECL", "S", "NP", "VP"))
let rules = native_list_append(rules, make_rule3("S-QUEST", "S", "Aux", "NP", "VP"))
let rules = native_list_append(rules, make_rule("S-IMP", "S", "VP"))
let rules = native_list_append(rules, make_rule2("NP-DET-N", "NP", "Det", "N"))
let rules = native_list_append(rules, make_rule3("NP-DET-ADJ-N","NP", "Det", "Adj", "N"))
let rules = native_list_append(rules, make_rule("NP-PRON", "NP", "Pron"))
let rules = native_list_append(rules, make_rule("NP-N", "NP", "N"))
let rules = native_list_append(rules, make_rule("VP-V", "VP", "V"))
let rules = native_list_append(rules, make_rule2("VP-V-NP", "VP", "V", "NP"))
let rules = native_list_append(rules, make_rule2("VP-V-PP", "VP", "V", "PP"))
let rules = native_list_append(rules, make_rule3("VP-V-NP-PP", "VP", "V", "NP", "PP"))
let rules = native_list_append(rules, make_rule2("VP-AUX-V", "VP", "Aux", "V"))
let rules = native_list_append(rules, make_rule3("VP-AUX-V-NP", "VP", "Aux", "V", "NP"))
let rules = native_list_append(rules, make_rule2("PP-P-NP", "PP", "P", "NP"))
return rules
}
fn get_rules() -> [[String]] {
return build_rules()
}
fn find_rule(rule_id_str: String) -> [String] {
let rules: [[String]] = get_rules()
let n: Int = native_list_len(rules)
let i: Int = 0
while i < n {
let rule: [String] = native_list_get(rules, i)
let id: String = native_list_get(rule, 0)
if str_eq(id, rule_id_str) {
return rule
}
let i = i + 1
}
let empty: [String] = native_list_empty()
return empty
}
// Tree node construction
fn make_leaf(label: String, word: String) -> String {
return "(" + label + " " + word + ")"
}
fn make_node1(label: String, child0: String) -> String {
return "(" + label + " _ " + child0 + ")"
}
fn make_node2(label: String, child0: String, child1: String) -> String {
return "(" + label + " _ " + child0 + " " + child1 + ")"
}
fn make_node3(label: String, child0: String, child1: String, child2: String) -> String {
return "(" + label + " _ " + child0 + " " + child1 + " " + child2 + ")"
}
fn make_node4(label: String, child0: String, child1: String, child2: String, child3: String) -> String {
return "(" + label + " _ " + child0 + " " + child1 + " " + child2 + " " + child3 + ")"
}
// Tree rendering
fn nlg_is_ws(c: String) -> Bool {
if str_eq(c, " ") { return true }
if str_eq(c, "\t") { return true }
if str_eq(c, "\n") { return true }
return false
}
fn skip_ws(s: String, pos: Int) -> Int {
let n: Int = str_len(s)
let i: Int = pos
let running: Bool = true
while running {
if i >= n {
let running = false
} else {
let c: String = str_slice(s, i, i + 1)
if nlg_is_ws(c) {
let i = i + 1
} else {
let running = false
}
}
}
return i
}
fn scan_token(s: String, start: Int) -> [String] {
let n: Int = str_len(s)
let i: Int = start
let running: Bool = true
while running {
if i >= n {
let running = false
} else {
let c: String = str_slice(s, i, i + 1)
if nlg_is_ws(c) {
let running = false
} else {
if str_eq(c, "(") {
let running = false
} else {
if str_eq(c, ")") {
let running = false
} else {
let i = i + 1
}
}
}
}
}
let tok: String = str_slice(s, start, i)
let result: [String] = native_list_empty()
let result = native_list_append(result, tok)
let result = native_list_append(result, int_to_str(i))
return result
}
fn render_tree(tree: String) -> String {
let words: [String] = native_list_empty()
let n: Int = str_len(tree)
let i: Int = 0
let prev_was_open: Bool = false
while i < n {
let c: String = str_slice(tree, i, i + 1)
if str_eq(c, "(") {
let prev_was_open = true
let i = i + 1
} else {
if str_eq(c, ")") {
let prev_was_open = false
let i = i + 1
} else {
if nlg_is_ws(c) {
let i = i + 1
} else {
let tok_info: [String] = scan_token(tree, i)
let tok: String = native_list_get(tok_info, 0)
let new_i: Int = str_to_int(native_list_get(tok_info, 1))
let i = new_i
if prev_was_open {
let prev_was_open = false
} else {
if !str_eq(tok, "_") {
let words = native_list_append(words, tok)
}
}
}
}
}
}
return str_join(words, " ")
}
// Word-order engine
// gram_word_order: returns the word order string from a profile.
fn gram_word_order(profile: [String]) -> String {
return lang_word_order(profile)
}
// gram_order_constituents: order Subject, Verb, Object tokens according to the
// language profile's word_order.
//
// subj, verb, obj: surface strings (may be empty).
// Returns a space-joined string in the correct order.
//
// Supported orders: SVO, SOV, VSO, VOS, OVS, OSV, free (defaults to SVO).
fn gram_order_constituents(subj: String, verb: String, obj: String, profile: [String]) -> String {
let order: String = gram_word_order(profile)
let parts: [String] = native_list_empty()
if str_eq(order, "SVO") {
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
return str_join(parts, " ")
}
if str_eq(order, "SOV") {
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
return str_join(parts, " ")
}
if str_eq(order, "VSO") {
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
return str_join(parts, " ")
}
if str_eq(order, "VOS") {
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
return str_join(parts, " ")
}
if str_eq(order, "OVS") {
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
return str_join(parts, " ")
}
if str_eq(order, "OSV") {
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
return str_join(parts, " ")
}
// "free" and unknown: use SVO as the neutral citation order.
if !str_eq(subj, "") { let parts = native_list_append(parts, subj) }
if !str_eq(verb, "") { let parts = native_list_append(parts, verb) }
if !str_eq(obj, "") { let parts = native_list_append(parts, obj) }
return str_join(parts, " ")
}
// gram_build_vp: construct a verb phrase surface string.
//
// verb: main verb surface form.
// aux: auxiliary surface form (empty if none).
// profile: language profile.
//
// In SVO/VSO/VOS languages the auxiliary precedes the main verb.
// In SOV languages the verb cluster appears at the end; we keep aux before V
// as a reasonable default for the auxiliary-final constructions in those languages.
fn gram_build_vp(verb: String, aux: String, profile: [String]) -> String {
if str_eq(aux, "") {
return verb
}
return aux + " " + verb
}
// gram_question_strategy: returns the question formation strategy for a language.
//
// "do-support" - English: "Do you see?" do-auxiliary inserted, verb stays base
// "particle" - Japanese: sentence-final appended
// "intonation" - Mandarin, Spanish: rising intonation only, word order unchanged
// "inversion" - French, German: subject-verb inversion
fn gram_question_strategy(profile: [String]) -> String {
let code: String = lang_get(profile, "code")
if str_eq(code, "en") { return "do-support" }
if str_eq(code, "ja") { return "particle" }
if str_eq(code, "zh") { return "intonation" }
if str_eq(code, "es") { return "intonation" }
if str_eq(code, "fr") { return "inversion" }
if str_eq(code, "de") { return "inversion" }
if str_eq(code, "ar") { return "intonation" }
if str_eq(code, "hi") { return "particle" }
if str_eq(code, "ru") { return "intonation" }
if str_eq(code, "fi") { return "particle" }
if str_eq(code, "sw") { return "intonation" }
if str_eq(code, "la") { return "intonation" } // Latin: word order marks Q (VSO or -ne suffix)
if str_eq(code, "he") { return "intonation" } // Modern Hebrew: rising intonation
if str_eq(code, "grc") { return "intonation" } // Ancient Greek: ἆρα particle or intonation
if str_eq(code, "ang") { return "intonation" } // Old English: hwæþer particle or intonation
if str_eq(code, "sa") { return "intonation" } // Sanskrit: kim particle or intonation
if str_eq(code, "got") { return "intonation" } // Gothic: ibai particle or intonation
if str_eq(code, "non") { return "intonation" } // Old Norse: hvárr particle or intonation
if str_eq(code, "enm") { return "do-support" } // Middle English: do-support emerging
if str_eq(code, "pi") { return "intonation" } // Pali: kim particle or intonation
// Unknown: default to intonation (safest never wrong, just flat)
return "intonation"
}
// NP and PP assembly
//
// These functions are profile-aware but the logic is the same across languages
// because we work with pre-assembled strings (Engram vocabulary supplies
// language-specific forms before these functions see them).
fn is_pronoun(word: String) -> Bool {
if str_eq(word, "I") { return true }
if str_eq(word, "you") { return true }
if str_eq(word, "he") { return true }
if str_eq(word, "she") { return true }
if str_eq(word, "it") { return true }
if str_eq(word, "we") { return true }
if str_eq(word, "they") { return true }
if str_eq(word, "me") { return true }
if str_eq(word, "him") { return true }
if str_eq(word, "her") { return true }
if str_eq(word, "us") { return true }
if str_eq(word, "them") { return true }
return false
}
// build_np: assemble a noun phrase tree from a referent string.
// profile parameter reserved for future case-marking / article agreement.
fn build_np(referent: String, slots: [String]) -> String {
if is_pronoun(referent) {
return make_node1("NP", make_leaf("Pron", referent))
}
let parts: [String] = str_split(referent, " ")
let np: Int = native_list_len(parts)
if np == 1 {
return make_node1("NP", make_leaf("N", referent))
}
if np == 2 {
let det: String = native_list_get(parts, 0)
let noun: String = native_list_get(parts, 1)
return make_node2("NP", make_leaf("Det", det), make_leaf("N", noun))
}
if np == 3 {
let det: String = native_list_get(parts, 0)
let adj: String = native_list_get(parts, 1)
let noun: String = native_list_get(parts, 2)
return make_node3("NP", make_leaf("Det", det), make_leaf("Adj", adj), make_leaf("N", noun))
}
return make_node1("NP", make_leaf("N", referent))
}
// build_pp: assemble a prepositional phrase tree from a "PREP NP" string.
// For postpositional languages (ja, hi, ko) the slot value is expected to be
// already pre-assembled with the postposition in the correct position by the
// caller (vocabulary lookup from Engram supplies the right surface form).
fn build_pp(loc: String) -> String {
let parts: [String] = str_split(loc, " ")
let n: Int = native_list_len(parts)
if n < 2 {
return make_leaf("PP", loc)
}
let prep: String = native_list_get(parts, 0)
let np_parts: [String] = native_list_empty()
let i: Int = 1
while i < n {
let np_parts = native_list_append(np_parts, native_list_get(parts, i))
let i = i + 1
}
let np_str: String = str_join(np_parts, " ")
let np_tree: String = build_np(np_str, native_list_empty())
return make_node2("PP", make_leaf("P", prep), np_tree)
}
// VP tree construction
fn build_vp_body(slots: [String]) -> String {
let verb_surf: String = slots_get(slots, "verb_surf")
let patient: String = slots_get(slots, "patient")
let loc: String = slots_get(slots, "location")
if !str_eq(patient, "") {
let obj_np: String = build_np(patient, slots)
if !str_eq(loc, "") {
let pp: String = build_pp(loc)
return make_node3("VP", make_leaf("V", verb_surf), obj_np, pp)
}
return make_node2("VP", make_leaf("V", verb_surf), obj_np)
}
if !str_eq(loc, "") {
let pp: String = build_pp(loc)
return make_node2("VP", make_leaf("V", verb_surf), pp)
}
return make_node1("VP", make_leaf("V", verb_surf))
}
fn build_vp_from_slots(slots: [String]) -> String {
let aux_surf: String = slots_get(slots, "aux_surf")
if !str_eq(aux_surf, "") {
let verb_surf: String = slots_get(slots, "verb_surf")
let patient: String = slots_get(slots, "patient")
let loc: String = slots_get(slots, "location")
if !str_eq(patient, "") {
let obj_np: String = build_np(patient, slots)
return make_node3("VP", make_leaf("Aux", aux_surf), make_leaf("V", verb_surf), obj_np)
}
return make_node2("VP", make_leaf("Aux", aux_surf), make_leaf("V", verb_surf))
}
return build_vp_body(slots)
}
// Tree generator
fn generate_tree(rule_id_str: String, slots: [String]) -> String {
let rule: [String] = find_rule(rule_id_str)
let n: Int = native_list_len(rule)
if n == 0 {
return make_leaf("ERR", "unknown-rule")
}
let lhs: String = native_list_get(rule, 1)
if str_eq(rule_id_str, "S-DECL") {
let agent: String = slots_get(slots, "agent")
let np_tree: String = build_np(agent, slots)
let vp_tree: String = build_vp_from_slots(slots)
return make_node2("S", np_tree, vp_tree)
}
if str_eq(rule_id_str, "S-QUEST") {
let agent: String = slots_get(slots, "agent")
let np_tree: String = build_np(agent, slots)
let vp_tree: String = build_vp_body(slots)
let aux_surf: String = slots_get(slots, "aux_surf")
return make_node3("S", make_leaf("Aux", aux_surf), np_tree, vp_tree)
}
if str_eq(rule_id_str, "S-IMP") {
let vp_tree: String = build_vp_from_slots(slots)
return make_node1("S", vp_tree)
}
return make_leaf(lhs, "?")
}
+38
View File
@@ -0,0 +1,38 @@
// auto-generated by elc --emit-header — do not edit
extern fn slots_get(slots: [String], key: String) -> String
extern fn slots_set(slots: [String], key: String, val: String) -> [String]
extern fn make_slots(k0: String, v0: String) -> [String]
extern fn make_slots2(k0: String, v0: String, k1: String, v1: String) -> [String]
extern fn make_slots3(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String) -> [String]
extern fn make_slots4(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String) -> [String]
extern fn make_slots5(k0: String, v0: String, k1: String, v1: String, k2: String, v2: String, k3: String, v3: String, k4: String, v4: String) -> [String]
extern fn rule_id(rule: [String]) -> String
extern fn rule_lhs(rule: [String]) -> String
extern fn rule_rhs_len(rule: [String]) -> Int
extern fn rule_rhs(rule: [String], idx: Int) -> String
extern fn make_rule(id: String, lhs: String, r0: String) -> [String]
extern fn make_rule2(id: String, lhs: String, r0: String, r1: String) -> [String]
extern fn make_rule3(id: String, lhs: String, r0: String, r1: String, r2: String) -> [String]
extern fn make_rule4(id: String, lhs: String, r0: String, r1: String, r2: String, r3: String) -> [String]
extern fn build_rules() -> [[String]]
extern fn get_rules() -> [[String]]
extern fn find_rule(rule_id_str: String) -> [String]
extern fn make_leaf(label: String, word: String) -> String
extern fn make_node1(label: String, child0: String) -> String
extern fn make_node2(label: String, child0: String, child1: String) -> String
extern fn make_node3(label: String, child0: String, child1: String, child2: String) -> String
extern fn make_node4(label: String, child0: String, child1: String, child2: String, child3: String) -> String
extern fn nlg_is_ws(c: String) -> Bool
extern fn skip_ws(s: String, pos: Int) -> Int
extern fn scan_token(s: String, start: Int) -> [String]
extern fn render_tree(tree: String) -> String
extern fn gram_word_order(profile: [String]) -> String
extern fn gram_order_constituents(subj: String, verb: String, obj: String, profile: [String]) -> String
extern fn gram_build_vp(verb: String, aux: String, profile: [String]) -> String
extern fn gram_question_strategy(profile: [String]) -> String
extern fn is_pronoun(word: String) -> Bool
extern fn build_np(referent: String, slots: [String]) -> String
extern fn build_pp(loc: String) -> String
extern fn build_vp_body(slots: [String]) -> String
extern fn build_vp_from_slots(slots: [String]) -> String
extern fn generate_tree(rule_id_str: String, slots: [String]) -> String
+65
View File
@@ -0,0 +1,65 @@
// image-demo.el - Drive the native PNG surface: plan a scene from a small
// meaning phrase (incl. a NEG frame) and emit a byte-valid 64x64 PNG whose
// palette is read from elp/faculty/sig/scene.basis.
fn img_frame(relation: String, polarity: String, confidence: String, importance: String, salience: String, subj_id: String) -> [String] {
let f: [String] = native_list_empty()
let f: [String] = native_list_append(f, "relation")
let f: [String] = native_list_append(f, relation)
let f: [String] = native_list_append(f, "polarity")
let f: [String] = native_list_append(f, polarity)
let f: [String] = native_list_append(f, "confidence")
let f: [String] = native_list_append(f, confidence)
let f: [String] = native_list_append(f, "importance")
let f: [String] = native_list_append(f, importance)
let f: [String] = native_list_append(f, "salience")
let f: [String] = native_list_append(f, salience)
let f: [String] = native_list_append(f, "subj_id")
let f: [String] = native_list_append(f, subj_id)
return f
}
fn rgb_str(c: [Int]) -> String {
return int_to_str(native_list_get(c, 0)) + "," + int_to_str(native_list_get(c, 1)) + "," + int_to_str(native_list_get(c, 2))
}
fn run_image() -> Int {
fs_mkdir("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out")
let table: [Int] = crc_table()
println("crc_table[1]=" + int_to_str(native_list_get(table, 1)) + " (expect 1996959894 / 0x77073096)")
let basis: [String] = basis_load("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/sig/scene.basis")
let warm: [Int] = parse_rgb(basis_field(basis, "warm"))
let cool: [Int] = parse_rgb(basis_field(basis, "cool"))
let bg: [Int] = parse_rgb(basis_field(basis, "bg"))
println("basis warm=" + rgb_str(warm) + " cool=" + rgb_str(cool) + " bg=" + rgb_str(bg) + " (read from scene.basis)")
let frames: [[String]] = native_list_empty()
let frames: [[String]] = native_list_append(frames, img_frame("agent", "aff", "0.9", "0.8", "0", "s1"))
let frames: [[String]] = native_list_append(frames, img_frame("theme", "aff", "0.7", "0.6", "1", "s2"))
let frames: [[String]] = native_list_append(frames, img_frame("cause", "aff", "0.8", "0.9", "0", "s3"))
let frames: [[String]] = native_list_append(frames, img_frame("negation", "neg", "0.85", "0.7", "1", "s4"))
let frames: [[String]] = native_list_append(frames, img_frame("goal", "aff", "0.6", "0.5", "0", "s5"))
let frames: [[String]] = native_list_append(frames, img_frame("result", "aff", "0.95", "1.0", "1", "s6"))
let shapes: [[Int]] = plan_scene(frames, warm, cool)
let ns: Int = native_list_len(shapes)
println("planned " + int_to_str(ns) + " shapes:")
let si: Int = 0
while si < ns {
let sh: [Int] = native_list_get(shapes, si)
let pol: String = surface_get(native_list_get(frames, si), "polarity")
println(" shape " + int_to_str(si) + " type=" + int_to_str(native_list_get(sh, 0)) + " x=" + int_to_str(native_list_get(sh, 1)) + " y=" + int_to_str(native_list_get(sh, 2)) + " size=" + int_to_str(native_list_get(sh, 3)) + " rgb=" + int_to_str(native_list_get(sh, 4)) + "," + int_to_str(native_list_get(sh, 5)) + "," + int_to_str(native_list_get(sh, 6)) + " polarity=" + pol)
let si: Int = si + 1
}
let raw: [Int] = rasterize(64, 64, shapes, bg)
println("rasterized raw (filtered scanlines) bytes=" + int_to_str(native_list_len(raw)) + " (expect 12352)")
let png: [Int] = png_build(64, 64, raw, table)
let plen: Int = native_list_len(png)
let ok: Int = png_write("/Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/scene.png", png)
println("PNG bytes=" + int_to_str(plen) + " -> /Users/will/Development/neuron-technologies/foundation/el/.claude/worktrees/agent-aaf04b0a9714c4070/elp/faculty/out/scene.png (write_ok=" + int_to_str(ok) + ")")
return plen
}
println("image-demo returned png_bytes=" + int_to_str(run_image()))
+412
View File
@@ -0,0 +1,412 @@
// image-surface.el - Native own-core raster PNG surface (the image efferent
// twin of audio). Renders a 64x64 RGB scene deterministically from a frame's
// meaning-geometry, then serialises a byte-valid PNG entirely own-core:
// 8-byte magic, IHDR, IDAT (zlib STORED/uncompressed DEFLATE + Adler32), IEND,
// with a per-chunk CRC32 computed via software xor32 (EL has no bitwise ops).
//
// The RGB palette basis is read from elp/faculty/sig/scene.basis (data, not
// literals) - the same read-from-learned discipline as the audio signatures.
// Integer-only throughout; pixels are composed functionally (painter's order)
// so no list mutation is needed.
// -- small int/parse helpers (self-contained) ----------------------------------
fn i_str_to_int(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let v: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
if c >= 48 {
if c < 58 {
let v: Int = v * 10 + (c - 48)
}
}
let i: Int = i + 1
}
return v
}
fn basis_load(path: String) -> [String] {
return str_split(fs_read(path), "\n")
}
fn basis_field(lines: [String], key: String) -> String {
let pref: String = key + ": "
let n: Int = native_list_len(lines)
let plen: Int = str_len(pref)
let i: Int = 0
while i < n {
let ln: String = native_list_get(lines, i)
if str_starts_with(ln, pref) {
return str_slice(ln, plen, str_len(ln))
}
let i: Int = i + 1
}
return ""
}
fn parse_rgb(csv: String) -> [Int] {
let parts: [String] = str_split(csv, ",")
let out: [Int] = native_list_empty()
let n: Int = native_list_len(parts)
let i: Int = 0
while i < n {
let v: Int = i_str_to_int(native_list_get(parts, i))
let out: [Int] = native_list_append(out, v)
let i: Int = i + 1
}
return out
}
// -- software 32-bit XOR (no bitwise ops in EL) --------------------------------
fn xor32(a: Int, b: Int) -> Int {
let r: Int = 0
let bit: Int = 1
let i: Int = 0
while i < 32 {
let abit: Int = (a / bit) % 2
let bbit: Int = (b / bit) % 2
if abit != bbit {
let add: Int = bit
let r: Int = r + add
}
let bit: Int = bit * 2
let i: Int = i + 1
}
return r
}
// -- CRC32 (table-driven, table built with xor32) ------------------------------
fn crc_table() -> [Int] {
let t: [Int] = native_list_empty()
let n: Int = 0
while n < 256 {
let c: Int = n
let k: Int = 0
while k < 8 {
if c % 2 == 1 {
let h: Int = c / 2
let c: Int = xor32(h, 3988292384)
} else {
let c: Int = c / 2
}
let k: Int = k + 1
}
let t: [Int] = native_list_append(t, c)
let n: Int = n + 1
}
return t
}
fn crc32_of(bytes: [Int], table: [Int]) -> Int {
let crc: Int = 4294967295
let n: Int = native_list_len(bytes)
let i: Int = 0
while i < n {
let b: Int = native_list_get(bytes, i)
let lo: Int = crc % 256
let idx: Int = xor32(lo, b) % 256
let tv: Int = native_list_get(table, idx)
let hi: Int = crc / 256
let crc: Int = xor32(hi, tv)
let i: Int = i + 1
}
return xor32(crc, 4294967295)
}
// -- Adler32 (for the zlib trailer) --------------------------------------------
fn adler32_of(bytes: [Int]) -> Int {
let a: Int = 1
let b: Int = 0
let n: Int = native_list_len(bytes)
let i: Int = 0
while i < n {
let byte: Int = native_list_get(bytes, i)
let a: Int = (a + byte) % 65521
let b: Int = (b + a) % 65521
let i: Int = i + 1
}
return b * 65536 + a
}
// -- byte-list append helpers --------------------------------------------------
fn app_u32be(dst: [Int], v: Int) -> [Int] {
let dst: [Int] = native_list_append(dst, (v / 16777216) % 256)
let dst: [Int] = native_list_append(dst, (v / 65536) % 256)
let dst: [Int] = native_list_append(dst, (v / 256) % 256)
let dst: [Int] = native_list_append(dst, v % 256)
return dst
}
fn app_tag(dst: [Int], s: String) -> [Int] {
let n: Int = str_len(s)
let i: Int = 0
while i < n {
let dst: [Int] = native_list_append(dst, str_char_code(s, i))
let i: Int = i + 1
}
return dst
}
fn app_all(dst: [Int], src: [Int]) -> [Int] {
let n: Int = native_list_len(src)
let i: Int = 0
while i < n {
let dst: [Int] = native_list_append(dst, native_list_get(src, i))
let i: Int = i + 1
}
return dst
}
// -- plan: frame meaning-geometry -> shape atoms -------------------------------
// shape = [type, x, y, size, r, g, b] (type 0=rect 1=disc 2=triangle)
fn charsum(s: String) -> Int {
let n: Int = str_len(s)
let i: Int = 0
let acc: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
let acc: Int = acc + c
let i: Int = i + 1
}
return acc
}
fn micro_of(s: String) -> Int {
let dot: Int = str_index_of(s, ".")
if dot < 0 { return i_str_to_int(s) * 1000000 }
let n: Int = str_len(s)
let fp: String = str_slice(s, dot + 1, n)
let ip: String = str_slice(s, 0, dot)
let iv: Int = i_str_to_int(ip)
let fv: Int = 0
let scale: Int = 100000
let fl: Int = str_len(fp)
let i: Int = 0
while i < 6 {
let d: Int = 0
if i < fl { let d: Int = str_char_code(fp, i) - 48 }
let fv: Int = fv + d * scale
let scale: Int = scale / 10
let i: Int = i + 1
}
return iv * 1000000 + fv
}
fn plan_scene(frames: [[String]], warm: [Int], cool: [Int]) -> [[Int]] {
let shapes: [[Int]] = native_list_empty()
let nf: Int = native_list_len(frames)
let fi: Int = 0
while fi < nf {
let fr: [String] = native_list_get(frames, fi)
let relation: String = surface_get(fr, "relation")
let polarity: String = surface_get(fr, "polarity")
let confidence: String = surface_get(fr, "confidence")
let importance: String = surface_get(fr, "importance")
let salience: String = surface_get(fr, "salience")
// relation -> shape type
let stype: Int = charsum(relation) % 3
// confidence -> size (8..22)
let cmi: Int = micro_of(confidence)
let size: Int = 8 + cmi / 71428
// salience -> y
let sal: Int = i_str_to_int(salience)
let y: Int = 6 + sal * 26
// subj_id/index -> x
let x: Int = 4 + (fi * 10) % 48
// polarity -> warm/cool base color
let br: Int = native_list_get(warm, 0)
let bg2: Int = native_list_get(warm, 1)
let bb: Int = native_list_get(warm, 2)
if str_eq(polarity, "neg") {
let br: Int = native_list_get(cool, 0)
let bg2: Int = native_list_get(cool, 1)
let bb: Int = native_list_get(cool, 2)
}
// importance -> brightness (500..1000 permille)
let imi: Int = micro_of(importance)
let bpm: Int = 500 + imi / 2000
let r: Int = br * bpm / 1000
let g: Int = bg2 * bpm / 1000
let b: Int = bb * bpm / 1000
let sh: [Int] = native_list_empty()
let sh: [Int] = native_list_append(sh, stype)
let sh: [Int] = native_list_append(sh, x)
let sh: [Int] = native_list_append(sh, y)
let sh: [Int] = native_list_append(sh, size)
let sh: [Int] = native_list_append(sh, r)
let sh: [Int] = native_list_append(sh, g)
let sh: [Int] = native_list_append(sh, b)
let shapes: [[Int]] = native_list_append(shapes, sh)
let fi: Int = fi + 1
}
return shapes
}
// covers: is (px,py) inside this shape?
fn covers(sh: [Int], px: Int, py: Int) -> Bool {
let stype: Int = native_list_get(sh, 0)
let sx: Int = native_list_get(sh, 1)
let sy: Int = native_list_get(sh, 2)
let size: Int = native_list_get(sh, 3)
let cx: Int = sx + size / 2
if stype == 0 {
if px >= sx {
if px < sx + size {
if py >= sy {
if py < sy + size {
return true
}
}
}
}
return false
}
if stype == 1 {
let rad: Int = size / 2
let dx: Int = px - cx
let dy: Int = py - (sy + rad)
if dx * dx + dy * dy <= rad * rad {
return true
}
return false
}
// triangle: apex at top (sy), base at sy+size
if py >= sy {
if py < sy + size {
let dyv: Int = py - sy
let halfw: Int = dyv / 2
let dxv: Int = px - cx
let adx: Int = dxv
if adx < 0 { let adx: Int = 0 - dxv }
if adx <= halfw {
return true
}
}
}
return false
}
// pixel_color: painter's algorithm - last covering shape wins. Returns [r,g,b].
fn pixel_color(px: Int, py: Int, shapes: [[Int]], bg: [Int]) -> [Int] {
let r: Int = native_list_get(bg, 0)
let g: Int = native_list_get(bg, 1)
let b: Int = native_list_get(bg, 2)
let n: Int = native_list_len(shapes)
let i: Int = 0
while i < n {
let sh: [Int] = native_list_get(shapes, i)
if covers(sh, px, py) {
let r: Int = native_list_get(sh, 4)
let g: Int = native_list_get(sh, 5)
let b: Int = native_list_get(sh, 6)
}
let i: Int = i + 1
}
let out: [Int] = native_list_empty()
let out: [Int] = native_list_append(out, r)
let out: [Int] = native_list_append(out, g)
let out: [Int] = native_list_append(out, b)
return out
}
// rasterize: build the raw (filtered) scanline byte stream, filter byte 0 / row.
fn rasterize(w: Int, h: Int, shapes: [[Int]], bg: [Int]) -> [Int] {
let raw: [Int] = native_list_empty()
let y: Int = 0
while y < h {
let raw: [Int] = native_list_append(raw, 0)
let x: Int = 0
while x < w {
let col: [Int] = pixel_color(x, y, shapes, bg)
let raw: [Int] = native_list_append(raw, native_list_get(col, 0))
let raw: [Int] = native_list_append(raw, native_list_get(col, 1))
let raw: [Int] = native_list_append(raw, native_list_get(col, 2))
let x: Int = x + 1
}
let y: Int = y + 1
}
return raw
}
// zlib stream with a single STORED (uncompressed) DEFLATE block + Adler32.
fn zlib_store(raw: [Int]) -> [Int] {
let z: [Int] = native_list_empty()
let z: [Int] = native_list_append(z, 120)
let z: [Int] = native_list_append(z, 1)
let z: [Int] = native_list_append(z, 1)
let len: Int = native_list_len(raw)
let nlen: Int = 65535 - len
let z: [Int] = native_list_append(z, len % 256)
let z: [Int] = native_list_append(z, (len / 256) % 256)
let z: [Int] = native_list_append(z, nlen % 256)
let z: [Int] = native_list_append(z, (nlen / 256) % 256)
let z: [Int] = app_all(z, raw)
let ad: Int = adler32_of(raw)
let z: [Int] = app_u32be(z, ad)
return z
}
// append a full PNG chunk: length + (type+data) + crc32(type+data).
fn app_chunk(png: [Int], type_and_data: [Int], table: [Int]) -> [Int] {
let total: Int = native_list_len(type_and_data)
let dlen: Int = total - 4
let png: [Int] = app_u32be(png, dlen)
let png: [Int] = app_all(png, type_and_data)
let crc: Int = crc32_of(type_and_data, table)
let png: [Int] = app_u32be(png, crc)
return png
}
fn png_build(w: Int, h: Int, raw: [Int], table: [Int]) -> [Int] {
let png: [Int] = native_list_empty()
// 8-byte signature
let png: [Int] = native_list_append(png, 137)
let png: [Int] = native_list_append(png, 80)
let png: [Int] = native_list_append(png, 78)
let png: [Int] = native_list_append(png, 71)
let png: [Int] = native_list_append(png, 13)
let png: [Int] = native_list_append(png, 10)
let png: [Int] = native_list_append(png, 26)
let png: [Int] = native_list_append(png, 10)
// IHDR
let ihdr: [Int] = native_list_empty()
let ihdr: [Int] = app_tag(ihdr, "IHDR")
let ihdr: [Int] = app_u32be(ihdr, w)
let ihdr: [Int] = app_u32be(ihdr, h)
let ihdr: [Int] = native_list_append(ihdr, 8)
let ihdr: [Int] = native_list_append(ihdr, 2)
let ihdr: [Int] = native_list_append(ihdr, 0)
let ihdr: [Int] = native_list_append(ihdr, 0)
let ihdr: [Int] = native_list_append(ihdr, 0)
let png: [Int] = app_chunk(png, ihdr, table)
// IDAT
let z: [Int] = zlib_store(raw)
let idat: [Int] = native_list_empty()
let idat: [Int] = app_tag(idat, "IDAT")
let idat: [Int] = app_all(idat, z)
let png: [Int] = app_chunk(png, idat, table)
// IEND
let iend: [Int] = native_list_empty()
let iend: [Int] = app_tag(iend, "IEND")
let png: [Int] = app_chunk(png, iend, table)
return png
}
fn png_write(path: String, png: [Int]) -> Int {
let n: Int = native_list_len(png)
let buf: String = __str_alloc(n)
let i: Int = 0
while i < n {
let buf: String = __str_set_char(buf, i, native_list_get(png, i))
let i: Int = i + 1
}
let ok: Int = fs_write_bytes(path, buf, n)
return ok
}
+72
View File
@@ -0,0 +1,72 @@
;;; lang_profile_ca.el — Catalan language profile for ELP.
;;; Mirrors lang_profile_it / _es / _pt; keys the realizer's construction switches.
;;; Catalan is the CLOSEST Romance sibling to the shared engine (~85% conceptual
;;; reuse). The deltas: PRONOMS FEBLES with four position allomorphs, l'-elision,
;;; del/al/pel contractions, the periphrastic preterite (vaig+INF), and NO
;;; essere/avere split (perfect aux is always HAVER; ser/estar is only the copula).
(lang_profile_ca
(language "Catalan")
(iso639 "ca")
(family "Romance")
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop yes) ; null subjects default; overt pronoun = emphatic
(obligatory-subject no)
(grammatical-gender yes) ; m/f; full NP agreement (art + adj + participle)
(do-support no)
(subject-aux-inversion no) ; yes/no Q = declarative order + '?'; no inversion
(article-selection "el/la/l'/els/les ; un/una/uns/unes") ; l'-ELISION:
; el/la -> l' before vowel or (silent) h, glued to
; the next word (l'home, l'illa); de -> d' before vowel
(article-drives-contraction yes) ; article choice feeds prep+article contraction
(adjective-position "postnominal-default + small prenominal class") ; bo/bon,
; mal, gran, nou, vell, primer, molt... prenominal
(question-punct plain) ; ? and ! only (no inverted ¿ ¡)
;; ── MANDATORY prep+article contractions ────────────────────────────────
(contractions ((de el del) (de els dels)
(a el al) (a els als)
(per el pel) (per els pels)))
(contraction-mandatory yes) ; *de el -> del obligatory
(contraction-blocked-before-elision yes) ; de l'home / a l'home (NO *del home)
;; ── clitic system: PRONOMS FEBLES (the headline delta) ──────────────────
(clitics yes)
(clitic-allomorphy four-position) ; per pronoun, form varies by position+onset:
; reinforced (em, et, el) proclitic before a consonant
; elided (m', t', l', n') proclitic before a vowel/h
; full (-me, -lo, -li) enclitic after a consonant/-r
; reduced ('m, 't, 'l, 'ns) enclitic after a vowel
(clitic-placement ((finite proclitic) ; el veig, no m'ho dóna
(imperative-affirmative enclitic) ; dóna'm, digues-me
(imperative-negative present-subjunctive) ; no parlis (delta)
(infinitive enclitic) ; ajudar-me, veure'l
(gerund enclitic))) ; fent-ho
(clitic-combination ((me el "me'l") (te el "te'l") (se el "se'l")
(me la "me la") (me en "me'n")
(li el "l'hi") (li en "n'hi"))) ; dative+accusative clusters
(clitic-particles (hi en ho)) ; locative hi, partitive/genitive en, neuter ho
;; ── verb / aspect system ───────────────────────────────────────────────
(finite-agreement "person+number (6-way)")
(tenses (present imperfet preterit-simple perifrastic-preterit futur
condicional subjuntiu-present subjuntiu-imperfet imperatiu))
(periphrastic-preterite "vaig/vas/va/vam/vau/van + INFINITIVE") ; << hallmark CA
; (vaig cantar = 'I sang'); coexists w/ synthetic pret.
(compound-past "pretèrit perfet = haver(present) + participle")
(perfect-aux "HAVER only") ; << NO essere/avere split (simpler than IT)
(participle-agreement ((haver preceding-acc-clitic))) ; les he vistes; else invariable
(progressive-aux "estar + gerundi")
(copula "ser / estar") ; ser: identity/essential/origin; estar:
; location + transient state (estic cansat, és a casa)
(passive-aux "ser (+ per-agent)")
(future inflectional) ; cantaré, serà
(comparative "més/menys ADJ que")
;; ── SACRED safety bar (shared with es/pt/it/en) ────────────────────────
(negation-faithful yes) ; polarity never dropped/inverted; unplaceable -> FLAG
(negation "no (preverbal) + optional 'pas' + concord") ; no...res/
; ningú/mai/cap/gens/enlloc
(negative-concord yes) ; preverbal negative subject (ningú) keeps 'no'
(neg-reinforcer pas)) ; optional (no ho faré pas)
+41
View File
@@ -0,0 +1,41 @@
;;; lang_profile_de.el — German language profile for ELP.
;;; Mirrors lang_profile_en / lang_profile_es. Keys the realizer's construction
;;; switches. German is the largest Germanic delta from the EN engine: V2 word
;;; order, four morphological cases, and separable-prefix verbs.
(lang_profile_de
(language "German")
(iso639 "de")
(family "Germanic")
(neighbor-base "en") ; realized by extending the English (Germanic) engine
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop no) ; obligatory subject in finite clauses
(obligatory-subject yes)
(grammatical-gender (m f n)) ; three genders; drives article + adj declension
(case-system (nom acc dat gen)) ; four cases on articles/adjs/nouns
(word-order V2) ; finite verb 2nd in main clause
(subordinate-order verb-final) ; "..., dass er den Hund SIEHT."
(separable-verbs yes) ; aufstehen -> "steht ... auf"; ppart "aufgestanden"
(do-support no) ; German negates/questions the finite verb directly
(subject-verb-inversion yes) ; yes/no Q fronts finite verb; wh-Q fills Vorfeld
(article-selection "der/die/das + ein/kein") ; declined by case x gender x number
(adjective-position prenominal)
(adjective-declension (strong weak mixed)) ; chosen by the determiner type
(noun-capitalization yes)
;; ── verb / aspect system ───────────────────────────────────────────────
(finite-agreement "person-and-number") ; full present/past paradigm
(auxiliary-order (modal tense-aux perfect passive main))
(perfect-aux (haben sein)) ; sein for intransitive motion/change verbs
(passive-aux "werden")
(future "werden + infinitive")
(comparative "synthetic (-er / -st, with umlaut)")
;; ── negation ───────────────────────────────────────────────────────────
(negation-markers (nicht kein)) ; kein- negates an indefinite NP; nicht else
(negation-faithful yes) ; SACRED: polarity never dropped/inverted -> FLAG
;; ── lexicon provenance ─────────────────────────────────────────────────
(lexicon-source "UniMorph deu (primary) + kaikki.org German (gender override)")
(lexicon-license "CC-BY-SA 3.0 / GFDL"))
+41
View File
@@ -0,0 +1,41 @@
;;; lang_profile_en.el — English language profile for ELP.
;;; Mirrors lang_profile_es / lang_profile_pt; keys the realizer's construction
;;; switches. English is typologically distinct from the Romance builds, so the
;;; flags differ where the grammar differs.
(lang_profile_en
(language "English")
(iso639 "en")
(family "Germanic")
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop no) ; OBLIGATORY subjects — missing subject is FLAGGED
(obligatory-subject yes)
(grammatical-gender no) ; natural gender only (he/she/it), no NP agreement
(do-support yes) ; negation & questions of lexical verbs insert do/does/did
(subject-aux-inversion yes) ; yes/no + non-subject wh questions invert the operator
(article-selection "a/an/the") ; a/an resolved PHONOLOGICALLY (an hour, a university)
(adjective-position prenominal) ; attributive adjectives precede the noun; invariant
(has-tag-questions yes) ; "...doesn't he?" — operator + reversed polarity
(has-there-existential yes) ; "there is/are/have been ..."
(possessive-clitic "'s") ; saxon genitive; plural in -s -> bare apostrophe
(question-punct plain) ; ? and ! only (no inverted marks)
;; ── verb / aspect system ───────────────────────────────────────────────
(finite-agreement "3sg-present-only") ; only 3sg present -s (+ suppletive be)
(auxiliary-order (modal perfect progressive passive main))
(perfect-aux "have") ; have + past participle
(progressive-aux "be") ; be + present participle
(passive-aux "be") ; be + past participle (+ by-agent)
(future "will + base") ; no inflectional future
(comparative "synthetic-or-periphrastic") ; -er/-est vs more/most by syllables
;; ── SACRED safety bar (shared with es/pt) ──────────────────────────────
(negation-faithful yes) ; polarity never dropped/inverted; unplaceable -> FLAG
;; ── DIALECT overlay (post-realization, one core -> US/UK/AU) ────────────
(dialect US) ; default; profile field switches the overlay
(dialects (US UK AU))
(dialect-canonical US) ; core is authored in US orthography
(dialect-overlay "dialect_en.to_dialect") ; orthography + lexis + grammar prefs
(dialect-covers (spelling lexis collective-agreement gotten/got)))
+45
View File
@@ -0,0 +1,45 @@
;;; lang_profile_es.el — Spanish language profile for ELP.
;;; Keys the realizer's construction switches. Mirrors lang_profile_en / _pt.
(lang_profile_es
(language "Spanish")
(iso639 "es")
(family "Romance")
;; -- core typology flags -------------------------------------------------
(pro-drop yes) ; subjects routinely dropped; agreement carries person
(obligatory-subject no)
(grammatical-gender yes) ; m/f on every noun; article+adjective AGREE
(gender-source lexicon); REAL per-noun gender from UniMorph — NOT a heuristic
(do-support no)
(subject-aux-inversion no) ; questions by intonation/punctuation, not inversion
(question-strategy intonation)
(article-selection "el/la/los/las un/una/unos/unas")
(stressed-a-rule yes) ; fem sg noun in stressed a-/ha- takes el/un (el agua)
(adjective-position postnominal) ; default post; a few prenominal + apocope
(adjective-agreement "gender+number")
(question-punct inverted) ; opening ¿ ¡ required
;; -- MANDATORY CONTRACTIONS (coordinator quality bar) --------------------
(contractions ((de el "del") (a el "al")))
(contraction-mandatory yes) ; 'de el'/'a el' MUST surface as del/al
;; -- verb / aspect system ------------------------------------------------
(verb-classes (ar er ir))
(tenses (present preterite imperfect future conditional))
(moods (ind sbjv imp))
(finite-agreement "person+number (6 slots)")
(perfect-aux "haber") ; haber + past participle (invariant -o)
(progressive-aux "estar") ; estar + gerund
(passive-aux "ser") ; ser + participle (agrees) + por-agent
(copula-split "ser/estar") ; permanent vs stage-level
(future "infinitive + é/ás/á/emos/éis/án")
;; -- clitics / government ------------------------------------------------
(object-clitics yes) ; me te lo la le nos os los las; proclisis/enclisis
(clitic-order "se II I III (le+lo -> se lo)")
(enclisis "imperative/infinitive/gerund + accent repair (dá+me+lo->dámelo)")
(verb-prep-government yes) ; verbs select prep (protestar+contra, escapar+de)
;; -- SACRED safety bar (shared with en/pt) -------------------------------
(negation-faithful yes)) ; polarity never dropped/inverted; unplaceable -> FLAG
+74
View File
@@ -0,0 +1,74 @@
;;; lang_profile_fr.el — French language profile for ELP.
;;; Mirrors lang_profile_it / lang_profile_es; keys the realizer's construction
;;; switches. French is a Romance sibling (~54% of the realizer code and the whole
;;; clause-engine architecture reused), but carries the family's biggest surface
;;; deltas: NOT pro-drop, DISCONTINUOUS negation, and an orthography/phonology
;;; mismatch (elision, liaison) that makes exact-match genuinely hard.
(lang_profile_fr
(language "French")
(iso639 "fr")
(family "Romance")
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop no) ; << French-specific: subject clitic OBLIGATORY
(obligatory-subject yes) ; je/tu/il/elle/nous/vous/ils/elles always overt
(grammatical-gender yes) ; m/f; full NP agreement (art + adj + participle)
(do-support no)
(subject-aux-inversion optional) ; est-ce que (default) OR clitic inversion (vas-tu)
(article-selection "le/la/l'/les ; un/une/des ; PARTITIVE du/de la/de l'/des")
(article-drives-contraction yes) ; à+le=au, de+le=du feed off article choice
(adjective-position "postnominal-default + prenominal-BAGS") ; beau/bon/grand/
; petit/jeune/vieux/nouveau + ordinals prenominal
; (beau->bel, nouveau->nouvel, vieux->vieil / vowel)
(question-punct "space-before") ; French typography: ' ?' ' !' (no ¿¡)
;; ── elision (orthography/phonology mismatch — French-specific) ──────────
(elision ((le l') (la l') (je j') (ne n') (de d') (que qu')
(me m') (te t') (se s') (ce c'))) ; before vowel / h-muet
(elision-h-muet yes) ; l'homme, l'hôpital (h-aspiré exception list kept)
(liaison noted-not-modeled) ; phonological, not written in surface
;; ── MANDATORY prep+article contractions ────────────────────────────────
(contractions ((à le au) (à les aux) (de le du) (de les des)))
(contraction-mandatory yes) ; *à le -> au obligatory; à la / à l' uncontracted
(partitive ((m-sg du) (f-sg "de la") (vowel "de l'") (pl des)))
(partitive-under-neg "de") ; << gap in current build: 'ne … pas de pain'
;; ── clitic system ──────────────────────────────────────────────────────
(clitics yes)
(clitic-order (me te se nous vous | le la les | lui leur | y | en))
(clitic-placement ((finite proclitic) ; je le lui donne
(imperative-affirmative enclitic-hyphen) ; donne-le-moi
(imperative-negative "ne+proclitic+verb+pas") ; ne le donne pas
(infinitive enclitic))) ; PARTIAL: clitic-climbing
; onto infinitive under modal
(clitic-imperative-shift ((me moi) (te toi))) ; final me/te -> moi/toi (donne-moi)
(clitic-particles (y en)) ; locative y, partitive/genitive en
;; ── verb / aspect system ───────────────────────────────────────────────
(finite-agreement "person+number (written; many homophones)")
(tenses (présent imparfait passé-simple futur conditionnel
subjonctif-présent subjonctif-imparfait impératif))
(compound-past "passé-composé = aux(present) + participe passé")
(perfect-aux "être/avoir (LEXICAL selection)") ; << French-specific
(etre-aux-class "intransitive motion/change (aller venir arriver partir
entrer sortir monter descendre naître mourir rester
tomber retourner passer devenir revenir rentrer) + ALL
pronominal verbs")
(participle-agreement ((être subject) ; elle est allée / elles venues
(avoir preceding-direct-object))) ; je les ai vus
(progressive "être en train de + infinitif") ; no dedicated aux
(copula "être (single; no ser/estar, no essere/stare)")
(passive-aux "être (+ par-agent)")
(future inflectional) ; parlera, sera
(comparative "plus/moins ADJ que")
(superlative "le/la plus ADJ (de …)") ; PARTIAL word-order in build
;; ── SACRED safety bar (shared with es/pt/it/en) ────────────────────────
(negation-faithful yes) ; polarity never dropped/inverted; unplaceable -> FLAG
(negation "DISCONTINUOUS: ne (preverbal) … pas/jamais/rien/personne/
plus/guère/que (postverbal)") ; << biggest structural delta
(negation-ne-elides yes) ; ne -> n' before vowel (n'ai pas vu)
(negation-passe-composé "ne + aux + pas + participe") ; n'ai pas vu
(negative-concord partial)) ; personne/rien as arguments post-participle
+70
View File
@@ -0,0 +1,70 @@
;;; lang_profile_it.el — Italian language profile for ELP.
;;; Mirrors lang_profile_es / lang_profile_pt; keys the realizer's construction
;;; switches. Italian is a Romance sibling, so ~85% of the flags match ES/PT; the
;;; essere/avere auxiliary split and phonological article selection are the deltas.
(lang_profile_it
(language "Italian")
(iso639 "it")
(family "Romance")
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop yes) ; null subjects default; overt pronoun = emphatic
(obligatory-subject no)
(grammatical-gender yes) ; m/f; full NP agreement (art + adj + participle)
(do-support no)
(subject-aux-inversion no) ; yes/no Q = declarative order + '?'; no inversion
(article-selection "il/lo/l'/i/gli + la/l'/le ; un/uno/un'/una") ; PHONOLOGICAL:
; lo/gli/uno before s+cons, z, gn, ps, pn, x, y, i+V;
; l'/un' before a vowel (elision, glued to next word)
(article-drives-contraction yes) ; article choice feeds the prep+art contraction
(adjective-position "postnominal-default + prenominal-class") ; bello/buono/grande
; /nuovo/vecchio/primo... prenominal (with apocope)
(question-punct plain) ; ? and ! only (no inverted ¿ ¡)
;; ── MANDATORY prep+article contractions ────────────────────────────────
(contractions ((di il del) (di lo dello) (di la della) (di i dei)
(di gli degli) (di le delle) (di l' dell')
(a il al) (a lo allo) (a la alla) (a i ai) (a gli agli)
(a le alle) (a l' all')
(da il dal) (da la dalla) (da gli dagli) (da l' dall')
(in il nel) (in la nella) (in gli negli) (in l' nell')
(su il sul) (su la sulla) (su gli sugli) (su l' sull')))
(contraction-mandatory yes) ; *di il -> del is obligatory, never uncontracted
(prep-no-contract (per tra fra)) ; per la strada (NOT *perla)
;; ── clitic system ──────────────────────────────────────────────────────
(clitics yes)
(clitic-placement ((finite proclitic) ; lo vedo, non me lo dà
(imperative-affirmative enclitic) ; dammelo, guardalo
(imperative-negative-tu non+infinitive) ; non parlare / non lo fare
(infinitive enclitic) ; vederlo, aiutarmi (drop -e)
(gerund enclitic))) ; dandolo
(clitic-combination ((mi lo "me lo") (ti lo "te lo") (ci lo "ce lo")
(vi lo "ve lo") (si lo "se lo")
(gli lo "glielo") (le lo "glielo"))) ; glielo = ONE word
(clitic-particles (ci ne)) ; locative ci, partitive ne
(raddoppiamento (da fa di va sta)) ; monosyllabic imper double clitic: dammelo
;; ── verb / aspect system ───────────────────────────────────────────────
(finite-agreement "person+number (6-way)")
(tenses (presente imperfetto passato-remoto futuro condizionale
congiuntivo-presente congiuntivo-imperfetto imperativo))
(compound-past "passato-prossimo = aux(present) + participle")
(perfect-aux "essere/avere (LEXICAL selection)") ; << Italian-specific
(essere-aux-class unaccusative) ; motion/change-of-state/copular/pronominal
; (andare venire nascere morire diventare piacere
; + ALL reflexives) -> essere
(participle-agreement ((essere subject) ; è andata / sono arrivati
(avere preceding-acc-clitic))) ; li ho visti
(progressive-aux "stare + gerundio") ; sto parlando
(copula "essere (default) / stare (state: sto bene)")
(passive-aux "essere / venire (+ da-agent)")
(future inflectional) ; parlerò, sarà
(comparative "più/meno ADJ di")
;; ── SACRED safety bar (shared with es/pt/en) ───────────────────────────
(negation-faithful yes) ; polarity never dropped/inverted; unplaceable -> FLAG
(negation "non (preverbal) + concord") ; non...niente/nessuno/mai/più
(negative-concord yes) ; preverbal negative word (nessuno/niente) suppresses non
(neg-adverb-position between-aux-and-participle)) ; non ho MAI visto
+30
View File
@@ -0,0 +1,30 @@
;;; lang_profile_la.el — Latin language profile for ELP.
;;; Keys the realizer's construction switches. Companion to morphology-la.el.
(lang_profile_la
(language "Latin")
(iso639 "la")
(family "Italic")
;; -- core typology flags -------------------------------------------------
(pro-drop yes) ; person carried by verb ending; subjects dropped
(obligatory-subject no)
(grammatical-gender yes) ; m/f/n; adjective AGREES in case+gender+number
(gender-source lexicon) ; REAL per-noun gender from UniMorph lat
(articles none) ; Latin has no articles
(case-system yes) ; NOM GEN DAT ACC ABL VOC (+ rare LOC)
(cases (nom gen dat acc abl voc))
(word-order "SOV (default; free order, case-marked)")
(adjective-position "either (case agreement carries the link)")
(adjective-agreement "case+gender+number")
;; -- verb / aspect system ------------------------------------------------
(verb-classes (1 2 3 3io 4)) ; four conjugations + i-stem 3rd
(tenses (present imperfect future perfect pluperfect futureperfect))
(moods (indicative subjunctive imperative infinitive))
(voices (active passive))
(finite-agreement "person+number (6 slots)")
(citation "principal parts: pres-1sg / pres-inf / perf-participle")
;; -- SACRED safety bar ---------------------------------------------------
(negation-faithful yes)) ; polarity never dropped/inverted
+40
View File
@@ -0,0 +1,40 @@
;;; lang_profile_pt.el — Portuguese language profile for ELP.
;;; Keys the realizer's construction switches. Mirrors lang_profile_es.
(lang_profile_pt
(language "Portuguese")
(iso639 "pt")
(family "Romance")
;; -- core typology flags -------------------------------------------------
(pro-drop yes) ; subjects routinely dropped; agreement carries person
(obligatory-subject no)
(grammatical-gender yes) ; m/f on every noun; article+adjective AGREE
(gender-source lexicon) ; REAL per-noun gender from UniMorph por / kaikki
(do-support no)
(subject-aux-inversion no)
(question-strategy intonation)
(article-selection "o/a/os/as um/uma/uns/umas")
(adjective-position postnominal)
(adjective-agreement "gender+number")
;; -- MANDATORY CONTRACTIONS (prep + article) -----------------------------
(contractions ((de o "do") (de a "da") (em o "no") (em a "na")
(a o "ao") (a a "à") (por o "pelo") (por a "pela")))
(contraction-mandatory yes)
;; -- verb / aspect system ------------------------------------------------
(verb-classes (ar er ir))
(tenses (present preterite imperfect future conditional))
(moods (ind sbjv imp))
(finite-agreement "person+number (6 slots)")
(perfect-aux "ter") ; ter + past participle
(copula-split "ser/estar")
(personal-infinitive yes) ; distinctive PT inflected infinitive
;; -- clitics / government ------------------------------------------------
(object-clitics yes) ; mesoclisis/enclisis/proclisis by context
(verb-prep-government yes)
;; -- SACRED safety bar ---------------------------------------------------
(negation-faithful yes))
+71
View File
@@ -0,0 +1,71 @@
;;; lang_profile_ro.el — Romanian language profile for ELP.
;;; Romanian is the BIG typological delta of the Romance family. The verb/clause
;;; engine and the SACRED negation contract mirror the ES/PT/IT core, but the
;;; NOMINAL system is genuinely new: a SUFFIXED definite article, preserved CASE,
;;; a NEUTER gender, and a VOCATIVE. Those flags mark where the shared engine was
;;; extended rather than reused.
(lang_profile_ro
(language "Romanian")
(iso639 "ro")
(family "Romance (Eastern / Balkan)")
;; ── core typology flags ────────────────────────────────────────────────
(pro-drop yes) ; null subjects default; overt pronoun = emphatic
(obligatory-subject no)
(grammatical-gender yes) ; m / f / NEUTER (n)
(neuter-gender yes) ; << ROMANIAN-SPECIFIC: masc-agreeing SG, fem-agreeing PL
; (un tren nou / două trenuri noi)
(do-support no)
(subject-aux-inversion no) ; yes/no Q = declarative order + '?'
(question-punct plain) ; ? and ! only
;; ── SUFFIXED DEFINITE ARTICLE (the headline engine extension) ───────────
(definite-article suffixed) ; << UNIQUE IN ROMANCE: enclitic on the noun
(definite-forms ((m/n sg "-ul / -le / -l : om->omul, câine->câinele, codru->codrul")
(f sg "-a / -ea / -ua : casă->casa, carte->cartea, stea->steaua")
(m pl "-i : oameni->oamenii")
(f/n pl "-le : case->casele, trenuri->trenurile")))
(article-host ((no-prenom-adj noun) ; omul bun
(prenom-adj adjective))) ; bunul om (adj carries the article)
(indefinite-article ((m/n "un") (f "o") (pl "niște") (gen/dat-pl "unor")))
;; ── CASE (preserved; NOM/ACC vs GEN/DAT) ────────────────────────────────
(case (nom/acc gen/dat vocative)) ; << ROMANIAN-SPECIFIC
(case-syncretism "nom=acc ; gen=dat")
(genitive-marking "gen/dat definite: -lui (m/n), -ei/-i (f), -lor (pl)")
(genitival-article ((m sg "al") (f sg "a") (m pl "ai") (f/n pl "ale"))) ; o carte a lui
(possession "definite-head + gen/dat possessor: casa băiatului")
(vocative ((m sg "-ule/-e : omule, băiete") (f sg "-o : Mario, fato")
(pl "-lor")))
;; ── verb / aspect system ────────────────────────────────────────────────
(finite-agreement "person+number (6-way)")
(tenses (prezent imperfect perfect-simplu conjunctiv-prezent
imperativ (periphrastic: perfect-compus viitor conditional)))
(compound-past "perfectul compus = a-avea-clitic + INVARIABLE participle")
(perfect-aux "a avea (am/ai/a/am/ați/au) — ONE auxiliary for ALL verbs")
(perfect-aux-split no) ; << SIMPLER than Italian: no essere/avere selection
(participle-agreement none) ; invariable in the perfect compus (agrees only as
; an adjective / in the passive)
(future "voi/vei/va/vom/veți/vor + infinitive (viitor literar)")
(conditional "aș/ai/ar/am/ați/ar + infinitive")
(subjunctive "conjunctiv: particle 'să' + subjunctive present")
(modal-complement "modal + să + subjunctive (vreau să merg, poți să ajuți)")
(copula "a fi")
(passive "a fi + participle (participle AGREES like an adjective)")
(comparative "mai / mai puțin ADJ decât")
;; ── clitic system (partial — see honest gaps) ───────────────────────────
(clitics yes)
(clitic-set ((acc te îl o ne îi le) (dat îmi îți îi ne le)
(refl te se ne se)))
(clitic-placement ((finite proclitic) ; îmi place, o văd
(perfect-compus elision) ; << m-am, l-am, i-am (PARTIAL)
(imperative-affirmative enclitic))) ; dă-mi (PARTIAL)
;; ── SACRED safety bar (shared with es/pt/it/en) ─────────────────────────
(negation-faithful yes) ; polarity never dropped/inverted; unplaceable -> FLAG
(negation "nu (single preverbal marker) + concord")
(negative-concord yes) ; nu … nimic / nimeni / niciodată / niciun
(negative-imperative "nu + INFINITIVE : nu pleca! (KNOWN GAP: uses imperative stem)"))
+761
View File
@@ -0,0 +1,761 @@
// big language-profile for testing
fn lang_profile_big0(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big0(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big0("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big1(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big1(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big1("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big2(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big2(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big2("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big3(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big3(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big3("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big4(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big4(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big4("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big5(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big5(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big5("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big6(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big6(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big6("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big7(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big7(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big7("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big8(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big8(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big8("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big9(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big9(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big9("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big10(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big10(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big10("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big11(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big11(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big11("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big12(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big12(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big12("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big13(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big13(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big13("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big14(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big14(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big14("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big15(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big15(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big15("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big16(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big16(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big16("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big17(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big17(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big17("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big18(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big18(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big18("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
fn lang_profile_big19(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
fn lang_get_big19(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
fn lang_profile_en() -> [String] {
return lang_profile_big19("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
+353
View File
@@ -0,0 +1,353 @@
// language-profile.el - Language profile data and accessors.
//
// A language profile is a slot map ([String] key-value list) describing the
// typological properties of a natural language. The engine reads these
// properties to drive morphology, word-order, and question-formation without
// any per-language code paths.
//
// Profile slot keys:
// code - ISO 639-1 code: "en", "ja", "ar", "zh", "de", "fr", "es", "sw", "hi", "ru", etc.
// word_order - "SVO" | "SOV" | "VSO" | "VOS" | "OVS" | "OSV" | "free"
// morph_type - "isolating" | "agglutinative" | "fusional" | "polysynthetic"
// has_case - "true" | "false"
// has_gender - "true" | "false"
// script_dir - "ltr" | "rtl" | "ttb"
// agreement - semicolon-separated features: "number;person" | "number;person;gender;case" | "none"
// null_subject - "true" | "false" (pro-drop: subject may be omitted)
// Constructor
fn lang_profile(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String] {
let r: [String] = native_list_empty()
let r = native_list_append(r, "code")
let r = native_list_append(r, code)
let r = native_list_append(r, "word_order")
let r = native_list_append(r, word_order)
let r = native_list_append(r, "morph_type")
let r = native_list_append(r, morph_type)
let r = native_list_append(r, "has_case")
let r = native_list_append(r, has_case)
let r = native_list_append(r, "has_gender")
let r = native_list_append(r, has_gender)
let r = native_list_append(r, "script_dir")
let r = native_list_append(r, script_dir)
let r = native_list_append(r, "agreement")
let r = native_list_append(r, agreement)
let r = native_list_append(r, "null_subject")
let r = native_list_append(r, null_subject)
return r
}
// Accessor
fn lang_get(profile: [String], key: String) -> String {
let n: Int = native_list_len(profile)
let i: Int = 0
while i < n - 1 {
let k: String = native_list_get(profile, i)
if str_eq(k, key) {
return native_list_get(profile, i + 1)
}
let i = i + 2
}
return ""
}
// Built-in profiles
//
// Each profile encodes typological facts about one language. These are data,
// not separate code paths. Adding a new language means adding a new profile
// and loading its vocabulary/suffix tables into the Engram - no engine changes.
// English: SVO, fusional, no grammatical case (nominative/accusative collapsed),
// no grammatical gender, left-to-right, agreement on number and person,
// obligatory subject (no pro-drop).
fn lang_profile_en() -> [String] {
return lang_profile("en", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
// Japanese: SOV, agglutinative, grammatical relations marked by postpositions
// (not inflectional case), no grammatical gender, left-to-right, no agreement
// morphology on verbs, pro-drop (null subject frequent).
fn lang_profile_ja() -> [String] {
return lang_profile("ja", "SOV", "agglutinative", "false", "false", "ltr", "none", "true")
}
// Arabic: VSO, fusional, full case system, grammatical gender (masc/fem),
// right-to-left script, agreement on number, person, gender, and case,
// pro-drop (subject agreement marking on verb allows subject omission).
fn lang_profile_ar() -> [String] {
return lang_profile("ar", "VSO", "fusional", "true", "true", "rtl", "number;person;gender;case", "true")
}
// Mandarin Chinese: SVO, isolating (no morphological inflection), no case,
// no grammatical gender, left-to-right, no agreement (no morphological marking),
// null subject allowed in discourse context.
fn lang_profile_zh() -> [String] {
return lang_profile("zh", "SVO", "isolating", "false", "false", "ltr", "none", "true")
}
// German: V2 (second-position verb, base SOV in subordinate clauses), fusional,
// four-case system, three grammatical genders, left-to-right, agreement on
// number, person, gender, and case, obligatory subject.
fn lang_profile_de() -> [String] {
return lang_profile("de", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "false")
}
// Spanish: SVO, fusional, no morphological case (but object clitics exist),
// grammatical gender (masc/fem), left-to-right, agreement on number, person,
// and gender, pro-drop (rich verbal agreement allows subject omission).
fn lang_profile_es() -> [String] {
return lang_profile("es", "SVO", "fusional", "false", "true", "ltr", "number;person;gender", "true")
}
// Finnish: SOV, agglutinative, fifteen grammatical cases, no grammatical gender,
// left-to-right, agreement on number, person, and case, no pro-drop (subject
// required in finite clauses).
fn lang_profile_fi() -> [String] {
return lang_profile("fi", "SOV", "agglutinative", "true", "false", "ltr", "number;person;case", "false")
}
// Swahili: SVO, agglutinative, noun-class system (15+ classes replacing gender),
// no case inflection, left-to-right, agreement driven by noun class and number,
// pro-drop (subject prefix on verb can stand alone).
fn lang_profile_sw() -> [String] {
return lang_profile("sw", "SVO", "agglutinative", "false", "false", "ltr", "noun-class;number", "true")
}
// Hindi: SOV, fusional, case-marked postpositional system, grammatical gender
// (masc/fem), left-to-right (Devanagari script still ltr), agreement on number,
// person, gender, and case, pro-drop (subject frequently dropped).
fn lang_profile_hi() -> [String] {
return lang_profile("hi", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Russian: free word order (pragmatically determined), fusional, six-case system,
// three grammatical genders, left-to-right (Cyrillic), agreement on number,
// person, gender, and case, no pro-drop (subject required).
fn lang_profile_ru() -> [String] {
return lang_profile("ru", "free", "fusional", "true", "true", "ltr", "number;person;gender;case", "false")
}
// French: SVO, fusional, no morphological case (but clitic object pronouns),
// two grammatical genders (masc/fem), left-to-right, agreement on number,
// person, and gender, no pro-drop.
fn lang_profile_fr() -> [String] {
return lang_profile("fr", "SVO", "fusional", "false", "true", "ltr", "number;person;gender", "false")
}
// Latin: SOV (highly free word order), fusional, six-case system (nom/gen/dat/acc/abl/voc),
// three genders (masc/fem/neut), left-to-right, rich agreement on number, person, gender,
// and case, pro-drop (subject expressed in verb ending).
fn lang_profile_la() -> [String] {
return lang_profile("la", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Hebrew (Modern): SVO, Semitic trilateral root morphology, two genders (masc/fem),
// two numbers (singular/plural; dual vestigial), right-to-left (Hebrew script),
// agreement on number, person, gender; zero copula in present tense; no grammatical cases.
fn lang_profile_he() -> [String] {
return lang_profile("he", "SVO", "semitic", "true", "false", "rtl", "number;person;gender", "true")
}
// Sanskrit: SOV/free, highly fusional, 3 genders, 8 cases, 3 numbers (sg/du/pl),
// Devanagari script, rich verb system (10 classes, 9 tenses/moods), pro-drop.
fn lang_profile_sa() -> [String] {
return lang_profile("sa", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Gothic: SOV, fusional, 3 genders, 4 cases, singular/plural,
// Gothic alphabet (romanized as þ/ƕ/ai/au/ei), strong and weak classes, pro-drop.
fn lang_profile_got() -> [String] {
return lang_profile("got", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Old Norse: free/SOV, fusional, 3 genders, 4 cases, singular/plural,
// definite article as noun suffix (-inn/-in/-it), strong and weak classes, pro-drop.
fn lang_profile_non() -> [String] {
return lang_profile("non", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Middle English (ca. 11001500): SVO emerging, mostly lost case system,
// -es plural/genitive, strong and weak verbs, no grammatical gender on nouns.
fn lang_profile_enm() -> [String] {
return lang_profile("enm", "SVO", "fusional", "false", "false", "ltr", "number;person", "false")
}
// Pali: SOV, fusional (simplified Sanskrit), 3 genders, 8 cases, sg/pl,
// Latin transliteration with IAST diacritics, Buddhist canonical language.
fn lang_profile_pi() -> [String] {
return lang_profile("pi", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Ancient Greek: free/SOV word order, highly fusional, 3 genders, 5 cases (nom/acc/gen/dat/voc),
// singular/dual/plural, polytonic Greek script (Unicode), complex verb system with aspect
// (imperfective/perfective), augment in past tenses, pro-drop.
fn lang_profile_grc() -> [String] {
return lang_profile("grc", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case;aspect", "true")
}
// Old English (Anglo-Saxon): SOV/V2, fusional, 3 genders, 4 cases (nom/acc/gen/dat),
// singular/plural, Latin alphabet + þ/ð/ƿ/æ, strong and weak noun/verb classes, pro-drop.
fn lang_profile_ang() -> [String] {
return lang_profile("ang", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Old French (ca. 10001300 CE): SVO/V2, fusional, two-case system (nominative/oblique),
// two genders (masculine/feminine), left-to-right, agreement on number, person, gender,
// and case, no pro-drop (subject generally required).
fn lang_profile_fro() -> [String] {
return lang_profile("fro", "SVO", "fusional", "true", "true", "ltr", "number;person;gender;case", "false")
}
// Old High German (ca. 7501050 CE): SOV/V2, fusional, four-case system, three genders,
// left-to-right, agreement on number, person, gender, and case, pro-drop.
fn lang_profile_goh() -> [String] {
return lang_profile("goh", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Old Irish (ca. 600900 CE): VSO, fusional, case system, three genders,
// left-to-right, agreement on number, person, gender, and case, pro-drop.
fn lang_profile_sga() -> [String] {
return lang_profile("sga", "VSO", "fusional", "true", "true", "ltr", "number;person;gender;case", "true")
}
// Tocharian B (ca. 5001000 CE): SOV, fusional, case system, two genders,
// left-to-right, agreement on number, person, gender, and case, no pro-drop.
fn lang_profile_txb() -> [String] {
return lang_profile("txb", "SOV", "fusional", "true", "true", "ltr", "number;person;gender;case", "false")
}
// Old Persian (ca. 525330 BCE): SOV, fusional, 8-case system, no grammatical gender,
// left-to-right, agreement on number, person, and case, pro-drop.
fn lang_profile_peo() -> [String] {
return lang_profile("peo", "SOV", "fusional", "true", "false", "ltr", "number;person;case", "true")
}
// Akkadian (Old Babylonian period, ca. 19001600 BCE): VSO, fusional, 3-case system
// (nominative/accusative/genitive with mimation), two genders, left-to-right,
// agreement on number, person, gender, and case, no pro-drop.
fn lang_profile_akk() -> [String] {
return lang_profile("akk", "VSO", "fusional", "true", "true", "ltr", "number;person;gender;case", "false")
}
// Ugaritic (ca. 14001200 BCE): VSO, Semitic trilateral root morphology, 3-case system,
// two genders, left-to-right (cuneiform alphabetic script), agreement on number, person,
// gender, and case, no pro-drop.
fn lang_profile_uga() -> [String] {
return lang_profile("uga", "VSO", "semitic", "true", "true", "ltr", "number;person;gender;case", "false")
}
// Ancient Egyptian / Middle Egyptian (ca. 21001300 BCE): SVO, agglutinative,
// no morphological case (word order + prepositions), two genders, left-to-right,
// agreement on number, person, and gender, pro-drop (zero copula in present).
fn lang_profile_egy() -> [String] {
return lang_profile("egy", "SVO", "agglutinative", "false", "true", "ltr", "number;person;gender", "true")
}
// Sumerian (ca. 30002000 BCE): SOV, agglutinative, ergative-absolutive case system,
// no grammatical gender (animacy distinction instead), left-to-right, agreement on
// number and person, pro-drop.
fn lang_profile_sux() -> [String] {
return lang_profile("sux", "SOV", "agglutinative", "true", "false", "ltr", "number;person", "true")
}
// Ge'ez (Classical Ethiopic, ca. 4th7th century CE): SOV, Semitic trilateral root
// morphology, two genders (masc/fem), Ethiopic/Fidel script (ltr), agreement on
// number, person, and gender, pro-drop (subject inflection on verb).
fn lang_profile_gez() -> [String] {
return lang_profile("gez", "SOV", "semitic", "true", "true", "ltr", "number;person;gender", "true")
}
// Coptic (Sahidic dialect, ca. 3rd11th century CE): SVO, agglutinative, no
// morphological case, two genders (masc/fem), left-to-right (Coptic alphabet),
// agreement on number and gender via bound subject pronouns, no pro-drop (explicit
// subject prefix required on every verb).
fn lang_profile_cop() -> [String] {
return lang_profile("cop", "SVO", "agglutinative", "false", "true", "ltr", "number;person;gender", "false")
}
// Dispatch: code -> profile
fn lang_from_code(code: String) -> [String] {
if str_eq(code, "en") { return lang_profile_en() }
if str_eq(code, "ja") { return lang_profile_ja() }
if str_eq(code, "ar") { return lang_profile_ar() }
if str_eq(code, "zh") { return lang_profile_zh() }
if str_eq(code, "de") { return lang_profile_de() }
if str_eq(code, "es") { return lang_profile_es() }
if str_eq(code, "fi") { return lang_profile_fi() }
if str_eq(code, "sw") { return lang_profile_sw() }
if str_eq(code, "hi") { return lang_profile_hi() }
if str_eq(code, "ru") { return lang_profile_ru() }
if str_eq(code, "fr") { return lang_profile_fr() }
if str_eq(code, "la") { return lang_profile_la() }
if str_eq(code, "he") { return lang_profile_he() }
if str_eq(code, "grc") { return lang_profile_grc() }
if str_eq(code, "ang") { return lang_profile_ang() }
if str_eq(code, "sa") { return lang_profile_sa() }
if str_eq(code, "got") { return lang_profile_got() }
if str_eq(code, "non") { return lang_profile_non() }
if str_eq(code, "enm") { return lang_profile_enm() }
if str_eq(code, "pi") { return lang_profile_pi() }
if str_eq(code, "fro") { return lang_profile_fro() }
if str_eq(code, "goh") { return lang_profile_goh() }
if str_eq(code, "sga") { return lang_profile_sga() }
if str_eq(code, "txb") { return lang_profile_txb() }
if str_eq(code, "peo") { return lang_profile_peo() }
if str_eq(code, "akk") { return lang_profile_akk() }
if str_eq(code, "uga") { return lang_profile_uga() }
if str_eq(code, "egy") { return lang_profile_egy() }
if str_eq(code, "sux") { return lang_profile_sux() }
if str_eq(code, "gez") { return lang_profile_gez() }
if str_eq(code, "cop") { return lang_profile_cop() }
// Unknown code: fall back to English profile
return lang_profile_en()
}
// English default - backward compatibility entry point.
fn lang_default() -> [String] {
return lang_profile_en()
}
// Typed convenience predicates
fn lang_is_isolating(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "morph_type"), "isolating")
}
fn lang_is_agglutinative(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "morph_type"), "agglutinative")
}
fn lang_is_fusional(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "morph_type"), "fusional")
}
fn lang_is_polysynthetic(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "morph_type"), "polysynthetic")
}
fn lang_is_rtl(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "script_dir"), "rtl")
}
fn lang_has_null_subject(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "null_subject"), "true")
}
fn lang_has_case(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "has_case"), "true")
}
fn lang_has_gender(profile: [String]) -> Bool {
return str_eq(lang_get(profile, "has_gender"), "true")
}
fn lang_word_order(profile: [String]) -> String {
return lang_get(profile, "word_order")
}
fn lang_code(profile: [String]) -> String {
return lang_get(profile, "code")
}
+46
View File
@@ -0,0 +1,46 @@
// auto-generated by elc --emit-header — do not edit
extern fn lang_profile(code: String, word_order: String, morph_type: String, has_case: String, has_gender: String, script_dir: String, agreement: String, null_subject: String) -> [String]
extern fn lang_get(profile: [String], key: String) -> String
extern fn lang_profile_en() -> [String]
extern fn lang_profile_ja() -> [String]
extern fn lang_profile_ar() -> [String]
extern fn lang_profile_zh() -> [String]
extern fn lang_profile_de() -> [String]
extern fn lang_profile_es() -> [String]
extern fn lang_profile_fi() -> [String]
extern fn lang_profile_sw() -> [String]
extern fn lang_profile_hi() -> [String]
extern fn lang_profile_ru() -> [String]
extern fn lang_profile_fr() -> [String]
extern fn lang_profile_la() -> [String]
extern fn lang_profile_he() -> [String]
extern fn lang_profile_sa() -> [String]
extern fn lang_profile_got() -> [String]
extern fn lang_profile_non() -> [String]
extern fn lang_profile_enm() -> [String]
extern fn lang_profile_pi() -> [String]
extern fn lang_profile_grc() -> [String]
extern fn lang_profile_ang() -> [String]
extern fn lang_profile_fro() -> [String]
extern fn lang_profile_goh() -> [String]
extern fn lang_profile_sga() -> [String]
extern fn lang_profile_txb() -> [String]
extern fn lang_profile_peo() -> [String]
extern fn lang_profile_akk() -> [String]
extern fn lang_profile_uga() -> [String]
extern fn lang_profile_egy() -> [String]
extern fn lang_profile_sux() -> [String]
extern fn lang_profile_gez() -> [String]
extern fn lang_profile_cop() -> [String]
extern fn lang_from_code(code: String) -> [String]
extern fn lang_default() -> [String]
extern fn lang_is_isolating(profile: [String]) -> Bool
extern fn lang_is_agglutinative(profile: [String]) -> Bool
extern fn lang_is_fusional(profile: [String]) -> Bool
extern fn lang_is_polysynthetic(profile: [String]) -> Bool
extern fn lang_is_rtl(profile: [String]) -> Bool
extern fn lang_has_null_subject(profile: [String]) -> Bool
extern fn lang_has_case(profile: [String]) -> Bool
extern fn lang_has_gender(profile: [String]) -> Bool
extern fn lang_word_order(profile: [String]) -> String
extern fn lang_code(profile: [String]) -> String
+40
View File
@@ -0,0 +1,40 @@
import "language-profile.el"
extern fn es_pluralize(noun: String) -> String
extern fn es_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn fr_pluralize(noun: String) -> String
extern fn fr_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn de_noun_plural(noun: String, gender: String) -> String
extern fn de_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn ru_noun_case(noun: String, gender: String, gram_case: String, number: String) -> String
extern fn ru_conjugate(verb: String, tense: String, person: String, number: String, gender: String) -> String
extern fn ja_conjugate(dict_form: String, form: String) -> String
extern fn fi_apply_case(noun: String, gram_case: String, number: String) -> String
extern fn fi_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn ar_sound_plural(noun: String, gender: String) -> String
extern fn ar_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
extern fn hi_noun_direct(noun: String, gender: String, number: String) -> String
extern fn hi_gender(noun: String) -> String
extern fn hi_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
extern fn sw_noun_plural(noun: String) -> String
extern fn sw_conjugate(verb: String, person: String, number: String, noun_class: String, tense: String) -> String
extern fn la_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn he_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
extern fn grc_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn ang_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn sa_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn got_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn non_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn enm_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn pi_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn fro_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn goh_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn sga_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn txb_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn peo_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn akk_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn uga_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn egy_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn sux_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn gez_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn cop_conjugate(verb: String, tense: String, person: String, number: String) -> String
+3
View File
@@ -0,0 +1,3 @@
fn morph_tiny(x: String) -> String {
return x
}
+528
View File
@@ -0,0 +1,528 @@
// morphology-akk.el - Akkadian morphology for the NLG engine.
// 𒀭𒂗𒍪 Akkadian (akkadû), the language of Babylon and Assyria.
//
// Implements Old Babylonian Akkadian verb conjugation (G-stem / Grundstamm),
// noun declension with mimation, and noun-phrase construction.
//
// Akkadian is the oldest attested Semitic language (ca. 2800100 BCE).
// It uses cuneiform script; we work in standard Latin transliteration
// (Old Babylonian dialect the classical prestige form).
//
// Language profile:
// code=akk, name=Akkadian, morph_type=semitic, word_order=VSO/SOV,
// script=cuneiform (transliterated), family=semitic/east-semitic
//
// Key grammatical facts:
// - Semitic trilateral root system: words built from 3-consonant roots
// by inserting vowel patterns (e.g. root p-r-s iparras "he decides")
// - Grammatical gender: masculine / feminine (no neuter)
// - Cases: nominative (-um), accusative (-am), genitive (-im) "mimation"
// - Number: singular / plural (dual is vestigial in verbs)
// - Verb stems: G (basic), D (intensive), Š (causative), N (passive);
// this file implements G-stem throughout
// - Two main tense/aspect systems:
// Present-future (iparras pattern): action in progress or future
// Perfect (iptaras pattern): completed action with present relevance
// Stative (paris pattern): resultant state, often adjectival
// - No definite or indefinite article; case endings convey
// determination contextually
// - Copula: bašû (to exist/be)
//
// Verb conjugation conventions:
// person: "first" | "second" | "third"
// gender: "m" | "f"
// number: "singular" | "plural"
// tense: "present" | "perfect" | "stative"
//
// Noun declension conventions:
// gram_case: "nom" | "acc" | "gen"
// number: "singular" | "plural"
// gender: "m" | "f" (passed to akk_decline for gender-specific forms)
//
// Verbs covered (G-stem infinitive, transliterated):
// "bašû" to exist / be (copula)
// "alāku" to go
// "amāru" to see
// "qabû" to say
// "epēšu" to do / make
//
// Nouns covered with known mimation forms:
// "šarrum" king
// "awīlum" man / person
// "bītum" house
// "ilum" god
//
// Depends on: morphology.el (str_eq, str_len, str_slice, str_ends_with)
// String helpers
import "morphology.el"
fn akk_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn akk_str_len(s: String) -> Int {
return str_len(s)
}
fn akk_str_drop_last(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len {
return ""
}
return str_slice(s, 0, len - n)
}
// Slot index
//
// Maps person × number to a 0-based slot for table lookups.
// Akkadian verb agreement does not distinguish gender in 1st person,
// and the 2nd person often conflates masc/fem in some paradigms.
// We use a 6-cell paradigm matching the most common OB presentation:
//
// 0 = 1sg (I)
// 1 = 2sg (you sg)
// 2 = 3sg m (he)
// 3 = 3sg f (she)
// 4 = 1pl (we)
// 5 = 3pl (they)
//
// Note: 2pl is rare / vestigial in attested OB texts; omitted here.
fn akk_slot(person: String, number: String) -> Int {
if str_eq(person, "first") {
if str_eq(number, "plural") { return 4 }
return 0
}
if str_eq(person, "second") {
return 1
}
// third
if str_eq(number, "plural") { return 5 }
return 2 // default: 3sg masc; caller may override with gender check below
}
// akk_slot_g: gender-aware slot for third person singular.
// Returns 3 (3sg fem) when person=third, number=singular, gender=f.
fn akk_slot_g(person: String, gender: String, number: String) -> Int {
let base: Int = akk_slot(person, number)
if str_eq(person, "third") {
if str_eq(number, "singular") {
if str_eq(gender, "f") { return 3 }
}
}
return base
}
// Copula: bašû to exist / be
//
// bašû is suppletive and highly irregular.
// Present: ibašši (3sg m/f), abašši (1sg), tabašši (2sg)
// Stative: bašī (3sg m), bašiat (3sg f), bašāku (1sg)
// Perfect: not commonly attested in G-stem; use present forms as fallback.
fn akk_copula_present(slot: Int) -> String {
if slot == 0 { return "abašši" } // 1sg
if slot == 1 { return "tabašši" } // 2sg
if slot == 2 { return "ibašši" } // 3sg m
if slot == 3 { return "ibašši" } // 3sg f (same form in attested OB)
if slot == 4 { return "nibašši" } // 1pl
return "ibaššū" // 3pl
}
fn akk_copula_stative(slot: Int) -> String {
if slot == 0 { return "bašāku" } // 1sg (stative 1sg: -āku suffix)
if slot == 1 { return "bašāta" } // 2sg (-āta suffix)
if slot == 2 { return "bašī" } // 3sg m (unmarked base)
if slot == 3 { return "bašiat" } // 3sg f (-at suffix)
if slot == 4 { return "bašānu" } // 1pl (-ānu suffix)
return "bašū" // 3pl ( suffix)
}
fn akk_is_copula(verb: String) -> Bool {
if str_eq(verb, "bašû") { return true }
if str_eq(verb, "bashu") { return true }
if str_eq(verb, "be") { return true }
return false
}
fn akk_conjugate_copula(tense: String, slot: Int) -> String {
if str_eq(tense, "stative") { return akk_copula_stative(slot) }
// present and perfect both fall back to present forms for bašû
return akk_copula_present(slot)
}
// alāku to go
//
// Irregular: present stem is illak- (not the expected alakk-).
// Present: illak (3sg), allak (1sg), tallak (2sg), nillak (1pl), illaku (3pl)
// Perfect: ittalk- forms (less common, use illak- + perf marker)
// Stative: use present as proxy
fn akk_alaku_present(slot: Int) -> String {
if slot == 0 { return "allak" } // 1sg
if slot == 1 { return "tallak" } // 2sg
if slot == 2 { return "illak" } // 3sg m
if slot == 3 { return "tallak" } // 3sg f (same as 2sg OB pattern)
if slot == 4 { return "nillak" } // 1pl
return "illaku" // 3pl
}
fn akk_alaku_perfect(slot: Int) -> String {
if slot == 0 { return "ittalak" } // 1sg
if slot == 1 { return "tattalak" } // 2sg
if slot == 2 { return "ittalak" } // 3sg m
if slot == 3 { return "tattalak" } // 3sg f
if slot == 4 { return "nittalak" } // 1pl
return "ittalku" // 3pl
}
// amāru to see
//
// Present (immar-): immar (3sg), ammar (1sg), tammar (2sg)
// Perfect (imtamar-): imtamar (3sg), amtamar (1sg), tamtamar (2sg)
fn akk_amaru_present(slot: Int) -> String {
if slot == 0 { return "ammar" } // 1sg
if slot == 1 { return "tammar" } // 2sg
if slot == 2 { return "immar" } // 3sg m
if slot == 3 { return "tammar" } // 3sg f
if slot == 4 { return "nimmar" } // 1pl
return "immaru" // 3pl
}
fn akk_amaru_perfect(slot: Int) -> String {
if slot == 0 { return "amtamar" } // 1sg
if slot == 1 { return "tamtamar" } // 2sg
if slot == 2 { return "imtamar" } // 3sg m
if slot == 3 { return "tamtamar" } // 3sg f
if slot == 4 { return "nimtamar" } // 1pl
return "imtamaru" // 3pl
}
fn akk_amaru_stative(slot: Int) -> String {
// amāru stative: 3sg "amir" (the one who saw / he has seen)
if slot == 0 { return "amrāku" }
if slot == 1 { return "amrāta" }
if slot == 2 { return "amir" }
if slot == 3 { return "amrat" }
if slot == 4 { return "amrānu" }
return "amrū"
}
// qabû to say / speak
//
// Present: iqabbi (3sg), aqabbi (1sg), taqabbi (2sg)
// Perfect: iqtabi (3sg), aqtabi (1sg), taqtabi (2sg)
fn akk_qabu_present(slot: Int) -> String {
if slot == 0 { return "aqabbi" } // 1sg
if slot == 1 { return "taqabbi" } // 2sg
if slot == 2 { return "iqabbi" } // 3sg m
if slot == 3 { return "taqabbi" } // 3sg f
if slot == 4 { return "niqabbi" } // 1pl
return "iqabbû" // 3pl
}
fn akk_qabu_perfect(slot: Int) -> String {
if slot == 0 { return "aqtabi" } // 1sg
if slot == 1 { return "taqtabi" } // 2sg
if slot == 2 { return "iqtabi" } // 3sg m
if slot == 3 { return "taqtabi" } // 3sg f
if slot == 4 { return "niqtabi" } // 1pl
return "iqtabû" // 3pl
}
fn akk_qabu_stative(slot: Int) -> String {
if slot == 0 { return "qabāku" }
if slot == 1 { return "qabāta" }
if slot == 2 { return "qabi" }
if slot == 3 { return "qabiat" }
if slot == 4 { return "qabānu" }
return "qabû"
}
// epēšu to do / make
//
// Present (ieppuš / eppuš): ieppuš (3sg), eppuš (1sg), teppuš (2sg)
// Perfect: iptešu forms
fn akk_epesu_present(slot: Int) -> String {
if slot == 0 { return "eppuš" } // 1sg
if slot == 1 { return "teppuš" } // 2sg
if slot == 2 { return "ieppuš" } // 3sg m
if slot == 3 { return "teppuš" } // 3sg f
if slot == 4 { return "neppuš" } // 1pl
return "ieppušu" // 3pl
}
fn akk_epesu_perfect(slot: Int) -> String {
if slot == 0 { return "iptešu" } // 1sg (irregular: root ʿ-p-š)
if slot == 1 { return "taptešu" } // 2sg
if slot == 2 { return "iptešu" } // 3sg m
if slot == 3 { return "taptešu" } // 3sg f
if slot == 4 { return "niptešu" } // 1pl
return "iptešū" // 3pl
}
fn akk_epesu_stative(slot: Int) -> String {
if slot == 0 { return "epšāku" }
if slot == 1 { return "epšāta" }
if slot == 2 { return "epuš" }
if slot == 3 { return "epšat" }
if slot == 4 { return "epšānu" }
return "epšū"
}
// Regular G-stem paradigms (iparras model)
//
// For regular verbs not in the irregular table, we apply the standard
// OB G-stem paradigm using a caller-supplied present stem and perfect stem.
// The stems must be pre-computed by the caller (or vocabulary layer).
//
// iparras (present) endings by slot:
// 1sg: a- prefix
// 2sg: ta- prefix
// 3sg m: i- prefix
// 3sg f: ta- prefix (same prefix as 2sg)
// 1pl: ni- prefix
// 3pl: i- prefix + suffix
//
// For the generic fallback we use "iparras" as the model template.
fn akk_regular_present(stem: String, slot: Int) -> String {
// stem is the 3sg m form (i-prefix already present in conventional citation)
// We rebuild from the bare root portion by stripping/adding prefixes.
// Simplification: return prefixed forms using the provided present-3sg string.
if slot == 0 { return "a" + stem } // 1sg: a + stem (strip i-, add a-)
if slot == 1 { return "ta" + stem } // 2sg
if slot == 2 { return "i" + stem } // 3sg m
if slot == 3 { return "ta" + stem } // 3sg f
if slot == 4 { return "ni" + stem } // 1pl
return "i" + stem + "u" // 3pl: i + stem +
}
fn akk_regular_perfect(stem: String, slot: Int) -> String {
// Perfect (iptaras) uses infix -ta- after first root consonant.
// stem here is the 3sg perfect form; we apply person endings.
if slot == 0 { return "a" + stem } // 1sg
if slot == 1 { return "ta" + stem } // 2sg
if slot == 2 { return "i" + stem } // 3sg m
if slot == 3 { return "ta" + stem } // 3sg f
if slot == 4 { return "ni" + stem } // 1pl
return "i" + stem + "u" // 3pl
}
fn akk_regular_stative(stem: String, slot: Int) -> String {
// Stative (paris): 3sg m has zero ending; others take person suffixes.
if slot == 0 { return stem + "āku" } // 1sg
if slot == 1 { return stem + "āta" } // 2sg
if slot == 2 { return stem } // 3sg m: bare stem
if slot == 3 { return stem + "at" } // 3sg f
if slot == 4 { return stem + "ānu" } // 1pl
return stem + "ū" // 3pl
}
// Known-verb dispatcher
fn akk_known_verb(verb: String, tense: String, slot: Int) -> String {
// bašû to be / exist
if str_eq(verb, "bašû") {
return akk_conjugate_copula(tense, slot)
}
if str_eq(verb, "bashu") {
return akk_conjugate_copula(tense, slot)
}
// alāku to go
if str_eq(verb, "alāku") {
if str_eq(tense, "perfect") { return akk_alaku_perfect(slot) }
if str_eq(tense, "stative") { return akk_alaku_present(slot) }
return akk_alaku_present(slot)
}
if str_eq(verb, "alaku") {
if str_eq(tense, "perfect") { return akk_alaku_perfect(slot) }
return akk_alaku_present(slot)
}
// amāru to see
if str_eq(verb, "amāru") {
if str_eq(tense, "perfect") { return akk_amaru_perfect(slot) }
if str_eq(tense, "stative") { return akk_amaru_stative(slot) }
return akk_amaru_present(slot)
}
if str_eq(verb, "amaru") {
if str_eq(tense, "perfect") { return akk_amaru_perfect(slot) }
if str_eq(tense, "stative") { return akk_amaru_stative(slot) }
return akk_amaru_present(slot)
}
// qabû to say
if str_eq(verb, "qabû") {
if str_eq(tense, "perfect") { return akk_qabu_perfect(slot) }
if str_eq(tense, "stative") { return akk_qabu_stative(slot) }
return akk_qabu_present(slot)
}
if str_eq(verb, "qabu") {
if str_eq(tense, "perfect") { return akk_qabu_perfect(slot) }
if str_eq(tense, "stative") { return akk_qabu_stative(slot) }
return akk_qabu_present(slot)
}
// epēšu to do / make
if str_eq(verb, "epēšu") {
if str_eq(tense, "perfect") { return akk_epesu_perfect(slot) }
if str_eq(tense, "stative") { return akk_epesu_stative(slot) }
return akk_epesu_present(slot)
}
if str_eq(verb, "epesu") {
if str_eq(tense, "perfect") { return akk_epesu_perfect(slot) }
if str_eq(tense, "stative") { return akk_epesu_stative(slot) }
return akk_epesu_present(slot)
}
return ""
}
// Main conjugation entry point
//
// akk_conjugate: conjugate an Akkadian verb (G-stem).
//
// verb: G-stem infinitive (transliterated, e.g. "alāku", "amāru")
// tense: "present" | "perfect" | "stative"
// person: "first" | "second" | "third"
// number: "singular" | "plural"
//
// Returns:
// - Inflected form for known verbs
// - verb unchanged as safe fallback for unknown verbs
fn akk_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let slot: Int = akk_slot(person, number)
// Copula shortcut
if akk_is_copula(verb) {
return akk_conjugate_copula(tense, slot)
}
// Known-verb table
let known: String = akk_known_verb(verb, tense, slot)
if !str_eq(known, "") {
return known
}
// Unknown verb: safe fallback
return verb
}
// Noun declension
//
// akk_decline: decline an Akkadian noun for gram_case and number.
//
// Mimation: OB nouns bear final -m in all case endings (mimation).
// The base noun (dictionary form) is the nominative singular with mimation.
// We strip the nominative -um ending (if present) to obtain the bare stem,
// then apply the requested ending.
//
// Masculine case endings (singular):
// Nominative: -um
// Accusative: -am
// Genitive: -im
//
// Masculine case endings (plural):
// Nominative: -ūtum (or in construct)
// Accusative/Genitive: -ātim (or in construct)
//
// Feminine nouns (identified by -tum nom sg ending):
// Sg nominative: -tum, accusative: -tam, genitive: -tim
// Pl nominative: -ātum, genitive/accusative: -ātim
//
// Known irregular stems (the vocabulary layer should pass dictionary forms):
// šarrum stem: šarr-
// awīlum stem: awīl-
// bītum stem: bīt-
// ilum stem: il-
fn akk_strip_nom(noun: String) -> String {
// Strip -um (masc nom sg mimation ending) to get bare stem
if akk_str_ends(noun, "um") {
return akk_str_drop_last(noun, 2)
}
// Strip -tum (fem nom sg)
if akk_str_ends(noun, "tum") {
return akk_str_drop_last(noun, 3)
}
// Already a bare stem or unusual form: return as-is
return noun
}
fn akk_is_fem(noun: String) -> Bool {
// Feminine nouns in OB typically end in -tum (nom sg)
if akk_str_ends(noun, "tum") { return true }
if akk_str_ends(noun, "tam") { return true }
if akk_str_ends(noun, "tim") { return true }
return false
}
fn akk_decline(noun: String, gram_case: String, number: String) -> String {
let fem: Bool = akk_is_fem(noun)
let stem: String = akk_strip_nom(noun)
if str_eq(number, "singular") {
if fem {
if str_eq(gram_case, "nom") { return stem + "tum" }
if str_eq(gram_case, "acc") { return stem + "tam" }
if str_eq(gram_case, "gen") { return stem + "tim" }
return stem + "tum"
}
// Masculine
if str_eq(gram_case, "nom") { return stem + "um" }
if str_eq(gram_case, "acc") { return stem + "am" }
if str_eq(gram_case, "gen") { return stem + "im" }
return stem + "um"
}
// Plural
if fem {
if str_eq(gram_case, "nom") { return stem + "ātum" }
// acc and gen merge in the oblique plural
return stem + "ātim"
}
// Masculine plural
if str_eq(gram_case, "nom") { return stem + "ūtum" }
return stem + "ātim"
}
// Noun phrase
//
// akk_noun_phrase: produce the surface noun phrase.
//
// Akkadian has no definite or indefinite article. Determination is conveyed
// by context, word order, and the genitive construct chain (status constructus).
// The definite parameter is accepted but has no surface effect: the declined
// noun is returned in either case.
//
// noun: dictionary form (nominative singular with mimation, e.g. "šarrum")
// gram_case: "nom" | "acc" | "gen"
// number: "singular" | "plural"
// definite: "true" | "false" (no surface effect in Akkadian)
fn akk_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String {
return akk_decline(noun, gram_case, number)
}
// Canonical verb mapping
//
// akk_map_canonical: map cross-lingual English canonical verb labels to
// their Akkadian G-stem infinitive equivalents.
fn akk_map_canonical(verb: String) -> String {
if str_eq(verb, "be") { return "bašû" }
if str_eq(verb, "go") { return "alāku" }
if str_eq(verb, "see") { return "amāru" }
if str_eq(verb, "say") { return "qabû" }
if str_eq(verb, "speak") { return "qabû" }
if str_eq(verb, "do") { return "epēšu" }
if str_eq(verb, "make") { return "epēšu" }
return verb
}
+31
View File
@@ -0,0 +1,31 @@
// auto-generated by elc --emit-header — do not edit
extern fn akk_str_ends(s: String, suf: String) -> Bool
extern fn akk_str_len(s: String) -> Int
extern fn akk_str_drop_last(s: String, n: Int) -> String
extern fn akk_slot(person: String, number: String) -> Int
extern fn akk_slot_g(person: String, gender: String, number: String) -> Int
extern fn akk_copula_present(slot: Int) -> String
extern fn akk_copula_stative(slot: Int) -> String
extern fn akk_is_copula(verb: String) -> Bool
extern fn akk_conjugate_copula(tense: String, slot: Int) -> String
extern fn akk_alaku_present(slot: Int) -> String
extern fn akk_alaku_perfect(slot: Int) -> String
extern fn akk_amaru_present(slot: Int) -> String
extern fn akk_amaru_perfect(slot: Int) -> String
extern fn akk_amaru_stative(slot: Int) -> String
extern fn akk_qabu_present(slot: Int) -> String
extern fn akk_qabu_perfect(slot: Int) -> String
extern fn akk_qabu_stative(slot: Int) -> String
extern fn akk_epesu_present(slot: Int) -> String
extern fn akk_epesu_perfect(slot: Int) -> String
extern fn akk_epesu_stative(slot: Int) -> String
extern fn akk_regular_present(stem: String, slot: Int) -> String
extern fn akk_regular_perfect(stem: String, slot: Int) -> String
extern fn akk_regular_stative(stem: String, slot: Int) -> String
extern fn akk_known_verb(verb: String, tense: String, slot: Int) -> String
extern fn akk_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn akk_strip_nom(noun: String) -> String
extern fn akk_is_fem(noun: String) -> Bool
extern fn akk_decline(noun: String, gram_case: String, number: String) -> String
extern fn akk_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
extern fn akk_map_canonical(verb: String) -> String
+752
View File
@@ -0,0 +1,752 @@
// morphology-ang.el - Old English (Anglo-Saxon) morphology for the NLG engine.
//
// Implements Old English verb conjugation, noun declension, and the definite
// article/demonstrative pronoun. Designed as a companion to morphology.el and
// called by the engine when the language profile code is "ang".
//
// Language profile: code=ang, name=Old English, morph_type=fusional,
// word_order=SOV, question_strategy=intonation, script=latin, family=germanic.
//
// Typology note: Old English is a synthetic Germanic language with four
// grammatical cases (nominative, accusative, genitive, dative), three genders,
// and strong/weak noun and verb classes. Strong verbs form their past tense by
// internal vowel change (ablaut); weak verbs use a dental (-de/-ode) suffix.
// Long vowels are marked with a macron (ā ē ī ō ū) and are preserved in all
// string literals; ǣ, æ, þ, ð, and ƿ (wynn) are used where historically
// appropriate. V2 (verb-second) word order applies in main clauses but is not
// enforced by this module the realizer handles constituent ordering.
//
// Verb conjugation covered:
// Tenses: present, past
// Persons: first/second/third × singular/plural (slots 0-5)
// Classes: weak (regular -ian), strong irregular table
// Irregulars: wesan/beon (be), habban (have), gān (go), cuman (come),
// secgan (say), sēon (see), dōn (do), willan (want), magan (can)
// Canonical map: "be" -> "wesan" (past) / "beon" (present)
//
// Noun declension covered:
// Strong masc a-stem (cyning pattern): nom/acc -, gen -es, dat -e; pl -as/-a/-um
// Strong neut a-stem (word pattern): sg same as masc; pl nom/acc -∅
// Weak n-stem (nama pattern): sg nom -a, obl -an; pl -an/-ena/-um
//
// Article: simplified demonstrative/article forms for masculine, feminine,
// neuter (se/sēo/þæt), fully declined.
//
// Depends on: morphology.el (str_ends_with, str_len, str_slice, str_eq)
// String helpers
import "morphology.el"
fn ang_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn ang_str_drop_last(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len {
return ""
}
return str_slice(s, 0, len - n)
}
fn ang_str_last_char(s: String) -> String {
let n: Int = str_len(s)
if n == 0 {
return ""
}
return str_slice(s, n - 1, n)
}
fn ang_str_last2(s: String) -> String {
let n: Int = str_len(s)
if n < 2 {
return s
}
return str_slice(s, n - 2, n)
}
// Person/number slot
//
// Maps person × number to a 0-based index for paradigm tables.
// 0 = 1st singular (ic)
// 1 = 2nd singular (þū)
// 2 = 3rd singular (hē/hēo/hit)
// 3 = 1st plural ()
// 4 = 2nd plural ()
// 5 = 3rd plural (hīe)
//
// Old English also has a dual (wit, git) not handled; dual falls through
// to plural.
fn ang_slot(person: String, number: String) -> Int {
if str_eq(person, "first") {
if str_eq(number, "singular") { return 0 }
return 3
}
if str_eq(person, "second") {
if str_eq(number, "singular") { return 1 }
return 4
}
// third
if str_eq(number, "singular") { return 2 }
return 5
}
// Canonical verb mapping
//
// The semantic layer may pass English canonical labels. Map to Old English
// citation (infinitive) forms. "be" maps to "beon" for present and "wesan"
// for past the caller selects tense, so we map "be" to "beon" and handle
// the past-tense wesan forms inside the conjugation function.
fn ang_map_canonical(verb: String) -> String {
if str_eq(verb, "be") { return "beon" }
if str_eq(verb, "have") { return "habban" }
if str_eq(verb, "go") { return "gān" }
if str_eq(verb, "come") { return "cuman" }
if str_eq(verb, "say") { return "secgan" }
if str_eq(verb, "see") { return "sēon" }
if str_eq(verb, "do") { return "dōn" }
if str_eq(verb, "want") { return "willan" }
if str_eq(verb, "will") { return "willan" }
if str_eq(verb, "can") { return "magan" }
if str_eq(verb, "know") { return "witan" }
if str_eq(verb, "give") { return "giefan" }
if str_eq(verb, "take") { return "niman" }
if str_eq(verb, "find") { return "findan" }
if str_eq(verb, "make") { return "macian" }
return verb
}
// Irregular: wesan (to be past tense forms)
//
// Past: wæs wǣre wæs wǣron wǣron wǣron
fn ang_wesan_past(slot: Int) -> String {
if slot == 0 { return "wæs" }
if slot == 1 { return "wǣre" }
if slot == 2 { return "wæs" }
if slot == 3 { return "wǣron" }
if slot == 4 { return "wǣron" }
return "wǣron"
}
// Irregular: beon (to be present / habitual / future)
//
// Present: bēo bist biþ bēoþ bēoþ bēoþ
//
// The present indicative of "wesan" is eom/eart/is/sind that paradigm is
// also provided below for completeness and for callers who specifically request
// wesan present.
fn ang_beon_present(slot: Int) -> String {
if slot == 0 { return "bēo" }
if slot == 1 { return "bist" }
if slot == 2 { return "biþ" }
if slot == 3 { return "bēoþ" }
if slot == 4 { return "bēoþ" }
return "bēoþ"
}
// Irregular: wesan present (eom/eart/is/sind)
//
// Present: eom eart is sind/sindon sind sind
fn ang_wesan_present(slot: Int) -> String {
if slot == 0 { return "eom" }
if slot == 1 { return "eart" }
if slot == 2 { return "is" }
if slot == 3 { return "sind" }
if slot == 4 { return "sind" }
return "sind"
}
// Irregular: habban (to have)
//
// Present: hæbbe hæfst hæfþ habbað habbað habbað
// Past: hæfde hæfdest hæfde hæfdon hæfdon hæfdon
fn ang_habban_present(slot: Int) -> String {
if slot == 0 { return "hæbbe" }
if slot == 1 { return "hæfst" }
if slot == 2 { return "hæfþ" }
if slot == 3 { return "habbað" }
if slot == 4 { return "habbað" }
return "habbað"
}
fn ang_habban_past(slot: Int) -> String {
if slot == 0 { return "hæfde" }
if slot == 1 { return "hæfdest" }
if slot == 2 { return "hæfde" }
if slot == 3 { return "hæfdon" }
if slot == 4 { return "hæfdon" }
return "hæfdon"
}
// Irregular: gān (to go)
//
// Present: gǣst gǣþ gāð gāð gāð
// Past: ēode ēodest ēode ēodon ēodon ēodon
fn ang_gan_present(slot: Int) -> String {
if slot == 0 { return "" }
if slot == 1 { return "gǣst" }
if slot == 2 { return "gǣþ" }
if slot == 3 { return "gāð" }
if slot == 4 { return "gāð" }
return "gāð"
}
fn ang_gan_past(slot: Int) -> String {
if slot == 0 { return "ēode" }
if slot == 1 { return "ēodest" }
if slot == 2 { return "ēode" }
if slot == 3 { return "ēodon" }
if slot == 4 { return "ēodon" }
return "ēodon"
}
// Irregular: cuman (to come)
//
// Present: cume cymst cymþ cumað cumað cumað
// Past: cōm cōme cōm cōmon cōmon cōmon
fn ang_cuman_present(slot: Int) -> String {
if slot == 0 { return "cume" }
if slot == 1 { return "cymst" }
if slot == 2 { return "cymþ" }
if slot == 3 { return "cumað" }
if slot == 4 { return "cumað" }
return "cumað"
}
fn ang_cuman_past(slot: Int) -> String {
if slot == 0 { return "cōm" }
if slot == 1 { return "cōme" }
if slot == 2 { return "cōm" }
if slot == 3 { return "cōmon" }
if slot == 4 { return "cōmon" }
return "cōmon"
}
// Irregular: secgan (to say)
//
// Present: secge sagast sagað secgað secgað secgað
// Past: sægde sægdest sægde sægdon sægdon sægdon
fn ang_secgan_present(slot: Int) -> String {
if slot == 0 { return "secge" }
if slot == 1 { return "sagast" }
if slot == 2 { return "sagað" }
if slot == 3 { return "secgað" }
if slot == 4 { return "secgað" }
return "secgað"
}
fn ang_secgan_past(slot: Int) -> String {
if slot == 0 { return "sægde" }
if slot == 1 { return "sægdest" }
if slot == 2 { return "sægde" }
if slot == 3 { return "sægdon" }
if slot == 4 { return "sægdon" }
return "sægdon"
}
// Irregular: sēon (to see)
//
// Present: sēo siehst siehþ sēoð sēoð sēoð
// Past: seah sāwe seah sāwon sāwon sāwon
fn ang_seon_present(slot: Int) -> String {
if slot == 0 { return "sēo" }
if slot == 1 { return "siehst" }
if slot == 2 { return "siehþ" }
if slot == 3 { return "sēoð" }
if slot == 4 { return "sēoð" }
return "sēoð"
}
fn ang_seon_past(slot: Int) -> String {
if slot == 0 { return "seah" }
if slot == 1 { return "sāwe" }
if slot == 2 { return "seah" }
if slot == 3 { return "sāwon" }
if slot == 4 { return "sāwon" }
return "sāwon"
}
// Irregular: dōn (to do)
//
// Present: dēst dēþ dōð dōð dōð
// Past: dyde dydest dyde dydon dydon dydon
fn ang_don_present(slot: Int) -> String {
if slot == 0 { return "" }
if slot == 1 { return "dēst" }
if slot == 2 { return "dēþ" }
if slot == 3 { return "dōð" }
if slot == 4 { return "dōð" }
return "dōð"
}
fn ang_don_past(slot: Int) -> String {
if slot == 0 { return "dyde" }
if slot == 1 { return "dydest" }
if slot == 2 { return "dyde" }
if slot == 3 { return "dydon" }
if slot == 4 { return "dydon" }
return "dydon"
}
// Irregular: willan (to want / will)
//
// Present: wille wilt wile willað willað willað
// Past: wolde woldest wolde woldon woldon woldon
fn ang_willan_present(slot: Int) -> String {
if slot == 0 { return "wille" }
if slot == 1 { return "wilt" }
if slot == 2 { return "wile" }
if slot == 3 { return "willað" }
if slot == 4 { return "willað" }
return "willað"
}
fn ang_willan_past(slot: Int) -> String {
if slot == 0 { return "wolde" }
if slot == 1 { return "woldest" }
if slot == 2 { return "wolde" }
if slot == 3 { return "woldon" }
if slot == 4 { return "woldon" }
return "woldon"
}
// Irregular: magan (to be able / can)
//
// Present: mæg meaht mæg magon magon magon
// Past: meahte meahtest meahte meahton meahton meahton
fn ang_magan_present(slot: Int) -> String {
if slot == 0 { return "mæg" }
if slot == 1 { return "meaht" }
if slot == 2 { return "mæg" }
if slot == 3 { return "magon" }
if slot == 4 { return "magon" }
return "magon"
}
fn ang_magan_past(slot: Int) -> String {
if slot == 0 { return "meahte" }
if slot == 1 { return "meahtest" }
if slot == 2 { return "meahte" }
if slot == 3 { return "meahton" }
if slot == 4 { return "meahton" }
return "meahton"
}
// Irregular: witan (to know)
//
// Present: wāt wāst wāt witon witon witon
// Past: wisse/wiste wissest wisse wisson wisson wisson
fn ang_witan_present(slot: Int) -> String {
if slot == 0 { return "wāt" }
if slot == 1 { return "wāst" }
if slot == 2 { return "wāt" }
if slot == 3 { return "witon" }
if slot == 4 { return "witon" }
return "witon"
}
fn ang_witan_past(slot: Int) -> String {
if slot == 0 { return "wisse" }
if slot == 1 { return "wissest" }
if slot == 2 { return "wisse" }
if slot == 3 { return "wisson" }
if slot == 4 { return "wisson" }
return "wisson"
}
// Weak verb: present-tense endings
//
// Weak verbs with -ian infinitives form their present tense as:
// stem + -e, -est, -eþ, -aþ, -aþ, -aþ
//
// The stem is the infinitive with -ian stripped (or -an for class-2 verbs).
fn ang_weak_present_ending(slot: Int) -> String {
if slot == 0 { return "e" }
if slot == 1 { return "est" }
if slot == 2 { return "" }
if slot == 3 { return "" }
if slot == 4 { return "" }
return ""
}
// Weak verb: past-tense ending selection
//
// Class 1 (-ian with short stem): past -ede (e.g. nerian -> nerede)
// Class 2 (-ian with long/heavy stem): past -ode (e.g. macian -> macode)
// Class 3 (-ian, small group): past -de (e.g. habban -> hæfde irregular)
//
// Heuristic: if the stem length is 1 char, use -ede; otherwise use -ode.
// This is a simplification; correct assignment requires lexical class marking.
//
// For the past, all persons in the plural share -on, and all singulars share
// the same dental-suffixed stem.
fn ang_weak_past_stem(stem: String) -> String {
let slen: Int = str_len(stem)
if slen <= 2 {
return stem + "ede"
}
return stem + "ode"
}
fn ang_weak_past(stem: String, slot: Int) -> String {
let pstem: String = ang_weak_past_stem(stem)
if slot == 0 { return pstem }
if slot == 1 { return pstem + "st" }
if slot == 2 { return pstem }
if slot == 3 { return ang_str_drop_last(pstem, 1) + "on" }
if slot == 4 { return ang_str_drop_last(pstem, 1) + "on" }
return ang_str_drop_last(pstem, 1) + "on"
}
// Stem extraction for weak verbs
//
// Strip the infinitive ending to recover the stem:
// -ian -> strip 3 chars (nerian -> ner-, macian -> mac-)
// -an -> strip 2 chars (habban -> habb-; fallback for non -ian)
// otherwise: return as-is
fn ang_weak_stem(verb: String) -> String {
if ang_str_ends(verb, "ian") {
return ang_str_drop_last(verb, 3)
}
if ang_str_ends(verb, "an") {
return ang_str_drop_last(verb, 2)
}
return verb
}
// ang_conjugate: main conjugation entry point
//
// verb: Old English infinitive or English canonical label
// tense: "present" | "past"
// person: "first" | "second" | "third"
// number: "singular" | "plural"
//
// Strategy:
// 1. Map canonical English labels to OE verbs.
// 2. Check the full irregular table.
// 3. Fall back to weak conjugation for unknown -ian/-an verbs.
// 4. Return the base form if nothing matches.
fn ang_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let v: String = ang_map_canonical(verb)
let slot: Int = ang_slot(person, number)
// Irregulars
// beon: present-tense "be" (habitual/future/general)
if str_eq(v, "beon") {
if str_eq(tense, "present") { return ang_beon_present(slot) }
// past: use wesan past forms
return ang_wesan_past(slot)
}
// wesan: past "be" and present "be" (existential/stative)
if str_eq(v, "wesan") {
if str_eq(tense, "present") { return ang_wesan_present(slot) }
return ang_wesan_past(slot)
}
if str_eq(v, "habban") {
if str_eq(tense, "present") { return ang_habban_present(slot) }
return ang_habban_past(slot)
}
if str_eq(v, "gān") {
if str_eq(tense, "present") { return ang_gan_present(slot) }
return ang_gan_past(slot)
}
if str_eq(v, "cuman") {
if str_eq(tense, "present") { return ang_cuman_present(slot) }
return ang_cuman_past(slot)
}
if str_eq(v, "secgan") {
if str_eq(tense, "present") { return ang_secgan_present(slot) }
return ang_secgan_past(slot)
}
if str_eq(v, "sēon") {
if str_eq(tense, "present") { return ang_seon_present(slot) }
return ang_seon_past(slot)
}
if str_eq(v, "dōn") {
if str_eq(tense, "present") { return ang_don_present(slot) }
return ang_don_past(slot)
}
if str_eq(v, "willan") {
if str_eq(tense, "present") { return ang_willan_present(slot) }
return ang_willan_past(slot)
}
if str_eq(v, "magan") {
if str_eq(tense, "present") { return ang_magan_present(slot) }
return ang_magan_past(slot)
}
if str_eq(v, "witan") {
if str_eq(tense, "present") { return ang_witan_present(slot) }
return ang_witan_past(slot)
}
// Regular weak conjugation
let stem: String = ang_weak_stem(v)
if str_eq(tense, "present") {
return stem + ang_weak_present_ending(slot)
}
if str_eq(tense, "past") {
return ang_weak_past(stem, slot)
}
// Unknown tense: return infinitive
return v
}
// Noun declension class detection
//
// Infer the declension class from the nominative singular form and an optional
// gender hint. Without a full lexicon, ending-based heuristics are used:
//
// ends in -a -> weak n-stem (nama pattern)
// ends in -e (long) -> may be various; default to strong masc a-stem
// any other ending -> strong a-stem; gender distinguishes masc vs neut
//
// The caller may pass gender as a hint:
// "masculine" | "feminine" | "neuter" | "" (empty = infer)
//
// For simplicity this module handles three paradigms:
// "strong_masc" a-stem masculine (cyning, mann)
// "strong_neut" a-stem neuter (word, scip)
// "weak" n-stem (nama, ēage)
fn ang_declension(noun: String, gender: String) -> String {
if ang_str_ends(noun, "a") { return "weak" }
if str_eq(gender, "neuter") { return "strong_neut" }
return "strong_masc"
}
// Strong masculine a-stem (cyning pattern)
//
// Stem: the noun as given (nom sg lacks an inflectional ending in this class).
//
// Singular: nom - acc - gen -es dat -e
// Plural: nom -as acc -as gen -a dat -um
fn ang_decline_strong_masc(noun: String, gram_case: String, number: String) -> String {
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return noun }
if str_eq(gram_case, "accusative") { return noun }
if str_eq(gram_case, "genitive") { return noun + "es" }
if str_eq(gram_case, "dative") { return noun + "e" }
return noun
}
// plural
if str_eq(gram_case, "nominative") { return noun + "as" }
if str_eq(gram_case, "accusative") { return noun + "as" }
if str_eq(gram_case, "genitive") { return noun + "a" }
if str_eq(gram_case, "dative") { return noun + "um" }
return noun + "as"
}
// Strong neuter a-stem (word pattern)
//
// Singular: same as strong masc
// Plural: nom/acc - gen -a dat -um
fn ang_decline_strong_neut(noun: String, gram_case: String, number: String) -> String {
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return noun }
if str_eq(gram_case, "accusative") { return noun }
if str_eq(gram_case, "genitive") { return noun + "es" }
if str_eq(gram_case, "dative") { return noun + "e" }
return noun
}
// plural: neuters have zero ending in nom/acc
if str_eq(gram_case, "nominative") { return noun }
if str_eq(gram_case, "accusative") { return noun }
if str_eq(gram_case, "genitive") { return noun + "a" }
if str_eq(gram_case, "dative") { return noun + "um" }
return noun
}
// Weak n-stem (nama pattern)
//
// The nom sg ends in -a; the oblique stem is formed by stripping -a and adding
// -an. Plural genitive is -ena.
//
// Singular: nom -a acc -an gen -an dat -an
// Plural: nom -an acc -an gen -ena dat -um
fn ang_decline_weak(noun: String, gram_case: String, number: String) -> String {
// Oblique stem: strip the final -a
let stem: String = ang_str_drop_last(noun, 1)
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return noun }
if str_eq(gram_case, "accusative") { return stem + "an" }
if str_eq(gram_case, "genitive") { return stem + "an" }
if str_eq(gram_case, "dative") { return stem + "an" }
return noun
}
// plural
if str_eq(gram_case, "nominative") { return stem + "an" }
if str_eq(gram_case, "accusative") { return stem + "an" }
if str_eq(gram_case, "genitive") { return stem + "ena" }
if str_eq(gram_case, "dative") { return stem + "um" }
return stem + "an"
}
// ang_decline: main declension entry point
//
// noun: nominative singular Old English noun (e.g. "cyning", "word", "nama")
// gram_case: "nominative" | "accusative" | "genitive" | "dative"
// number: "singular" | "plural"
// gender: "masculine" | "neuter" | "feminine" | "" (empty triggers inference)
//
// Returns the inflected form. Falls back to the nominative singular for any
// unrecognised combination.
fn ang_decline(noun: String, gram_case: String, number: String, gender: String) -> String {
let decl: String = ang_declension(noun, gender)
if str_eq(decl, "strong_masc") {
return ang_decline_strong_masc(noun, gram_case, number)
}
if str_eq(decl, "strong_neut") {
return ang_decline_strong_neut(noun, gram_case, number)
}
if str_eq(decl, "weak") {
return ang_decline_weak(noun, gram_case, number)
}
// Unknown: return nominative unchanged
return noun
}
// Definite article / demonstrative: se/sēo/þæt
//
// Old English used the demonstrative pronoun se/sēo/þæt as a definite article.
// The full paradigm (gender × case × number) is given below.
//
// Masculine:
// sg: nom se acc þone gen þæs dat þǣm
// pl: nom þā acc þā gen þāra dat þǣm
//
// Feminine:
// sg: nom sēo acc þā gen þǣre dat þǣre
// pl: nom þā acc þā gen þāra dat þǣm
//
// Neuter:
// sg: nom þæt acc þæt gen þæs dat þǣm
// pl: nom þā acc þā gen þāra dat þǣm
fn ang_article_masculine(gram_case: String, number: String) -> String {
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return "se" }
if str_eq(gram_case, "accusative") { return "þone" }
if str_eq(gram_case, "genitive") { return "þæs" }
if str_eq(gram_case, "dative") { return "þǣm" }
return "se"
}
// plural
if str_eq(gram_case, "nominative") { return "þā" }
if str_eq(gram_case, "accusative") { return "þā" }
if str_eq(gram_case, "genitive") { return "þāra" }
if str_eq(gram_case, "dative") { return "þǣm" }
return "þā"
}
fn ang_article_feminine(gram_case: String, number: String) -> String {
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return "sēo" }
if str_eq(gram_case, "accusative") { return "þā" }
if str_eq(gram_case, "genitive") { return "þǣre" }
if str_eq(gram_case, "dative") { return "þǣre" }
return "sēo"
}
// plural
if str_eq(gram_case, "nominative") { return "þā" }
if str_eq(gram_case, "accusative") { return "þā" }
if str_eq(gram_case, "genitive") { return "þāra" }
if str_eq(gram_case, "dative") { return "þǣm" }
return "þā"
}
fn ang_article_neuter(gram_case: String, number: String) -> String {
if str_eq(number, "singular") {
if str_eq(gram_case, "nominative") { return "þæt" }
if str_eq(gram_case, "accusative") { return "þæt" }
if str_eq(gram_case, "genitive") { return "þæs" }
if str_eq(gram_case, "dative") { return "þǣm" }
return "þæt"
}
// plural
if str_eq(gram_case, "nominative") { return "þā" }
if str_eq(gram_case, "accusative") { return "þā" }
if str_eq(gram_case, "genitive") { return "þāra" }
if str_eq(gram_case, "dative") { return "þǣm" }
return "þā"
}
fn ang_article(gender: String, gram_case: String, number: String) -> String {
if str_eq(gender, "masculine") { return ang_article_masculine(gram_case, number) }
if str_eq(gender, "feminine") { return ang_article_feminine(gram_case, number) }
// neuter
return ang_article_neuter(gram_case, number)
}
// Gender inference from noun form
//
// A last-resort heuristic when the caller provides no gender hint.
// -a ending strongly suggests weak masculine or neuter (but most -a nouns are
// masculine weak). Without a full lexicon, masculine is the safe default.
fn ang_infer_gender(noun: String) -> String {
if ang_str_ends(noun, "u") { return "feminine" }
if ang_str_ends(noun, "e") { return "feminine" }
return "masculine"
}
// ang_noun_phrase: noun phrase builder
//
// Produces a declined noun with optional definite article (demonstrative)
// prepended. When gender is empty ("") it is inferred from the noun form.
//
// noun: nominative singular Old English noun
// gram_case: "nominative" | "accusative" | "genitive" | "dative"
// number: "singular" | "plural"
// definite: "true" | "false"
fn ang_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String {
let gender: String = ang_infer_gender(noun)
let declined: String = ang_decline(noun, gram_case, number, gender)
if str_eq(definite, "true") {
let art: String = ang_article(gender, gram_case, number)
return art + " " + declined
}
return declined
}
+44
View File
@@ -0,0 +1,44 @@
// auto-generated by elc --emit-header — do not edit
extern fn ang_str_ends(s: String, suf: String) -> Bool
extern fn ang_str_drop_last(s: String, n: Int) -> String
extern fn ang_str_last_char(s: String) -> String
extern fn ang_str_last2(s: String) -> String
extern fn ang_slot(person: String, number: String) -> Int
extern fn ang_map_canonical(verb: String) -> String
extern fn ang_wesan_past(slot: Int) -> String
extern fn ang_beon_present(slot: Int) -> String
extern fn ang_wesan_present(slot: Int) -> String
extern fn ang_habban_present(slot: Int) -> String
extern fn ang_habban_past(slot: Int) -> String
extern fn ang_gan_present(slot: Int) -> String
extern fn ang_gan_past(slot: Int) -> String
extern fn ang_cuman_present(slot: Int) -> String
extern fn ang_cuman_past(slot: Int) -> String
extern fn ang_secgan_present(slot: Int) -> String
extern fn ang_secgan_past(slot: Int) -> String
extern fn ang_seon_present(slot: Int) -> String
extern fn ang_seon_past(slot: Int) -> String
extern fn ang_don_present(slot: Int) -> String
extern fn ang_don_past(slot: Int) -> String
extern fn ang_willan_present(slot: Int) -> String
extern fn ang_willan_past(slot: Int) -> String
extern fn ang_magan_present(slot: Int) -> String
extern fn ang_magan_past(slot: Int) -> String
extern fn ang_witan_present(slot: Int) -> String
extern fn ang_witan_past(slot: Int) -> String
extern fn ang_weak_present_ending(slot: Int) -> String
extern fn ang_weak_past_stem(stem: String) -> String
extern fn ang_weak_past(stem: String, slot: Int) -> String
extern fn ang_weak_stem(verb: String) -> String
extern fn ang_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn ang_declension(noun: String, gender: String) -> String
extern fn ang_decline_strong_masc(noun: String, gram_case: String, number: String) -> String
extern fn ang_decline_strong_neut(noun: String, gram_case: String, number: String) -> String
extern fn ang_decline_weak(noun: String, gram_case: String, number: String) -> String
extern fn ang_decline(noun: String, gram_case: String, number: String, gender: String) -> String
extern fn ang_article_masculine(gram_case: String, number: String) -> String
extern fn ang_article_feminine(gram_case: String, number: String) -> String
extern fn ang_article_neuter(gram_case: String, number: String) -> String
extern fn ang_article(gender: String, gram_case: String, number: String) -> String
extern fn ang_infer_gender(noun: String) -> String
extern fn ang_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
+729
View File
@@ -0,0 +1,729 @@
// morphology-ar.el - Arabic morphology for the NLG engine.
//
// Implements Arabic verb conjugation, noun inflection (gram_case, gender, number,
// definiteness), and definite-article attachment with sun/moon letter handling.
//
// Arabic is a Semitic language with a trilateral root system: most words derive
// from 3-consonant roots by inserting vowel patterns (أوزان awzan) around the
// root consonants. Verb conjugation is realised as prefix + stem + suffix.
//
// Strategy: the engine takes the 3ms perfect (past tense) form as the canonical
// dictionary key (e.g. كَتَبَ kataba) and applies affix patterns to derive all
// other conjugated forms for Form I (الفعل المجرد) regular verbs. A lookup
// table covers essential irregular and hollow verbs.
//
// Verb tenses covered: "past" (perfect/الماضي), "present" (imperfect/المضارع),
// "future" (سَيَفْعَلُ = sa- + imperfect).
// Persons: first/second/third × masculine/feminine × singular/plural (+ dual stubs).
// Gender params: "m" (masculine) | "f" (feminine).
//
// Depends on: morphology.el (str_ends_with, str_len, str_slice, str_eq, str_drop_last concept)
// String helpers
import "morphology.el"
fn ar_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn ar_str_len(s: String) -> Int {
return str_len(s)
}
fn ar_str_drop_last(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len {
return ""
}
return str_slice(s, 0, len - n)
}
fn ar_str_last_char(s: String) -> String {
let n: Int = str_len(s)
if n == 0 {
return ""
}
return str_slice(s, n - 1, n)
}
// Slot index
//
// Maps person × gender × number to a 0-based slot for table lookups.
// Slot layout (10 cells, matching classical Arabic conjugation paradigm):
// 0 = 3ms (he)
// 1 = 3fs (she)
// 2 = 2ms (you m sg)
// 3 = 2fs (you f sg)
// 4 = 1s (I)
// 5 = 3mp (they m pl)
// 6 = 3fp (they f pl)
// 7 = 2mp (you m pl)
// 8 = 2fp (you f pl)
// 9 = 1p (we)
fn ar_slot(person: String, gender: String, number: String) -> Int {
if str_eq(person, "third") {
if str_eq(number, "singular") {
if str_eq(gender, "f") { return 1 }
return 0
}
// plural
if str_eq(gender, "f") { return 6 }
return 5
}
if str_eq(person, "second") {
if str_eq(number, "singular") {
if str_eq(gender, "f") { return 3 }
return 2
}
// plural
if str_eq(gender, "f") { return 8 }
return 7
}
// first
if str_eq(number, "plural") { return 9 }
return 4
}
// Perfect (past) suffixes
//
// Form I perfect: root-past-stem (e.g. كَتَبَ kataba) + suffix.
// The 3ms form IS the base (no suffix added). All other persons add a suffix
// that replaces or follows the final short vowel of the base.
//
// Pattern (dropping the final -a of the 3ms base, then adding):
// 3ms: -a (base as given)
// 3fs: -at
// 2ms: -ta
// 2fs: -ti
// 1s: -tu
// 3mp: -uu
// 3fp: -na
// 2mp: -tum
// 2fp: -tunna
// 1p: -naa
//
// The base passed to ar_conjugate_form1 is the full 3ms form (ends in -a).
// For suffixed forms we drop the final vowel character (1 byte = the -a) then
// apply the suffix. In Arabic script the final short vowel (fatha ـَ) on the
// last consonant of the base is part of the grapheme cluster of that consonant;
// for our stored strings the form كَتَبَ is stored with the final fatha attached
// to the ب. The suffix strings already include the vowel that replaces it, so
// we drop 1 character from the base.
//
// For simplicity the suffixes below are given as Arabic transliteration that
// the El string system handles as UTF-8. The actual Arabic forms are stored
// as UTF-8 Arabic script literals.
//
// Returns the suffix string (including the vowel carried on the junction
// consonant for suffixed forms). Returns "" for 3ms (base is the full form).
fn ar_perfect_suffix(slot: Int) -> String {
if slot == 0 { return "" } // 3ms: base is already complete
if slot == 1 { return "ت" } // 3fs: -at (تْ taa saakina)
if slot == 2 { return "تَ" } // 2ms: -ta
if slot == 3 { return "تِ" } // 2fs: -ti
if slot == 4 { return "تُ" } // 1s: -tu
if slot == 5 { return "وا" } // 3mp: -uu (واو + alif farika)
if slot == 6 { return "نَ" } // 3fp: -na
if slot == 7 { return "تُمْ" } // 2mp: -tum
if slot == 8 { return "تُنَّ" } // 2fp: -tunna
return "نَا" // 1p: -naa (9)
}
// Imperfect (present) prefixes
//
// Form I imperfect: prefix + middle vowel pattern + suffix.
// Prefix depends on person (and for 1s the prefix is أَ).
fn ar_imperfect_prefix(slot: Int) -> String {
if slot == 0 { return "يَ" } // 3ms: ya-
if slot == 1 { return "تَ" } // 3fs: ta-
if slot == 2 { return "تَ" } // 2ms: ta-
if slot == 3 { return "تَ" } // 2fs: ta-
if slot == 4 { return "أَ" } // 1s: a-
if slot == 5 { return "يَ" } // 3mp: ya-
if slot == 6 { return "يَ" } // 3fp: ya-
if slot == 7 { return "تَ" } // 2mp: ta-
if slot == 8 { return "تَ" } // 2fp: ta-
return "نَ" // 1p: na- (9)
}
// Imperfect (present) suffixes
//
// Standard Form I imperfect yaf'ulu / yaf'alu / yaf'ilu vowel class.
// The stem vowel is encoded in the verb's imperfect stem (stored in the lookup
// table or derived from the base). The suffix encodes number/gender/person.
//
// Suffix pattern (after the u-class stem: yaktubu):
// 3ms: -u (yaktub-u)
// 3fs: -u (taktub-u)
// 2ms: -u (taktub-u)
// 2fs: -iina (taktub-iina)
// 1s: -u (aktub-u)
// 3mp: -uuna (yaktub-uuna)
// 3fp: -na (yaktub-na)
// 2mp: -uuna (taktub-uuna)
// 2fp: -na (taktub-na)
// 1p: -u (naktub-u)
fn ar_imperfect_suffix(slot: Int) -> String {
if slot == 0 { return "ُ" } // 3ms: -u
if slot == 1 { return "ُ" } // 3fs: -u
if slot == 2 { return "ُ" } // 2ms: -u
if slot == 3 { return "ِينَ" } // 2fs: -iina
if slot == 4 { return "ُ" } // 1s: -u
if slot == 5 { return "ُونَ" } // 3mp: -uuna
if slot == 6 { return "نَ" } // 3fp: -na
if slot == 7 { return "ُونَ" } // 2mp: -uuna
if slot == 8 { return "نَ" } // 2fp: -na
return "ُ" // 1p: -u (9)
}
// Form I conjugation
//
// ar_conjugate_form1: conjugate a regular Form I verb.
//
// past_base: the 3ms perfect form (e.g. "كَتَبَ")
// present_stem: the imperfect stem without prefix (e.g. "كْتُبُ" for yaktubu)
// This is the middle part after stripping the prefix: for يَكْتُبُ
// the stem = "كْتُبُ". We strip the final -u vowel diacritic
// (1 char) from the stem and re-add via the suffix.
// tense: "past" | "present" | "future"
// slot: ar_slot result
fn ar_conjugate_form1(past_base: String, present_stem: String, tense: String, slot: Int) -> String {
if str_eq(tense, "past") {
// 3ms: return base as-is
if slot == 0 { return past_base }
// All other forms: drop final character of base (the short -a vowel mark
// on the last root consonant), then append the suffix.
let suf: String = ar_perfect_suffix(slot)
// Drop the last character (the fatha diacritic or final vowel-letter)
let stem: String = ar_str_drop_last(past_base, 1)
return stem + suf
}
if str_eq(tense, "present") {
let pre: String = ar_imperfect_prefix(slot)
let suf: String = ar_imperfect_suffix(slot)
// present_stem already includes the medial vowel pattern (e.g. "كْتُبُ")
// Drop its final character (the -u diacritic) before adding the suffix.
let mid: String = ar_str_drop_last(present_stem, 1)
return pre + mid + suf
}
if str_eq(tense, "future") {
// Future = سَ (sa-) + imperfect 3ms form
let pres_3ms: String = ar_conjugate_form1(past_base, present_stem, "present", 0)
return "سَ" + pres_3ms
}
// Unknown tense: return base form
return past_base
}
// Irregular verb lookup table
//
// Returns the inflected form for verbs that cannot be derived by Form I rules,
// or "" if the verb is not in the table.
//
// Covered verbs (by their 3ms past / dictionary key):
// كَانَ kaana to be (hollow verb, waw-medial)
// ذَهَبَ dhahaba to go (Form I, regular; explicit table for certainty)
// جَاءَ jaa'a to come (hamzated + defective)
// قَالَ qaala to say (hollow verb, waw-medial)
// رَأَى ra'aa to see (hamzated + defective)
// أَكَلَ akala to eat (hamzated initial)
// شَرِبَ shariba to drink (Form I i-class)
// عَرَفَ arafa to know (Form I a-class)
// أَرَادَ araada to want (Form IV hollow)
// اِسْتَطَاعَ istata'a can/be able (Form X)
// فَعَلَ fa'ala to do/act (Form I; paradigm verb)
// أَخَذَ akhadha to take (hamzated initial)
// عَمِلَ amila to work (Form I i-class)
//
// For each verb: [past_3ms, past_3fs, past_2ms, past_2fs, past_1s,
// past_3mp, past_3fp, past_2mp, past_2fp, past_1p,
// pres_3ms, pres_3fs, pres_2ms, pres_2fs, pres_1s,
// pres_3mp, pres_3fp, pres_2mp, pres_2fp, pres_1p]
fn ar_irregular_kaana(slot: Int, tense: String) -> String {
// كَانَ to be
if str_eq(tense, "past") {
if slot == 0 { return "كَانَ" }
if slot == 1 { return "كَانَتْ" }
if slot == 2 { return "كُنْتَ" }
if slot == 3 { return "كُنْتِ" }
if slot == 4 { return "كُنْتُ" }
if slot == 5 { return "كَانُوا" }
if slot == 6 { return "كُنَّ" }
if slot == 7 { return "كُنْتُمْ" }
if slot == 8 { return "كُنْتُنَّ" }
return "كُنَّا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يَكُونُ" }
if slot == 1 { return "تَكُونُ" }
if slot == 2 { return "تَكُونُ" }
if slot == 3 { return "تَكُونِينَ" }
if slot == 4 { return "أَكُونُ" }
if slot == 5 { return "يَكُونُونَ" }
if slot == 6 { return "يَكُنَّ" }
if slot == 7 { return "تَكُونُونَ" }
if slot == 8 { return "تَكُنَّ" }
return "نَكُونُ"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_kaana(slot, "present")
return "سَ" + pres
}
return "كَانَ"
}
fn ar_irregular_qaala(slot: Int, tense: String) -> String {
// قَالَ to say (hollow waw-medial)
if str_eq(tense, "past") {
if slot == 0 { return "قَالَ" }
if slot == 1 { return "قَالَتْ" }
if slot == 2 { return "قُلْتَ" }
if slot == 3 { return "قُلْتِ" }
if slot == 4 { return "قُلْتُ" }
if slot == 5 { return "قَالُوا" }
if slot == 6 { return "قُلْنَ" }
if slot == 7 { return "قُلْتُمْ" }
if slot == 8 { return "قُلْتُنَّ" }
return "قُلْنَا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يَقُولُ" }
if slot == 1 { return "تَقُولُ" }
if slot == 2 { return "تَقُولُ" }
if slot == 3 { return "تَقُولِينَ" }
if slot == 4 { return "أَقُولُ" }
if slot == 5 { return "يَقُولُونَ" }
if slot == 6 { return "يَقُلْنَ" }
if slot == 7 { return "تَقُولُونَ" }
if slot == 8 { return "تَقُلْنَ" }
return "نَقُولُ"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_qaala(slot, "present")
return "سَ" + pres
}
return "قَالَ"
}
fn ar_irregular_jaa(slot: Int, tense: String) -> String {
// جَاءَ to come (hamzated defective)
if str_eq(tense, "past") {
if slot == 0 { return "جَاءَ" }
if slot == 1 { return "جَاءَتْ" }
if slot == 2 { return "جِئْتَ" }
if slot == 3 { return "جِئْتِ" }
if slot == 4 { return "جِئْتُ" }
if slot == 5 { return "جَاءُوا" }
if slot == 6 { return "جِئْنَ" }
if slot == 7 { return "جِئْتُمْ" }
if slot == 8 { return "جِئْتُنَّ" }
return "جِئْنَا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يَجِيءُ" }
if slot == 1 { return "تَجِيءُ" }
if slot == 2 { return "تَجِيءُ" }
if slot == 3 { return "تَجِيئِينَ" }
if slot == 4 { return "أَجِيءُ" }
if slot == 5 { return "يَجِيئُونَ" }
if slot == 6 { return "يَجِئْنَ" }
if slot == 7 { return "تَجِيئُونَ" }
if slot == 8 { return "تَجِئْنَ" }
return "نَجِيءُ"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_jaa(slot, "present")
return "سَ" + pres
}
return "جَاءَ"
}
fn ar_irregular_raaa(slot: Int, tense: String) -> String {
// رَأَى to see (hamzated defective)
if str_eq(tense, "past") {
if slot == 0 { return "رَأَى" }
if slot == 1 { return "رَأَتْ" }
if slot == 2 { return "رَأَيْتَ" }
if slot == 3 { return "رَأَيْتِ" }
if slot == 4 { return "رَأَيْتُ" }
if slot == 5 { return "رَأَوْا" }
if slot == 6 { return "رَأَيْنَ" }
if slot == 7 { return "رَأَيْتُمْ" }
if slot == 8 { return "رَأَيْتُنَّ" }
return "رَأَيْنَا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يَرَى" }
if slot == 1 { return "تَرَى" }
if slot == 2 { return "تَرَى" }
if slot == 3 { return "تَرَيْنَ" }
if slot == 4 { return "أَرَى" }
if slot == 5 { return "يَرَوْنَ" }
if slot == 6 { return "يَرَيْنَ" }
if slot == 7 { return "تَرَوْنَ" }
if slot == 8 { return "تَرَيْنَ" }
return "نَرَى"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_raaa(slot, "present")
return "سَ" + pres
}
return "رَأَى"
}
fn ar_irregular_araada(slot: Int, tense: String) -> String {
// أَرَادَ to want (Form IV hollow)
if str_eq(tense, "past") {
if slot == 0 { return "أَرَادَ" }
if slot == 1 { return "أَرَادَتْ" }
if slot == 2 { return "أَرَدْتَ" }
if slot == 3 { return "أَرَدْتِ" }
if slot == 4 { return "أَرَدْتُ" }
if slot == 5 { return "أَرَادُوا" }
if slot == 6 { return "أَرَدْنَ" }
if slot == 7 { return "أَرَدْتُمْ" }
if slot == 8 { return "أَرَدْتُنَّ" }
return "أَرَدْنَا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يُرِيدُ" }
if slot == 1 { return "تُرِيدُ" }
if slot == 2 { return "تُرِيدُ" }
if slot == 3 { return "تُرِيدِينَ" }
if slot == 4 { return "أُرِيدُ" }
if slot == 5 { return "يُرِيدُونَ" }
if slot == 6 { return "يُرِدْنَ" }
if slot == 7 { return "تُرِيدُونَ" }
if slot == 8 { return "تُرِدْنَ" }
return "نُرِيدُ"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_araada(slot, "present")
return "سَ" + pres
}
return "أَرَادَ"
}
fn ar_irregular_istata(slot: Int, tense: String) -> String {
// اِسْتَطَاعَ can / be able (Form X hollow)
if str_eq(tense, "past") {
if slot == 0 { return "اِسْتَطَاعَ" }
if slot == 1 { return "اِسْتَطَاعَتْ" }
if slot == 2 { return "اِسْتَطَعْتَ" }
if slot == 3 { return "اِسْتَطَعْتِ" }
if slot == 4 { return "اِسْتَطَعْتُ" }
if slot == 5 { return "اِسْتَطَاعُوا" }
if slot == 6 { return "اِسْتَطَعْنَ" }
if slot == 7 { return "اِسْتَطَعْتُمْ" }
if slot == 8 { return "اِسْتَطَعْتُنَّ" }
return "اِسْتَطَعْنَا"
}
if str_eq(tense, "present") {
if slot == 0 { return "يَسْتَطِيعُ" }
if slot == 1 { return "تَسْتَطِيعُ" }
if slot == 2 { return "تَسْتَطِيعُ" }
if slot == 3 { return "تَسْتَطِيعِينَ" }
if slot == 4 { return "أَسْتَطِيعُ" }
if slot == 5 { return "يَسْتَطِيعُونَ" }
if slot == 6 { return "يَسْتَطِعْنَ" }
if slot == 7 { return "تَسْتَطِيعُونَ" }
if slot == 8 { return "تَسْتَطِعْنَ" }
return "نَسْتَطِيعُ"
}
if str_eq(tense, "future") {
let pres: String = ar_irregular_istata(slot, "present")
return "سَ" + pres
}
return "اِسْتَطَاعَ"
}
// Irregular verb dispatcher
//
// ar_irregular: returns the inflected form if verb is in the lookup table,
// or "" if not found (caller should use Form I rules).
//
// verb: 3ms past form (dictionary key) as Arabic string
// tense: "past" | "present" | "future"
// slot: ar_slot result
fn ar_irregular(verb: String, tense: String, slot: Int) -> String {
if str_eq(verb, "كَانَ") { return ar_irregular_kaana(slot, tense) }
if str_eq(verb, "قَالَ") { return ar_irregular_qaala(slot, tense) }
if str_eq(verb, "جَاءَ") { return ar_irregular_jaa(slot, tense) }
if str_eq(verb, "رَأَى") { return ar_irregular_raaa(slot, tense) }
if str_eq(verb, "أَرَادَ") { return ar_irregular_araada(slot, tense) }
if str_eq(verb, "اِسْتَطَاعَ") { return ar_irregular_istata(slot, tense) }
return ""
}
// Regular Form I verb table
//
// For regular Form I verbs that would be correctly generated by ar_conjugate_form1
// but whose imperfect stem must be looked up (Arabic verbs have three vowel
// classes for the imperfect medial vowel: a, i, u فَعَلَ/يَفْعَلُ,
// فَعِلَ/يَفْعَلُ, فَعَلَ/يَفْعُلُ). We store the present stem for each.
//
// Returns present_stem (the imperfect without prefix, e.g. "كْتُبُ" for yaktubu),
// or "" if not in table.
fn ar_present_stem(verb: String) -> String {
if str_eq(verb, "كَتَبَ") { return "كْتُبُ" } // kataba -> yaktubu (u-class)
if str_eq(verb, "ذَهَبَ") { return "ذْهَبُ" } // dhahaba -> yadhhabu (a-class)
if str_eq(verb, "أَكَلَ") { return "أْكُلُ" } // akala -> yaakulu (u-class)
if str_eq(verb, "شَرِبَ") { return "شْرَبُ" } // shariba -> yashrabu (a-class)
if str_eq(verb, "عَرَفَ") { return "عْرِفُ" } // arafa -> yarifu (i-class)
if str_eq(verb, "فَعَلَ") { return "فْعَلُ" } // fa'ala -> yaf'alu (a-class)
if str_eq(verb, "أَخَذَ") { return "أْخُذُ" } // akhadha -> yaakhudhu (u-class)
if str_eq(verb, "عَمِلَ") { return "عْمَلُ" } // amila -> ya'malu (a-class)
if str_eq(verb, "دَرَسَ") { return "دْرُسُ" } // darasa -> yadrusu (u-class)
if str_eq(verb, "فَهِمَ") { return "فْهَمُ" } // fahima -> yafhamu (a-class)
if str_eq(verb, "سَمِعَ") { return "سْمَعُ" } // sami'a -> yasma'u (a-class)
if str_eq(verb, "جَلَسَ") { return "جْلِسُ" } // jalasa -> yajlisu (i-class)
if str_eq(verb, "فَتَحَ") { return "فْتَحُ" } // fataha -> yaftahu (a-class)
if str_eq(verb, "خَرَجَ") { return "خْرُجُ" } // kharaja -> yakhruju (u-class)
if str_eq(verb, "دَخَلَ") { return "دْخُلُ" } // dakhala -> yadkhulu (u-class)
if str_eq(verb, "وَجَدَ") { return "جِدُ" } // wajada -> yajidu (i-class, waw-initial)
if str_eq(verb, "صَنَعَ") { return "صْنَعُ" } // sana'a -> yasna'u (a-class)
if str_eq(verb, "رَجَعَ") { return "رْجِعُ" } // raja'a -> yarji'u (i-class)
if str_eq(verb, "وَقَفَ") { return "قِفُ" } // waqafa -> yaqifu (i-class, waw-initial)
if str_eq(verb, "قَرَأَ") { return "قْرَأُ" } // qara'a -> yaqra'u (a-class)
if str_eq(verb, "كَذَبَ") { return "كْذِبُ" } // kadhaba -> yakdhibu (i-class)
return ""
}
// Main conjugation dispatcher
//
// ar_conjugate: conjugate an Arabic verb.
//
// verb: 3ms perfect form (dictionary key), e.g. "كَتَبَ"
// tense: "past" | "present" | "future"
// person: "first" | "second" | "third"
// gender: "m" | "f"
// number: "singular" | "plural"
fn ar_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String {
let slot: Int = ar_slot(person, gender, number)
// 1. Check irregular table
let irreg: String = ar_irregular(verb, tense, slot)
if !str_eq(irreg, "") {
return irreg
}
// 2. Look up present stem for regular Form I
let present_stem: String = ar_present_stem(verb)
if !str_eq(present_stem, "") {
return ar_conjugate_form1(verb, present_stem, tense, slot)
}
// 3. Fallback: return base form (3ms past) unknown verb
return verb
}
// Definite article
//
// ar_definite_article: prefix ال (al-) to a noun with sun/moon letter handling.
//
// Sun letters (الحروف الشمسية) cause the lam of the article to assimilate to
// the first letter of the noun. Moon letters (الحروف القمرية) do not.
//
// Sun letters (Unicode Arabic code points):
// ت ث د ذ ر ز س ش ص ض ط ظ ل ن
//
// Moon letters (all others):
// أ ب ج ح خ ع غ ف ق ك م ه و ي
//
// In Arabic orthography the assimilation is shown with a shadda on the sun letter.
// Here we return "ال" (al-) for moon letters and the assimilated form for sun
// letters. The noun is prefixed with the article; the article lam is replaced
// by a shadda on the sun consonant.
fn ar_is_sun_letter(c: String) -> Bool {
if str_eq(c, "ت") { return true }
if str_eq(c, "ث") { return true }
if str_eq(c, "د") { return true }
if str_eq(c, "ذ") { return true }
if str_eq(c, "ر") { return true }
if str_eq(c, "ز") { return true }
if str_eq(c, "س") { return true }
if str_eq(c, "ش") { return true }
if str_eq(c, "ص") { return true }
if str_eq(c, "ض") { return true }
if str_eq(c, "ط") { return true }
if str_eq(c, "ظ") { return true }
if str_eq(c, "ل") { return true }
if str_eq(c, "ن") { return true }
return false
}
fn ar_definite_article(noun: String) -> String {
// Extract first character to determine sun/moon
let n: Int = ar_str_len(noun)
if n == 0 {
return noun
}
let first: String = str_slice(noun, 0, 1)
if ar_is_sun_letter(first) {
// Sun letter: article lam assimilates -> الـ + shadda on first letter
// Written as: أَلْ + first + shadda + rest
// We represent this as "ال" + first_with_shadda + rest_of_noun
// The shadda diacritic (U+0651) attaches to the sun letter.
let shadda: String = "ّ"
let rest: String = str_slice(noun, 1, n)
return "ال" + first + shadda + rest
}
// Moon letter: simple al- prefix
return "ال" + noun
}
// Case endings
//
// ar_case_ending: return the short vowel ending for a noun given its gram_case
// and definiteness.
//
// case: "nom" | "acc" | "gen"
// definite: "true" | "false"
//
// Indefinite endings carry nunation (tanwin):
// nom: -un (ٌ)
// acc: -an (ً)
// gen: -in (ٍ)
//
// Definite endings are single short vowels:
// nom: -u (ُ)
// acc: -a (َ)
// gen: -i (ِ)
fn ar_case_ending(kase: String, definite: String) -> String {
let is_def: Bool = str_eq(definite, "true")
if str_eq(kase, "nom") {
if is_def { return "ُ" }
return "ٌ"
}
if str_eq(kase, "acc") {
if is_def { return "َ" }
return "ً"
}
if str_eq(kase, "gen") {
if is_def { return "ِ" }
return "ٍ"
}
return ""
}
// Gender inference
//
// ar_gender: infer gender from noun form.
// Returns "f" for nouns ending in taa marbuta (ة or ـة), otherwise "m".
// This covers the most reliable heuristic; broken plurals and loanwords may
// vary but are handled by explicit lookup in the Engram.
fn ar_gender(noun: String) -> String {
if ar_str_ends(noun, "ة") { return "f" }
if ar_str_ends(noun, "ـة") { return "f" }
return "m"
}
// Sound plurals
//
// ar_sound_plural: form the sound masculine or feminine plural.
//
// Sound masculine plural (جمع المذكر السالم):
// nom: -uuna (ونَ)
// acc/gen: -iina (ينَ)
//
// Sound feminine plural (جمع المؤنث السالم):
// Remove final ة (taa marbuta) if present, then add -aat (اتٌ/اتُ).
//
// This function returns the base plural form (without case ending) suitable
// for passing to ar_noun_form. For masculine plural case variation, callers
// should use ar_masc_pl_ending.
fn ar_masc_pl_ending(kase: String) -> String {
if str_eq(kase, "nom") { return "ونَ" }
// acc and gen both use -iina in sound masculine plural
return "ينَ"
}
fn ar_sound_plural(noun: String, gender: String) -> String {
if str_eq(gender, "f") {
// Feminine sound plural: drop ة, add ات
if ar_str_ends(noun, "ة") {
let base: String = ar_str_drop_last(noun, 1)
return base + "ات"
}
return noun + "ات"
}
// Masculine sound plural (nominative form as default): -uuna
return noun + "ون"
}
// Full noun inflection
//
// ar_noun_form: produce the inflected noun form.
//
// noun: base (singular) noun string
// gender: "m" | "f" (pass "" to infer from noun ending)
// kase: "nom" | "acc" | "gen" | "" (no case ending added)
// number: "singular" | "plural"
// definite: "true" | "false"
//
// For plurals, the function applies the sound plural (broken plurals are
// language-external and must be supplied via Engram vocabulary nodes).
fn ar_noun_form(noun: String, gender: String, kase: String, number: String, definite: String) -> String {
// Resolve gender
let g: String = gender
if str_eq(g, "") {
let g = ar_gender(noun)
}
// Build the stem (with definiteness and number)
let stem: String = noun
if str_eq(number, "plural") {
if str_eq(g, "m") {
// Masculine sound plural: stem + case-dependent ending
let pl_suf: String = ar_masc_pl_ending(kase)
if str_eq(definite, "true") {
let def_stem: String = ar_definite_article(noun)
return def_stem + pl_suf
}
return noun + pl_suf
}
// Feminine plural: drop ة, add ات + case ending
let fem_pl: String = ar_sound_plural(noun, "f")
let case_end: String = ar_case_ending(kase, definite)
if str_eq(definite, "true") {
return ar_definite_article(fem_pl) + case_end
}
return fem_pl + case_end
}
// Singular
let case_end: String = ar_case_ending(kase, definite)
if str_eq(definite, "true") {
let def_stem: String = ar_definite_article(noun)
return def_stem + case_end
}
return noun + case_end
}
// Convenience: verb inflect entry point
//
// ar_verb_form: thin wrapper matching the signature style of the main engine.
// Accepts gender as part of person encoding: "third_m" | "third_f" | "first" | "second_m" | "second_f".
// Alternatively accepts explicit gender param.
fn ar_verb_form(verb: String, tense: String, person: String, number: String) -> String {
// Default gender to masculine
return ar_conjugate(verb, tense, person, "m", number)
}
+27
View File
@@ -0,0 +1,27 @@
// auto-generated by elc --emit-header — do not edit
extern fn ar_str_ends(s: String, suf: String) -> Bool
extern fn ar_str_len(s: String) -> Int
extern fn ar_str_drop_last(s: String, n: Int) -> String
extern fn ar_str_last_char(s: String) -> String
extern fn ar_slot(person: String, gender: String, number: String) -> Int
extern fn ar_perfect_suffix(slot: Int) -> String
extern fn ar_imperfect_prefix(slot: Int) -> String
extern fn ar_imperfect_suffix(slot: Int) -> String
extern fn ar_conjugate_form1(past_base: String, present_stem: String, tense: String, slot: Int) -> String
extern fn ar_irregular_kaana(slot: Int, tense: String) -> String
extern fn ar_irregular_qaala(slot: Int, tense: String) -> String
extern fn ar_irregular_jaa(slot: Int, tense: String) -> String
extern fn ar_irregular_raaa(slot: Int, tense: String) -> String
extern fn ar_irregular_araada(slot: Int, tense: String) -> String
extern fn ar_irregular_istata(slot: Int, tense: String) -> String
extern fn ar_irregular(verb: String, tense: String, slot: Int) -> String
extern fn ar_present_stem(verb: String) -> String
extern fn ar_conjugate(verb: String, tense: String, person: String, gender: String, number: String) -> String
extern fn ar_is_sun_letter(c: String) -> Bool
extern fn ar_definite_article(noun: String) -> String
extern fn ar_case_ending(kase: String, definite: String) -> String
extern fn ar_gender(noun: String) -> String
extern fn ar_masc_pl_ending(kase: String) -> String
extern fn ar_sound_plural(noun: String, gender: String) -> String
extern fn ar_noun_form(noun: String, gender: String, kase: String, number: String, definite: String) -> String
extern fn ar_verb_form(verb: String, tense: String, person: String, number: String) -> String
+577
View File
@@ -0,0 +1,577 @@
// morphology-cop.el - Coptic (Sahidic dialect) morphology for the NLG engine.
//
// Implements Coptic verb conjugation (bipartite and tripartite patterns), noun
// phrase assembly with definite and indefinite articles, and noun number marking.
// Designed as a companion to morphology.el; called when language code is "cop".
//
// Language profile: code=cop, name=Coptic, morph_type=agglutinative,
// word_order=SVO, question_strategy=particle, script=coptic, family=afro-asiatic-egyptian.
//
// Script: Coptic uses the Greek alphabet plus seven additional letters borrowed
// from Demotic Egyptian. All Coptic-script characters in this file use their
// correct Unicode code points (Coptic block U+2C80U+2CFF; Coptic letters also
// appear in the Greek block: ϣ U+03E3, ϥ U+03E5, ϩ U+03E9, ϫ U+03EB, ϭ U+03ED).
//
// The El runtime stores strings as byte arrays. String literals with Coptic
// Unicode characters are encoded as UTF-8 and compared via str_eq byte equality.
// The runtime limitation on non-ASCII *output display* does not affect internal
// string logic str_eq and concatenation work correctly.
//
// Grammatical notes (Sahidic Coptic, ca. 2001000 CE):
// - SVO word order (Greek influence; reversed from classical Egyptian)
// - Definite articles prefixed directly to the noun (no space):
// p- (masc sg), t- (fem sg), n- (plural) definite
// ou- (sg indefinite), hen- (pl indefinite)
// - Grammatical gender: masculine / feminine (still active)
// - No case endings grammatical role expressed by word order + prepositions
// - Verb tense/aspect expressed by conjugation base (bipartite pattern):
// Present I: pronoun prefix + verb stem ("f-bwk" = he goes)
// Perfect: a- + pronoun prefix + verb ("a-f-bwk" = he went)
// Future: pronoun prefix + na- + verb ("f-na-bwk" = he will go)
// - Pronoun prefixes (Sahidic used as subject markers in bipartite conjugation):
// 1sg: a-/t- (full: ⲁⲛⲟⲕ) 2sg m: k- 2sg f: te-
// 3sg m: f- 3sg f: s-
// 1pl: n- 2pl: teten- 3pl: se-
// - Copula: "pe" (m sg), "te" (f sg), "ne" (pl); zero copula for adj predicates
// - "to be/become": ϣωπε (Sahidic; present: fϣoop / sϣoop; past: afϣwpe)
//
// Verbs covered (Sahidic transliteration / Coptic script):
// ϣωπε (shwpe) to be / become bwk to go
// nau to see jw to say / speak
// di to give
//
// Canonical English Coptic mapping:
// "be" ϣωπε / zero copula "go" bwk
// "see" nau "say" jw
// "give" di
//
// Persons/numbers covered:
// person: "first" | "second" | "third"
// gender: "m" | "f" (relevant for 2sg and 3sg pronoun prefix selection)
// number: "singular" | "plural"
//
// Depends on: morphology.el (str_eq, str_len, str_slice, str_ends_with)
// String helpers
import "morphology.el"
fn cop_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn cop_str_len(s: String) -> Int {
return str_len(s)
}
fn cop_drop(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len { return "" }
return str_slice(s, 0, len - n)
}
fn cop_last_char(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "" }
return str_slice(s, n - 1, n)
}
// Person/number slot
//
// Maps person × number to a 0-based index used in paradigm tables.
// Gender is not encoded in the slot index here; it is passed separately to
// cop_subject_prefix where it matters (2sg and 3sg distinction).
//
// Slot layout:
// 0 = 1st singular (ⲁⲛⲟⲕ anok)
// 1 = 2nd singular (ⲛⲧⲟⲕ/ⲛⲧⲟ ntok/nto) gender resolved in cop_subject_prefix
// 2 = 3rd singular (ⲛⲧⲟϥ/ⲛⲧⲟⲥ ntof/ntos) gender resolved in cop_subject_prefix
// 3 = 1st plural (ⲁⲛⲟⲛ anon)
// 4 = 2nd plural (ⲛⲧⲱⲧⲉⲛ ntwten)
// 5 = 3rd plural (ⲛⲧⲟⲩ ntou)
fn cop_slot(person: String, number: String) -> Int {
if str_eq(person, "first") {
if str_eq(number, "singular") { return 0 }
return 3
}
if str_eq(person, "second") {
if str_eq(number, "singular") { return 1 }
return 4
}
// third
if str_eq(number, "singular") { return 2 }
return 5
}
// Subject pronoun prefixes
//
// Coptic bipartite conjugation uses short pronoun prefixes attached directly to
// the verb stem (or to the tense base in tripartite). These are the Sahidic
// bound subject pronouns.
//
// Full independent pronouns (for reference):
// 1sg: ⲁⲛⲟⲕ (anok) prefix: ⲁ- / ⲧ- (varies by tense base)
// 2sg m: ⲛⲧⲟⲕ (ntok) prefix: ⲕ-
// 2sg f: ⲛⲧⲟ (nto) prefix: ⲧⲉ-
// 3sg m: ⲛⲧⲟϥ (ntof) prefix: ϥ-
// 3sg f: ⲛⲧⲟⲥ (ntos) prefix: -
// 1pl: ⲁⲛⲟⲛ (anon) prefix: ⲛ-
// 2pl: ⲛⲧⲱⲧⲉⲛ (ntwten) prefix: ⲧⲉⲧⲉⲛ-
// 3pl: ⲛⲧⲟⲩ (ntou) prefix: ⲥⲉ-
//
// cop_subject_prefix returns the short bound prefix used in bipartite conjugation.
// For the perfect (a-prefix tense base), the subject prefix follows "a-" directly.
fn cop_subject_prefix(person: String, number: String) -> String {
if str_eq(person, "first") {
if str_eq(number, "singular") { return "" }
return ""
}
if str_eq(person, "second") {
if str_eq(number, "singular") { return "" }
return "ⲧⲉⲧⲉⲛ"
}
// third
if str_eq(number, "singular") { return "ϥ" }
return "ⲥⲉ"
}
// cop_subject_prefix_gendered: like cop_subject_prefix but handles the
// 2sg feminine (ⲧⲉ-) and 3sg feminine (-) distinction.
fn cop_subject_prefix_gendered(person: String, gender: String, number: String) -> String {
if str_eq(person, "first") {
if str_eq(number, "singular") { return "" }
return ""
}
if str_eq(person, "second") {
if str_eq(number, "singular") {
if str_eq(gender, "f") { return "ⲧⲉ" }
return ""
}
return "ⲧⲉⲧⲉⲛ"
}
// third person
if str_eq(number, "singular") {
if str_eq(gender, "f") { return "" }
return "ϥ"
}
return "ⲥⲉ"
}
// Copula
//
// The Coptic nominal/adjectival copula is a standalone particle that agrees with
// the gender and number of the subject:
// Masculine sg: ⲡⲉ (pe)
// Feminine sg: ⲧⲉ (te)
// Plural: ⲛⲉ (ne)
//
// For adjective predicates in the present tense, the copula is often zero
// (following the inherited Egyptian zero-copula rule). This engine returns ""
// for the present adjective predicate and the full copula particle otherwise.
fn cop_copula_particle(gender: String, number: String) -> String {
if str_eq(number, "plural") { return "ⲛⲉ" }
if str_eq(gender, "f") { return "ⲧⲉ" }
return "ⲡⲉ"
}
// Verb: ϣωπε (to be / become)
//
// ϣωπε is the Sahidic verb meaning "to be" or "to become". It is used as a
// substantive/existential copula. For adjective predicate sentences the zero
// copula is preferred (inherited from Egyptian).
//
// Sahidic forms:
// Present I (bipartite): prefix + ϣⲟⲟⲡ (e.g. ϥϣⲟⲟⲡ "he is/exists")
// Perfect (a- base): + prefix + ϣⲱⲡⲉ (e.g. ⲁϥϣⲱⲡⲉ "he became")
// Future (na- infix): prefix + ⲛⲁϣⲱⲡⲉ (e.g. ϥⲛⲁϣⲱⲡⲉ "he will become")
//
// Note: ϣⲟⲟⲡ (shoop) is the present stem; ϣⲱⲡⲉ (shwpe) is the infinitive/perfect stem.
fn cop_shwpe_present(prefix: String) -> String {
return prefix + "ϣⲟⲟⲡ"
}
fn cop_shwpe_perfect(prefix: String) -> String {
return "" + prefix + "ϣⲱⲡⲉ"
}
fn cop_shwpe_future(prefix: String) -> String {
return prefix + "ⲛⲁϣⲱⲡⲉ"
}
// Verb: bwk (to go) written ⲃⲱⲕ
//
// A common strong verb. The standard bipartite/tripartite pattern applies.
// Present: prefix + ⲃⲱⲕ (e.g. ϥⲃⲱⲕ "he goes")
// Perfect: + prefix + ⲃⲱⲕ (e.g. ⲁϥⲃⲱⲕ "he went")
// Future: prefix + ⲛⲁⲃⲱⲕ (e.g. ϥⲛⲁⲃⲱⲕ "he will go")
fn cop_bwk_present(prefix: String) -> String {
return prefix + "ⲃⲱⲕ"
}
fn cop_bwk_perfect(prefix: String) -> String {
return "" + prefix + "ⲃⲱⲕ"
}
fn cop_bwk_future(prefix: String) -> String {
return prefix + "ⲛⲁⲃⲱⲕ"
}
// Verb: nau (to see) written ⲛⲁⲩ
//
// nau is a biconsonantal verb. Regular bipartite conjugation:
// Present: prefix + ⲛⲁⲩ (e.g. ϥⲛⲁⲩ "he sees")
// Perfect: + prefix + ⲛⲁⲩ (e.g. ⲁϥⲛⲁⲩ "he saw")
// Future: prefix + ⲛⲁⲛⲁⲩ (e.g. ϥⲛⲁⲛⲁⲩ "he will see")
//
// Note: the future prefix "na-" followed by "nau" produces "nanau" standard.
fn cop_nau_present(prefix: String) -> String {
return prefix + "ⲛⲁⲩ"
}
fn cop_nau_perfect(prefix: String) -> String {
return "" + prefix + "ⲛⲁⲩ"
}
fn cop_nau_future(prefix: String) -> String {
return prefix + "ⲛⲁⲛⲁⲩ"
}
// Verb: jw (to say / speak) written ϫⲱ
//
// ϫⲱ is the Sahidic verb for "to say". Bipartite pattern:
// Present: prefix + ϫⲱ (e.g. ϥϫⲱ "he says")
// Perfect: + prefix + ϫⲱ (e.g. ⲁϥϫⲱ "he said")
// Future: prefix + ⲛⲁϫⲱ (e.g. ϥⲛⲁϫⲱ "he will say")
fn cop_jw_present(prefix: String) -> String {
return prefix + "ϫⲱ"
}
fn cop_jw_perfect(prefix: String) -> String {
return "" + prefix + "ϫⲱ"
}
fn cop_jw_future(prefix: String) -> String {
return prefix + "ⲛⲁϫⲱ"
}
// Verb: di (to give) written ϯ
//
// ϯ (ti/di) is a monosyllabic verb meaning "to give". It is very common in
// Coptic texts. Bipartite pattern:
// Present: prefix + ϯ (e.g. ϥϯ "he gives")
// Perfect: + prefix + ϯ (e.g. ⲁϥϯ "he gave")
// Future: prefix + ⲛⲁϯ (e.g. ϥⲛⲁϯ "he will give")
fn cop_di_present(prefix: String) -> String {
return prefix + "ϯ"
}
fn cop_di_perfect(prefix: String) -> String {
return "" + prefix + "ϯ"
}
fn cop_di_future(prefix: String) -> String {
return prefix + "ⲛⲁϯ"
}
// Copula detection
fn cop_is_copula(verb: String) -> Bool {
if str_eq(verb, "ϣωπε") { return true }
if str_eq(verb, "shwpe") { return true }
if str_eq(verb, "be") { return true }
return false
}
// Known-verb dispatcher
//
// Returns the inflected form for a known verb given the subject prefix string
// and tense. Returns "" if the verb is not in the table.
fn cop_known_verb_prefixed(verb: String, tense: String, prefix: String) -> String {
// ϣωπε / shwpe / "be" to be / become
if str_eq(verb, "ϣωπε") {
if str_eq(tense, "present") { return cop_shwpe_present(prefix) }
if str_eq(tense, "past") { return cop_shwpe_perfect(prefix) }
if str_eq(tense, "future") { return cop_shwpe_future(prefix) }
return cop_shwpe_present(prefix)
}
if str_eq(verb, "shwpe") {
if str_eq(tense, "present") { return cop_shwpe_present(prefix) }
if str_eq(tense, "past") { return cop_shwpe_perfect(prefix) }
if str_eq(tense, "future") { return cop_shwpe_future(prefix) }
return cop_shwpe_present(prefix)
}
// bwk / ⲃⲱⲕ to go
if str_eq(verb, "bwk") {
if str_eq(tense, "present") { return cop_bwk_present(prefix) }
if str_eq(tense, "past") { return cop_bwk_perfect(prefix) }
if str_eq(tense, "future") { return cop_bwk_future(prefix) }
return cop_bwk_present(prefix)
}
if str_eq(verb, "ⲃⲱⲕ") {
if str_eq(tense, "present") { return cop_bwk_present(prefix) }
if str_eq(tense, "past") { return cop_bwk_perfect(prefix) }
if str_eq(tense, "future") { return cop_bwk_future(prefix) }
return cop_bwk_present(prefix)
}
if str_eq(verb, "go") {
if str_eq(tense, "present") { return cop_bwk_present(prefix) }
if str_eq(tense, "past") { return cop_bwk_perfect(prefix) }
if str_eq(tense, "future") { return cop_bwk_future(prefix) }
return cop_bwk_present(prefix)
}
// nau / ⲛⲁⲩ to see
if str_eq(verb, "nau") {
if str_eq(tense, "present") { return cop_nau_present(prefix) }
if str_eq(tense, "past") { return cop_nau_perfect(prefix) }
if str_eq(tense, "future") { return cop_nau_future(prefix) }
return cop_nau_present(prefix)
}
if str_eq(verb, "ⲛⲁⲩ") {
if str_eq(tense, "present") { return cop_nau_present(prefix) }
if str_eq(tense, "past") { return cop_nau_perfect(prefix) }
if str_eq(tense, "future") { return cop_nau_future(prefix) }
return cop_nau_present(prefix)
}
if str_eq(verb, "see") {
if str_eq(tense, "present") { return cop_nau_present(prefix) }
if str_eq(tense, "past") { return cop_nau_perfect(prefix) }
if str_eq(tense, "future") { return cop_nau_future(prefix) }
return cop_nau_present(prefix)
}
// jw / ϫⲱ to say / speak
if str_eq(verb, "jw") {
if str_eq(tense, "present") { return cop_jw_present(prefix) }
if str_eq(tense, "past") { return cop_jw_perfect(prefix) }
if str_eq(tense, "future") { return cop_jw_future(prefix) }
return cop_jw_present(prefix)
}
if str_eq(verb, "ϫⲱ") {
if str_eq(tense, "present") { return cop_jw_present(prefix) }
if str_eq(tense, "past") { return cop_jw_perfect(prefix) }
if str_eq(tense, "future") { return cop_jw_future(prefix) }
return cop_jw_present(prefix)
}
if str_eq(verb, "say") {
if str_eq(tense, "present") { return cop_jw_present(prefix) }
if str_eq(tense, "past") { return cop_jw_perfect(prefix) }
if str_eq(tense, "future") { return cop_jw_future(prefix) }
return cop_jw_present(prefix)
}
// di / ϯ to give
if str_eq(verb, "di") {
if str_eq(tense, "present") { return cop_di_present(prefix) }
if str_eq(tense, "past") { return cop_di_perfect(prefix) }
if str_eq(tense, "future") { return cop_di_future(prefix) }
return cop_di_present(prefix)
}
if str_eq(verb, "ϯ") {
if str_eq(tense, "present") { return cop_di_present(prefix) }
if str_eq(tense, "past") { return cop_di_perfect(prefix) }
if str_eq(tense, "future") { return cop_di_future(prefix) }
return cop_di_present(prefix)
}
if str_eq(verb, "give") {
if str_eq(tense, "present") { return cop_di_present(prefix) }
if str_eq(tense, "past") { return cop_di_perfect(prefix) }
if str_eq(tense, "future") { return cop_di_future(prefix) }
return cop_di_present(prefix)
}
// Verb not in table
return ""
}
// Regular verb conjugation
//
// For verbs not in the explicit table, apply the productive bipartite pattern:
// Present: prefix + stem
// Perfect: + prefix + stem
// Future: prefix + ⲛⲁ + stem
fn cop_regular_present(prefix: String, stem: String) -> String {
return prefix + stem
}
fn cop_regular_perfect(prefix: String, stem: String) -> String {
return "" + prefix + stem
}
fn cop_regular_future(prefix: String, stem: String) -> String {
return prefix + "ⲛⲁ" + stem
}
// cop_conjugate: main conjugation entry point
//
// verb: Coptic verb (Sahidic stem, transliterated, or English canonical label)
// tense: "present" | "past" | "future"
// person: "first" | "second" | "third"
// number: "singular" | "plural"
//
// Returns the fully conjugated form with subject prefix embedded.
// Zero copula ("") is returned for present "be" (adj predicate context).
// For unknown verbs the regular bipartite pattern is applied as a productive fallback.
fn cop_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let prefix: String = cop_subject_prefix(person, number)
// Handle "be" canonical zero copula in present; ϣωπε otherwise
if str_eq(verb, "be") {
if str_eq(tense, "present") { return "" }
if str_eq(tense, "past") { return cop_shwpe_perfect(prefix) }
if str_eq(tense, "future") { return cop_shwpe_future(prefix) }
return ""
}
// Try the known-verb table
let known: String = cop_known_verb_prefixed(verb, tense, prefix)
if !str_eq(known, "") {
return known
}
// Regular productive bipartite conjugation
if str_eq(tense, "present") { return cop_regular_present(prefix, verb) }
if str_eq(tense, "past") { return cop_regular_perfect(prefix, verb) }
if str_eq(tense, "future") { return cop_regular_future(prefix, verb) }
// Unknown tense: return verb as safe fallback
return verb
}
// Article system
//
// cop_article: return the Coptic article string for the given gender/number/definiteness.
//
// Definite articles (prefixed directly to noun, no space):
// Masculine singular: ⲡ- (p-)
// Feminine singular: ⲧ- (t-)
// Plural (both): ⲛ- (n-)
//
// Indefinite articles:
// Singular (both genders): ⲟⲩ- (ou-)
// Plural: ϩⲉⲛ- (hen-)
//
// gender: "m" | "f"
// number: "singular" | "plural"
// definite: "true" | "false"
//
// Returns the article prefix string (to be concatenated with the noun).
fn cop_article(gender: String, number: String, definite: String) -> String {
if str_eq(definite, "true") {
if str_eq(number, "plural") { return "" }
if str_eq(gender, "f") { return "" }
return ""
}
// Indefinite
if str_eq(number, "plural") { return "ϩⲉⲛ" }
return "ⲟⲩ"
}
// Noun number
//
// cop_decline: return the noun in the appropriate number form.
//
// Coptic nouns have no case endings. Grammatical role is expressed entirely by
// word order and prepositions. The gram_case parameter is accepted for API
// symmetry with other morphology modules but has no effect.
//
// Plural formation:
// Coptic plural morphology is highly irregular (inherited from Egyptian and
// influenced by Greek loanwords). Common patterns:
// - Many nouns show no suffix change plurality is indicated only by the plural article ⲛ-.
// - Some nouns take -ⲟⲟⲩⲉ (-ooue): e.g. ϩⲟ (face) ϩⲟⲟⲩⲉ
// - Greek loanwords often add -ⲟⲥ / -ⲟⲩ in Greek fashion
//
// This function implements:
// - No suffix change (base form) as the productive default the article carries number.
// - Words ending in (a common Coptic nominal ending) may take -ⲟⲟⲩⲉ in the plural;
// this suffix is applied only when the caller explicitly requests plural and the
// noun ends in (productive pattern).
// Vocabulary-layer irregular plurals should be stored in vocabulary-cop.el and
// passed already inflected.
fn cop_decline(noun: String, gram_case: String, number: String) -> String {
if str_eq(number, "singular") { return noun }
// Plural: if noun ends in ⲉ, attempt -ooue suffix (common productive pattern)
if cop_str_ends(noun, "") {
let stem: String = cop_drop(noun, 1)
return stem + "ⲟⲟⲩⲉ"
}
// Default: base form (article carries the plural signal)
return noun
}
// Noun phrase assembly
//
// cop_noun_phrase: build a complete Coptic noun phrase.
//
// noun: base noun (Coptic script or transliteration)
// gram_case: accepted for API symmetry; has no effect (Coptic is caseless)
// number: "singular" | "plural"
// definite: "true" | "false"
//
// The article is prefixed directly to the noun with no intervening space,
// following standard Coptic orthographic convention.
// Gender defaults to masculine when not determinable from context; the caller
// should supply the declined noun already in its correct form if gender-sensitive
// plural forms are needed.
fn cop_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String {
let form: String = cop_decline(noun, gram_case, number)
// Infer gender from number: if plural, gender is moot for the article (always ⲛ-)
// For singular, default to masculine (caller provides gender via article if known)
let art: String = cop_article("m", number, definite)
if str_eq(definite, "true") {
return art + form
}
if str_eq(definite, "false") {
// Indefinite article + noun (no space Coptic convention for proclitic articles)
return art + form
}
return form
}
// cop_noun_phrase_gendered: noun phrase with explicit gender for correct article selection.
//
// gender: "m" | "f"
fn cop_noun_phrase_gendered(noun: String, gram_case: String, number: String, definite: String, gender: String) -> String {
let form: String = cop_decline(noun, gram_case, number)
let art: String = cop_article(gender, number, definite)
if str_eq(definite, "true") {
return art + form
}
if str_eq(definite, "false") {
return art + form
}
return form
}
// Canonical verb mapping
//
// cop_map_canonical: map cross-lingual English canonical verb labels to their
// Sahidic Coptic equivalents before dispatching to cop_conjugate.
fn cop_map_canonical(verb: String) -> String {
if str_eq(verb, "be") { return "be" }
if str_eq(verb, "go") { return "bwk" }
if str_eq(verb, "see") { return "nau" }
if str_eq(verb, "say") { return "jw" }
if str_eq(verb, "speak") { return "jw" }
if str_eq(verb, "give") { return "di" }
// Unknown: return as-is; cop_conjugate will apply the regular pattern
return verb
}
+35
View File
@@ -0,0 +1,35 @@
// auto-generated by elc --emit-header — do not edit
extern fn cop_str_ends(s: String, suf: String) -> Bool
extern fn cop_str_len(s: String) -> Int
extern fn cop_drop(s: String, n: Int) -> String
extern fn cop_last_char(s: String) -> String
extern fn cop_slot(person: String, number: String) -> Int
extern fn cop_subject_prefix(person: String, number: String) -> String
extern fn cop_subject_prefix_gendered(person: String, gender: String, number: String) -> String
extern fn cop_copula_particle(gender: String, number: String) -> String
extern fn cop_shwpe_present(prefix: String) -> String
extern fn cop_shwpe_perfect(prefix: String) -> String
extern fn cop_shwpe_future(prefix: String) -> String
extern fn cop_bwk_present(prefix: String) -> String
extern fn cop_bwk_perfect(prefix: String) -> String
extern fn cop_bwk_future(prefix: String) -> String
extern fn cop_nau_present(prefix: String) -> String
extern fn cop_nau_perfect(prefix: String) -> String
extern fn cop_nau_future(prefix: String) -> String
extern fn cop_jw_present(prefix: String) -> String
extern fn cop_jw_perfect(prefix: String) -> String
extern fn cop_jw_future(prefix: String) -> String
extern fn cop_di_present(prefix: String) -> String
extern fn cop_di_perfect(prefix: String) -> String
extern fn cop_di_future(prefix: String) -> String
extern fn cop_is_copula(verb: String) -> Bool
extern fn cop_known_verb_prefixed(verb: String, tense: String, prefix: String) -> String
extern fn cop_regular_present(prefix: String, stem: String) -> String
extern fn cop_regular_perfect(prefix: String, stem: String) -> String
extern fn cop_regular_future(prefix: String, stem: String) -> String
extern fn cop_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn cop_article(gender: String, number: String, definite: String) -> String
extern fn cop_decline(noun: String, gram_case: String, number: String) -> String
extern fn cop_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
extern fn cop_noun_phrase_gendered(noun: String, gram_case: String, number: String, definite: String, gender: String) -> String
extern fn cop_map_canonical(verb: String) -> String
+814
View File
@@ -0,0 +1,814 @@
// morphology-de.el - German morphology: articles, adjective endings, noun
// plurals, and verb conjugation.
//
// German is a fusional language with:
// - 4 grammatical cases: nominative, accusative, dative, genitive
// - 3 genders: masculine (m), feminine (f), neuter (n)
// - 2 numbers: singular, plural
// - Strong and weak verb classes
//
// Conventions used throughout:
// gender: "m" | "f" | "n"
// case: "nom" | "acc" | "dat" | "gen"
// number: "sg" | "pl"
// person: "1" | "2" | "3"
// tense: "present" | "past" | "future"
// article_type: "def" | "indef" | "none"
//
// Depends on: language-profile (str_eq, str_len, str_slice, str_drop_last,
// str_ends_with)
// Definite articles (der-words)
//
// Masc Fem Neut Plural
// Nom: der die das die
// Acc: den die das die
// Dat: dem der dem den
// Gen: des der des der
import "morphology.el"
fn de_article_def(gender: String, gram_case: String, number: String) -> String {
if str_eq(number, "pl") {
if str_eq(gram_case, "nom") { return "die" }
if str_eq(gram_case, "acc") { return "die" }
if str_eq(gram_case, "dat") { return "den" }
if str_eq(gram_case, "gen") { return "der" }
return "die"
}
if str_eq(gender, "m") {
if str_eq(gram_case, "nom") { return "der" }
if str_eq(gram_case, "acc") { return "den" }
if str_eq(gram_case, "dat") { return "dem" }
if str_eq(gram_case, "gen") { return "des" }
return "der"
}
if str_eq(gender, "f") {
if str_eq(gram_case, "nom") { return "die" }
if str_eq(gram_case, "acc") { return "die" }
if str_eq(gram_case, "dat") { return "der" }
if str_eq(gram_case, "gen") { return "der" }
return "die"
}
// neuter
if str_eq(gram_case, "nom") { return "das" }
if str_eq(gram_case, "acc") { return "das" }
if str_eq(gram_case, "dat") { return "dem" }
if str_eq(gram_case, "gen") { return "des" }
return "das"
}
// Indefinite articles (ein-words)
//
// Masc Fem Neut Plural
// Nom: ein eine ein
// Acc: einen eine ein
// Dat: einem einer einem
// Gen: eines einer eines
fn de_article_indef(gender: String, gram_case: String, number: String) -> String {
if str_eq(number, "pl") {
// Indefinite article has no plural form
return ""
}
if str_eq(gender, "m") {
if str_eq(gram_case, "nom") { return "ein" }
if str_eq(gram_case, "acc") { return "einen" }
if str_eq(gram_case, "dat") { return "einem" }
if str_eq(gram_case, "gen") { return "eines" }
return "ein"
}
if str_eq(gender, "f") {
if str_eq(gram_case, "nom") { return "eine" }
if str_eq(gram_case, "acc") { return "eine" }
if str_eq(gram_case, "dat") { return "einer" }
if str_eq(gram_case, "gen") { return "einer" }
return "eine"
}
// neuter
if str_eq(gram_case, "nom") { return "ein" }
if str_eq(gram_case, "acc") { return "ein" }
if str_eq(gram_case, "dat") { return "einem" }
if str_eq(gram_case, "gen") { return "eines" }
return "ein"
}
// de_article: unified article dispatch.
// definite: "def" | "indef" | "none"
fn de_article(gender: String, gram_case: String, number: String, definite: String) -> String {
if str_eq(definite, "def") { return de_article_def(gender, gram_case, number) }
if str_eq(definite, "indef") { return de_article_indef(gender, gram_case, number) }
return ""
}
// Adjective endings
//
// Weak endings (after a definite article or der-word):
//
// Masc Fem Neut Plural
// Nom: -e -e -e -en
// Acc: -en -e -e -en
// Dat: -en -en -en -en
// Gen: -en -en -en -en
//
// Mixed endings (after ein-words with no marking, i.e. indef article):
//
// Masc Fem Neut Plural
// Nom: -er -e -es -en
// Acc: -en -e -es -en
// Dat: -en -en -en -en
// Gen: -en -en -en -en
//
// Strong endings (no preceding article):
//
// Masc Fem Neut Plural
// Nom: -er -e -es -e
// Acc: -en -e -es -e
// Dat: -em -er -em -en
// Gen: -en -er -en -er
//
// article_type: "def" | "indef" | "none"
fn de_adj_ending(gender: String, gram_case: String, number: String, article_type: String) -> String {
if str_eq(article_type, "def") {
// Weak declension
if str_eq(number, "pl") {
return "en"
}
if str_eq(gender, "m") {
if str_eq(gram_case, "nom") { return "e" }
return "en"
}
if str_eq(gender, "f") {
if str_eq(gram_case, "nom") { return "e" }
if str_eq(gram_case, "acc") { return "e" }
return "en"
}
// neuter
if str_eq(gram_case, "nom") { return "e" }
if str_eq(gram_case, "acc") { return "e" }
return "en"
}
if str_eq(article_type, "indef") {
// Mixed declension
if str_eq(number, "pl") {
return "en"
}
if str_eq(gender, "m") {
if str_eq(gram_case, "nom") { return "er" }
return "en"
}
if str_eq(gender, "f") {
if str_eq(gram_case, "nom") { return "e" }
if str_eq(gram_case, "acc") { return "e" }
return "en"
}
// neuter
if str_eq(gram_case, "nom") { return "es" }
if str_eq(gram_case, "acc") { return "es" }
return "en"
}
// Strong declension (no article)
if str_eq(number, "pl") {
if str_eq(gram_case, "nom") { return "e" }
if str_eq(gram_case, "acc") { return "e" }
if str_eq(gram_case, "dat") { return "en" }
if str_eq(gram_case, "gen") { return "er" }
return "e"
}
if str_eq(gender, "m") {
if str_eq(gram_case, "nom") { return "er" }
if str_eq(gram_case, "acc") { return "en" }
if str_eq(gram_case, "dat") { return "em" }
if str_eq(gram_case, "gen") { return "en" }
return "er"
}
if str_eq(gender, "f") {
if str_eq(gram_case, "nom") { return "e" }
if str_eq(gram_case, "acc") { return "e" }
if str_eq(gram_case, "dat") { return "er" }
if str_eq(gram_case, "gen") { return "er" }
return "e"
}
// neuter
if str_eq(gram_case, "nom") { return "es" }
if str_eq(gram_case, "acc") { return "es" }
if str_eq(gram_case, "dat") { return "em" }
if str_eq(gram_case, "gen") { return "en" }
return "es"
}
// Noun plural formation
//
// Major patterns, keyed on lemma. Where a noun is known irregular, the full
// plural is returned. Otherwise a productive heuristic by gender and ending
// is applied:
//
// Masculine hard nouns +e (der Tag Tage)
// Feminine nouns in -e +n (die Katze Katzen)
// Feminine nouns +en (die Frau Frauen)
// Neuter nouns in -chen/-lein (das Mädchen Mädchen)
// Neuter nouns in -um -um +en (das Zentrum Zentren)
// Loanwords in -a,-o,-i +s (das Auto Autos)
// Default +e
fn de_noun_plural(noun: String, gender: String) -> String {
// Lexical irregulars
if str_eq(noun, "Mann") { return "Männer" }
if str_eq(noun, "Kind") { return "Kinder" }
if str_eq(noun, "Haus") { return "Häuser" }
if str_eq(noun, "Buch") { return "Bücher" }
if str_eq(noun, "Mutter") { return "Mütter" }
if str_eq(noun, "Vater") { return "Väter" }
if str_eq(noun, "Bruder") { return "Brüder" }
if str_eq(noun, "Tochter") { return "Töchter" }
if str_eq(noun, "Nacht") { return "Nächte" }
if str_eq(noun, "Stadt") { return "Städte" }
if str_eq(noun, "Wort") { return "Wörter" }
if str_eq(noun, "Gott") { return "Götter" }
if str_eq(noun, "Wald") { return "Wälder" }
if str_eq(noun, "Band") { return "Bände" }
if str_eq(noun, "Hund") { return "Hunde" }
if str_eq(noun, "Baum") { return "Bäume" }
if str_eq(noun, "Raum") { return "Räume" }
if str_eq(noun, "Traum") { return "Träume" }
if str_eq(noun, "Zug") { return "Züge" }
if str_eq(noun, "Flug") { return "Flüge" }
if str_eq(noun, "Fuß") { return "Füße" }
if str_eq(noun, "Gruß") { return "Grüße" }
if str_eq(noun, "Geist") { return "Geister" }
if str_eq(noun, "Schwanz") { return "Schwänze" }
if str_eq(noun, "Stuhl") { return "Stühle" }
if str_eq(noun, "Stuhl") { return "Stühle" }
if str_eq(noun, "Sohn") { return "Söhne" }
if str_eq(noun, "Ton") { return "Töne" }
if str_eq(noun, "Fluss") { return "Flüsse" }
if str_eq(noun, "Frau") { return "Frauen" }
if str_eq(noun, "Straße") { return "Straßen" }
if str_eq(noun, "Schule") { return "Schulen" }
if str_eq(noun, "Blume") { return "Blumen" }
if str_eq(noun, "Katze") { return "Katzen" }
if str_eq(noun, "Sprache") { return "Sprachen" }
if str_eq(noun, "Kirche") { return "Kirchen" }
if str_eq(noun, "Tür") { return "Türen" }
if str_eq(noun, "Uhr") { return "Uhren" }
if str_eq(noun, "Zahl") { return "Zahlen" }
if str_eq(noun, "Wahl") { return "Wahlen" }
if str_eq(noun, "Bahn") { return "Bahnen" }
if str_eq(noun, "Zahn") { return "Zähne" }
if str_eq(noun, "Nase") { return "Nasen" }
if str_eq(noun, "Maus") { return "Mäuse" }
if str_eq(noun, "Mädchen") { return "Mädchen" }
if str_eq(noun, "Messer") { return "Messer" }
if str_eq(noun, "Fenster") { return "Fenster" }
if str_eq(noun, "Zimmer") { return "Zimmer" }
if str_eq(noun, "Wasser") { return "Wasser" }
if str_eq(noun, "Bett") { return "Betten" }
if str_eq(noun, "Auto") { return "Autos" }
if str_eq(noun, "Kino") { return "Kinos" }
if str_eq(noun, "Radio") { return "Radios" }
if str_eq(noun, "Foto") { return "Fotos" }
if str_eq(noun, "Cafe") { return "Cafes" }
if str_eq(noun, "Zentrum") { return "Zentren" }
if str_eq(noun, "Museum") { return "Museen" }
if str_eq(noun, "Gymnasium") { return "Gymnasien" }
if str_eq(noun, "Studium") { return "Studien" }
if str_eq(noun, "Datum") { return "Daten" }
// Productive heuristics
// Nouns ending in -chen or -lein: no change (diminutives)
if str_ends_with(noun, "chen") { return noun }
if str_ends_with(noun, "lein") { return noun }
// Nouns ending in -um: replace with -en
if str_ends_with(noun, "um") {
return str_drop_last(noun, 2) + "en"
}
// Loanwords ending in vowel or -s: add -s
if str_ends_with(noun, "a") { return noun + "s" }
if str_ends_with(noun, "o") { return noun + "s" }
if str_ends_with(noun, "i") { return noun + "s" }
if str_ends_with(noun, "u") { return noun + "s" }
if str_ends_with(noun, "y") { return noun + "s" }
// Feminine nouns ending in -e: add -n
if str_eq(gender, "f") {
if str_ends_with(noun, "e") {
return noun + "n"
}
// Feminine nouns ending in -in: add -nen
if str_ends_with(noun, "in") {
return noun + "nen"
}
// Most other feminines: add -en
return noun + "en"
}
// Neuter and masculine: default to +e
return noun + "e"
}
// Noun case endings
//
// In German, noun case inflection is mostly carried by the article and
// adjective. The noun itself only changes in two regular situations:
// - Genitive singular masculine/neuter: -(e)s
// - Dative plural: -n (if not already ending in -n or -s)
//
// Irregular genitive forms (e.g. N-declension: Herr Herrn) are
// handled per-lemma in de_case_ending.
fn de_case_ending(noun: String, gender: String, gram_case: String, number: String) -> String {
// N-declension masculines (weak nouns): all non-nominative singular forms + all plural add -(e)n
if str_eq(noun, "Herr") {
if str_eq(number, "sg") {
if str_eq(gram_case, "nom") { return "Herr" }
return "Herrn"
}
return "Herren"
}
if str_eq(noun, "Mensch") {
if str_eq(number, "sg") {
if str_eq(gram_case, "nom") { return "Mensch" }
return "Menschen"
}
return "Menschen"
}
if str_eq(noun, "Student") {
if str_eq(number, "sg") {
if str_eq(gram_case, "nom") { return "Student" }
return "Studenten"
}
return "Studenten"
}
if str_eq(noun, "Kollege") {
if str_eq(number, "sg") {
if str_eq(gram_case, "nom") { return "Kollege" }
return "Kollegen"
}
return "Kollegen"
}
if str_eq(noun, "Name") {
if str_eq(number, "sg") {
if str_eq(gram_case, "nom") { return "Name" }
if str_eq(gram_case, "gen") { return "Namens" }
return "Namen"
}
return "Namen"
}
// Regular masculine/neuter: genitive singular gets -(e)s
if str_eq(number, "sg") {
if str_eq(gram_case, "gen") {
if str_eq(gender, "m") {
if str_ends_with(noun, "s") { return noun + "es" }
if str_ends_with(noun, "x") { return noun + "es" }
if str_ends_with(noun, "z") { return noun + "es" }
if str_ends_with(noun, "sch") { return noun + "es" }
return noun + "s"
}
if str_eq(gender, "n") {
if str_ends_with(noun, "s") { return noun + "es" }
if str_ends_with(noun, "x") { return noun + "es" }
if str_ends_with(noun, "z") { return noun + "es" }
return noun + "s"
}
}
// All other singular cases: noun unchanged
return noun
}
// Plural dative: add -n unless already ending in -n or -s
if str_eq(gram_case, "dat") {
let pl: String = de_noun_plural(noun, gender)
if str_ends_with(pl, "n") { return pl }
if str_ends_with(pl, "s") { return pl }
return pl + "n"
}
// All other plural cases: return the standard plural form
return de_noun_plural(noun, gender)
}
// Weak verb conjugation
//
// Model: machen (mach-)
//
// Present:
// 1sg ich mache 2sg du machst 3sg er/sie/es macht
// 1pl wir machen 2pl ihr macht 3pl sie machen
//
// Past (Präteritum):
// 1sg ich machte 2sg du machtest 3sg er/sie/es machte
// 1pl wir machten 2pl ihr machtet 3pl sie machten
fn de_conjugate_weak(stem: String, tense: String, person: String, number: String) -> String {
if str_eq(tense, "present") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return stem + "e" }
if str_eq(person, "2") {
// Stems ending in -t or -d insert -e- before -st
if str_ends_with(stem, "t") { return stem + "est" }
if str_ends_with(stem, "d") { return stem + "est" }
return stem + "st"
}
// 3sg
if str_ends_with(stem, "t") { return stem + "et" }
if str_ends_with(stem, "d") { return stem + "et" }
return stem + "t"
}
// plural
if str_eq(person, "1") { return stem + "en" }
if str_eq(person, "2") {
if str_ends_with(stem, "t") { return stem + "et" }
if str_ends_with(stem, "d") { return stem + "et" }
return stem + "t"
}
// 3pl
return stem + "en"
}
if str_eq(tense, "past") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return stem + "te" }
if str_eq(person, "2") { return stem + "test" }
return stem + "te"
}
if str_eq(person, "1") { return stem + "ten" }
if str_eq(person, "2") { return stem + "tet" }
return stem + "ten"
}
// Future: werden + infinitive caller must prepend the auxiliary
return stem + "en"
}
// Strong / irregular verb present-tense forms
//
// Returns the correct surface form if the verb is irregular, or "" if unknown.
// Only present-tense irregulars are encoded here because past tense for strong
// verbs is stored as a separate stem (Ablaut) see de_conjugate.
fn de_irregular_present(verb: String, person: String, number: String) -> String {
// sein fully irregular
if str_eq(verb, "sein") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "bin" }
if str_eq(person, "2") { return "bist" }
return "ist"
}
if str_eq(person, "1") { return "sind" }
if str_eq(person, "2") { return "seid" }
return "sind"
}
// haben
if str_eq(verb, "haben") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "habe" }
if str_eq(person, "2") { return "hast" }
return "hat"
}
if str_eq(person, "1") { return "haben" }
if str_eq(person, "2") { return "habt" }
return "haben"
}
// werden
if str_eq(verb, "werden") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "werde" }
if str_eq(person, "2") { return "wirst" }
return "wird"
}
if str_eq(person, "1") { return "werden" }
if str_eq(person, "2") { return "werdet" }
return "werden"
}
// gehen
if str_eq(verb, "gehen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "gehe" }
if str_eq(person, "2") { return "gehst" }
return "geht"
}
if str_eq(person, "1") { return "gehen" }
if str_eq(person, "2") { return "geht" }
return "gehen"
}
// kommen
if str_eq(verb, "kommen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "komme" }
if str_eq(person, "2") { return "kommst" }
return "kommt"
}
if str_eq(person, "1") { return "kommen" }
if str_eq(person, "2") { return "kommt" }
return "kommen"
}
// sehen vowel change eie in 2sg/3sg
if str_eq(verb, "sehen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "sehe" }
if str_eq(person, "2") { return "siehst" }
return "sieht"
}
if str_eq(person, "1") { return "sehen" }
if str_eq(person, "2") { return "seht" }
return "sehen"
}
// essen vowel change ei in 2sg/3sg
if str_eq(verb, "essen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "esse" }
if str_eq(person, "2") { return "isst" }
return "isst"
}
if str_eq(person, "1") { return "essen" }
if str_eq(person, "2") { return "esst" }
return "essen"
}
// geben vowel change ei in 2sg/3sg
if str_eq(verb, "geben") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "gebe" }
if str_eq(person, "2") { return "gibst" }
return "gibt"
}
if str_eq(person, "1") { return "geben" }
if str_eq(person, "2") { return "gebt" }
return "geben"
}
// nehmen vowel change ei + consonant change in 2sg/3sg
if str_eq(verb, "nehmen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "nehme" }
if str_eq(person, "2") { return "nimmst" }
return "nimmt"
}
if str_eq(person, "1") { return "nehmen" }
if str_eq(person, "2") { return "nehmt" }
return "nehmen"
}
// fahren vowel change aä in 2sg/3sg
if str_eq(verb, "fahren") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "fahre" }
if str_eq(person, "2") { return "fährst" }
return "fährt"
}
if str_eq(person, "1") { return "fahren" }
if str_eq(person, "2") { return "fahrt" }
return "fahren"
}
// laufen vowel change auäu in 2sg/3sg
if str_eq(verb, "laufen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "laufe" }
if str_eq(person, "2") { return "läufst" }
return "läuft"
}
if str_eq(person, "1") { return "laufen" }
if str_eq(person, "2") { return "lauft" }
return "laufen"
}
// wissen irregular throughout
if str_eq(verb, "wissen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "weiß" }
if str_eq(person, "2") { return "weißt" }
return "weiß"
}
if str_eq(person, "1") { return "wissen" }
if str_eq(person, "2") { return "wisst" }
return "wissen"
}
// können modal
if str_eq(verb, "können") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "kann" }
if str_eq(person, "2") { return "kannst" }
return "kann"
}
if str_eq(person, "1") { return "können" }
if str_eq(person, "2") { return "könnt" }
return "können"
}
// müssen modal
if str_eq(verb, "müssen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "muss" }
if str_eq(person, "2") { return "musst" }
return "muss"
}
if str_eq(person, "1") { return "müssen" }
if str_eq(person, "2") { return "müsst" }
return "müssen"
}
// wollen modal
if str_eq(verb, "wollen") {
if str_eq(number, "sg") {
if str_eq(person, "1") { return "will" }
if str_eq(person, "2") { return "willst" }
return "will"
}
if str_eq(person, "1") { return "wollen" }
if str_eq(person, "2") { return "wollt" }
return "wollen"
}
// Unknown: signal caller to fall through to weak conjugation
return ""
}
// Strong verb past-tense (Präteritum) Ablaut stems
//
// Returns the past stem for strong verbs (Ablaut form), or "" if unknown/weak.
// The past-tense endings for strong verbs differ from weak:
// 1sg/3sg: bare stem (no ending)
// 2sg: stem + -st
// 1pl/3pl: stem + -en
// 2pl: stem + -t
fn de_strong_past_stem(verb: String) -> String {
if str_eq(verb, "gehen") { return "ging" }
if str_eq(verb, "kommen") { return "kam" }
if str_eq(verb, "sehen") { return "sah" }
if str_eq(verb, "geben") { return "gab" }
if str_eq(verb, "nehmen") { return "nahm" }
if str_eq(verb, "fahren") { return "fuhr" }
if str_eq(verb, "laufen") { return "lief" }
if str_eq(verb, "schreiben") { return "schrieb" }
if str_eq(verb, "bleiben") { return "blieb" }
if str_eq(verb, "steigen") { return "stieg" }
if str_eq(verb, "lesen") { return "las" }
if str_eq(verb, "sprechen") { return "sprach" }
if str_eq(verb, "treffen") { return "traf" }
if str_eq(verb, "essen") { return "" }
if str_eq(verb, "trinken") { return "trank" }
if str_eq(verb, "finden") { return "fand" }
if str_eq(verb, "denken") { return "dachte" }
if str_eq(verb, "bringen") { return "brachte" }
if str_eq(verb, "stehen") { return "stand" }
if str_eq(verb, "liegen") { return "lag" }
if str_eq(verb, "sitzen") { return "saß" }
if str_eq(verb, "fallen") { return "fiel" }
if str_eq(verb, "halten") { return "hielt" }
if str_eq(verb, "rufen") { return "rief" }
if str_eq(verb, "tragen") { return "trug" }
if str_eq(verb, "schlagen") { return "schlug" }
if str_eq(verb, "ziehen") { return "zog" }
if str_eq(verb, "wachsen") { return "wuchs" }
if str_eq(verb, "helfen") { return "half" }
if str_eq(verb, "werfen") { return "warf" }
return ""
}
// Normalization helpers
//
// The realizer sends long-form labels ("singular", "first").
// German morphology uses short forms ("sg", "1"). Normalize on entry.
fn de_norm_number(number: String) -> String {
if str_eq(number, "singular") { return "sg" }
if str_eq(number, "plural") { return "pl" }
return number
}
fn de_norm_person(person: String) -> String {
if str_eq(person, "first") { return "1" }
if str_eq(person, "second") { return "2" }
if str_eq(person, "third") { return "3" }
return person
}
// Unified German verb conjugation
//
// tense: "present" | "past" | "future"
// person: "1" | "2" | "3" (also accepts "first" | "second" | "third")
// number: "sg" | "pl" (also accepts "singular" | "plural")
fn de_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let number = de_norm_number(number)
let person = de_norm_person(person)
// Future tense: werden (conjugated) + infinitive
if str_eq(tense, "future") {
let aux: String = de_irregular_present("werden", person, number)
return aux + " " + verb
}
// sein past is also fully irregular
if str_eq(verb, "sein") {
if str_eq(tense, "present") {
return de_irregular_present("sein", person, number)
}
// Past (war)
if str_eq(number, "sg") {
if str_eq(person, "1") { return "war" }
if str_eq(person, "2") { return "warst" }
return "war"
}
if str_eq(person, "1") { return "waren" }
if str_eq(person, "2") { return "wart" }
return "waren"
}
// haben past: hatte
if str_eq(verb, "haben") {
if str_eq(tense, "present") {
return de_irregular_present("haben", person, number)
}
if str_eq(number, "sg") {
if str_eq(person, "1") { return "hatte" }
if str_eq(person, "2") { return "hattest" }
return "hatte"
}
if str_eq(person, "1") { return "hatten" }
if str_eq(person, "2") { return "hattet" }
return "hatten"
}
// wissen past: wusste (mixed/irregular)
if str_eq(verb, "wissen") {
if str_eq(tense, "present") {
return de_irregular_present("wissen", person, number)
}
if str_eq(number, "sg") {
if str_eq(person, "1") { return "wusste" }
if str_eq(person, "2") { return "wusstest" }
return "wusste"
}
if str_eq(person, "1") { return "wussten" }
if str_eq(person, "2") { return "wusstet" }
return "wussten"
}
// Modals: können, müssen, wollen past uses weak -te suffix on preterite stem
if str_eq(verb, "können") {
if str_eq(tense, "present") {
return de_irregular_present("können", person, number)
}
return de_conjugate_weak("konnt", "past", person, number)
}
if str_eq(verb, "müssen") {
if str_eq(tense, "present") {
return de_irregular_present("müssen", person, number)
}
return de_conjugate_weak("musst", "past", person, number)
}
if str_eq(verb, "wollen") {
if str_eq(tense, "present") {
return de_irregular_present("wollen", person, number)
}
return de_conjugate_weak("wollt", "past", person, number)
}
// Present: try irregular table first
if str_eq(tense, "present") {
let irr: String = de_irregular_present(verb, person, number)
if !str_eq(irr, "") {
return irr
}
// Fall through to weak conjugation using infinitive stem (drop -en)
let stem: String = str_drop_last(verb, 2)
return de_conjugate_weak(stem, "present", person, number)
}
// Past: try strong Ablaut first
if str_eq(tense, "past") {
let ps: String = de_strong_past_stem(verb)
if !str_eq(ps, "") {
// Strong past endings: 1sg/3sg bare, 2sg+st, 1pl/3pl+en, 2pl+t
if str_eq(number, "sg") {
if str_eq(person, "1") { return ps }
if str_eq(person, "2") { return ps + "st" }
return ps
}
if str_eq(person, "1") { return ps + "en" }
if str_eq(person, "2") { return ps + "t" }
return ps + "en"
}
// Weak past
let stem: String = str_drop_last(verb, 2)
return de_conjugate_weak(stem, "past", person, number)
}
// Fallback: return infinitive
return verb
}
+13
View File
@@ -0,0 +1,13 @@
// auto-generated by elc --emit-header — do not edit
extern fn de_article_def(gender: String, gram_case: String, number: String) -> String
extern fn de_article_indef(gender: String, gram_case: String, number: String) -> String
extern fn de_article(gender: String, gram_case: String, number: String, definite: String) -> String
extern fn de_adj_ending(gender: String, gram_case: String, number: String, article_type: String) -> String
extern fn de_noun_plural(noun: String, gender: String) -> String
extern fn de_case_ending(noun: String, gender: String, gram_case: String, number: String) -> String
extern fn de_conjugate_weak(stem: String, tense: String, person: String, number: String) -> String
extern fn de_irregular_present(verb: String, person: String, number: String) -> String
extern fn de_strong_past_stem(verb: String) -> String
extern fn de_norm_number(number: String) -> String
extern fn de_norm_person(person: String) -> String
extern fn de_conjugate(verb: String, tense: String, person: String, number: String) -> String
+571
View File
@@ -0,0 +1,571 @@
// morphology-egy.el - Ancient Egyptian (Middle Egyptian) morphology for the NLG engine.
//
// Implements Middle Egyptian verb conjugation (sdm=f / sdm.n=f paradigm),
// noun number marking, suffix pronouns, and noun phrase assembly.
// Designed as a companion to morphology.el; called when language code is "egy".
//
// Language profile: code=egy, name=Ancient Egyptian, morph_type=agglutinative,
// word_order=SVO (Middle Egyptian nominal sentences), question_strategy=particle,
// script=hieroglyphic (transliterated here as ASCII), family=afro-asiatic-egyptian.
//
// Script note: Classical transliteration uses special characters ( š q ).
// This engine uses a safe ASCII mapping:
// A = (aleph/glottal stop) a = (ayin)
// H = (h with dot) x = (velar fricative)
// X = (emphatic h) sh = š (sh sound)
// q = q (emphatic k) T = (tj sound)
// D = (dj sound)
// This mapping keeps all string literals ASCII-safe for the El runtime.
//
// Grammatical notes (Middle Egyptian, ca. 20001300 BCE):
// - Aspectual system: Imperfective (sdm=f), Perfective (sdm.n=f), Prospective
// - "tense" labels used here: "present" (imperfective), "past" (perfective),
// "future" (prospective/sdm.xr=f), "infinitive"
// - Two grammatical genders: masculine (unmarked) and feminine (suffix -t)
// - Number: singular (unmarked), dual (-wy masc / -ty fem), plural (-w masc / -wt fem)
// - No case endings syntactic role expressed by word order and prepositions
// - No definite/indefinite article in Middle Egyptian (Late Egyptian introduced pꜣ/tꜣ/nꜣ)
// - Zero copula: adjectival predicates need no verb "to be" ("nfr sw" = "he is good")
// - Suffix pronouns attach directly to the verb stem with = (e.g. sdm=f "he hears")
//
// Persons/numbers covered (suffix pronoun paradigm):
// person: "first" | "second" | "third"
// gender: "m" | "f" (relevant for 2sg, 3sg; 1sg and plurals often unmarked)
// number: "singular" | "dual" | "plural"
//
// Verbs covered (ASCII transliteration gloss):
// wnn to be/exist (copular auxiliary)
// rdi / di to give
// mAA to see
// Dd to say
// Sm to go
// iri to do / make
// sdm to hear (the paradigm verb for the sdm=f construction)
//
// Canonical English Egyptian mapping:
// "be" wnn / zero copula "give" rdi
// "see" mAA "say" Dd
// "go" Sm "do" iri
// "make" iri "hear" sdm
//
// Depends on: morphology.el (str_eq, str_len, str_slice, str_ends_with)
// String helpers
import "morphology.el"
fn egy_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn egy_str_len(s: String) -> Int {
return str_len(s)
}
fn egy_drop(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len { return "" }
return str_slice(s, 0, len - n)
}
fn egy_last_char(s: String) -> String {
let n: Int = str_len(s)
if n == 0 { return "" }
return str_slice(s, n - 1, n)
}
// Person/number slot
//
// Maps person × gender × number to a 0-based index used in paradigm tables.
// Egyptian suffix pronouns distinguish gender in 2nd and 3rd person singular.
//
// Slot layout:
// 0 = 1sg (=i)
// 1 = 2sg masc (=k)
// 2 = 2sg fem (=T)
// 3 = 3sg masc (=f)
// 4 = 3sg fem (=s)
// 5 = 1pl (=n)
// 6 = 2pl (=Tn)
// 7 = 3pl (=sn)
// 8 = 1du / 2du / 3du (=sny simplified; dual pronouns are rare in sources)
//
// Dual falls through to slot 8 (a single dual pronoun slot for all persons).
fn egy_slot(person: String, number: String) -> Int {
if str_eq(number, "dual") { return 8 }
if str_eq(person, "first") {
if str_eq(number, "plural") { return 5 }
return 0
}
if str_eq(person, "second") {
if str_eq(number, "plural") { return 6 }
return 1
}
// third person
if str_eq(number, "plural") { return 7 }
return 3
}
// egy_slot_with_gender: slot variant that factors in gender for 2sg and 3sg.
fn egy_slot_with_gender(person: String, gender: String, number: String) -> Int {
if str_eq(number, "dual") { return 8 }
if str_eq(person, "first") {
if str_eq(number, "plural") { return 5 }
return 0
}
if str_eq(person, "second") {
if str_eq(number, "plural") { return 6 }
if str_eq(gender, "f") { return 2 }
return 1
}
// third person
if str_eq(number, "plural") { return 7 }
if str_eq(gender, "f") { return 4 }
return 3
}
// Suffix pronouns
//
// Egyptian suffix pronouns attach to verbs, nouns, and prepositions.
// Written with = before the pronoun in transliteration (e.g. =f = "his / he").
//
// Standard Middle Egyptian paradigm:
// 1sg: =i ("I / me / my")
// 2sg m: =k ("you / your" masc)
// 2sg f: =T ("you / your" fem, in classical)
// 3sg m: =f ("he / him / his")
// 3sg f: =s ("she / her")
// 1pl: =n ("we / us / our")
// 2pl: =Tn ("you all / your" plural)
// 3pl: =sn ("they / them / their")
// dual: =sny (simplified dual rare in Middle Egyptian texts)
fn egy_conjugate_pronoun(person: String, number: String) -> String {
let slot: Int = egy_slot(person, number)
if slot == 0 { return "=i" }
if slot == 1 { return "=k" }
if slot == 5 { return "=n" }
if slot == 6 { return "=Tn" }
if slot == 7 { return "=sn" }
if slot == 8 { return "=sny" }
// slots 24 need gender; default to masc for slot 3
return "=f"
}
fn egy_suffix_pronoun(slot: Int) -> String {
if slot == 0 { return "=i" }
if slot == 1 { return "=k" }
if slot == 2 { return "=T" }
if slot == 3 { return "=f" }
if slot == 4 { return "=s" }
if slot == 5 { return "=n" }
if slot == 6 { return "=Tn" }
if slot == 7 { return "=sn" }
// dual (slot 8)
return "=sny"
}
// Copula detection
//
// In Middle Egyptian the verb "to be" as predicate is often omitted in the
// present (zero copula for adjective predicates). The auxiliary wnn is used
// for existence / substantive "to be" and in subordinate clauses.
// Canonical English label "be" maps to zero copula in the present.
fn egy_is_copula(verb: String) -> Bool {
if str_eq(verb, "wnn") { return true }
if str_eq(verb, "be") { return true }
return false
}
// Copula conjugation
//
// Present ("imperfective"): zero copula for adjectival predicate return "".
// The auxiliary iw...wnn is used in certain syntactic environments but the
// bare zero is the canonical Middle Egyptian form.
// Past ("perfective"): wnn.n (with perfective suffix .n)
// Future ("prospective"): wnn.xr (prospective form, simplified)
fn egy_conjugate_copula(tense: String, slot: Int) -> String {
if str_eq(tense, "present") { return "" }
if str_eq(tense, "past") {
return "wnn.n" + egy_suffix_pronoun(slot)
}
if str_eq(tense, "future") {
return "wnn.xr" + egy_suffix_pronoun(slot)
}
if str_eq(tense, "infinitive") { return "wnn" }
// Default: zero copula
return ""
}
// Irregular verb: rdi / di (to give)
//
// rdi is the full form; di is the common abbreviated written form.
// Imperfective (present): di=f (3sg m), with full pronoun for other persons.
// Perfective (past): di.n=f
// Prospective (future): di.xr=f
// Infinitive: rdi
fn egy_rdi_present(slot: Int) -> String {
return "di" + egy_suffix_pronoun(slot)
}
fn egy_rdi_past(slot: Int) -> String {
return "di.n" + egy_suffix_pronoun(slot)
}
fn egy_rdi_future(slot: Int) -> String {
return "di.xr" + egy_suffix_pronoun(slot)
}
// Irregular verb: mAA (to see)
//
// mAA is a geminated root (m-AA).
// Present: mAA=f; Past: mAA.n=f; Future: mAA.xr=f
fn egy_mAA_present(slot: Int) -> String {
return "mAA" + egy_suffix_pronoun(slot)
}
fn egy_mAA_past(slot: Int) -> String {
return "mAA.n" + egy_suffix_pronoun(slot)
}
fn egy_mAA_future(slot: Int) -> String {
return "mAA.xr" + egy_suffix_pronoun(slot)
}
// Irregular verb: Dd (to say)
//
// Present: Dd=f; Past: Dd.n=f; Future: Dd.xr=f
// Infinitive: Dd
fn egy_Dd_present(slot: Int) -> String {
return "Dd" + egy_suffix_pronoun(slot)
}
fn egy_Dd_past(slot: Int) -> String {
return "Dd.n" + egy_suffix_pronoun(slot)
}
fn egy_Dd_future(slot: Int) -> String {
return "Dd.xr" + egy_suffix_pronoun(slot)
}
// Irregular verb: Sm (to go)
//
// Present: Sm=f; Past: Sm.n=f; Future: Sm.xr=f
// (Note: the verb Smt "to go" appears in texts; Sm is the most common short form.)
fn egy_Sm_present(slot: Int) -> String {
return "Sm" + egy_suffix_pronoun(slot)
}
fn egy_Sm_past(slot: Int) -> String {
return "Sm.n" + egy_suffix_pronoun(slot)
}
fn egy_Sm_future(slot: Int) -> String {
return "Sm.xr" + egy_suffix_pronoun(slot)
}
// Irregular verb: iri (to do / make)
//
// iri has a contracted 3-radical stem ir- before pronouns.
// Present: ir=f; Past: ir.n=f; Future: ir.xr=f
// Infinitive: iri
fn egy_iri_present(slot: Int) -> String {
return "ir" + egy_suffix_pronoun(slot)
}
fn egy_iri_past(slot: Int) -> String {
return "ir.n" + egy_suffix_pronoun(slot)
}
fn egy_iri_future(slot: Int) -> String {
return "ir.xr" + egy_suffix_pronoun(slot)
}
// Regular verb: sdm (to hear)
//
// sdm (to hear) is the paradigm verb used in grammar textbooks to illustrate
// all Egyptian verb forms. The sdm=f construction names the imperfective suffix
// verb pattern itself.
// Present: sdm=f; Past: sdm.n=f; Future: sdm.xr=f
// Infinitive: sdm
fn egy_sdm_present(slot: Int) -> String {
return "sdm" + egy_suffix_pronoun(slot)
}
fn egy_sdm_past(slot: Int) -> String {
return "sdm.n" + egy_suffix_pronoun(slot)
}
fn egy_sdm_future(slot: Int) -> String {
return "sdm.xr" + egy_suffix_pronoun(slot)
}
// Known-verb dispatcher
//
// Returns the inflected form for a known verb, or "" if unknown.
// Accepts both canonical English labels and Egyptian transliterations.
fn egy_known_verb(verb: String, tense: String, slot: Int) -> String {
// rdi / di to give
if str_eq(verb, "rdi") {
if str_eq(tense, "present") { return egy_rdi_present(slot) }
if str_eq(tense, "past") { return egy_rdi_past(slot) }
if str_eq(tense, "future") { return egy_rdi_future(slot) }
if str_eq(tense, "infinitive") { return "rdi" }
return egy_rdi_present(slot)
}
if str_eq(verb, "di") {
if str_eq(tense, "present") { return egy_rdi_present(slot) }
if str_eq(tense, "past") { return egy_rdi_past(slot) }
if str_eq(tense, "future") { return egy_rdi_future(slot) }
if str_eq(tense, "infinitive") { return "rdi" }
return egy_rdi_present(slot)
}
if str_eq(verb, "give") {
if str_eq(tense, "present") { return egy_rdi_present(slot) }
if str_eq(tense, "past") { return egy_rdi_past(slot) }
if str_eq(tense, "future") { return egy_rdi_future(slot) }
if str_eq(tense, "infinitive") { return "rdi" }
return egy_rdi_present(slot)
}
// mAA to see
if str_eq(verb, "mAA") {
if str_eq(tense, "present") { return egy_mAA_present(slot) }
if str_eq(tense, "past") { return egy_mAA_past(slot) }
if str_eq(tense, "future") { return egy_mAA_future(slot) }
if str_eq(tense, "infinitive") { return "mAA" }
return egy_mAA_present(slot)
}
if str_eq(verb, "see") {
if str_eq(tense, "present") { return egy_mAA_present(slot) }
if str_eq(tense, "past") { return egy_mAA_past(slot) }
if str_eq(tense, "future") { return egy_mAA_future(slot) }
if str_eq(tense, "infinitive") { return "mAA" }
return egy_mAA_present(slot)
}
// Dd to say
if str_eq(verb, "Dd") {
if str_eq(tense, "present") { return egy_Dd_present(slot) }
if str_eq(tense, "past") { return egy_Dd_past(slot) }
if str_eq(tense, "future") { return egy_Dd_future(slot) }
if str_eq(tense, "infinitive") { return "Dd" }
return egy_Dd_present(slot)
}
if str_eq(verb, "say") {
if str_eq(tense, "present") { return egy_Dd_present(slot) }
if str_eq(tense, "past") { return egy_Dd_past(slot) }
if str_eq(tense, "future") { return egy_Dd_future(slot) }
if str_eq(tense, "infinitive") { return "Dd" }
return egy_Dd_present(slot)
}
// Sm to go
if str_eq(verb, "Sm") {
if str_eq(tense, "present") { return egy_Sm_present(slot) }
if str_eq(tense, "past") { return egy_Sm_past(slot) }
if str_eq(tense, "future") { return egy_Sm_future(slot) }
if str_eq(tense, "infinitive") { return "Sm" }
return egy_Sm_present(slot)
}
if str_eq(verb, "go") {
if str_eq(tense, "present") { return egy_Sm_present(slot) }
if str_eq(tense, "past") { return egy_Sm_past(slot) }
if str_eq(tense, "future") { return egy_Sm_future(slot) }
if str_eq(tense, "infinitive") { return "Sm" }
return egy_Sm_present(slot)
}
// iri to do / make
if str_eq(verb, "iri") {
if str_eq(tense, "present") { return egy_iri_present(slot) }
if str_eq(tense, "past") { return egy_iri_past(slot) }
if str_eq(tense, "future") { return egy_iri_future(slot) }
if str_eq(tense, "infinitive") { return "iri" }
return egy_iri_present(slot)
}
if str_eq(verb, "do") {
if str_eq(tense, "present") { return egy_iri_present(slot) }
if str_eq(tense, "past") { return egy_iri_past(slot) }
if str_eq(tense, "future") { return egy_iri_future(slot) }
if str_eq(tense, "infinitive") { return "iri" }
return egy_iri_present(slot)
}
if str_eq(verb, "make") {
if str_eq(tense, "present") { return egy_iri_present(slot) }
if str_eq(tense, "past") { return egy_iri_past(slot) }
if str_eq(tense, "future") { return egy_iri_future(slot) }
if str_eq(tense, "infinitive") { return "iri" }
return egy_iri_present(slot)
}
// sdm to hear
if str_eq(verb, "sdm") {
if str_eq(tense, "present") { return egy_sdm_present(slot) }
if str_eq(tense, "past") { return egy_sdm_past(slot) }
if str_eq(tense, "future") { return egy_sdm_future(slot) }
if str_eq(tense, "infinitive") { return "sdm" }
return egy_sdm_present(slot)
}
if str_eq(verb, "hear") {
if str_eq(tense, "present") { return egy_sdm_present(slot) }
if str_eq(tense, "past") { return egy_sdm_past(slot) }
if str_eq(tense, "future") { return egy_sdm_future(slot) }
if str_eq(tense, "infinitive") { return "sdm" }
return egy_sdm_present(slot)
}
// Verb not in table
return ""
}
// Regular verb conjugation
//
// For verbs not in the explicit table, apply the productive suffix-verb pattern:
// Present (imperfective sdm=f): stem + pronoun suffix
// Past (perfective sdm.n=f): stem + ".n" + pronoun suffix
// Future (prospective sdm.xr=f): stem + ".xr" + pronoun suffix
// Infinitive: stem unchanged
//
// This covers the vast majority of strong (sound) verb roots.
fn egy_regular_present(stem: String, slot: Int) -> String {
return stem + egy_suffix_pronoun(slot)
}
fn egy_regular_past(stem: String, slot: Int) -> String {
return stem + ".n" + egy_suffix_pronoun(slot)
}
fn egy_regular_future(stem: String, slot: Int) -> String {
return stem + ".xr" + egy_suffix_pronoun(slot)
}
// egy_conjugate: main conjugation entry point
//
// verb: Egyptian verb (ASCII transliteration) or English canonical label
// tense: "present" | "past" | "future" | "infinitive"
// person: "first" | "second" | "third"
// number: "singular" | "dual" | "plural"
//
// Returns:
// - "" for present copula (zero copula caller omits the verb)
// - inflected form (stem + .n + suffix, etc.) for all other cases
// - verb + regular suffix for unknown verbs (productive fallback)
fn egy_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let slot: Int = egy_slot(person, number)
// Handle copula (wnn / "be")
if egy_is_copula(verb) {
return egy_conjugate_copula(tense, slot)
}
// Try the known-verb table
let known: String = egy_known_verb(verb, tense, slot)
if !str_eq(known, "") {
return known
}
// Infinitive: return unchanged
if str_eq(tense, "infinitive") { return verb }
// Regular verb: apply productive sdm=f / sdm.n=f pattern
if str_eq(tense, "present") { return egy_regular_present(verb, slot) }
if str_eq(tense, "past") { return egy_regular_past(verb, slot) }
if str_eq(tense, "future") { return egy_regular_future(verb, slot) }
// Unknown tense: return verb unchanged as safe fallback
return verb
}
// Noun number marking
//
// Middle Egyptian nouns are invariant for case syntactic role is expressed by
// word order and prepositions, not noun endings. Number is marked by suffix:
//
// Singular: base form (no suffix)
// Dual: masc + wy / fem + ty (wy and ty in ASCII transliteration)
// Plural: masc + w / fem + wt
//
// Many common nouns have suppletive or irregular plurals (recorded in the
// vocabulary layer). This function implements the productive regular pattern.
//
// gram_case: accepted for API symmetry but has no effect (Egyptian is caseless).
fn egy_decline(noun: String, gram_case: String, number: String) -> String {
if str_eq(number, "singular") { return noun }
if str_eq(number, "dual") {
// Feminine dual: if noun ends in t (feminine marker), replace with ty
if egy_str_ends(noun, "t") {
let stem: String = egy_drop(noun, 1)
return stem + "ty"
}
return noun + "wy"
}
// Plural
if egy_str_ends(noun, "t") {
// Feminine noun: add wt
return noun + "wt"
}
// Masculine noun: add w
return noun + "w"
}
// Feminine derivation
//
// egy_fem: derive the feminine form of a noun or adjective by appending -t.
//
// In Middle Egyptian, the feminine gender marker is the suffix -t (written with
// the bread-loaf hieroglyph, Gardiner X1). If the base already ends in -t the
// form is returned unchanged to avoid double-suffixing.
fn egy_fem(noun: String) -> String {
if egy_str_ends(noun, "t") { return noun }
return noun + "t"
}
// Noun phrase assembly
//
// egy_noun_phrase: return the surface form of a noun phrase.
//
// noun: base noun (ASCII transliteration)
// gram_case: passed for API symmetry; has no effect (Egyptian is caseless)
// number: "singular" | "dual" | "plural"
// definite: "true" | "false" Middle Egyptian has no article; parameter accepted
// for API symmetry. Late Egyptian pꜣ/tꜣ/nꜣ articles are not implemented
// here (they would require knowing the gender of each noun).
//
// Returns the noun in its correct number form.
fn egy_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String {
return egy_decline(noun, gram_case, number)
}
// Canonical verb mapping
//
// egy_map_canonical: map cross-lingual English canonical verb labels to their
// Middle Egyptian equivalents before dispatching to egy_conjugate.
fn egy_map_canonical(verb: String) -> String {
if str_eq(verb, "be") { return "wnn" }
if str_eq(verb, "give") { return "rdi" }
if str_eq(verb, "see") { return "mAA" }
if str_eq(verb, "say") { return "Dd" }
if str_eq(verb, "go") { return "Sm" }
if str_eq(verb, "do") { return "iri" }
if str_eq(verb, "make") { return "iri" }
if str_eq(verb, "hear") { return "sdm" }
// Unknown: return as-is; egy_conjugate will apply the regular pattern
return verb
}
+38
View File
@@ -0,0 +1,38 @@
// auto-generated by elc --emit-header — do not edit
extern fn egy_str_ends(s: String, suf: String) -> Bool
extern fn egy_str_len(s: String) -> Int
extern fn egy_drop(s: String, n: Int) -> String
extern fn egy_last_char(s: String) -> String
extern fn egy_slot(person: String, number: String) -> Int
extern fn egy_slot_with_gender(person: String, gender: String, number: String) -> Int
extern fn egy_conjugate_pronoun(person: String, number: String) -> String
extern fn egy_suffix_pronoun(slot: Int) -> String
extern fn egy_is_copula(verb: String) -> Bool
extern fn egy_conjugate_copula(tense: String, slot: Int) -> String
extern fn egy_rdi_present(slot: Int) -> String
extern fn egy_rdi_past(slot: Int) -> String
extern fn egy_rdi_future(slot: Int) -> String
extern fn egy_mAA_present(slot: Int) -> String
extern fn egy_mAA_past(slot: Int) -> String
extern fn egy_mAA_future(slot: Int) -> String
extern fn egy_Dd_present(slot: Int) -> String
extern fn egy_Dd_past(slot: Int) -> String
extern fn egy_Dd_future(slot: Int) -> String
extern fn egy_Sm_present(slot: Int) -> String
extern fn egy_Sm_past(slot: Int) -> String
extern fn egy_Sm_future(slot: Int) -> String
extern fn egy_iri_present(slot: Int) -> String
extern fn egy_iri_past(slot: Int) -> String
extern fn egy_iri_future(slot: Int) -> String
extern fn egy_sdm_present(slot: Int) -> String
extern fn egy_sdm_past(slot: Int) -> String
extern fn egy_sdm_future(slot: Int) -> String
extern fn egy_known_verb(verb: String, tense: String, slot: Int) -> String
extern fn egy_regular_present(stem: String, slot: Int) -> String
extern fn egy_regular_past(stem: String, slot: Int) -> String
extern fn egy_regular_future(stem: String, slot: Int) -> String
extern fn egy_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn egy_decline(noun: String, gram_case: String, number: String) -> String
extern fn egy_fem(noun: String) -> String
extern fn egy_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
extern fn egy_map_canonical(verb: String) -> String
+451
View File
@@ -0,0 +1,451 @@
// morphology-enm.el - Middle English morphology for the NLG engine.
//
// Implements Middle English verb conjugation and noun declension for the
// ca. 1100-1500 CE period (Chaucerian English). Designed as a companion to
// morphology.el and called by the engine when the language profile code is "enm".
//
// Language profile: code=enm, name=Middle English, morph_type=analytic,
// word_order=SVO, question_strategy=inversion, script=latin, family=germanic.
//
// Verb conjugation covered:
// Tenses: present, past
// Persons: first/second/third x singular/plural (slots 0-5)
// Classes: weak (productive: -est 2sg, -eth 3sg, -en pl; past: -ede/-de/-te)
// Irregulars: been/ben (be), han/haven (have), goon (go), seen (see),
// seyn/seyen (say), comen (come), maken (make)
// Canonical map: "be" -> "been"
//
// Noun declension covered:
// Middle English has largely lost case endings. This module handles:
// - nominative (base form)
// - genitive singular (+es)
// - plural (+es default; irregular forms for common words)
// Common irregulars: man->men, child->children, ox->oxen, foot->feet,
// tooth->teeth
//
// Article formation:
// Definite: "the" prepended
// Indefinite: "a" or "an" based on the first letter of the noun phrase
//
// Depends on: morphology.el (str_ends_with, str_len, str_slice, str_eq)
// String helpers
import "morphology.el"
fn enm_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn enm_drop(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len { return "" }
return str_slice(s, 0, len - n)
}
fn enm_first_char(s: String) -> String {
if str_len(s) == 0 { return "" }
return str_slice(s, 0, 1)
}
// Person/number slot
//
// Maps person x number to a 0-based paradigm slot.
// 0 = 1st singular (I / ich)
// 1 = 2nd singular (thou)
// 2 = 3rd singular (he / she / it)
// 3 = 1st plural (we)
// 4 = 2nd plural (ye)
// 5 = 3rd plural (they)
fn enm_slot(person: String, number: String) -> Int {
if str_eq(person, "first") {
if str_eq(number, "singular") { return 0 }
return 3
}
if str_eq(person, "second") {
if str_eq(number, "singular") { return 1 }
return 4
}
// third
if str_eq(number, "singular") { return 2 }
return 5
}
// Irregular verb tables
//
// Each irregular verb has a present and past paradigm of six forms (slots 0-5).
// Forms are in Middle English spelling, close to Chaucerian usage.
fn enm_been_present(slot: Int) -> String {
if slot == 0 { return "am" }
if slot == 1 { return "art" }
if slot == 2 { return "is" }
if slot == 3 { return "aren" }
if slot == 4 { return "been" }
return "been"
}
fn enm_been_past(slot: Int) -> String {
if slot == 0 { return "was" }
if slot == 1 { return "were" }
if slot == 2 { return "was" }
if slot == 3 { return "were" }
if slot == 4 { return "were" }
return "were"
}
fn enm_haven_present(slot: Int) -> String {
if slot == 0 { return "have" }
if slot == 1 { return "hast" }
if slot == 2 { return "hath" }
if slot == 3 { return "have" }
if slot == 4 { return "have" }
return "have"
}
fn enm_haven_past(slot: Int) -> String {
if slot == 0 { return "hadde" }
if slot == 1 { return "haddest" }
if slot == 2 { return "hadde" }
if slot == 3 { return "hadden" }
if slot == 4 { return "hadden" }
return "hadden"
}
fn enm_goon_present(slot: Int) -> String {
if slot == 0 { return "go" }
if slot == 1 { return "goost" }
if slot == 2 { return "gooth" }
if slot == 3 { return "goon" }
if slot == 4 { return "goon" }
return "goon"
}
fn enm_goon_past(slot: Int) -> String {
if slot == 0 { return "wente" }
if slot == 1 { return "wentest" }
if slot == 2 { return "wente" }
if slot == 3 { return "wenten" }
if slot == 4 { return "wenten" }
return "wenten"
}
fn enm_seen_present(slot: Int) -> String {
if slot == 0 { return "see" }
if slot == 1 { return "seest" }
if slot == 2 { return "seeth" }
if slot == 3 { return "seen" }
if slot == 4 { return "seen" }
return "seen"
}
fn enm_seen_past(slot: Int) -> String {
if slot == 0 { return "saugh" }
if slot == 1 { return "sawest" }
if slot == 2 { return "saugh" }
if slot == 3 { return "sawen" }
if slot == 4 { return "sawen" }
return "sawen"
}
fn enm_seyen_present(slot: Int) -> String {
if slot == 0 { return "seye" }
if slot == 1 { return "seyst" }
if slot == 2 { return "seith" }
if slot == 3 { return "seyen" }
if slot == 4 { return "seyen" }
return "seyen"
}
fn enm_seyen_past(slot: Int) -> String {
if slot == 0 { return "seide" }
if slot == 1 { return "seidest" }
if slot == 2 { return "seide" }
if slot == 3 { return "seiden" }
if slot == 4 { return "seiden" }
return "seiden"
}
fn enm_comen_present(slot: Int) -> String {
if slot == 0 { return "come" }
if slot == 1 { return "comest" }
if slot == 2 { return "cometh" }
if slot == 3 { return "comen" }
if slot == 4 { return "comen" }
return "comen"
}
fn enm_comen_past(slot: Int) -> String {
if slot == 0 { return "cam" }
if slot == 1 { return "come" }
if slot == 2 { return "cam" }
if slot == 3 { return "comen" }
if slot == 4 { return "comen" }
return "comen"
}
fn enm_maken_present(slot: Int) -> String {
if slot == 0 { return "make" }
if slot == 1 { return "makest" }
if slot == 2 { return "maketh" }
if slot == 3 { return "maken" }
if slot == 4 { return "maken" }
return "maken"
}
fn enm_maken_past(slot: Int) -> String {
if slot == 0 { return "made" }
if slot == 1 { return "madest" }
if slot == 2 { return "made" }
if slot == 3 { return "maden" }
if slot == 4 { return "maden" }
return "maden"
}
// Canonical verb mapping
//
// Maps English semantic labels to Middle English infinitives so the semantic
// layer can request forms without knowing the target-language lexeme.
fn enm_map_canonical(verb: String) -> String {
if str_eq(verb, "be") { return "been" }
if str_eq(verb, "have") { return "haven" }
if str_eq(verb, "go") { return "goon" }
if str_eq(verb, "see") { return "seen" }
if str_eq(verb, "say") { return "seyen" }
if str_eq(verb, "come") { return "comen" }
if str_eq(verb, "make") { return "maken" }
return verb
}
// Weak verb stem derivation
//
// For weak verbs the present stem is the infinitive minus any trailing -en or -e.
// Past tense suffix: most common is -ede (after unvoiced consonants often -te,
// after voiced -de). This module uses -ede as the default productive past suffix.
//
// Present:
// slot 0: stem (I love)
// slot 1: stem + est (thou lovest)
// slot 2: stem + eth (he loveth)
// slot 3: stem + en (we loven)
// slot 4: stem + en (ye loven)
// slot 5: stem + en (they loven)
//
// Past:
// slot 0: stem + ede (I lovede)
// slot 1: stem + edest (thou lovedest)
// slot 2: stem + ede (he lovede)
// slot 3..5: stem + eden (we loveden)
fn enm_weak_stem(verb: String) -> String {
if enm_str_ends(verb, "en") { return enm_drop(verb, 2) }
if enm_str_ends(verb, "e") { return enm_drop(verb, 1) }
return verb
}
fn enm_weak_present(stem: String, slot: Int) -> String {
if slot == 0 { return stem + "e" }
if slot == 1 { return stem + "est" }
if slot == 2 { return stem + "eth" }
if slot == 3 { return stem + "en" }
if slot == 4 { return stem + "en" }
return stem + "en"
}
fn enm_weak_past(stem: String, slot: Int) -> String {
if slot == 0 { return stem + "ede" }
if slot == 1 { return stem + "edest" }
if slot == 2 { return stem + "ede" }
if slot == 3 { return stem + "eden" }
if slot == 4 { return stem + "eden" }
return stem + "eden"
}
// enm_conjugate: main conjugation entry point
//
// verb: Middle English infinitive (e.g. "loven", "been") or English canonical
// tense: "present" | "past"
// person: "first" | "second" | "third"
// number: "singular" | "plural"
//
// Returns the inflected form. Unknown tenses fall back to the infinitive rather
// than crashing.
fn enm_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let v: String = enm_map_canonical(verb)
let slot: Int = enm_slot(person, number)
// Irregulars
if str_eq(v, "been") {
if str_eq(tense, "present") { return enm_been_present(slot) }
if str_eq(tense, "past") { return enm_been_past(slot) }
return v
}
if str_eq(v, "haven") {
if str_eq(tense, "present") { return enm_haven_present(slot) }
if str_eq(tense, "past") { return enm_haven_past(slot) }
return v
}
if str_eq(v, "goon") {
if str_eq(tense, "present") { return enm_goon_present(slot) }
if str_eq(tense, "past") { return enm_goon_past(slot) }
return v
}
if str_eq(v, "seen") {
if str_eq(tense, "present") { return enm_seen_present(slot) }
if str_eq(tense, "past") { return enm_seen_past(slot) }
return v
}
if str_eq(v, "seyen") {
if str_eq(tense, "present") { return enm_seyen_present(slot) }
if str_eq(tense, "past") { return enm_seyen_past(slot) }
return v
}
if str_eq(v, "comen") {
if str_eq(tense, "present") { return enm_comen_present(slot) }
if str_eq(tense, "past") { return enm_comen_past(slot) }
return v
}
if str_eq(v, "maken") {
if str_eq(tense, "present") { return enm_maken_present(slot) }
if str_eq(tense, "past") { return enm_maken_past(slot) }
return v
}
// Regular weak verb
let stem: String = enm_weak_stem(v)
if str_eq(tense, "present") { return enm_weak_present(stem, slot) }
if str_eq(tense, "past") { return enm_weak_past(stem, slot) }
// Unknown tense: return infinitive unchanged
return v
}
// Noun plural irregulars
//
// Returns the suppletive plural form for nouns with non-productive plurals, or ""
// if the noun takes the regular -es plural.
//
// Covered: man, woman, child, ox, foot, tooth, goose, mouse, louse
// These mirror patterns still visible in Modern English, present in ME too.
fn enm_irregular_plural(noun: String) -> String {
if str_eq(noun, "man") { return "men" }
if str_eq(noun, "woman") { return "wommen" }
if str_eq(noun, "child") { return "children" }
if str_eq(noun, "ox") { return "oxen" }
if str_eq(noun, "foot") { return "feet" }
if str_eq(noun, "tooth") { return "teeth" }
if str_eq(noun, "goose") { return "gees" }
if str_eq(noun, "mouse") { return "mees" }
if str_eq(noun, "louse") { return "lees" }
return ""
}
// Regular plural formation
//
// Default: append -es. For nouns already ending in -e, append just -s.
// For nouns ending in -s, -x, -sh, -ch: the -es is still appropriate but
// in ME spelling the forms vary; we use the simple +es rule uniformly.
fn enm_make_plural(noun: String) -> String {
// Check suppletive irregular first
let irreg: String = enm_irregular_plural(noun)
if !str_eq(irreg, "") { return irreg }
// Noun ends in -e: just add -s to avoid double vowel
if enm_str_ends(noun, "e") { return noun + "s" }
// Default: +es
return noun + "es"
}
// enm_decline: main declension entry point
//
// Middle English has largely lost case morphology. This function handles the
// three practically relevant categories:
// nominative base form (used also for accusative and dative)
// genitive base form + es (possessive)
// plural irregular or base + es
//
// noun: base nominative form (e.g. "knyght", "man", "lond")
// gram_case: "nominative" | "accusative" | "dative" | "genitive" | "plural"
// ("accusative" and "dative" return the nominative form)
// number: "singular" | "plural"
//
// Returns the inflected form.
fn enm_decline(noun: String, gram_case: String, number: String) -> String {
// Plural number overrides gram_case for the plural form
if str_eq(number, "plural") {
return enm_make_plural(noun)
}
// Singular
if str_eq(gram_case, "genitive") {
// Genitive singular: +es (even after -e: "the kinges court")
return noun + "es"
}
// Nominative, accusative, dative all the same in ME
return noun
}
// Article selection
//
// Middle English uses "the" (definite) and "a" / "an" (indefinite).
// The indefinite article is "an" before a vowel-initial word, "a" otherwise.
// Vowel check is on the first character of the noun phrase word.
fn enm_is_vowel_initial(s: String) -> Bool {
let c: String = enm_first_char(s)
if str_eq(c, "a") { return true }
if str_eq(c, "e") { return true }
if str_eq(c, "i") { return true }
if str_eq(c, "o") { return true }
if str_eq(c, "u") { return true }
// ME also treated initial h as effectively silent in some dialects;
// we conservatively treat h-initial as consonant-initial.
return false
}
fn enm_indef_article(noun_phrase: String) -> String {
if enm_is_vowel_initial(noun_phrase) { return "an" }
return "a"
}
// enm_noun_phrase: noun phrase builder
//
// Constructs a full noun phrase with the appropriate article.
//
// noun: base nominative singular form (e.g. "knyght", "man", "lond")
// gram_case: "nominative" | "accusative" | "dative" | "genitive" | "plural"
// number: "singular" | "plural"
// definite: "true" | "false" (string comparison)
//
// Returns the complete noun phrase string (article + declined noun).
fn enm_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String {
let form: String = enm_decline(noun, gram_case, number)
if str_eq(definite, "true") {
return "the " + form
}
// Indefinite article only makes sense for singular; plural takes no article
if str_eq(number, "plural") {
return form
}
let art: String = enm_indef_article(form)
return art + " " + form
}
+30
View File
@@ -0,0 +1,30 @@
// auto-generated by elc --emit-header — do not edit
extern fn enm_str_ends(s: String, suf: String) -> Bool
extern fn enm_drop(s: String, n: Int) -> String
extern fn enm_first_char(s: String) -> String
extern fn enm_slot(person: String, number: String) -> Int
extern fn enm_been_present(slot: Int) -> String
extern fn enm_been_past(slot: Int) -> String
extern fn enm_haven_present(slot: Int) -> String
extern fn enm_haven_past(slot: Int) -> String
extern fn enm_goon_present(slot: Int) -> String
extern fn enm_goon_past(slot: Int) -> String
extern fn enm_seen_present(slot: Int) -> String
extern fn enm_seen_past(slot: Int) -> String
extern fn enm_seyen_present(slot: Int) -> String
extern fn enm_seyen_past(slot: Int) -> String
extern fn enm_comen_present(slot: Int) -> String
extern fn enm_comen_past(slot: Int) -> String
extern fn enm_maken_present(slot: Int) -> String
extern fn enm_maken_past(slot: Int) -> String
extern fn enm_map_canonical(verb: String) -> String
extern fn enm_weak_stem(verb: String) -> String
extern fn enm_weak_present(stem: String, slot: Int) -> String
extern fn enm_weak_past(stem: String, slot: Int) -> String
extern fn enm_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn enm_irregular_plural(noun: String) -> String
extern fn enm_make_plural(noun: String) -> String
extern fn enm_decline(noun: String, gram_case: String, number: String) -> String
extern fn enm_is_vowel_initial(s: String) -> Bool
extern fn enm_indef_article(noun_phrase: String) -> String
extern fn enm_noun_phrase(noun: String, gram_case: String, number: String, definite: String) -> String
+716
View File
@@ -0,0 +1,716 @@
// morphology-es.el - Spanish morphology for the NLG engine.
//
// Implements fusional Spanish verb conjugation, noun pluralization, gender
// inference, and article agreement. Designed as a companion to morphology.el
// and called by the engine when the language profile code is "es".
//
// Verb tenses covered: present, preterite (past), future, imperfect.
// Persons: first/second/third × singular/plural (1s 2s 3s 1p 2p 3p).
// Verb classes: -ar, -er, -ir (regular) + a core set of common irregulars.
//
// Depends on: morphology.el (str_ends, str_drop_last, str_last_char, str_last2, str_last3, is_vowel)
// String helpers (local, matching morphology.el conventions)
import "morphology.el"
fn es_str_ends(s: String, suf: String) -> Bool {
return str_ends_with(s, suf)
}
fn es_str_drop_last(s: String, n: Int) -> String {
let len: Int = str_len(s)
if n >= len {
return ""
}
return str_slice(s, 0, len - n)
}
fn es_str_last_char(s: String) -> String {
let n: Int = str_len(s)
if n == 0 {
return ""
}
return str_slice(s, n - 1, n)
}
fn es_str_last2(s: String) -> String {
let n: Int = str_len(s)
if n < 2 {
return s
}
return str_slice(s, n - 2, n)
}
fn es_str_last3(s: String) -> String {
let n: Int = str_len(s)
if n < 3 {
return s
}
return str_slice(s, n - 3, n)
}
// Verb class detection
//
// Spanish verbs fall into three conjugation classes defined by the infinitive
// ending: -ar, -er, -ir. The stem is the infinitive minus those two characters.
fn es_verb_class(base: String) -> String {
if es_str_ends(base, "ar") { return "ar" }
if es_str_ends(base, "er") { return "er" }
if es_str_ends(base, "ir") { return "ir" }
return "ar"
}
fn es_stem(base: String) -> String {
return es_str_drop_last(base, 2)
}
// Person/number index
//
// Maps person × number to a 0-based slot index used inside paradigm tables.
// 0 = 1s, 1 = 2s, 2 = 3s, 3 = 1p, 4 = 2p, 5 = 3p
fn es_slot(person: String, number: String) -> Int {
if str_eq(person, "first") {
if str_eq(number, "singular") { return 0 }
return 3
}
if str_eq(person, "second") {
if str_eq(number, "singular") { return 1 }
return 4
}
// third
if str_eq(number, "singular") { return 2 }
return 5
}
// Irregular present tense
//
// Returns the fully-inflected form if the verb is irregular in the present
// tense for the given person/number slot, otherwise returns "".
//
// ser: soy, eres, es, somos, sois, son
// estar: estoy, estás, está, estamos, estáis, están
// tener: tengo, tienes, tiene, tenemos, tenéis, tienen
// hacer: hago, haces, hace, hacemos, hacéis, hacen
// ir: voy, vas, va, vamos, vais, van
// ver: veo, ves, ve, vemos, veis, ven
// dar: doy, das, da, damos, dais, dan
// saber: sé, sabes, sabe, sabemos, sabéis, saben
// poder: puedo, puedes, puede, podemos, podéis, pueden
// querer: quiero, quieres, quiere, queremos, queréis, quieren
// venir: vengo, vienes, viene, venimos, venís, vienen
// decir: digo, dices, dice, decimos, decís, dicen
// haber: he, has, ha, hemos, habéis, han
fn es_irregular_present(verb: String, person: String, number: String) -> String {
let slot: Int = es_slot(person, number)
if str_eq(verb, "ser") {
if slot == 0 { return "soy" }
if slot == 1 { return "eres" }
if slot == 2 { return "es" }
if slot == 3 { return "somos" }
if slot == 4 { return "sois" }
return "son"
}
if str_eq(verb, "estar") {
if slot == 0 { return "estoy" }
if slot == 1 { return "estás" }
if slot == 2 { return "está" }
if slot == 3 { return "estamos" }
if slot == 4 { return "estáis" }
return "están"
}
if str_eq(verb, "tener") {
if slot == 0 { return "tengo" }
if slot == 1 { return "tienes" }
if slot == 2 { return "tiene" }
if slot == 3 { return "tenemos" }
if slot == 4 { return "tenéis" }
return "tienen"
}
if str_eq(verb, "hacer") {
if slot == 0 { return "hago" }
if slot == 1 { return "haces" }
if slot == 2 { return "hace" }
if slot == 3 { return "hacemos" }
if slot == 4 { return "hacéis" }
return "hacen"
}
if str_eq(verb, "ir") {
if slot == 0 { return "voy" }
if slot == 1 { return "vas" }
if slot == 2 { return "va" }
if slot == 3 { return "vamos" }
if slot == 4 { return "vais" }
return "van"
}
if str_eq(verb, "ver") {
if slot == 0 { return "veo" }
if slot == 1 { return "ves" }
if slot == 2 { return "ve" }
if slot == 3 { return "vemos" }
if slot == 4 { return "veis" }
return "ven"
}
if str_eq(verb, "dar") {
if slot == 0 { return "doy" }
if slot == 1 { return "das" }
if slot == 2 { return "da" }
if slot == 3 { return "damos" }
if slot == 4 { return "dais" }
return "dan"
}
if str_eq(verb, "saber") {
if slot == 0 { return "" }
if slot == 1 { return "sabes" }
if slot == 2 { return "sabe" }
if slot == 3 { return "sabemos" }
if slot == 4 { return "sabéis" }
return "saben"
}
if str_eq(verb, "poder") {
if slot == 0 { return "puedo" }
if slot == 1 { return "puedes" }
if slot == 2 { return "puede" }
if slot == 3 { return "podemos" }
if slot == 4 { return "podéis" }
return "pueden"
}
if str_eq(verb, "querer") {
if slot == 0 { return "quiero" }
if slot == 1 { return "quieres" }
if slot == 2 { return "quiere" }
if slot == 3 { return "queremos" }
if slot == 4 { return "queréis" }
return "quieren"
}
if str_eq(verb, "venir") {
if slot == 0 { return "vengo" }
if slot == 1 { return "vienes" }
if slot == 2 { return "viene" }
if slot == 3 { return "venimos" }
if slot == 4 { return "venís" }
return "vienen"
}
if str_eq(verb, "decir") {
if slot == 0 { return "digo" }
if slot == 1 { return "dices" }
if slot == 2 { return "dice" }
if slot == 3 { return "decimos" }
if slot == 4 { return "decís" }
return "dicen"
}
if str_eq(verb, "haber") {
if slot == 0 { return "he" }
if slot == 1 { return "has" }
if slot == 2 { return "ha" }
if slot == 3 { return "hemos" }
if slot == 4 { return "habéis" }
return "han"
}
return ""
}
// Irregular preterite tense
//
// Returns the inflected preterite form for irregular verbs, or "" if regular.
//
// ser/ir (same preterite): fui, fuiste, fue, fuimos, fuisteis, fueron
// tener: tuve, tuviste, tuvo, tuvimos, tuvisteis, tuvieron
// hacer: hice, hiciste, hizo, hicimos, hicisteis, hicieron
// estar: estuve, estuviste, estuvo, estuvimos, estuvisteis, estuvieron
// dar: di, diste, dio, dimos, disteis, dieron
// saber: supe, supiste, supo, supimos, supisteis, supieron
// poder: pude, pudiste, pudo, pudimos, pudisteis, pudieron
// querer: quise, quisiste, quiso, quisimos, quisisteis, quisieron
// venir: vine, viniste, vino, vinimos, vinisteis, vinieron
// decir: dije, dijiste, dijo, dijimos, dijisteis, dijeron
// haber: hube, hubiste, hubo, hubimos, hubisteis, hubieron
// ver: vi, viste, vio, vimos, visteis, vieron
fn es_irregular_preterite(verb: String, person: String, number: String) -> String {
let slot: Int = es_slot(person, number)
if str_eq(verb, "ser") {
if slot == 0 { return "fui" }
if slot == 1 { return "fuiste" }
if slot == 2 { return "fue" }
if slot == 3 { return "fuimos" }
if slot == 4 { return "fuisteis" }
return "fueron"
}
if str_eq(verb, "ir") {
if slot == 0 { return "fui" }
if slot == 1 { return "fuiste" }
if slot == 2 { return "fue" }
if slot == 3 { return "fuimos" }
if slot == 4 { return "fuisteis" }
return "fueron"
}
if str_eq(verb, "tener") {
if slot == 0 { return "tuve" }
if slot == 1 { return "tuviste" }
if slot == 2 { return "tuvo" }
if slot == 3 { return "tuvimos" }
if slot == 4 { return "tuvisteis" }
return "tuvieron"
}
if str_eq(verb, "hacer") {
if slot == 0 { return "hice" }
if slot == 1 { return "hiciste" }
if slot == 2 { return "hizo" }
if slot == 3 { return "hicimos" }
if slot == 4 { return "hicisteis" }
return "hicieron"
}
if str_eq(verb, "estar") {
if slot == 0 { return "estuve" }
if slot == 1 { return "estuviste" }
if slot == 2 { return "estuvo" }
if slot == 3 { return "estuvimos" }
if slot == 4 { return "estuvisteis" }
return "estuvieron"
}
if str_eq(verb, "dar") {
if slot == 0 { return "di" }
if slot == 1 { return "diste" }
if slot == 2 { return "dio" }
if slot == 3 { return "dimos" }
if slot == 4 { return "disteis" }
return "dieron"
}
if str_eq(verb, "saber") {
if slot == 0 { return "supe" }
if slot == 1 { return "supiste" }
if slot == 2 { return "supo" }
if slot == 3 { return "supimos" }
if slot == 4 { return "supisteis" }
return "supieron"
}
if str_eq(verb, "poder") {
if slot == 0 { return "pude" }
if slot == 1 { return "pudiste" }
if slot == 2 { return "pudo" }
if slot == 3 { return "pudimos" }
if slot == 4 { return "pudisteis" }
return "pudieron"
}
if str_eq(verb, "querer") {
if slot == 0 { return "quise" }
if slot == 1 { return "quisiste" }
if slot == 2 { return "quiso" }
if slot == 3 { return "quisimos" }
if slot == 4 { return "quisisteis" }
return "quisieron"
}
if str_eq(verb, "venir") {
if slot == 0 { return "vine" }
if slot == 1 { return "viniste" }
if slot == 2 { return "vino" }
if slot == 3 { return "vinimos" }
if slot == 4 { return "vinisteis" }
return "vinieron"
}
if str_eq(verb, "decir") {
if slot == 0 { return "dije" }
if slot == 1 { return "dijiste" }
if slot == 2 { return "dijo" }
if slot == 3 { return "dijimos" }
if slot == 4 { return "dijisteis" }
return "dijeron"
}
if str_eq(verb, "haber") {
if slot == 0 { return "hube" }
if slot == 1 { return "hubiste" }
if slot == 2 { return "hubo" }
if slot == 3 { return "hubimos" }
if slot == 4 { return "hubisteis" }
return "hubieron"
}
if str_eq(verb, "ver") {
if slot == 0 { return "vi" }
if slot == 1 { return "viste" }
if slot == 2 { return "vio" }
if slot == 3 { return "vimos" }
if slot == 4 { return "visteis" }
return "vieron"
}
return ""
}
// Irregular imperfect tense
//
// Only three verbs are truly irregular in the imperfect:
// ser: era, eras, era, éramos, erais, eran
// ir: iba, ibas, iba, íbamos, ibais, iban
// ver: veía, veías, veía, veíamos, veíais, veían
fn es_irregular_imperfect(verb: String, person: String, number: String) -> String {
let slot: Int = es_slot(person, number)
if str_eq(verb, "ser") {
if slot == 0 { return "era" }
if slot == 1 { return "eras" }
if slot == 2 { return "era" }
if slot == 3 { return "éramos" }
if slot == 4 { return "erais" }
return "eran"
}
if str_eq(verb, "ir") {
if slot == 0 { return "iba" }
if slot == 1 { return "ibas" }
if slot == 2 { return "iba" }
if slot == 3 { return "íbamos" }
if slot == 4 { return "ibais" }
return "iban"
}
if str_eq(verb, "ver") {
if slot == 0 { return "veía" }
if slot == 1 { return "veías" }
if slot == 2 { return "veía" }
if slot == 3 { return "veíamos" }
if slot == 4 { return "veíais" }
return "veían"
}
return ""
}
// Regular present conjugation
//
// -ar: -o, -as, -a, -amos, -áis, -an
// -er: -o, -es, -e, -emos, -éis, -en
// -ir: -o, -es, -e, -imos, -ís, -en
fn es_regular_present(stem: String, vclass: String, slot: Int) -> String {
if str_eq(vclass, "ar") {
if slot == 0 { return stem + "o" }
if slot == 1 { return stem + "as" }
if slot == 2 { return stem + "a" }
if slot == 3 { return stem + "amos" }
if slot == 4 { return stem + "áis" }
return stem + "an"
}
if str_eq(vclass, "er") {
if slot == 0 { return stem + "o" }
if slot == 1 { return stem + "es" }
if slot == 2 { return stem + "e" }
if slot == 3 { return stem + "emos" }
if slot == 4 { return stem + "éis" }
return stem + "en"
}
// -ir
if slot == 0 { return stem + "o" }
if slot == 1 { return stem + "es" }
if slot == 2 { return stem + "e" }
if slot == 3 { return stem + "imos" }
if slot == 4 { return stem + "ís" }
return stem + "en"
}
// Regular preterite conjugation
//
// -ar: -é, -aste, -ó, -amos, -asteis, -aron
// -er: -í, -iste, -ió, -imos, -isteis, -ieron
// -ir: -í, -iste, -ió, -imos, -isteis, -ieron
fn es_regular_preterite(stem: String, vclass: String, slot: Int) -> String {
if str_eq(vclass, "ar") {
if slot == 0 { return stem + "é" }
if slot == 1 { return stem + "aste" }
if slot == 2 { return stem + "ó" }
if slot == 3 { return stem + "amos" }
if slot == 4 { return stem + "asteis" }
return stem + "aron"
}
// -er and -ir share the same preterite endings
if slot == 0 { return stem + "í" }
if slot == 1 { return stem + "iste" }
if slot == 2 { return stem + "" }
if slot == 3 { return stem + "imos" }
if slot == 4 { return stem + "isteis" }
return stem + "ieron"
}
// Regular future conjugation
//
// Future is formed from the full infinitive + endings (all classes):
// -é, -ás, -á, -emos, -éis, -án
//
// No stem change; the infinitive is the future stem.
fn es_regular_future(base: String, slot: Int) -> String {
if slot == 0 { return base + "é" }
if slot == 1 { return base + "ás" }
if slot == 2 { return base + "á" }
if slot == 3 { return base + "emos" }
if slot == 4 { return base + "éis" }
return base + "án"
}
// Irregular future stems
//
// Some verbs contract or alter their infinitive for the future stem.
// Returns the irregular future stem, or "" if the verb uses the regular stem.
fn es_irregular_future_stem(verb: String) -> String {
if str_eq(verb, "tener") { return "tendr" }
if str_eq(verb, "hacer") { return "har" }
if str_eq(verb, "poder") { return "podr" }
if str_eq(verb, "querer") { return "querr" }
if str_eq(verb, "venir") { return "vendr" }
if str_eq(verb, "decir") { return "dir" }
if str_eq(verb, "haber") { return "habr" }
if str_eq(verb, "saber") { return "sabr" }
if str_eq(verb, "salir") { return "saldr" }
if str_eq(verb, "poner") { return "pondr" }
return ""
}
// Regular imperfect conjugation
//
// -ar: -aba, -abas, -aba, -ábamos, -abais, -aban
// -er/-ir: -ía, -ías, -ía, -íamos, -íais, -ían
fn es_regular_imperfect(stem: String, vclass: String, slot: Int) -> String {
if str_eq(vclass, "ar") {
if slot == 0 { return stem + "aba" }
if slot == 1 { return stem + "abas" }
if slot == 2 { return stem + "aba" }
if slot == 3 { return stem + "ábamos" }
if slot == 4 { return stem + "abais" }
return stem + "aban"
}
// -er and -ir
if slot == 0 { return stem + "ía" }
if slot == 1 { return stem + "ías" }
if slot == 2 { return stem + "ía" }
if slot == 3 { return stem + "íamos" }
if slot == 4 { return stem + "íais" }
return stem + "ían"
}
// Full conjugation entry point
//
// es_conjugate: conjugate a Spanish verb.
//
// verb: Spanish infinitive (e.g. "hablar", "ser", "tener")
// tense: "present" | "past" | "future" | "imperfect"
// (note: "past" maps to the preterite/indefinite past)
// person: "first" | "second" | "third"
// number: "singular" | "plural"
fn es_conjugate(verb: String, tense: String, person: String, number: String) -> String {
let slot: Int = es_slot(person, number)
if str_eq(tense, "present") {
let irreg: String = es_irregular_present(verb, person, number)
if !str_eq(irreg, "") {
return irreg
}
let vclass: String = es_verb_class(verb)
let stem: String = es_stem(verb)
return es_regular_present(stem, vclass, slot)
}
if str_eq(tense, "past") {
let irreg: String = es_irregular_preterite(verb, person, number)
if !str_eq(irreg, "") {
return irreg
}
let vclass: String = es_verb_class(verb)
let stem: String = es_stem(verb)
return es_regular_preterite(stem, vclass, slot)
}
if str_eq(tense, "future") {
let irreg_stem: String = es_irregular_future_stem(verb)
if !str_eq(irreg_stem, "") {
return es_regular_future(irreg_stem, slot)
}
return es_regular_future(verb, slot)
}
if str_eq(tense, "imperfect") {
let irreg: String = es_irregular_imperfect(verb, person, number)
if !str_eq(irreg, "") {
return irreg
}
let vclass: String = es_verb_class(verb)
let stem: String = es_stem(verb)
return es_regular_imperfect(stem, vclass, slot)
}
// Unknown tense: return infinitive unchanged
return verb
}
// Noun gender inference
//
// Returns "m" (masculine), "f" (feminine), or "unknown".
//
// Heuristics (not exhaustive cover most common patterns):
// ends in -o -> masculine (libro, gato, niño)
// ends in -a -> feminine (casa, mesa, niña)
// ends in -ión -> feminine (canción, nación)
// ends in -dad/-tad -> feminine (ciudad, libertad)
// ends in -umbre -> feminine (costumbre)
// ends in -sis -> feminine (crisis, tesis)
// ends in -ema/-ama -> masculine (problema, programa, tema, idioma)
// ends in -or -> masculine (color, amor, señor)
// ends in -aje -> masculine (viaje, paisaje)
// ends in -án/-ón -> masculine (avión check -ión first)
// otherwise -> unknown
fn es_gender(noun: String) -> String {
// -ión before -o so "avión" feminine (it ends -ión, not just -on)
if es_str_ends(noun, "ión") { return "f" }
if es_str_ends(noun, "dad") { return "f" }
if es_str_ends(noun, "tad") { return "f" }
if es_str_ends(noun, "umbre") { return "f" }
if es_str_ends(noun, "sis") { return "f" }
if es_str_ends(noun, "ema") { return "m" }
if es_str_ends(noun, "ama") { return "m" }
if es_str_ends(noun, "aje") { return "m" }
if es_str_ends(noun, "or") { return "m" }
if es_str_ends(noun, "o") { return "m" }
if es_str_ends(noun, "a") { return "f" }
return "unknown"
}
// Noun pluralization
//
// Rules (applied in order):
// ends in vowel (a e i o u) -> add -s
// ends in consonant -> add -es
// ends in -z -> replace -z with -ces
// ends in -s (unstressed) -> unchanged (e.g. "el lunes" -> "los lunes")
//
// Note: nouns ending in stressed vowel + s (e.g. "el autobús" "los autobuses")
// are handled by the consonant rule since -s is a consonant ending for pluralization
// purposes; but "el lunes" (days of week ending in -s) stay unchanged — this is
// an irregular class. The table below handles common invariant nouns.
fn es_invariant_plural(noun: String) -> String {
if str_eq(noun, "lunes") { return "lunes" }
if str_eq(noun, "martes") { return "martes" }
if str_eq(noun, "miércoles") { return "miércoles" }
if str_eq(noun, "jueves") { return "jueves" }
if str_eq(noun, "viernes") { return "viernes" }
if str_eq(noun, "crisis") { return "crisis" }
if str_eq(noun, "tesis") { return "tesis" }
if str_eq(noun, "análisis") { return "análisis" }
if str_eq(noun, "dosis") { return "dosis" }
if str_eq(noun, "virus") { return "virus" }
return ""
}
fn es_pluralize(noun: String) -> String {
let inv: String = es_invariant_plural(noun)
if !str_eq(inv, "") {
return inv
}
let last: String = es_str_last_char(noun)
// Ends in -z: replace with -ces
if str_eq(last, "z") {
return es_str_drop_last(noun, 1) + "ces"
}
// Ends in a vowel: add -s
if str_eq(last, "a") { return noun + "s" }
if str_eq(last, "e") { return noun + "s" }
if str_eq(last, "i") { return noun + "s" }
if str_eq(last, "o") { return noun + "s" }
if str_eq(last, "u") { return noun + "s" }
// Ends in consonant (including -s for stressed words like autobús): add -es
return noun + "es"
}
// Article agreement
//
// es_agree_article: return the correct Spanish article for a noun.
//
// noun: the noun (used for gender and number inference)
// definite: "true" for definite (el/la/los/las), "false" for indefinite (un/una/unos/unas)
// number: "singular" | "plural"
//
// Special case: feminine nouns beginning with stressed "a-" or "ha-" take
// masculine singular definite article: "el agua", "el hacha".
// This is handled by checking the noun's first character when gender is feminine.
fn es_starts_with_stressed_a(noun: String) -> Bool {
// Approximate: check if noun starts with "a" or "ha" (covers most cases)
// The accent on the first syllable is not detectable orthographically in
// general, so we apply the rule broadly for any feminine noun starting with
// "a" or "ha" in singular.
let n: Int = str_len(noun)
if n == 0 {
return false
}
let c0: String = str_slice(noun, 0, 1)
if str_eq(c0, "a") { return true }
if n >= 2 {
let c1: String = str_slice(noun, 1, 2)
if str_eq(c0, "h") {
if str_eq(c1, "a") { return true }
}
}
return false
}
fn es_agree_article(noun: String, definite: String, number: String) -> String {
let gender: String = es_gender(noun)
let is_plural: Bool = str_eq(number, "plural")
let is_def: Bool = str_eq(definite, "true")
if is_def {
if is_plural {
if str_eq(gender, "f") { return "las" }
return "los"
}
// singular
if str_eq(gender, "f") {
// el agua rule: feminine singular nouns starting with stressed "a"
if es_starts_with_stressed_a(noun) { return "el" }
return "la"
}
return "el"
}
// indefinite
if is_plural {
if str_eq(gender, "f") { return "unas" }
return "unos"
}
if str_eq(gender, "f") { return "una" }
return "un"
}
+23
View File
@@ -0,0 +1,23 @@
// auto-generated by elc --emit-header — do not edit
extern fn es_str_ends(s: String, suf: String) -> Bool
extern fn es_str_drop_last(s: String, n: Int) -> String
extern fn es_str_last_char(s: String) -> String
extern fn es_str_last2(s: String) -> String
extern fn es_str_last3(s: String) -> String
extern fn es_verb_class(base: String) -> String
extern fn es_stem(base: String) -> String
extern fn es_slot(person: String, number: String) -> Int
extern fn es_irregular_present(verb: String, person: String, number: String) -> String
extern fn es_irregular_preterite(verb: String, person: String, number: String) -> String
extern fn es_irregular_imperfect(verb: String, person: String, number: String) -> String
extern fn es_regular_present(stem: String, vclass: String, slot: Int) -> String
extern fn es_regular_preterite(stem: String, vclass: String, slot: Int) -> String
extern fn es_regular_future(base: String, slot: Int) -> String
extern fn es_irregular_future_stem(verb: String) -> String
extern fn es_regular_imperfect(stem: String, vclass: String, slot: Int) -> String
extern fn es_conjugate(verb: String, tense: String, person: String, number: String) -> String
extern fn es_gender(noun: String) -> String
extern fn es_invariant_plural(noun: String) -> String
extern fn es_pluralize(noun: String) -> String
extern fn es_starts_with_stressed_a(noun: String) -> Bool
extern fn es_agree_article(noun: String, definite: String, number: String) -> String

Some files were not shown because too many files have changed in this diff Show More