Compare commits

..

3 Commits

Author SHA1 Message Date
will.anderson 858e9ccd2d Merge pull request 'runtime: extract engram_text.c, and repair 10 harnesses that could not link' (#162) from fix/runtime-extract-text into fix/runtime-growth-guard 2026-08-17 00:57:00 +00:00
bigmerge 678dac5efc runtime: extract engram_text.c, and repair 10 harnesses that could not link
First concern moved out of el_runtime.c under the ratchet, and the move is
deliberately small: it exists to prove the mechanism end to end before anything
large depends on it.

engram_text.{c,h} — query tokenization, candidate-token hygiene, word-boundary
matching, and the text-damage signature. Four functions, moved verbatim; only
`static` was dropped and each doc comment travelled with the code. They touch no
EL value type and no engram store type: plain C over <ctype.h>/<string.h> over
char buffers. They were never el_runtime.c's business.

  el_runtime.c   20,527 -> 20,427 lines   (BUDGET max_lines ratcheted down)
  engram fns        279 -> 275            (BUDGET max_engram_fns ratcheted down)

The Stage 1 extension point worked as designed: adding the file to
lang/runtime/SOURCES was one line, and every build path picked it up. The
Stage 2 drift guard then caught that I had NOT added it to install.sh's
standalone list — the exact class of drift it was written for, on its first
real change, before the commit rather than after a broken SDK shipped.

WHY ONLY 100 LINES, AND WHAT ACTUALLY BLOCKS THE REST

Measured, not estimated: of 273 engram-domain functions in el_runtime.c
(~9,700 lines), only 75 (~1,058 lines) can move today, and they are scattered
rather than clustered. The blocker is a single fact:

  EngramNode, EngramEdge, EngramStore, EngramLayer, EngramWal and EngramIdSlot
  are typedef'd INSIDE el_runtime.c. No sibling can see them. engram_store.h
  defines a SEPARATE serializable "node view" struct and maps between the two.

So every engram function that takes an EngramNode* — which is most of them, 109
of 273 by direct type reference — cannot compile in engram_store.c until those
types move to a shared header. That extraction is the real Stage 3 enabler and
it deserves its own change: it touches the most load-bearing struct in the
system, and doing it in the same commit as a code move would make a regression
impossible to bisect.

REPAIRED: 10 engram harnesses that had silently stopped linking

Not new breakage from this move — verified against unmodified dev, where
el_runtime.c + engram_store.c alone already failed with undefined symbols.
They had been dead for as long as el_runtime.c has been calling into the
siblings, and nothing noticed because nothing ran them.

  run_m3_parity, run_m7_traversal, run_m35_hebb_persist,
  run_interoception_p0..p5   — now build from $(scripts/el-runtime-sources.sh)
  run_wal_tests              — its two TUs #include "el_runtime.c" directly, so
                               it links the SIBLINGS ONLY; adding el_runtime.c
                               to that link line would define every symbol twice

(That #include'd .c is worth recording: the runtime does have one, in
engram/test/test_wal.c and the generated test_failloud.c.)

Verified locally — every one of these was run, not assumed:
  * m3_parity ............ PASS, incl. ASan+UBSan clean across seed/on/reboot
  * m7_traversal ......... PASS
  * m35_hebb_persist ..... PASS   (the gate over the original prod hebb bug)
  * interoception p0..p5 . PASS   (all six)
  * wal_tests ............ 66 passed, 0 failed, + fail-loud exit check
  * self-host fixpoint ... byte-identical, AND the emitted C is byte-identical
                           to the pre-move compiler output — the move changes
                           nothing the compiler produces
  * engram/src/server.el . compiles and links
  * native suites ........ 8 of 13, unchanged from before the move; the same 5
                           pre-existing failures, no regression
  * both runtime guards .. green at the new, lower budget

Also fixes a block comment left unterminated by the extraction (the deleted
range carried its closing */), restoring the compile to its single pre-existing
-Wcomment warning.
2026-08-16 16:58:18 -05:00
bigmerge fe634c4582 runtime: put el_runtime.c on a ratchet, and actually run the guards
scripts/check-single-runtime.sh guards against el_runtime.c being COPIED — it
was written after a lagging fork shipped to prod and dropped learned hebb edges.
Nothing guarded against it GROWING. So it grew: 10,607 -> 20,527 lines, 94% in
3.5 months, the whole time under an explicit commit-message promise that it was
a temporary shim about to be deleted.

Worse, the copy guard was never wired in. Its own footer described the CI
wire-in as a TODO, and the TODO had never been done — the script existed but ran
nowhere, in no workflow and in no hook, so it had caught nothing for as long as
it has been in the tree. A guard that does not run is a comment.

This adds the missing guard and runs both.

  * lang/runtime/BUDGET — a RATCHET, not a limit. max_lines is set at the
    current 20,527 with NO headroom: the file cannot grow by one line. A second
    cap, max_engram_fns (279), counts top-level engram_/eg_/cog_ definitions in
    it — ~47.5% of the file is engram code and engram already owns six sibling
    .c files, so this is the scoreboard for moving it out. Both may only go DOWN.

  * scripts/check-runtime-growth.sh — enforces the ratchet, and three
    invariants that keep the multi-file runtime honest: every .c in
    lang/runtime/ is either in SOURCES or explicitly platform-optional (an
    unaccounted .c is compiled by nothing and is silently dead); install.sh's
    hardcoded download list matches SOURCES (it cannot call the helper — it
    runs where there is no checkout — so that copy is checked, not trusted);
    and an advisory nudge to lower the budget when you have earned it.

  * Both guards now run as early steps in ci-dev.yaml, ci-stage.yaml and
    sdk-release.yaml, and in .githooks/pre-commit.

The failure message is the point. The guard that existed said what was wrong but
not where the code should go, which makes it easy to "fix" by arguing with the
guard. This one names the destination: the concern-owning .c, or a new .c plus
one line in SOURCES, or c_source in a program's manifest.el — and it prints the
`nm` command that proves placement is link-time and that the shipped compiler
already links from ten translation units. Every runtime file except el_runtime.c
is deliberately uncapped, because that is where code is supposed to go.

Proven with negative controls, per lang/AGENTS.md step 5 — each shown FAILING:
  * +1 line to el_runtime.c                  -> FAIL (20528/20527)
  * +1 engram fn, net-zero lines             -> FAIL (280/279)
  * a new unaccounted lang/runtime/*.c       -> FAIL
  * engram_store.c removed from install.sh   -> FAIL, names the missing file
  * el_runtime.c truncated to 20,000 lines   -> PASS + "lower max_lines to 20000"
  * baseline, tree unmodified                -> OK, and both guards green

el_runtime.c is byte-identical after the controls; this commit changes zero
lines of it.
2026-08-16 16:48:04 -05:00
19 changed files with 119 additions and 4819 deletions
-6
View File
@@ -1,6 +0,0 @@
# organ: local device state and its own engram store — never production's
peripheral/.consent.json
peripheral/.resume.json
peripheral/.engram/
peripheral/organ
+13 -212
View File
@@ -23,26 +23,16 @@
// warning. The runtime takes an exclusive flock at startup and a second start // warning. The runtime takes an exclusive flock at startup and a second start
// is refused loudly with the holder's pid. // is refused loudly with the holder's pid.
// //
// guards: names WHAT the singleton protects this program's data directory. The // NOT declared here, on purpose: ENGRAM_DATA_DIR. Its resolution is owned by
// lock lives inside it, so the guard is keyed on the store and not on the word // engram_resolve_data_dir() (el_runtime.c), which defaults to $HOME/.neuron/engram
// "engram": two engrams against the same store cannot both run no matter how the // and fails LOUD rather than silently persisting to an ephemeral directory.
// environment is spelled, and two engrams against DIFFERENT stores are not each // Declaring a default for it here as well would put the data dir's fallback in
// other's business and are not refused. Until 2026-08-16 the lock was keyed on // two places which is precisely the defect this migration removes (until
// the program name and $TMPDIR, and both of those sentences were false. // 2026-08-15 the reseed backup path carried its own "/tmp/engram" default that
// // disagreed with the resolver, so the pre-destructive safety copy landed in /tmp).
// It names the resolver rather than restating its path, for the same reason
// ENGRAM_DATA_DIR is NOT declared as an `env` entry below: engram_resolve_data_dir()
// (el_runtime.c) owns that path it defaults to $HOME/.neuron/engram and fails
// LOUD rather than silently persisting to an ephemeral directory. Restating the
// default here would give the data dir two owners that can disagree, which is
// precisely the defect this migration removes (until 2026-08-15 the reseed backup
// path carried its own "/tmp/engram" default that disagreed with the resolver, so
// the pre-destructive safety copy landed in /tmp). A guard that resolved the path
// its own way could guard a directory the program never writes to.
// HOME is likewise not declared: it is a genuine environment read, not a knob. // HOME is likewise not declared: it is a genuine environment read, not a knob.
program "engram" { program "engram" {
singleton: "engram" singleton: "engram"
guards: engram_resolve_data_dir()
// Core server // Core server
env ENGRAM_BIND: String = ":8742" env ENGRAM_BIND: String = ":8742"
@@ -1479,16 +1469,9 @@ fn route_guide_summon(method: String, path: String, body: String) -> String {
// //
// The SINGLE NODE is the DEGENERATE n=1 case of this SAME operation not a // The SINGLE NODE is the DEGENERATE n=1 case of this SAME operation not a
// separate CRUD path: // separate CRUD path:
// write(signal) = realize(signal) reframe(region=, manifold) (route_write) // write(content) = reframe(region=, manifold=[1 node]) (route_write)
// supersede(id,new) = reframe(region={id}, manifold=[1 node]) (route_supersede) // supersede(id,new) = reframe(region={id}, manifold=[1 node]) (route_supersede)
// relate(a,b,rel) = the rebind sub-op in isolation (route_create_edge) // relate(a,b,rel) = the rebind sub-op in isolation (route_create_edge)
//
// CORRECTED 2026-08-16: write was documented above as
// "reframe(region=∅, manifold=[1 node])", and the "[1 node]" was not the design
// it was the DEFECT. A node is an OUTPUT of realization, never an INPUT to
// it. What arrives at an intake route is a SIGNAL, and how many nodes it
// becomes is for the realizer to say, not for the route to assume. See
// "INTAKE" below.
// The ONLY anti-pattern is decomposing a region-scale change into a LOOP of // The ONLY anti-pattern is decomposing a region-scale change into a LOOP of
// independent top-level per-node updates. Here the region is the unit: one // independent top-level per-node updates. Here the region is the unit: one
// isolate, one atomic set-replace, one persist, one verify iterating members // isolate, one atomic set-replace, one persist, one verify iterating members
@@ -1703,174 +1686,6 @@ fn reframe_core(region: [String], manifold: String, reason: String, do_rebind: I
",\"keystones_protected\":true}" ",\"keystones_protected\":true}"
} }
//
// INTAKE the ONE door: signal realization manifold store.
//
// THERE IS NO WRITE NODE. What arrives at an intake route is a SIGNAL. A node
// is an OUTPUT of realization, never an INPUT to it. route_write used to say:
//
// let manifold: String = "[" + body + "]" // the body IS a valid manifold node object
//
// and hand that to reframe_core. That is not a manifold it is the request
// body wearing the word, and the comment stated the wrong assumption out loud.
// It is why a compound signal landed as ONE flat node with ZERO edges. Measured
// before this change, on a cp -Rc clone:
// POST /api/write {"type":"memory","content":"A cathedral is stone holding a
// shape that stone alone would not hold."}
// {"ok":true,"inserted":1,"nodes_added":1,"edges_added":0,...}
// GET /api/neighbors/<new id> [] (read back out, not taken on trust)
//
// NOTHING IS DECOMPOSED HERE, AND NOTHING MAY EVER BE. transduce(signal,
// modality) IS the realization primitive (el_runtime.c: "Manifold",
// "Realizers + transduce"). It dispatches through the dlsym realizer registry,
// so ADDING A MODALITY IS REGISTERING A REALIZER never an edit to this file,
// and never a patch to the runtime. This function only carries what the
// primitive returns into the store, which is the one thing the engram's HTTP
// surface has never done: `grep -n 'transduce\|realize\|Manifold\|decompos'
// engram/src/server.el` returned exactly one line before this change, a comment.
//
// GENERAL BY CONSTRUCTION, NOT SPECIAL-CASED TO route_write. Five of the six
// intake doors (write, supersede, nodes, neuron/knowledge/capture,
// neuron/state-events) are the same hand-written "content string →
// engram_node_full → one flat node", differing ONLY in the node_type / tier /
// tags they hardcode. Those are parameters here, so each door can be moved onto
// this one function as it is transitioned. Only /api/write rides it in this
// pass; the rest are listed as remaining work.
//
// WHEN THERE IS NO ORGAN the signal is stored flat exactly as before, and the
// response SAYS SO ("realized":false, "organ":false). Silent flattening is the
// actual defect a caller could not distinguish "nothing decomposed me" from
// "I decomposed into one component". el_runtime.c draws the same line at
// registration time, between an absent organ and a broken one, for the same
// reason: those two must not look alike.
//
// Resolve a component KEY to the node id it was inserted as. Components are
// addressed BY KEY, never by index (el_runtime.c, "Manifold"), because the key
// is what survives persistence so relations are resolved by key too.
fn key_to_id(keys: [String], ids: [String], key: String) -> String {
let n: Int = el_list_len(keys)
let i: Int = 0
while i < n {
if str_eq(el_list_get(keys, i), key) { return el_list_get(ids, i) }
i = i + 1
}
return ""
}
fn intake_signal(signal: String, modality: String, region: [String],
nt_in: String, tier_in: String, tags: String,
reason: String, do_rebind: Int) -> String {
let n_before: Int = engram_node_count()
let e_before: Int = engram_edge_count()
let region_n: Int = el_list_len(region)
let tomb: String = if region_n > 0 { supersede_set(region, reason) } else { "" }
// Identity can never be minted through intake the same rule
// insert_manifold_json holds, applied at the one door instead of per-route.
let nt: String = if str_eq(nt_in, "") { "Memory" } else { nt_in }
if str_eq(nt, "self") { nt = "Memory" }
if str_eq(nt, "values") { nt = "Memory" }
let tier: String = if str_eq(tier_in, "") { "Working" } else { tier_in }
let has_organ: Int = realizer_has(modality)
let new_ids: [String] = el_list_empty()
let keys: [String] = el_list_empty()
let ncomp: Int = 0
let nrel: Int = 0
let realized: Int = 0
if has_organ > 0 {
let m: Manifold = transduce(signal, modality)
// A realizer that returns a bare Geometry transduces NOTHING by design
// (el_runtime.c) manifold_is() is the check, so a fingerprinting organ
// is not silently mistaken for a decomposing one.
if manifold_is(m) > 0 {
realized = 1
ncomp = manifold_size(m)
let i: Int = 0
let prev: String = ""
while i < ncomp {
let key: String = manifold_key(m, i)
let role: String = manifold_role(m, i)
// The component's OWN geometry, at its own width this is the
// whole point of a manifold over a fingerprint, and it is why
// node_attach_geometry is used rather than re-embedding the
// component's name as text.
let g: Geometry = manifold_geometry(m, i)
let ctags: String = "[\"component\",\"role:" + role + "\",\"modality:" + modality + "\"]"
let cid: String = engram_node_full(key, nt, key, 0.5, 0.5, 0.9, tier, ctags)
let landed: Int = node_attach_geometry(cid, g)
let freed: Int = geometry_free(g)
new_ids = el_list_append(new_ids, cid)
keys = el_list_append(keys, key)
// PRESERVED CONTRACT: manifold_member wires the inserted set
// into one connected sub-graph, exactly as insert_manifold_json
// already did. Not reinvented reused.
if !str_eq(prev, "") { engram_connect(prev, cid, 0.6, "manifold_member") }
prev = cid
i = i + 1
}
// THE RELATIONS ARE THE CONTENT. Relation weight IS the grounding
// (correspondence-and-censorship §1) it arrives on the edge from
// the realizer and nothing here computes or second-guesses it.
nrel = manifold_rel_count(m)
let j: Int = 0
while j < nrel {
let fk: String = manifold_rel_from(m, j)
let rn: String = manifold_rel_name(m, j)
let tk: String = manifold_rel_to(m, j)
let w: Float = manifold_rel_weight(m, j)
let fid: String = key_to_id(keys, new_ids, fk)
let tid: String = key_to_id(keys, new_ids, tk)
if !str_eq(fid, "") {
if !str_eq(tid, "") {
engram_connect(fid, tid, w, rn)
}
}
j = j + 1
}
let mfreed: Int = manifold_free(m)
}
}
// NO ORGAN: store the signal flat, as before but say so. This is the
// pre-existing behaviour preserved verbatim, not a new fallback path.
if realized == 0 {
let label: String = str_slice(signal, 0, 60)
let fid: String = engram_node_full(signal, nt, label, 0.5, 0.5, 0.9, tier, tags)
new_ids = el_list_append(new_ids, fid)
}
let inserted: Int = el_list_len(new_ids)
let bound: Int = if do_rebind > 0 { rebind_cosine(new_ids, tomb) } else { 0 }
let saved: Int = persist_canonical()
let new_csv: String = ""
let k: Int = 0
while k < inserted {
let sep: String = if k == 0 { "" } else { "," }
new_csv = new_csv + sep + "\"" + el_list_get(new_ids, k) + "\""
k = k + 1
}
let realized_s: String = if realized > 0 { "true" } else { "false" }
let organ_s: String = if has_organ > 0 { "true" } else { "false" }
return "{\"ok\":true,\"region_superseded\":" + int_to_str(region_n) +
",\"tombstone_id\":\"" + tomb + "\"" +
",\"inserted\":" + int_to_str(inserted) +
",\"new_ids\":[" + new_csv + "]" +
",\"edges_rebound\":" + int_to_str(bound) +
",\"realized\":" + realized_s +
",\"modality\":\"" + modality + "\"" +
",\"organ\":" + organ_s +
",\"components\":" + int_to_str(ncomp) +
",\"relations\":" + int_to_str(nrel) +
",\"nodes_added\":" + int_to_str(engram_node_count() - n_before) +
",\"edges_added\":" + int_to_str(engram_edge_count() - e_before) +
",\"node_count\":" + int_to_str(engram_node_count()) +
",\"edge_count\":" + int_to_str(engram_edge_count()) +
",\"keystones_protected\":true}"
}
// POST /api/reframe the universal set-based mutation. // POST /api/reframe the universal set-based mutation.
// Body: {vantage?, region_ids?(csv), k?, expand?, manifold(json array), reason?, rebind?} // Body: {vantage?, region_ids?(csv), k?, expand?, manifold(json array), reason?, rebind?}
// region_ids (explicit) wins; else cosine-isolate around vantage. // region_ids (explicit) wins; else cosine-isolate around vantage.
@@ -1907,32 +1722,18 @@ fn route_reframe(method: String, path: String, body: String) -> String {
return reframe_core(region, manifold, reason, do_rebind) return reframe_core(region, manifold, reason, do_rebind)
} }
// write INTAKE OF A SIGNAL. Not "reframe with a manifold of one node": the // write DEGENERATE n=1 of reframe: region=, manifold=[1 node]. The SAME
// route no longer decides how many nodes the signal is. It hands the signal to // reframe_core path. rebind off so the pure-add matches plain node creation.
// the realization primitive and stores whatever manifold comes back. // POST /api/write {content, node_type?, tier?, tags?}
//
// The line this replaces was:
// let manifold: String = "[" + body + "]" // the body IS a valid manifold node object
// which asserted that a request body is a manifold. It is not, and that single
// assertion is the whole measured defect (1 node, 0 edges, [] neighbors).
//
// rebind stays off so a pure add still matches plain node creation.
// POST /api/write {content, modality?, node_type?, tier?, tags?}
fn route_write(method: String, path: String, body: String) -> String { fn route_write(method: String, path: String, body: String) -> String {
let content: String = json_get_string(body, "content") let content: String = json_get_string(body, "content")
if str_eq(content, "") { return err_json("write: content required") } if str_eq(content, "") { return err_json("write: content required") }
let nt: String = json_get_string(body, "node_type") let nt: String = json_get_string(body, "node_type")
if str_eq(nt, "self") { return err_json("write: identity is write-protected") } if str_eq(nt, "self") { return err_json("write: identity is write-protected") }
if str_eq(nt, "values") { return err_json("write: identity is write-protected") } if str_eq(nt, "values") { return err_json("write: identity is write-protected") }
// The modality names which organ to sense with. It is data, never a branch:
// a new modality is a realizer_register call somewhere else in the program,
// not another endpoint and not another case here.
let mod_raw: String = json_get_string(body, "modality")
let modality: String = if str_eq(mod_raw, "") { "text" } else { mod_raw }
let tier: String = json_get_string(body, "tier")
let tags: String = json_get_raw(body, "tags")
let empty: [String] = el_list_empty() let empty: [String] = el_list_empty()
return intake_signal(content, modality, empty, nt, tier, tags, "write", 0) let manifold: String = "[" + body + "]" // the body IS a valid manifold node object
return reframe_core(empty, manifold, "write", 0)
} }
// supersede DEGENERATE n=1 of reframe: region={id}, manifold=[1 node]. The // supersede DEGENERATE n=1 of reframe: region={id}, manifold=[1 node]. The
+7 -11
View File
@@ -127,18 +127,14 @@ After changing any `.el` source in `el-compiler/src/` (run from the `lang/` dir)
```bash ```bash
# 1. Stage2: current elc compiles the (modified) compiler to C # 1. Stage2: current elc compiles the (modified) compiler to C
./dist/platform/elc elc-cli.el > elc-new.c ./dist/platform/elc elc-cli.el > elc-new.c
# 2. Build the new compiler. Link the WHOLE runtime set, not el_runtime.c alone: # 2. Build the new compiler. The C link target is el_runtime.c — it holds the
# el_runtime.c calls into engram_store / engram_vindex / eg_cosine_batch and # engram store + http/json/state impls the compiler output calls. el_runtime.c
# wraps el_seed.c, so a one-file link fails at `ld` with undefined symbols # self-hosts elc on its own; el_seed.c is the (aspirational) seed layer and does
# (verified 2026-08-16 — the previous single-file line in this doc is stale). # NOT compile standalone under clang (missing prototypes for the el_runtime.c
cc -std=c11 -O2 -I runtime -I$(brew --prefix openssl@3)/include \ # symbols it wraps — see caveat below), so link el_runtime.c here.
-L$(brew --prefix openssl@3)/lib \ cc -std=c11 -I runtime -lcurl -lpthread \
-o dist/platform/elc-new \ -o dist/platform/elc-new \
elc-new.c runtime/el_runtime.c runtime/el_seed.c \ elc-new.c runtime/el_runtime.c
runtime/engram_cognition.c runtime/engram_geometry.c runtime/engram_reason.c \
runtime/engram_store.c runtime/engram_verify.c runtime/engram_vindex.c \
runtime/eg_cosine_batch.c runtime/eg_cosine_batch_strategy_cpu.c \
-lcurl -lssl -lcrypto -lpthread -lm
# 3. Verify self-hosting FIXPOINT (stage3 == stage2 output, byte-identical): # 3. Verify self-hosting FIXPOINT (stage3 == stage2 output, byte-identical):
./dist/platform/elc-new elc-cli.el > elc-verify.c ./dist/platform/elc-new elc-cli.el > elc-verify.c
diff elc-new.c elc-verify.c # must be identical diff elc-new.c elc-verify.c # must be identical
BIN
View File
Binary file not shown.
+1 -16
View File
@@ -3295,12 +3295,6 @@ fn cgi_arg(value: String, has_value: Bool) -> String {
// exit before touching configuration, ports, or any data directory. // exit before touching configuration, ports, or any data directory.
// 2. config declarations resolve env-or-default, one declaration per entry. // 2. config declarations resolve env-or-default, one declaration per entry.
// 3. validate LAST report EVERY missing/ill-typed entry at once, then exit. // 3. validate LAST report EVERY missing/ill-typed entry at once, then exit.
//
// `singleton:` carries its `guards:` expression as its SECOND argument the
// state the lock protects, evaluated here at the process boundary. A singleton
// without one does not compile (see below): a lock keyed on the program's name
// rather than on its state refuses unrelated instances and permits concurrent
// ones, which is not a weaker guard but a wrong one.
fn el_bool_arg(b: Bool) -> String { fn el_bool_arg(b: Bool) -> String {
if b { return "EL_INT(1)" } if b { return "EL_INT(1)" }
return "EL_INT(0)" return "EL_INT(0)"
@@ -3312,16 +3306,7 @@ fn emit_program_init(stmt: Map<String, Any>) -> Void {
let has_singleton: Bool = stmt["has_singleton"] let has_singleton: Bool = stmt["has_singleton"]
if has_singleton { if has_singleton {
let sid: String = stmt["singleton"] let sid: String = stmt["singleton"]
let has_guards: Bool = stmt["has_guards"] emit_line(" el_singleton_acquire(EL_STR(" + c_str_lit(sid) + "));")
if has_guards {
let guards_c: String = cg_expr(stmt["guards"])
emit_line(" el_singleton_acquire(EL_STR(" + c_str_lit(sid) + "), " + guards_c + ");")
} else {
// Refuse at COMPILE time. The alternative emitting a name-keyed
// lock is the defect itself, and it fails silently in the direction
// that loses data.
emit_line("#error \"singleton '" + sid + "' declares no `guards:` — a singleton must name the state it protects, e.g. `guards: engram_resolve_data_dir()` (spec 18.2)\"")
}
} }
let entries = stmt["entries"] let entries = stmt["entries"]
let n: Int = native_list_len(entries) let n: Int = native_list_len(entries)
-29
View File
@@ -1976,18 +1976,6 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
// singleton: "id" process identity. The runtime takes an exclusive // singleton: "id" process identity. The runtime takes an exclusive
// lock at startup; a SECOND start is refused, loudly, // lock at startup; a SECOND start is refused, loudly,
// instead of two processes sharing one data dir. // instead of two processes sharing one data dir.
// guards: <expr> WHAT that singleton protects: an expression yielding
// the path of the guarded state directory, evaluated at
// startup. MANDATORY with `singleton:`, because a lock
// keyed on a program's NAME rather than on its STATE is
// not a guard measured 2026-08-16, the name-keyed
// version refused unrelated instances (different data
// dirs) AND permitted concurrent ones (same data dir,
// different $TMPDIR). It is an expression and not a
// string so a program can point at the resolver that
// already OWNS the path (§18.4) instead of restating
// its default here, which would give the path two
// owners that can disagree.
// env NAME: T = "d" one configuration entry. Its type and its default // env NAME: T = "d" one configuration entry. Its type and its default
// are declared ONCE, here, and resolved+validated // are declared ONCE, here, and resolved+validated
// before main() body runs. // before main() body runs.
@@ -2005,8 +1993,6 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
let p = expect(tokens, p, "LBrace") let p = expect(tokens, p, "LBrace")
let singleton = "" let singleton = ""
let has_singleton = false let has_singleton = false
let guards_node = { "expr": "Str", "value": "" }
let has_guards = false
let entries = native_list_empty() let entries = native_list_empty()
// Entry-scratch declared at loop-body level (not inside the branch) so // Entry-scratch declared at loop-body level (not inside the branch) so
// that inner `let` forms compile to assignment rather than a C-scoped // that inner `let` forms compile to assignment rather than a C-scoped
@@ -2061,18 +2047,6 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
"has_default": has_default, "has_default": has_default,
"required": erequired "required": erequired
}) })
} else {
if str_eq(fname, "guards") {
// guards: <expr> the STATE the singleton protects.
// Parsed as a full expression, not a string literal, so
// it can name the resolver that owns the path
// (`guards: engram_resolve_data_dir()`) rather than
// duplicating that resolver's default here.
let p = expect(tokens, p, "Colon")
let g_r = parse_expr(tokens, p)
let guards_node = g_r["node"]
let p = g_r["pos"]
let has_guards = true
} else { } else {
// scalar field: `name: "value"` // scalar field: `name: "value"`
let p = expect(tokens, p, "Colon") let p = expect(tokens, p, "Colon")
@@ -2083,7 +2057,6 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
let has_singleton = true let has_singleton = true
} }
} }
}
let k5 = tok_kind(tokens, p) let k5 = tok_kind(tokens, p)
if k5 == "Comma" { if k5 == "Comma" {
let p = p + 1 let p = p + 1
@@ -2097,8 +2070,6 @@ fn parse_stmt(tokens: [Any], pos: Int) -> Map<String, Any> {
"name": name, "name": name,
"singleton": singleton, "singleton": singleton,
"has_singleton": has_singleton, "has_singleton": has_singleton,
"guards": guards_node,
"has_guards": has_guards,
"entries": entries "entries": entries
}, p) }, p)
} }
-525
View File
@@ -1,525 +0,0 @@
/* el_audio_darwin.m — the SPEAKER realizer. El's native audio output on Darwin.
*
* WHY THIS FILE EXISTS.
*
* Neuron could already turn meaning into samples the render path in
* elp/src/speech.el superposes formant resonances over a glottal source and
* produces PCM. What it could not do was make a sound. Every path from those
* samples to the air ran outside the language: a 939-line Swift program
* (peripheral/src/periph.swift) that shelled out to /usr/bin/afplay. So the
* voice was not a capability of El or of Neuron. It was a separate binary
* standing next to them, and "speak" meant "ask that binary to speak."
*
* A speaker is not a language feature the way a string is, but it is exactly
* the kind of thing a runtime owns: a device. El already owns the filesystem,
* the network, the clock, and a graph. It should own the one output device that
* makes it audible. After this file, `speak` is an El operation.
*
* WHY IT IS A REALIZER AND NOT PURE EL.
*
* This is the boundary the whole design turns on. Everything ABOVE the sample
* buffer is arithmetic and belongs in El: formant geometry, superposition,
* envelopes, WAV framing, the voice signature. Everything in this file is the
* part that cannot be arithmetic handing a buffer to CoreAudio and waiting
* for the hardware to drain it. There is no way to express "the DAC has now
* played these samples" in El, and there should not be. So the split is: El
* computes the sound, the realizer emits it, and the realizer is as thin as it
* can possibly be it makes no decisions about content, it has no opinion
* about audio, and it cannot synthesize anything.
*
* The precedent is eg_cosine_batch_strategy_metal_hand.m: a platform-bound
* capability compiled as its OWN translation unit, declared in el_runtime.h,
* and linked in where the platform supports it. Deliberately NOT a patch to
* el_runtime.c adding a device to El must not mean editing the core runtime,
* for the same reason adding a modality must not (see el_runtime.c's realizer
* registry: a realizer is resolved by name, so new organs never touch the
* middle of the language). el_audio_null.c is the same two entry points for
* every platform that is not Darwin, so El code that speaks still links
* everywhere and simply reports that it has no speaker.
*
* WHY AudioQueue AND NOT afplay.
*
* afplay is a process. Using it means the sound Neuron makes is a file it wrote
* and asked something else to open which forces every utterance through the
* disk, cannot start until the whole utterance exists, and puts a fork/exec
* between the intent to speak and the sound. AudioQueue takes the samples
* directly out of memory. Nothing is written, nothing is spawned, and a caller
* that wants to stream can push buffers as it renders them.
*
* AudioToolbox ships with macOS, so this stays own-core: no cloud, no library
* to install, no model. The output is the local speaker and nothing leaves the
* machine there is no network path in this file at all, by construction.
*/
#import <AudioToolbox/AudioToolbox.h>
#import <Foundation/Foundation.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include "el_runtime.h"
/* Three buffers is the standard AudioQueue depth: one being played by the
* hardware, one queued behind it, one being refilled. Fewer risks a gap on a
* busy machine; more only adds latency before the first sound. */
#define EL_AQ_NBUF 3
#define EL_AQ_FRAMES 8192
typedef struct {
const int16_t* pcm;
int64_t frames;
int64_t pos;
volatile int inflight; /* buffers CoreAudio still owns */
volatile int drained; /* set once the last buffer has been played */
} ElAqState;
/* Called on an AudioQueue-internal thread each time a buffer finishes playing.
* Refills and re-enqueues while samples remain; when the source is exhausted it
* lets the buffer die and counts it out. `drained` flips only when the queue is
* holding nothing, which is what makes the play call synchronous without
* clipping the tail the same reason periph.swift used .dataPlayedBack rather
* than treating "consumed" as "heard". */
static void el_aq_callback(void* userData, AudioQueueRef q, AudioQueueBufferRef buf) {
ElAqState* st = (ElAqState*)userData;
int64_t remain = st->frames - st->pos;
if (remain <= 0) {
if (--st->inflight <= 0) st->drained = 1;
return;
}
int64_t n = remain < EL_AQ_FRAMES ? remain : EL_AQ_FRAMES;
memcpy(buf->mAudioData, st->pcm + st->pos, (size_t)n * sizeof(int16_t));
buf->mAudioDataByteSize = (UInt32)(n * (int64_t)sizeof(int16_t));
st->pos += n;
if (AudioQueueEnqueueBuffer(q, buf, 0, NULL) != noErr) {
if (--st->inflight <= 0) st->drained = 1;
}
}
/* Play a 16-bit mono PCM buffer out the default output device, blocking until
* the hardware has actually finished. Returns 1 on success, 0 on any failure
* never throws, never hangs indefinitely. */
static int el_audio_play_raw(const int16_t* pcm, int64_t frames, int32_t sample_rate) {
if (!pcm || frames <= 0 || sample_rate <= 0) return 0;
AudioStreamBasicDescription fmt;
memset(&fmt, 0, sizeof(fmt));
fmt.mSampleRate = (Float64)sample_rate;
fmt.mFormatID = kAudioFormatLinearPCM;
fmt.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
fmt.mFramesPerPacket = 1;
fmt.mChannelsPerFrame = 1;
fmt.mBitsPerChannel = 16;
fmt.mBytesPerFrame = 2;
fmt.mBytesPerPacket = 2;
ElAqState st;
memset(&st, 0, sizeof(st));
st.pcm = pcm;
st.frames = frames;
AudioQueueRef q = NULL;
/* NULL run loop => callbacks arrive on an AudioQueue-internal thread, so
* this function can simply wait rather than having to pump a run loop it
* does not own. El programs are not required to have one. */
if (AudioQueueNewOutput(&fmt, el_aq_callback, &st, NULL, NULL, 0, &q) != noErr || !q) {
return 0;
}
AudioQueueBufferRef bufs[EL_AQ_NBUF];
int prepared = 0;
for (int i = 0; i < EL_AQ_NBUF; i++) {
if (AudioQueueAllocateBuffer(q, EL_AQ_FRAMES * sizeof(int16_t), &bufs[i]) != noErr) break;
prepared++;
}
if (prepared == 0) { AudioQueueDispose(q, true); return 0; }
/* Prime: fill what we can before starting, so playback begins immediately
* rather than after the first underrun. */
for (int i = 0; i < prepared; i++) {
int64_t remain = st.frames - st.pos;
if (remain <= 0) break;
int64_t n = remain < EL_AQ_FRAMES ? remain : EL_AQ_FRAMES;
memcpy(bufs[i]->mAudioData, st.pcm + st.pos, (size_t)n * sizeof(int16_t));
bufs[i]->mAudioDataByteSize = (UInt32)(n * (int64_t)sizeof(int16_t));
st.pos += n;
if (AudioQueueEnqueueBuffer(q, bufs[i], 0, NULL) != noErr) break;
st.inflight++;
}
if (st.inflight == 0) { AudioQueueDispose(q, true); return 0; }
if (AudioQueueStart(q, NULL) != noErr) { AudioQueueDispose(q, true); return 0; }
/* Bound the wait by the material's own duration plus a margin. A speaker
* that wedges a program is worse than a speaker that gives up. */
double seconds = (double)frames / (double)sample_rate;
int64_t max_us = (int64_t)((seconds + 5.0) * 1000000.0);
int64_t waited = 0;
const int64_t tick = 5000; /* 5 ms */
while (!st.drained && waited < max_us) {
usleep((useconds_t)tick);
waited += tick;
}
AudioQueueStop(q, true);
AudioQueueDispose(q, true);
return st.drained ? 1 : 0;
}
/* ── El entry points ────────────────────────────────────────────────────────
* Declared in el_runtime.h; see there for the El-facing contract. */
/* 1 when this build has a real speaker behind it. El code should ask before
* speaking so the no-speaker case is a reported condition, not a silence that
* looks like success. */
el_val_t speaker_available(void) {
return (el_val_t)1;
}
el_val_t speaker_name(void) {
return EL_STR("coreaudio-audioqueue");
}
/* Play an El [Int] of 16-bit samples. Values are clamped, not wrapped: a
* render that overshoots should distort at the rails the way real clipping
* does, rather than invert phase and produce a sound nothing in the signal
* chain intended. */
el_val_t speaker_play_pcm16(el_val_t samples, el_val_t sample_rate) {
int64_t n = (int64_t)el_list_len(samples);
int32_t sr = (int32_t)sample_rate;
if (n <= 0 || sr <= 0) return (el_val_t)0;
int16_t* pcm = (int16_t*)malloc((size_t)n * sizeof(int16_t));
if (!pcm) return (el_val_t)0;
for (int64_t i = 0; i < n; i++) {
int64_t v = (int64_t)el_list_get(samples, (el_val_t)i);
if (v > 32767) v = 32767;
if (v < -32768) v = -32768;
pcm[i] = (int16_t)v;
}
int ok = el_audio_play_raw(pcm, n, sr);
free(pcm);
return (el_val_t)(ok ? 1 : 0);
}
/* ── Asynchronous playback ───────────────────────────────────────────────────
*
* converse needs this and a blocking play cannot give it. Barge-in means
* stopping ON THE SPOT when the user starts talking not at the end of the
* current buffer, and certainly not at the end of the utterance. So the async
* path keeps one queue alive, reports how far the hardware actually got, and
* can be halted mid-buffer.
*
* `played_frames` is what makes an interrupted utterance resumable at the
* sample rather than at the segment: it is the position the DAC reached, not
* the position we enqueued to, and those differ by up to the full queue depth.
*
* One utterance at a time. A second async play stops the first a mouth that
* can say two things at once is not a feature. */
static AudioQueueRef g_aq = NULL;
static ElAqState* g_aq_state = NULL;
static int16_t* g_aq_pcm = NULL;
static int32_t g_aq_sr = 0;
static void el_audio_teardown(void) {
if (g_aq) {
AudioQueueStop(g_aq, true);
AudioQueueDispose(g_aq, true);
g_aq = NULL;
}
free(g_aq_pcm); g_aq_pcm = NULL;
free(g_aq_state); g_aq_state = NULL;
g_aq_sr = 0;
}
el_val_t speaker_play_pcm16_async(el_val_t samples, el_val_t sample_rate) {
el_audio_teardown();
int64_t n = (int64_t)el_list_len(samples);
int32_t sr = (int32_t)sample_rate;
if (n <= 0 || sr <= 0) return (el_val_t)0;
g_aq_pcm = (int16_t*)malloc((size_t)n * sizeof(int16_t));
if (!g_aq_pcm) return (el_val_t)0;
for (int64_t i = 0; i < n; i++) {
int64_t v = (int64_t)el_list_get(samples, (el_val_t)i);
if (v > 32767) v = 32767;
if (v < -32768) v = -32768;
g_aq_pcm[i] = (int16_t)v;
}
g_aq_state = (ElAqState*)calloc(1, sizeof(ElAqState));
if (!g_aq_state) { el_audio_teardown(); return (el_val_t)0; }
g_aq_state->pcm = g_aq_pcm;
g_aq_state->frames = n;
g_aq_sr = sr;
AudioStreamBasicDescription fmt;
memset(&fmt, 0, sizeof(fmt));
fmt.mSampleRate = (Float64)sr;
fmt.mFormatID = kAudioFormatLinearPCM;
fmt.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
fmt.mFramesPerPacket = 1;
fmt.mChannelsPerFrame = 1;
fmt.mBitsPerChannel = 16;
fmt.mBytesPerFrame = 2;
fmt.mBytesPerPacket = 2;
if (AudioQueueNewOutput(&fmt, el_aq_callback, g_aq_state, NULL, NULL, 0, &g_aq) != noErr || !g_aq) {
el_audio_teardown();
return (el_val_t)0;
}
for (int i = 0; i < EL_AQ_NBUF; i++) {
int64_t remain = g_aq_state->frames - g_aq_state->pos;
if (remain <= 0) break;
AudioQueueBufferRef b = NULL;
if (AudioQueueAllocateBuffer(g_aq, EL_AQ_FRAMES * sizeof(int16_t), &b) != noErr) break;
int64_t k = remain < EL_AQ_FRAMES ? remain : EL_AQ_FRAMES;
memcpy(b->mAudioData, g_aq_state->pcm + g_aq_state->pos, (size_t)k * sizeof(int16_t));
b->mAudioDataByteSize = (UInt32)(k * (int64_t)sizeof(int16_t));
g_aq_state->pos += k;
if (AudioQueueEnqueueBuffer(g_aq, b, 0, NULL) != noErr) break;
g_aq_state->inflight++;
}
if (g_aq_state->inflight == 0) { el_audio_teardown(); return (el_val_t)0; }
if (AudioQueueStart(g_aq, NULL) != noErr) { el_audio_teardown(); return (el_val_t)0; }
return (el_val_t)1;
}
el_val_t speaker_playing(void) {
if (!g_aq || !g_aq_state) return (el_val_t)0;
return (el_val_t)(g_aq_state->drained ? 0 : 1);
}
/* Frames the DAC has actually rendered. AudioQueueGetCurrentTime's mSampleTime
* is relative to queue start, which is exactly the "where was I really" figure
* a resumable utterance needs. Falls back to the enqueued position if the
* timeline is unavailable (it is, briefly, right after start). */
el_val_t speaker_played_frames(void) {
if (!g_aq || !g_aq_state) return (el_val_t)0;
AudioTimeStamp ts;
memset(&ts, 0, sizeof(ts));
Boolean discontinuity = false;
if (AudioQueueGetCurrentTime(g_aq, NULL, &ts, &discontinuity) == noErr &&
(ts.mFlags & kAudioTimeStampSampleTimeValid)) {
int64_t played = (int64_t)ts.mSampleTime;
if (played < 0) played = 0;
if (played > g_aq_state->frames) played = g_aq_state->frames;
return (el_val_t)played;
}
return (el_val_t)g_aq_state->pos;
}
/* Pause where we are, keeping the queue and its position intact.
*
* This is the difference between barge-in and "finish the buffer". The moment
* the microphone hears speech, output must stop AT THAT SAMPLE a listener
* experiences even 200ms of continued talking as being talked over. Pause
* rather than stop because the interruption might turn out to be a backchannel
* ("mm-hm"), and the right response to a backchannel is to carry on as though
* nothing happened, which requires the queue to still be exactly where it was.
* A stop-and-restart would re-attack the buffer and be audible as a stutter. */
el_val_t speaker_pause(void) {
if (!g_aq) return (el_val_t)0;
return (el_val_t)(AudioQueuePause(g_aq) == noErr ? 1 : 0);
}
el_val_t speaker_resume(void) {
if (!g_aq) return (el_val_t)0;
return (el_val_t)(AudioQueueStart(g_aq, NULL) == noErr ? 1 : 0);
}
el_val_t speaker_stop(void) {
if (!g_aq) return (el_val_t)0;
/* immediate: do NOT let the queue finish what it is holding */
AudioQueueStop(g_aq, true);
el_audio_teardown();
return (el_val_t)1;
}
/* Decode a 16-bit RIFF/WAVE into a freshly malloc'd mono int16 buffer.
* Returns frames, or 0 on any failure; *out is set only on success. Shared by
* the blocking and async WAV paths. */
static int64_t el_wav_load(const char* path, int16_t** out, int32_t* out_sr) {
if (!path || !out) return 0;
FILE* f = fopen(path, "rb");
if (!f) return 0;
if (fseek(f, 0, SEEK_END) != 0) { fclose(f); return 0; }
long size = ftell(f);
if (size <= 44) { fclose(f); return 0; }
rewind(f);
unsigned char* d = (unsigned char*)malloc((size_t)size);
if (!d) { fclose(f); return 0; }
size_t got = fread(d, 1, (size_t)size, f);
fclose(f);
if (got != (size_t)size) { free(d); return 0; }
if (memcmp(d, "RIFF", 4) != 0 || memcmp(d + 8, "WAVE", 4) != 0) { free(d); return 0; }
int32_t sr = 0, channels = 0, bits = 0;
long dataOff = -1, dataLen = 0, o = 12;
/* Chunk-walk rather than assuming fmt-then-data at fixed offsets: recorders
* routinely interleave JUNK/FLLR padding, and a fixed-offset parser reads
* padding as audio. */
while (o + 8 <= size) {
long sz = (long)d[o+4] | ((long)d[o+5] << 8) | ((long)d[o+6] << 16) | ((long)d[o+7] << 24);
if (sz < 0) break;
if (memcmp(d + o, "fmt ", 4) == 0 && o + 24 <= size) {
channels = (int32_t)(d[o+10] | (d[o+11] << 8));
sr = (int32_t)((long)d[o+12] | ((long)d[o+13] << 8) | ((long)d[o+14] << 16) | ((long)d[o+15] << 24));
bits = (int32_t)(d[o+22] | (d[o+23] << 8));
} else if (memcmp(d + o, "data", 4) == 0) {
dataOff = o + 8;
dataLen = sz;
if (dataOff + dataLen > size) dataLen = size - dataOff;
}
o += 8 + sz + (sz & 1);
}
if (dataOff < 0 || sr <= 0 || bits != 16 || channels < 1 || dataLen <= 0) { free(d); return 0; }
long frames = dataLen / (2 * channels);
int16_t* pcm = (int16_t*)malloc((size_t)frames * sizeof(int16_t));
if (!pcm) { free(d); return 0; }
for (long i = 0; i < frames; i++) {
long b = dataOff + i * 2 * channels;
pcm[i] = (int16_t)((unsigned)d[b] | ((unsigned)d[b+1] << 8));
}
free(d);
*out = pcm;
if (out_sr) *out_sr = sr;
return (int64_t)frames;
}
/* Async WAV playback. converse speaks PRE-RENDERED segments and must keep
* listening while it does, so it needs the file on the queue without blocking
* and needs to be able to stop it mid-buffer. Going through the file rather
* than an El [Int] also avoids marshalling a million-element list per segment
* for audio the caller never intends to look at. */
el_val_t speaker_play_wav_async(el_val_t path) {
const char* p = EL_CSTR(path);
if (!p) return (el_val_t)0;
el_audio_teardown();
int32_t sr = 0;
int16_t* pcm = NULL;
int64_t frames = el_wav_load(p, &pcm, &sr);
if (frames <= 0 || !pcm) { free(pcm); return (el_val_t)0; }
g_aq_pcm = pcm;
g_aq_sr = sr;
g_aq_state = (ElAqState*)calloc(1, sizeof(ElAqState));
if (!g_aq_state) { el_audio_teardown(); return (el_val_t)0; }
g_aq_state->pcm = g_aq_pcm;
g_aq_state->frames = frames;
AudioStreamBasicDescription fmt;
memset(&fmt, 0, sizeof(fmt));
fmt.mSampleRate = (Float64)sr;
fmt.mFormatID = kAudioFormatLinearPCM;
fmt.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
fmt.mFramesPerPacket = 1;
fmt.mChannelsPerFrame = 1;
fmt.mBitsPerChannel = 16;
fmt.mBytesPerFrame = 2;
fmt.mBytesPerPacket = 2;
if (AudioQueueNewOutput(&fmt, el_aq_callback, g_aq_state, NULL, NULL, 0, &g_aq) != noErr || !g_aq) {
el_audio_teardown();
return (el_val_t)0;
}
for (int i = 0; i < EL_AQ_NBUF; i++) {
int64_t remain = g_aq_state->frames - g_aq_state->pos;
if (remain <= 0) break;
AudioQueueBufferRef b = NULL;
if (AudioQueueAllocateBuffer(g_aq, EL_AQ_FRAMES * sizeof(int16_t), &b) != noErr) break;
int64_t k = remain < EL_AQ_FRAMES ? remain : EL_AQ_FRAMES;
memcpy(b->mAudioData, g_aq_state->pcm + g_aq_state->pos, (size_t)k * sizeof(int16_t));
b->mAudioDataByteSize = (UInt32)(k * (int64_t)sizeof(int16_t));
g_aq_state->pos += k;
if (AudioQueueEnqueueBuffer(g_aq, b, 0, NULL) != noErr) break;
g_aq_state->inflight++;
}
if (g_aq_state->inflight == 0) { el_audio_teardown(); return (el_val_t)0; }
if (AudioQueueStart(g_aq, NULL) != noErr) { el_audio_teardown(); return (el_val_t)0; }
return (el_val_t)1;
}
/* Total frames and sample rate of a WAV, without playing it — wav-info, and the
* duration converse needs to compute progress through a segment. */
el_val_t wav_frames(el_val_t path) {
const char* p = EL_CSTR(path);
int16_t* pcm = NULL; int32_t sr = 0;
int64_t n = el_wav_load(p, &pcm, &sr);
free(pcm);
return (el_val_t)n;
}
el_val_t wav_rate(el_val_t path) {
const char* p = EL_CSTR(path);
int16_t* pcm = NULL; int32_t sr = 0;
int64_t n = el_wav_load(p, &pcm, &sr);
free(pcm);
return (el_val_t)(n > 0 ? sr : 0);
}
/* Play a 16-bit mono RIFF/WAVE file. Present because the render already knows
* how to write a WAV and a caller may reasonably want to hear one back without
* re-rendering it; the parse is deliberately minimal and chunk-walking, so the
* JUNK/FLLR padding that recorders emit does not defeat it. */
el_val_t speaker_play_wav(el_val_t path) {
const char* p = EL_CSTR(path);
if (!p) return (el_val_t)0;
FILE* f = fopen(p, "rb");
if (!f) return (el_val_t)0;
if (fseek(f, 0, SEEK_END) != 0) { fclose(f); return (el_val_t)0; }
long size = ftell(f);
if (size <= 44) { fclose(f); return (el_val_t)0; }
rewind(f);
unsigned char* d = (unsigned char*)malloc((size_t)size);
if (!d) { fclose(f); return (el_val_t)0; }
size_t got = fread(d, 1, (size_t)size, f);
fclose(f);
if (got != (size_t)size) { free(d); return (el_val_t)0; }
if (memcmp(d, "RIFF", 4) != 0 || memcmp(d + 8, "WAVE", 4) != 0) { free(d); return (el_val_t)0; }
int32_t sr = 0, channels = 0, bits = 0;
long dataOff = -1, dataLen = 0;
long o = 12;
while (o + 8 <= size) {
long sz = (long)d[o+4] | ((long)d[o+5] << 8) | ((long)d[o+6] << 16) | ((long)d[o+7] << 24);
if (sz < 0) break;
if (memcmp(d + o, "fmt ", 4) == 0 && o + 24 <= size) {
channels = (int32_t)(d[o+10] | (d[o+11] << 8));
sr = (int32_t)((long)d[o+12] | ((long)d[o+13] << 8) | ((long)d[o+14] << 16) | ((long)d[o+15] << 24));
bits = (int32_t)(d[o+22] | (d[o+23] << 8));
} else if (memcmp(d + o, "data", 4) == 0) {
dataOff = o + 8;
dataLen = sz;
if (dataOff + dataLen > size) dataLen = size - dataOff;
}
o += 8 + sz + (sz & 1);
}
if (dataOff < 0 || sr <= 0 || bits != 16 || channels < 1 || dataLen <= 0) { free(d); return (el_val_t)0; }
long frames = dataLen / (2 * channels);
int16_t* pcm = (int16_t*)malloc((size_t)frames * sizeof(int16_t));
if (!pcm) { free(d); return (el_val_t)0; }
/* Take channel 0; the organ is mono by design and downmixing would be an
* opinion about content this layer is not entitled to have. */
for (long i = 0; i < frames; i++) {
long b = dataOff + i * 2 * channels;
pcm[i] = (int16_t)((unsigned)d[b] | ((unsigned)d[b+1] << 8));
}
free(d);
int ok = el_audio_play_raw(pcm, frames, sr);
free(pcm);
return (el_val_t)(ok ? 1 : 0);
}
-841
View File
@@ -1,841 +0,0 @@
/* el_capture_darwin.m — the MICROPHONE and CAMERA realizers. El's afferent
* organ on Darwin: the two entry points through which the world gets in.
*
* WHY THIS FILE EXISTS, AND WHY IT IS A REALIZER RATHER THAN PURE EL.
*
* el_audio_darwin.m argued the efferent half of this: El can compute a sound
* but it cannot make one, because "the DAC has now played these samples" is not
* a fact any amount of arithmetic can produce. This file is the same argument
* run backwards. El can compute *about* a sound it can window it, take its
* autocorrelation, run Levinson-Durbin over that, find the formant peaks in the
* resulting all-pole envelope, and hand back a voiceprint but it cannot ASK.
* There is no expression in El, and there must not be, whose value is "the next
* 1024 frames the microphone hears" or "what the camera is pointed at right
* now." Those are not computed; they are *requested*, from an operating system
* that owns the device, mediates consent for it, and delivers the answer on a
* thread of its choosing whenever it feels like it. Asking is the one primitive
* operation here. Everything else in this file is bookkeeping around the ask.
*
* So the line is drawn exactly where el_audio_darwin.m drew it, at the sample
* buffer, and it is drawn on purpose:
*
* BELOW the line (here): open the device, honour the OS permission gate,
* install a tap or a frame delegate, convert whatever the hardware happens to
* emit into the one shape El asked for, and hand it up. No opinions about
* content. No analysis. No decisions.
*
* ABOVE the line (El): energy, zero-crossing rate, spectral centroid, F0 by
* autocorrelation, LPC, formants F1-F5, the compact descriptors, the
* scene-geometry grid, the yield-or-hold turn-taking decision. All of it is
* arithmetic over a buffer, all of it belongs in El, and none of it appears
* below. The reference this file ports peripheral/src/periph.swift held
* both halves, and that was the problem worth fixing: the descriptors were
* trapped in a 939-line binary standing next to the language instead of being
* written in it. Porting the *whole* of periph.swift down here would have
* reproduced that mistake in C. Only the ask came down.
*
* The precedent for the file's SHAPE is eg_cosine_batch_strategy_metal_hand.m:
* a platform-bound capability compiled as its own translation unit, declared in
* el_runtime.h, linked in where the platform supports it, and deliberately NOT
* a patch to the middle of el_runtime.c. Acquiring a device must not mean
* editing the language, for the same reason acquiring a modality must not (see
* the realizer registry: organs are resolved by name). el_peripheral_null.c is
* the same entry points everywhere else, so El code that listens still links on
* every platform and merely reports having no ear.
*
* FAIL CLOSED, ALWAYS.
*
* A capture path that returns plausible-looking zeros when it was denied is
* worse than one that returns nothing, because the caller cannot tell the
* difference between a silent room and a refused microphone. Every entry point
* here checks AVCaptureDevice's authorization status BEFORE touching hardware
* and returns the empty value an empty list, a 0 map on anything short of
* .authorized. mic_available() and camera_available() report that state WITHOUT
* prompting, so El can ask "may I?" without the act of asking being a prompt.
*
* NEVER HANG.
*
* Every wait in this file is bounded: 30s on a permission prompt (the user has
* to walk to the dialog), seconds+5 on a capture of `seconds`, 10s on a camera
* frame. An organ that wedges the program holding it is not an organ, it is a
* fault. Every path also tears the device down on the way out, including the
* failure paths, so a timed-out capture does not leave the mic light on.
*
* OWN-CORE AND LOCAL BY CONSTRUCTION.
*
* AVFoundation, CoreVideo, CoreGraphics and ImageIO ship with macOS. There is
* no third-party library here, no model, and the part that matters no
* network path of any kind. Samples and pixels move from local hardware into an
* El value and stop. periph.swift had a URLSession in it; this file has no
* socket, no URL, and nothing that could grow one without being obvious in
* review. Consent is enforced above this layer in El and below it by the OS;
* this layer's whole contribution to that is refusing to proceed.
*
* DISCLOSURE.
*
* Every actual device touch writes one line to stderr and flushes it, before
* the device opens. stderr and not stdout: a program that announces "I am about
* to open the microphone" on stdout has corrupted its own output, and the
* caller must be able to separate the answer from how it was obtained. One line
* per touch, no more a disclosure rail that spams is a rail people learn to
* ignore.
*/
#if defined(__APPLE__)
#import <Foundation/Foundation.h>
#import <AVFoundation/AVFoundation.h>
#import <CoreMedia/CoreMedia.h>
#import <CoreVideo/CoreVideo.h>
#import <CoreGraphics/CoreGraphics.h>
#import <ImageIO/ImageIO.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <unistd.h>
#include <pthread.h>
#include "el_runtime.h"
/* ── Disclosure ──────────────────────────────────────────────────────────────
* One flushed line per real device touch, on stderr. Flushed rather than
* buffered so the line reaches the terminal BEFORE the mic light comes on
* rather than whenever the buffer happens to drain. */
static void el_cap_disclose(const char* what) {
fprintf(stderr, " [peripheral] %s\n", what);
fflush(stderr);
}
/* ── Permission ──────────────────────────────────────────────────────────────
* authorizationStatus is a pure read of the TCC database: it never prompts and
* never blocks, which is what lets mic_available()/camera_available() answer
* honestly without the question itself becoming an event. requestAccess DOES
* prompt, so it lives behind its own entry point and nothing calls it
* implicitly. */
static int el_cap_authorized(AVMediaType media) {
@try {
return [AVCaptureDevice authorizationStatusForMediaType:media]
== AVAuthorizationStatusAuthorized ? 1 : 0;
} @catch (NSException* e) {
(void)e;
return 0;
}
}
static int el_cap_device_present(AVMediaType media) {
@try {
return [AVCaptureDevice defaultDeviceWithMediaType:media] != nil ? 1 : 0;
} @catch (NSException* e) {
(void)e;
return 0;
}
}
/* Prompt once and wait, bounded. 30 seconds is the same budget periph.swift
* used: long enough for a human to notice a dialog and decide, short enough
* that an unattended run fails rather than parks forever. A timeout is reported
* as "not granted", which is the safe reading we genuinely do not know that
* it was. */
static int el_cap_request(AVMediaType media) {
__block int granted = 0;
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
@try {
[AVCaptureDevice requestAccessForMediaType:media
completionHandler:^(BOOL ok) {
granted = ok ? 1 : 0;
dispatch_semaphore_signal(sem);
}];
} @catch (NSException* e) {
(void)e;
return 0;
}
if (dispatch_semaphore_wait(sem,
dispatch_time(DISPATCH_TIME_NOW, (int64_t)(30 * NSEC_PER_SEC))) != 0) {
return 0; /* timed out — treat as refused */
}
return granted;
}
/* ════════════════════════════════════════════════════════════════════════════
* MICROPHONE one-shot capture
* */
/* The sink the tap block writes into. An object rather than a static so two
* captures can never share state, and so ARC keeps it alive for exactly as long
* as the block that captured it. The lock is real, not decorative: the tap runs
* on an AVAudioEngine-internal thread and the waiter runs on the caller's. */
@interface ElCapMicSink : NSObject
@property (nonatomic, strong) NSMutableData* pcm;
@property (nonatomic, strong) NSLock* lock;
@end
@implementation ElCapMicSink
- (instancetype)init {
self = [super init];
if (self) {
_pcm = [NSMutableData data];
_lock = [[NSLock alloc] init];
}
return self;
}
@end
/* Capture `seconds` of mono 16-bit PCM at `sample_rate`.
*
* The hardware format is NOT assumed. A built-in mic will typically hand back
* float32 at 44.1 or 48 kHz, an aggregate device may be 8 channels at 96 kHz,
* and a caller asking for 16 kHz mono (which is what the formant path wants)
* gets 16 kHz mono either way. AVAudioConverter does the rate conversion and
* the downmix; doing it by hand would mean writing a resampler in the one file
* that is supposed to contain no arithmetic.
*
* The converter is built ONCE, outside the tap, because a sample-rate converter
* carries filter state across buffers rebuilding it per callback would put a
* discontinuity at every buffer boundary, which is audible and which would then
* show up in El's spectral descriptors as energy that was never in the room. */
el_val_t mic_capture_pcm16(el_val_t seconds, el_val_t sample_rate) {
el_val_t empty = el_list_empty();
if (!el_cap_authorized(AVMediaTypeAudio)) return empty;
int64_t secs = (int64_t)seconds;
int64_t sr = (int64_t)sample_rate;
if (secs <= 0 || sr <= 0) return empty;
/* Bound the ask. A caller that asks for a year of audio has made a mistake,
* and honouring it would mean an unkillable capture and an OOM. */
if (secs > 300) secs = 300;
if (sr > 384000) sr = 384000;
__block AVAudioEngine* engine = nil;
AVAudioInputNode* input = nil;
AVAudioFormat* hwFmt = nil;
int tapped = 0;
@try {
engine = [[AVAudioEngine alloc] init];
input = [engine inputNode];
hwFmt = [input inputFormatForBus:0];
} @catch (NSException* e) {
(void)e;
return empty;
}
if (!input || !hwFmt || hwFmt.sampleRate <= 0 || hwFmt.channelCount == 0) {
return empty;
}
/* Preferred target: mono int16 at the requested rate. If the converter
* refuses that pairing (some exotic input layouts will not downmix), fall
* back to keeping the hardware's channel count and taking channel 0 on the
* way out the organ is mono by design and inventing a downmix here would
* be an opinion about content this layer is not entitled to have. */
AVAudioFormat* outFmt =
[[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatInt16
sampleRate:(double)sr
channels:1
interleaved:YES];
AVAudioConverter* conv = outFmt ? [[AVAudioConverter alloc] initFromFormat:hwFmt
toFormat:outFmt] : nil;
AVAudioChannelCount outCh = 1;
if (!conv) {
outFmt = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatInt16
sampleRate:(double)sr
channels:hwFmt.channelCount
interleaved:YES];
conv = outFmt ? [[AVAudioConverter alloc] initFromFormat:hwFmt toFormat:outFmt] : nil;
outCh = hwFmt.channelCount;
}
if (!conv || !outFmt) return empty;
ElCapMicSink* sink = [[ElCapMicSink alloc] init];
const int64_t want = secs * sr; /* frames we are waiting for */
{
char msg[192];
snprintf(msg, sizeof(msg),
"MIC: opening the microphone for %llds -> %lld Hz mono PCM "
"(local, never egresses).", (long long)secs, (long long)sr);
el_cap_disclose(msg);
}
const double ratio = (double)sr / hwFmt.sampleRate;
@try {
[input installTapOnBus:0
bufferSize:4096
format:hwFmt
block:^(AVAudioPCMBuffer* _Nonnull buf, AVAudioTime* _Nonnull when) {
(void)when;
if (!buf || buf.frameLength == 0) return;
AVAudioFrameCount cap =
(AVAudioFrameCount)((double)buf.frameLength * ratio) + 1024;
AVAudioPCMBuffer* out =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:outFmt frameCapacity:cap];
if (!out) return;
__block BOOL fed = NO;
AVAudioConverterInputBlock feed =
^AVAudioBuffer* _Nullable (AVAudioPacketCount need,
AVAudioConverterInputStatus* _Nonnull status) {
(void)need;
if (fed) { *status = AVAudioConverterInputStatus_NoDataNow; return nil; }
fed = YES;
*status = AVAudioConverterInputStatus_HaveData;
return buf;
};
NSError* err = nil;
AVAudioConverterOutputStatus st =
[conv convertToBuffer:out error:&err withInputFromBlock:feed];
if (st == AVAudioConverterOutputStatus_Error || out.frameLength == 0) return;
const int16_t* src = out.int16ChannelData ? out.int16ChannelData[0] : NULL;
if (!src) return;
NSUInteger n = (NSUInteger)out.frameLength;
[sink.lock lock];
if (outCh == 1) {
[sink.pcm appendBytes:src length:n * sizeof(int16_t)];
} else {
/* Interleaved: stride to channel 0. */
for (NSUInteger i = 0; i < n; i++) {
int16_t v = src[i * outCh];
[sink.pcm appendBytes:&v length:sizeof(int16_t)];
}
}
[sink.lock unlock];
}];
tapped = 1;
[engine prepare];
NSError* startErr = nil;
if (![engine startAndReturnError:&startErr]) {
[input removeTapOnBus:0];
return empty;
}
} @catch (NSException* e) {
(void)e;
@try { if (tapped) [input removeTapOnBus:0]; } @catch (NSException* e2) { (void)e2; }
@try { [engine stop]; } @catch (NSException* e2) { (void)e2; }
return empty;
}
/* Wait for `want` frames, bounded by the material's own duration plus a
* margin. A device that stops producing must not become a hang. */
const int64_t deadline_us = (secs + 5) * 1000000;
int64_t waited_us = 0;
const int64_t tick_us = 5000;
for (;;) {
[sink.lock lock];
int64_t have = (int64_t)([sink.pcm length] / sizeof(int16_t));
[sink.lock unlock];
if (have >= want || waited_us >= deadline_us) break;
usleep((useconds_t)tick_us);
waited_us += tick_us;
}
@try { [input removeTapOnBus:0]; } @catch (NSException* e) { (void)e; }
@try { [engine stop]; } @catch (NSException* e) { (void)e; }
engine = nil;
/* Hand up exactly what was asked for, or everything we got if the device
* came up short. Never padded: silence we invented is indistinguishable
* from silence we heard, and El has no way to tell them apart afterwards. */
[sink.lock lock];
int64_t have = (int64_t)([sink.pcm length] / sizeof(int16_t));
int64_t n = have < want ? have : want;
const int16_t* pcm = (const int16_t*)[sink.pcm bytes];
el_val_t list = empty;
for (int64_t i = 0; i < n; i++) {
list = el_list_append(list, (el_val_t)(int64_t)pcm[i]);
}
[sink.lock unlock];
return list;
}
el_val_t mic_available(void) {
if (!el_cap_authorized(AVMediaTypeAudio)) return (el_val_t)0;
return (el_val_t)(el_cap_device_present(AVMediaTypeAudio) ? 1 : 0);
}
el_val_t mic_request_access(void) {
return (el_val_t)(el_cap_request(AVMediaTypeAudio) ? 1 : 0);
}
/* ════════════════════════════════════════════════════════════════════════════
* MICROPHONE live monitor (the full-duplex ear)
*
* converse needs to keep listening WHILE it speaks, which means the mic is open
* at the same time as the speaker. In a real room that is a feedback path:
* without cancellation Neuron hears its own voice, decides someone is talking,
* and barges in on itself. setVoiceProcessingEnabled: hands the input node to
* the OS voice-processing unit, which subtracts the known output signal from
* the input the single thing that makes barge-in work outside a headset.
*
* It is not always available (some aggregate and virtual devices refuse it), so
* failure to enable it is reported as a DISTINCT return value (2) rather than
* folded into success. The caller needs to know, because the correct response
* is to raise the VAD floor, and a caller that thinks AEC is on will set that
* floor far too low.
*
* The tap keeps only a running short-window RMS in a static behind a mutex.
* Deliberately not a queue of samples: this path is polled at ~50 Hz by a loop
* that only ever asks "is someone talking", and buffering audio nobody reads
* would be an unbounded allocation in the middle of a conversation.
* */
static AVAudioEngine* g_mon_engine = nil;
static int g_mon_running = 0;
static int g_mon_code = 0; /* what the successful start reported */
static double g_mon_rms = 0.0;
static pthread_mutex_t g_mon_lock = PTHREAD_MUTEX_INITIALIZER;
el_val_t mic_monitor_start(void) {
/* Idempotent, and it re-reports the ORIGINAL code rather than a bare 1: a
* caller that starts twice must not be told AEC is on when the first start
* already discovered it was not. */
if (g_mon_running) return (el_val_t)g_mon_code;
if (!el_cap_authorized(AVMediaTypeAudio)) return (el_val_t)0;
AVAudioEngine* engine = nil;
AVAudioInputNode* input = nil;
AVAudioFormat* fmt = nil;
int aec = 0;
int tapped = 0;
@try {
engine = [[AVAudioEngine alloc] init];
input = [engine inputNode];
} @catch (NSException* e) {
(void)e;
return (el_val_t)0;
}
if (!input) return (el_val_t)0;
/* Enable AEC BEFORE reading the format: the voice-processing unit imposes
* its own input format, and a tap installed with the pre-VP format would be
* rejected at start. */
@try {
NSError* vpErr = nil;
if ([input respondsToSelector:@selector(setVoiceProcessingEnabled:error:)]) {
aec = [input setVoiceProcessingEnabled:YES error:&vpErr] ? 1 : 0;
}
} @catch (NSException* e) {
(void)e;
aec = 0;
}
@try {
fmt = [input inputFormatForBus:0];
} @catch (NSException* e) {
(void)e;
return (el_val_t)0;
}
if (!fmt || fmt.sampleRate <= 0 || fmt.channelCount == 0) return (el_val_t)0;
el_cap_disclose(aec
? "MIC: opening the microphone for live monitoring, echo-cancelled (local)."
: "MIC: opening the microphone for live monitoring, NO echo cancellation (local).");
@try {
[input installTapOnBus:0
bufferSize:1024
format:fmt
block:^(AVAudioPCMBuffer* _Nonnull buf, AVAudioTime* _Nonnull when) {
(void)when;
if (!buf) return;
AVAudioFrameCount n = buf.frameLength;
if (n == 0) return;
double sum = 0.0;
/* Whatever the VP unit hands back — float32 is the norm, int16 and
* int32 are possible on odd hardware normalise to -1..1 so the
* Float El sees means the same thing on every device. */
if (buf.floatChannelData) {
const float* ch = buf.floatChannelData[0];
for (AVAudioFrameCount i = 0; i < n; i++) sum += (double)ch[i] * (double)ch[i];
} else if (buf.int16ChannelData) {
const int16_t* ch = buf.int16ChannelData[0];
for (AVAudioFrameCount i = 0; i < n; i++) {
double v = (double)ch[i] / 32768.0;
sum += v * v;
}
} else if (buf.int32ChannelData) {
const int32_t* ch = buf.int32ChannelData[0];
for (AVAudioFrameCount i = 0; i < n; i++) {
double v = (double)ch[i] / 2147483648.0;
sum += v * v;
}
} else {
return;
}
double rms = sqrt(sum / (double)n);
if (rms < 0.0) rms = 0.0;
if (rms > 1.0) rms = 1.0;
pthread_mutex_lock(&g_mon_lock);
g_mon_rms = rms;
pthread_mutex_unlock(&g_mon_lock);
}];
tapped = 1;
[engine prepare];
NSError* startErr = nil;
if (![engine startAndReturnError:&startErr]) {
[input removeTapOnBus:0];
return (el_val_t)0;
}
} @catch (NSException* e) {
(void)e;
@try { if (tapped) [input removeTapOnBus:0]; } @catch (NSException* e2) { (void)e2; }
@try { [engine stop]; } @catch (NSException* e2) { (void)e2; }
return (el_val_t)0;
}
pthread_mutex_lock(&g_mon_lock);
g_mon_rms = 0.0;
pthread_mutex_unlock(&g_mon_lock);
g_mon_engine = engine;
g_mon_running = 1;
g_mon_code = aec ? 1 : 2;
return (el_val_t)g_mon_code;
}
/* Float in 0..1. Reads the last window the tap computed; never blocks on the
* audio thread beyond the mutex, because this is polled inside a turn-taking
* loop where a stall IS a missed barge-in. */
el_val_t mic_monitor_rms(void) {
double rms = 0.0;
pthread_mutex_lock(&g_mon_lock);
rms = g_mon_rms;
pthread_mutex_unlock(&g_mon_lock);
return el_from_float(rms);
}
el_val_t mic_monitor_stop(void) {
AVAudioEngine* engine = g_mon_engine;
g_mon_engine = nil;
g_mon_running = 0;
g_mon_code = 0;
if (engine) {
@try { [[engine inputNode] removeTapOnBus:0]; } @catch (NSException* e) { (void)e; }
@try { [engine stop]; } @catch (NSException* e) { (void)e; }
}
pthread_mutex_lock(&g_mon_lock);
g_mon_rms = 0.0;
pthread_mutex_unlock(&g_mon_lock);
return (el_val_t)1;
}
/* ════════════════════════════════════════════════════════════════════════════
* CAMERA
* */
static void el_cap_free_bitmap(void* info, const void* data, size_t size) {
(void)info; (void)size;
free((void*)data);
}
/* CVPixelBuffer -> CGImage, own-core, no CoreImage.
*
* The output is pinned to 32BGRA at the AVCaptureVideoDataOutput (see below)
* precisely so this conversion can be a memcpy and a CGImageCreate. The
* alternative accepting the camera's native 2vuy/420v and colour-converting
* here would mean either pulling in CoreImage or writing a YUV->RGB matrix in
* the file that is supposed to contain no arithmetic. Asking the capture output
* for BGRA moves that work into AVFoundation, where it is already written and
* already hardware-accelerated.
*
* The rows are copied out rather than aliased because the CVPixelBuffer is
* recycled by the capture session the moment the delegate returns; a CGImage
* pointing at it would be pointing at the NEXT frame by the time anyone looked. */
static CGImageRef el_cap_cgimage_from_pixelbuffer(CVPixelBufferRef pb) {
if (!pb) return NULL;
if (CVPixelBufferGetPixelFormatType(pb) != kCVPixelFormatType_32BGRA) return NULL;
if (CVPixelBufferLockBaseAddress(pb, kCVPixelBufferLock_ReadOnly) != kCVReturnSuccess) return NULL;
size_t w = CVPixelBufferGetWidth(pb);
size_t h = CVPixelBufferGetHeight(pb);
size_t src_bpr = CVPixelBufferGetBytesPerRow(pb);
const uint8_t* base = (const uint8_t*)CVPixelBufferGetBaseAddress(pb);
CGImageRef img = NULL;
if (base && w > 0 && h > 0 && src_bpr >= w * 4) {
size_t dst_bpr = w * 4;
uint8_t* copy = (uint8_t*)malloc(dst_bpr * h);
if (copy) {
for (size_t y = 0; y < h; y++) {
memcpy(copy + y * dst_bpr, base + y * src_bpr, dst_bpr);
}
CGDataProviderRef dp =
CGDataProviderCreateWithData(NULL, copy, dst_bpr * h, el_cap_free_bitmap);
if (dp) {
CGColorSpaceRef cs = CGColorSpaceCreateDeviceRGB();
if (cs) {
img = CGImageCreate(w, h, 8, 32, dst_bpr, cs,
(CGBitmapInfo)(kCGBitmapByteOrder32Little |
kCGImageAlphaNoneSkipFirst),
dp, NULL, false, kCGRenderingIntentDefault);
CGColorSpaceRelease(cs);
}
CGDataProviderRelease(dp); /* provider owns `copy` from here */
} else {
free(copy);
}
}
}
CVPixelBufferUnlockBaseAddress(pb, kCVPixelBufferLock_ReadOnly);
return img;
}
/* The frame delegate. AVCaptureVideoDataOutput is used rather than
* AVCapturePhotoOutput for the same reason periph.swift used it: the photo path
* wants KVO and a session owned by an app object, and this runs in a plain CLI
* process with no run loop it can assume. A data output just calls back.
*
* The first frames are dropped on purpose. A camera that has just been powered
* on is still converging exposure and white balance, and the first frame is
* reliably darker and greener than the room. El's scene-geometry descriptors
* are brightness and mean-colour statistics, so handing up an unsettled frame
* would not produce a slightly worse answer, it would produce a confidently
* wrong one. */
@interface ElCapFrameGrabber : NSObject <AVCaptureVideoDataOutputSampleBufferDelegate> {
CGImageRef _img;
int _seen;
dispatch_semaphore_t _sem;
}
- (dispatch_semaphore_t)sem;
- (CGImageRef)takeImage; /* transfers ownership to the caller */
@end
@implementation ElCapFrameGrabber
- (instancetype)init {
self = [super init];
if (self) {
_img = NULL;
_seen = 0;
_sem = dispatch_semaphore_create(0);
}
return self;
}
- (dispatch_semaphore_t)sem { return _sem; }
- (CGImageRef)takeImage {
CGImageRef out = _img;
_img = NULL;
return out;
}
- (void)dealloc {
if (_img) { CGImageRelease(_img); _img = NULL; }
}
/* Runs on the serial delegate queue, so no lock is needed among callbacks; the
* waiter only reads _img after the semaphore has been signalled AND the session
* has been stopped, which orders it after the last callback. */
- (void)captureOutput:(AVCaptureOutput*)output
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection*)connection {
(void)output; (void)connection;
_seen++;
if (_img != NULL || _seen < 5) return; /* let exposure settle */
CVImageBufferRef pb = CMSampleBufferGetImageBuffer(sampleBuffer);
if (!pb) return;
CGImageRef img = el_cap_cgimage_from_pixelbuffer(pb);
if (!img) return;
_img = img;
dispatch_semaphore_signal(_sem);
}
@end
/* Bring the camera up, take exactly one settled frame, put it back down.
* Returns a +1 CGImageRef the caller releases, or NULL. Bounded at 10s: a
* camera held by another process, or one whose TCC grant was revoked between
* the check and the open, must fail rather than park. */
static CGImageRef el_cap_grab_frame(void) {
AVCaptureSession* session = nil;
AVCaptureVideoDataOutput* output = nil;
ElCapFrameGrabber* grabber = nil;
CGImageRef img = NULL;
@try {
AVCaptureDevice* dev = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
if (!dev) return NULL;
NSError* err = nil;
AVCaptureDeviceInput* in = [AVCaptureDeviceInput deviceInputWithDevice:dev error:&err];
if (!in) return NULL;
session = [[AVCaptureSession alloc] init];
session.sessionPreset = AVCaptureSessionPresetPhoto;
if (![session canAddInput:in]) return NULL;
[session addInput:in];
output = [[AVCaptureVideoDataOutput alloc] init];
output.alwaysDiscardsLateVideoFrames = YES;
/* Pin the pixel format so the CGImage conversion above stays a memcpy.
* Every macOS capture device advertises 32BGRA. */
output.videoSettings = @{ (id)kCVPixelBufferPixelFormatTypeKey :
@(kCVPixelFormatType_32BGRA) };
grabber = [[ElCapFrameGrabber alloc] init];
dispatch_queue_t q = dispatch_queue_create("el.capture.camera", DISPATCH_QUEUE_SERIAL);
[output setSampleBufferDelegate:grabber queue:q];
if (![session canAddOutput:output]) return NULL;
[session addOutput:output];
el_cap_disclose("CAMERA: opening the camera for one frame (local, never egresses).");
[session startRunning];
} @catch (NSException* e) {
(void)e;
@try { [session stopRunning]; } @catch (NSException* e2) { (void)e2; }
return NULL;
}
long timed_out = dispatch_semaphore_wait([grabber sem],
dispatch_time(DISPATCH_TIME_NOW, (int64_t)(10 * NSEC_PER_SEC)));
/* Stop first, then detach the delegate, then read. In that order the last
* callback has already returned by the time anyone touches the image. */
@try { [session stopRunning]; } @catch (NSException* e) { (void)e; }
@try { [output setSampleBufferDelegate:nil queue:NULL]; } @catch (NSException* e) { (void)e; }
if (timed_out == 0) img = [grabber takeImage];
return img;
}
el_val_t camera_available(void) {
if (!el_cap_authorized(AVMediaTypeVideo)) return (el_val_t)0;
return (el_val_t)(el_cap_device_present(AVMediaTypeVideo) ? 1 : 0);
}
el_val_t camera_request_access(void) {
return (el_val_t)(el_cap_request(AVMediaTypeVideo) ? 1 : 0);
}
/* Longest edge of the grid handed to El. 64 is not a resolution, it is a budget:
* a 1920x1080 frame is 6.2 MILLION packed RGB ints, and building that as an El
* list would cost more time and memory than everything El then does with it.
* The descriptors El computes over this mean colour, brightness, a 3x3
* luminance grid are region statistics, and region statistics do not get
* meaningfully better above a 64-wide grid. The TRUE frame dimensions are
* reported separately so nothing downstream has to guess what was thrown away. */
#define EL_CAP_GRID_MAX 64
/* One frame as Map{width, height, grid_w, grid_h, pixels:[Int]}.
*
* "pixels" is packed R,G,B with NO alpha three ints per grid cell, row-major
* from the TOP-LEFT. (CGBitmapContext lays its buffer out top row first and
* CGContextDrawImage does the flip, so row 0 here is the top of the frame, the
* same convention periph.swift's grid indexing assumed.) Alpha is dropped
* because a camera frame has none worth carrying and it would inflate the list
* by a third to say "opaque" six thousand times. */
el_val_t camera_capture_rgb(void) {
if (!el_cap_authorized(AVMediaTypeVideo)) return (el_val_t)0;
CGImageRef img = el_cap_grab_frame();
if (!img) return (el_val_t)0;
size_t w = CGImageGetWidth(img);
size_t h = CGImageGetHeight(img);
if (w == 0 || h == 0) { CGImageRelease(img); return (el_val_t)0; }
/* Preserve aspect ratio, longest edge capped. */
size_t gw = w, gh = h;
size_t longest = w > h ? w : h;
if (longest > EL_CAP_GRID_MAX) {
double s = (double)EL_CAP_GRID_MAX / (double)longest;
gw = (size_t)((double)w * s + 0.5);
gh = (size_t)((double)h * s + 0.5);
if (gw == 0) gw = 1;
if (gh == 0) gh = 1;
}
size_t bpr = gw * 4;
uint8_t* buf = (uint8_t*)calloc(1, bpr * gh);
if (!buf) { CGImageRelease(img); return (el_val_t)0; }
CGColorSpaceRef cs = CGColorSpaceCreateDeviceRGB();
CGContextRef ctx = cs ? CGBitmapContextCreate(buf, gw, gh, 8, bpr, cs,
(CGBitmapInfo)kCGImageAlphaPremultipliedLast)
: NULL;
if (cs) CGColorSpaceRelease(cs);
if (!ctx) { free(buf); CGImageRelease(img); return (el_val_t)0; }
/* Nearest-neighbour. This is a decimation for statistics, not a thumbnail
* for a human to look at; smoothing would only cost time and blur the very
* region boundaries the grid exists to measure. */
CGContextSetInterpolationQuality(ctx, kCGInterpolationNone);
CGContextDrawImage(ctx, CGRectMake(0, 0, (CGFloat)gw, (CGFloat)gh), img);
CGContextRelease(ctx);
CGImageRelease(img);
el_val_t pixels = el_list_empty();
for (size_t y = 0; y < gh; y++) {
const uint8_t* row = buf + y * bpr;
for (size_t x = 0; x < gw; x++) {
const uint8_t* p = row + x * 4; /* RGBA8, premultiplied-last */
pixels = el_list_append(pixels, (el_val_t)(int64_t)p[0]);
pixels = el_list_append(pixels, (el_val_t)(int64_t)p[1]);
pixels = el_list_append(pixels, (el_val_t)(int64_t)p[2]);
}
}
free(buf);
el_val_t m = el_map_new((el_val_t)0);
if (!m) return (el_val_t)0;
m = el_map_set(m, EL_STR("width"), (el_val_t)(int64_t)w);
m = el_map_set(m, EL_STR("height"), (el_val_t)(int64_t)h);
m = el_map_set(m, EL_STR("grid_w"), (el_val_t)(int64_t)gw);
m = el_map_set(m, EL_STR("grid_h"), (el_val_t)(int64_t)gh);
m = el_map_set(m, EL_STR("pixels"), pixels);
return m;
}
/* One frame to disk as JPEG, at FULL resolution — the opposite budget from
* camera_capture_rgb, and for the opposite reason. A file is not being walked
* element-by-element by an interpreter; it costs one ImageIO call and it is the
* artefact a human or a later pass will actually look at. The encoder is
* ImageIO's because a JPEG encoder is a codec, and re-implementing one in El
* would be a large amount of arithmetic that buys nothing: the point of keeping
* work in El is the reasoning, not the entropy coding. */
el_val_t camera_capture_jpeg(el_val_t path) {
const char* p = EL_CSTR(path);
if (!p || !*p) return (el_val_t)0;
if (!el_cap_authorized(AVMediaTypeVideo)) return (el_val_t)0;
CGImageRef img = el_cap_grab_frame();
if (!img) return (el_val_t)0;
int ok = 0;
@autoreleasepool {
NSString* ns = [NSString stringWithUTF8String:p];
NSURL* url = ns ? [NSURL fileURLWithPath:ns] : nil;
if (url) {
CGImageDestinationRef dst =
CGImageDestinationCreateWithURL((__bridge CFURLRef)url, CFSTR("public.jpeg"), 1, NULL);
if (dst) {
CGImageDestinationAddImage(dst, img, NULL);
ok = CGImageDestinationFinalize(dst) ? 1 : 0;
CFRelease(dst);
}
}
}
CGImageRelease(img);
return (el_val_t)ok;
}
#endif /* __APPLE__ */
-89
View File
@@ -1,89 +0,0 @@
/* el_peripheral_null.c — the no-device build of El's I/O organ.
*
* Every entry point declared in el_runtime.h's "Peripheral" block, implemented
* as an honest refusal. This is what a platform without an El audio/capture
* realizer links instead of el_audio_darwin.m + el_capture_darwin.m, so an El
* program that speaks or listens still COMPILES AND LINKS everywhere.
*
* The distinction that matters: these do not pretend. speaker_available() and
* mic_available() return 0, and every operation returns its failure sentinel.
* A program asking "can I speak here?" gets a truthful no, rather than a
* silence it would have to infer something from. Silent success is the failure
* mode this whole change exists to eliminate El spent this entire codebase's
* history writing WAV files full of zeros and reporting ok=true, and nobody
* caught it because nothing ever said "there is no sound here".
*
* Compiled INSTEAD OF the Darwin realizers, never alongside them the symbols
* are the same by design, which is the point: the El side never branches on
* platform, it branches on speaker_available().
*/
#include "el_runtime.h"
#if !defined(__APPLE__)
/* ── Speaker ─────────────────────────────────────────────────────────────── */
el_val_t speaker_available(void) { return (el_val_t)0; }
el_val_t speaker_name(void) { return EL_STR("none"); }
el_val_t speaker_play_pcm16(el_val_t samples, el_val_t sample_rate) {
(void)samples; (void)sample_rate;
return (el_val_t)0;
}
el_val_t speaker_play_wav(el_val_t path) {
(void)path;
return (el_val_t)0;
}
el_val_t speaker_play_pcm16_async(el_val_t samples, el_val_t sample_rate) {
(void)samples; (void)sample_rate;
return (el_val_t)0;
}
el_val_t speaker_play_wav_async(el_val_t path) {
(void)path;
return (el_val_t)0;
}
el_val_t speaker_pause(void) { return (el_val_t)0; }
el_val_t speaker_resume(void) { return (el_val_t)0; }
el_val_t speaker_playing(void) { return (el_val_t)0; }
el_val_t speaker_stop(void) { return (el_val_t)0; }
el_val_t speaker_played_frames(void) { return (el_val_t)0; }
/* WAV geometry is pure parsing and would work fine here, but reporting a
* duration for audio this build cannot play would invite a caller to sequence
* around a silence. Refuse consistently with the rest of the file. */
el_val_t wav_frames(el_val_t path) { (void)path; return (el_val_t)0; }
el_val_t wav_rate(el_val_t path) { (void)path; return (el_val_t)0; }
/* ── Microphone ──────────────────────────────────────────────────────────── */
el_val_t mic_available(void) { return (el_val_t)0; }
el_val_t mic_request_access(void) { return (el_val_t)0; }
/* Empty list, not 0: the contract says capture returns samples, and a caller
* iterating the result must find nothing rather than dereference a non-list. */
el_val_t mic_capture_pcm16(el_val_t seconds, el_val_t sample_rate) {
(void)seconds; (void)sample_rate;
return el_list_empty();
}
el_val_t mic_monitor_start(void) { return (el_val_t)0; }
el_val_t mic_monitor_rms(void) { return el_from_float(0.0); }
el_val_t mic_monitor_stop(void) { return (el_val_t)0; }
/* ── Camera ──────────────────────────────────────────────────────────────── */
el_val_t camera_available(void) { return (el_val_t)0; }
el_val_t camera_request_access(void) { return (el_val_t)0; }
el_val_t camera_capture_rgb(void) { return (el_val_t)0; }
el_val_t camera_capture_jpeg(el_val_t path) {
(void)path;
return (el_val_t)0;
}
#endif /* !__APPLE__ */
+18 -95
View File
@@ -19709,84 +19709,22 @@ void log_warn(el_val_t msg_v) {
* become a convention. */ * become a convention. */
static int el_singleton_fd = -1; static int el_singleton_fd = -1;
static char el_singleton_path[1024]; static char el_singleton_path[1024];
static char el_singleton_state[1024];
/* el_singleton_acquire — claim exclusive use of the guarded STATE, or refuse to static const char* el_singleton_dir(void) {
* start. Compiler-injected as the FIRST statement of main() for any program const char* d = getenv("EL_SINGLETON_DIR");
* whose `program` block declares `singleton:` (which must also declare if (d && *d) return d;
* `guards:` see lang/spec/language.md §18.2). d = getenv("TMPDIR");
* if (d && *d) return d;
* GUARD THE THING, NOT THE NAME. return "/tmp";
* }
* Until 2026-08-16 this lock was keyed on the program's NAME and on $TMPDIR
* `$EL_SINGLETON_DIR|$TMPDIR|/tmp` + `/el-singleton-<name>.lock` and never /* el_singleton_acquire — claim exclusive process identity, or refuse to start.
* consulted the state it claimed to protect. Its own refusal message said * Compiler-injected as the FIRST statement of main() for any program whose
* "Refusing to start a second instance against the same state" while it had not * `program` block declares `singleton:`. */
* looked at any state. Measured, it failed in BOTH directions: el_val_t el_singleton_acquire(el_val_t id_v) {
*
* - FALSE POSITIVE: two engrams against genuinely DIFFERENT data dirs could
* not coexist. The second was refused, naming the first's pid for sharing
* a name, not a store.
* - FALSE NEGATIVE (the dangerous one): `TMPDIR=/tmp/other` let a second
* instance start against the SAME data dir with no complaint. That is
* exactly the two-instance data-loss condition the guard exists to prevent,
* and the workaround was one environment variable.
*
* Both are one error: the identity of the resource had been replaced by a label
* for it. The fix is to put the lock file INSIDE the state it guards:
*
* <state>/.el-singleton-<id>.lock
*
* That placement is the whole mechanism, and it is why there is no hashing, no
* canonical-path registry, and no environment variable left to subvert:
*
* - Same directory => same file => same inode => the flock CONTENDS. There is
* no TMPDIR in the key, so there is nothing to change to get past it.
* - Different dirs => different files => no contention. Two stores are two
* stores; they were never in conflict and are no longer treated as if they
* were.
* - Different SPELLINGS of one directory trailing slash, `x/../x`, a symlink
* resolve to the same inode in the kernel's own path walk, so they contend
* without this code comparing strings at all. Path canonicalisation here is
* for the human-readable message, never for the decision.
*
* Kept, deliberately, from the version this replaces: it is an flock and not a
* pidfile (the kernel releases it on crash and on SIGKILL, so there is no stale
* state and therefore no "delete the lock file to get unstuck" ritual), and it
* reports the HOLDER'S PID (added because a stale process survived `pkill -f`
* and went on answering probes; "already running" is not actionable, a pid is).
*
* Changed: the message is now TRUE. It says "the same state" because the lock it
* failed to take lives in that state, and it names the state it checked. */
el_val_t el_singleton_acquire(el_val_t id_v, el_val_t state_v) {
const char* id = EL_CSTR(id_v); const char* id = EL_CSTR(id_v);
if (!id || !*id) return EL_NULL; if (!id || !*id) return EL_NULL;
/* A singleton with nothing to guard is the defect this function exists to
* remove; refuse rather than silently fall back to name-keying. The compiler
* rejects `singleton:` without `guards:`, so reaching this is a toolchain
* mismatch, not a user mistake say so. */
const char* state = EL_CSTR(state_v);
if (!state || !*state) {
fprintf(stderr,
"[el] FATAL: singleton '%s' was given no state to guard.\n"
"[el] A lock keyed on a program's NAME instead of on the state it\n"
"[el] protects is not a guard: it refuses unrelated instances and\n"
"[el] permits concurrent ones. Declare `guards: <path>` alongside\n"
"[el] `singleton:` in the program block (spec §18.2).\n", id);
exit(1);
}
/* Canonicalise so the operator is told WHICH directory was checked, in one
* spelling, whatever spelling they typed. This is a readability measure, not
* the mechanism: realpath() may fail (the directory may not exist yet) and
* correctness must not depend on it when it succeeds it names the same
* directory, and when it does not we fall back to the path as given and the
* kernel's own path walk still collapses the spellings at open() time. */
char* rp = realpath(state, NULL);
snprintf(el_singleton_state, sizeof(el_singleton_state), "%s", rp ? rp : state);
free(rp);
/* Sanitise the id into a filename. */ /* Sanitise the id into a filename. */
char safe[256]; char safe[256];
size_t si = 0; size_t si = 0;
@@ -19797,25 +19735,13 @@ el_val_t el_singleton_acquire(el_val_t id_v, el_val_t state_v) {
safe[si++] = (char)(ok ? c : '-'); safe[si++] = (char)(ok ? c : '-');
} }
safe[si] = '\0'; safe[si] = '\0';
/* THE MECHANISM: the lock lives inside the state it guards. Two spellings of
* one directory name one file; two directories name two files. Note there is
* no $TMPDIR and no $EL_SINGLETON_DIR in this path the escape hatch that
* made the guard bypassable is gone because there is nowhere left to put it. */
snprintf(el_singleton_path, sizeof(el_singleton_path), snprintf(el_singleton_path, sizeof(el_singleton_path),
"%s/.el-singleton-%s.lock", el_singleton_state, safe); "%s/el-singleton-%s.lock", el_singleton_dir(), safe);
int fd = open(el_singleton_path, O_RDWR | O_CREAT, 0644); int fd = open(el_singleton_path, O_RDWR | O_CREAT, 0644);
if (fd < 0) { if (fd < 0) {
/* Unguardable state. Refusing is the only honest option: starting anyway fprintf(stderr, "[el] FATAL: singleton '%s': cannot open lock file %s: %s\n",
* would mean running unguarded against exactly the store the guard is id, el_singleton_path, strerror(errno));
* here to protect. */
fprintf(stderr,
"[el] FATAL: singleton '%s': cannot open the lock inside the state it guards.\n"
"[el] state: %s\n"
"[el] lock: %s (%s)\n"
"[el] The guarded directory must exist and be writable. Refusing to\n"
"[el] start unguarded against it.\n",
id, el_singleton_state, el_singleton_path, strerror(errno));
exit(1); exit(1);
} }
if (flock(fd, LOCK_EX | LOCK_NB) != 0) { if (flock(fd, LOCK_EX | LOCK_NB) != 0) {
@@ -19831,14 +19757,11 @@ el_val_t el_singleton_acquire(el_val_t id_v, el_val_t state_v) {
fprintf(stderr, "[el] FATAL: another instance of '%s' is already running", id); fprintf(stderr, "[el] FATAL: another instance of '%s' is already running", id);
if (holder > 0) fprintf(stderr, " (pid %ld)", holder); if (holder > 0) fprintf(stderr, " (pid %ld)", holder);
fprintf(stderr, ".\n" fprintf(stderr, ".\n"
"[el] state: %s\n"
"[el] lock: %s\n" "[el] lock: %s\n"
"[el] Refusing to start a second instance against the same\n" "[el] Refusing to start a second instance against the same\n"
"[el] state. Two writers against one store is data loss, not a\n" "[el] state. Stop the running one and VERIFY it is gone\n"
"[el] warning. Stop the running one and VERIFY it is gone\n" "[el] (ps -p <pid>) before retrying.\n",
"[el] (ps -p %ld) before retrying — or point this instance at a\n" el_singleton_path);
"[el] different state, which is permitted and is not refused.\n",
el_singleton_state, el_singleton_path, holder > 0 ? holder : (long)0);
close(fd); close(fd);
exit(1); exit(1);
} }
+1 -87
View File
@@ -80,92 +80,6 @@ void println(el_val_t s);
void print(el_val_t s); void print(el_val_t s);
el_val_t readline(void); el_val_t readline(void);
/* stderr counterpart of println (defined in el_seed.c). El could write to
* stdout and nowhere else, which is right for a program's RESULT and wrong for
* everything about how that result was produced. Disclosure especially has to
* leave on a stream the caller can separate from the answer: a program that
* announces "I am about to open the microphone" on stdout has corrupted its own
* output. Flushed on every call, so a disclosure reaches the terminal BEFORE
* the device it describes is touched rather than whenever the buffer drains. */
void eprintln(el_val_t s);
/* ── Peripheral: the speaker, the microphone, the camera ─────────────────────
*
* El's I/O organ. Implemented per platform in its OWN translation unit
* el_audio_darwin.m / el_capture_darwin.m on Darwin, el_peripheral_null.c
* everywhere else so El code that speaks or listens links on every platform
* and merely reports having no device where there isn't one. Declared here and
* deliberately NOT implemented in el_runtime.c: acquiring a device must not
* mean editing the middle of the language, the same rule the realizer registry
* follows for modalities.
*
* These are the ONLY parts of the organ that are not El. Everything above the
* sample buffer WAV encode/decode, LPC autocorrelation, Levinson-Durbin,
* formant extraction, source-filter resynthesis, the compact descriptors, the
* converse decision loop is arithmetic, and arithmetic belongs in El. What
* remains here is what El cannot express: handing a buffer to the DAC and
* waiting for it to drain, and asking the OS for frames off a capture device.
*
* Local by construction: none of these entry points has a network path. Samples
* and pixels go to and from local hardware and nowhere else. Consent is
* enforced ABOVE this layer in El (peripheral/src/organ.el) for the Neuron-level
* grant, and BELOW it by the OS for TCC; capture fails closed on either. */
/* Speaker (efferent). speaker_play_pcm16 BLOCKS until the audio has actually
* been played rather than merely queued, so a caller can sequence utterances
* without guessing durations and without clipping each tail. */
el_val_t speaker_available(void); /* 1 if a real speaker backs this build */
el_val_t speaker_name(void); /* backend id, e.g. "coreaudio-audioqueue" */
el_val_t speaker_play_pcm16(el_val_t samples, el_val_t sample_rate); /* [Int] 16-bit mono; 1 ok */
el_val_t speaker_play_wav(el_val_t path); /* 16-bit mono RIFF/WAVE; 1 ok */
/* Asynchronous playback — required by converse, which must keep listening while
* it speaks and must be able to stop ON THE SPOT mid-buffer. A blocking play
* cannot be interrupted, and "finish the current buffer" is not barge-in.
* speaker_stop() halts output immediately; speaker_playing() reports whether
* the hardware is still going; speaker_played_frames() is how far it actually
* got, which is what makes an interrupted utterance resumable at the sample. */
el_val_t speaker_play_pcm16_async(el_val_t samples, el_val_t sample_rate);
el_val_t speaker_play_wav_async(el_val_t path);
el_val_t speaker_pause(void); /* stop AT THIS SAMPLE, keep position */
el_val_t speaker_resume(void); /* carry on from exactly there */
el_val_t speaker_playing(void);
el_val_t speaker_stop(void);
el_val_t speaker_played_frames(void);
/* WAV geometry without playing — wav-info, and the segment duration converse
* needs to turn elapsed time into progress. */
el_val_t wav_frames(el_val_t path);
el_val_t wav_rate(el_val_t path);
/* Microphone (afferent). Fails CLOSED: returns 0 unless the OS has granted
* capture access. mic_capture_pcm16 blocks for `seconds` and returns an [Int]
* of 16-bit mono samples at `sample_rate` the raw stream is handed to El and
* never written anywhere by this layer. mic_available() reports device +
* permission state without prompting. */
el_val_t mic_available(void); /* 1 device present AND OS-authorized */
el_val_t mic_request_access(void); /* prompt once; 1 if granted */
el_val_t mic_capture_pcm16(el_val_t seconds, el_val_t sample_rate); /* [Int], empty on refusal */
/* Live monitoring for full-duplex converse. mic_monitor_start enables the OS
* voice-processing unit (acoustic echo cancellation) so the microphone does not
* hear the speaker without AEC, Neuron barges in on its own voice and
* turn-taking is unusable in a real room. mic_monitor_rms returns the current
* short-window RMS as a Float in 0..1. */
el_val_t mic_monitor_start(void); /* 1 ok; 2 = started but AEC unavailable */
el_val_t mic_monitor_rms(void); /* Float */
el_val_t mic_monitor_stop(void);
/* Camera (afferent). Fails CLOSED like the microphone. camera_capture_rgb
* returns a Map with width/height and the frame as an [Int] of packed RGB
* bytes, so the descriptor arithmetic can happen in El rather than here.
* camera_capture_jpeg writes an encoded frame via ImageIO, which is a codec and
* not something El should re-implement. */
el_val_t camera_available(void);
el_val_t camera_request_access(void);
el_val_t camera_capture_rgb(void); /* Map{width,height,pixels:[Int]} or 0 */
el_val_t camera_capture_jpeg(el_val_t path); /* 1 ok */
/* ── String builtins ─────────────────────────────────────────────────────── */ /* ── String builtins ─────────────────────────────────────────────────────── */
el_val_t el_str_concat(el_val_t a, el_val_t b); el_val_t el_str_concat(el_val_t a, el_val_t b);
@@ -1177,7 +1091,7 @@ el_val_t __env_get(el_val_t key);
* All three are COMPILER-INJECTED at the head of main() they are not meant to * All three are COMPILER-INJECTED at the head of main() they are not meant to
* be written by hand, which is the point: the guarantee cannot be forgotten at a * be written by hand, which is the point: the guarantee cannot be forgotten at a
* call site because there is no call site. */ * call site because there is no call site. */
el_val_t el_singleton_acquire(el_val_t id, el_val_t state); /* §18.2 process identity — keyed on the guarded state */ el_val_t el_singleton_acquire(el_val_t id); /* §18.1 process identity */
el_val_t el_config_declare(el_val_t name, el_val_t type, el_val_t el_config_declare(el_val_t name, el_val_t type,
el_val_t deflt, el_val_t has_default, el_val_t deflt, el_val_t has_default,
el_val_t required); /* §18.2 config schema */ el_val_t required); /* §18.2 config schema */
+3 -130
View File
@@ -154,18 +154,9 @@ static void seed_request_start(void) {
* file still links on its own. */ * file still links on its own. */
__attribute__((weak)) void el_str_cache_flush(void); __attribute__((weak)) void el_str_cache_flush(void);
/* Byte-buffer capacity registry (defined below, next to the string
* primitives). The arena frees the pointers it tracked, so any capacity
* entry for those addresses must go with them otherwise a later malloc
* reusing the address would inherit a stale width. */
static void seed_cap_drop(const char* p);
static void seed_request_end(void) { static void seed_request_end(void) {
_seed_arena_on = 0; _seed_arena_on = 0;
for (size_t i = 0; i < _seed_arena.count; i++) { for (size_t i = 0; i < _seed_arena.count; i++) free(_seed_arena.ptrs[i]);
seed_cap_drop(_seed_arena.ptrs[i]);
free(_seed_arena.ptrs[i]);
}
_seed_arena.count = 0; _seed_arena.count = 0;
if (el_str_cache_flush) el_str_cache_flush(); /* freed pointers may be reused */ if (el_str_cache_flush) el_str_cache_flush(); /* freed pointers may be reused */
} }
@@ -197,114 +188,6 @@ static char* seed_strbuf(size_t n) {
static el_val_t seed_wrap_str(char* s) { return EL_STR(s); } static el_val_t seed_wrap_str(char* s) { return EL_STR(s); }
/* ── Byte-buffer capacity registry ────────────────────────────────────────────
* A String produced by __str_alloc is a fixed-size BYTE BUFFER, not text. Its
* length is the capacity it was asked for; strlen() is meaningless on it,
* because the buffer is zero-filled and binary content (PCM audio, RIFF
* headers, image rasters) contains NUL bytes by nature.
*
* Before this registry existed, __str_set_char bounds-checked the write index
* against strlen(p). For a freshly __str_alloc'd buffer strlen(p) == 0, so the
* check `idx >= len` rejected EVERY index and the function was a total no-op:
* every El program that built bytes this way wrote a file of pure zeros and
* still saw a success return. That is why El's own-core WAV writer emitted
* 55,244 silent bytes with a correct-looking header length and no header.
*
* The fix cannot be "trust the index", because that removes the bound. It also
* cannot be a length header stored behind the pointer, because __str_set_char
* accepts any String including a string literal in .rodata, where reading the
* bytes preceding the pointer is undefined and may fault. So capacity is kept
* in a side table keyed by the pointer itself: allocation registers, the arena
* sweep unregisters, and anything not registered keeps the exact strlen
* behaviour it had before. Text semantics are unchanged; byte buffers gain the
* bound they always should have had. */
typedef struct {
char* ptr; /* NULL = empty slot, (char*)1 = tombstone */
size_t cap;
} SeedCapEntry;
#define SEED_CAP_TOMB ((char*)1)
static _Thread_local SeedCapEntry* _seed_cap = NULL;
static _Thread_local size_t _seed_cap_mask = 0; /* table size - 1 */
static _Thread_local size_t _seed_cap_used = 0; /* live + tombstoned */
static size_t seed_cap_hash(const char* p) {
uintptr_t h = (uintptr_t)p >> 4; /* malloc alignment: low bits are dead */
h *= (uintptr_t)0x9E3779B97F4A7C15ull;
return (size_t)(h >> 32);
}
static void seed_cap_put(char* p, size_t cap);
static void seed_cap_grow(void) {
size_t old_size = _seed_cap_mask ? _seed_cap_mask + 1 : 0;
SeedCapEntry* old = _seed_cap;
size_t new_size = old_size ? old_size * 2 : 256;
SeedCapEntry* fresh = calloc(new_size, sizeof(SeedCapEntry));
if (!fresh) return; /* out of memory: keep old table */
_seed_cap = fresh;
_seed_cap_mask = new_size - 1;
_seed_cap_used = 0;
for (size_t i = 0; i < old_size; i++) {
if (old[i].ptr && old[i].ptr != SEED_CAP_TOMB) seed_cap_put(old[i].ptr, old[i].cap);
}
free(old);
}
static void seed_cap_put(char* p, size_t cap) {
if (!p) return;
if (!_seed_cap || (_seed_cap_used + 1) * 4 >= (_seed_cap_mask + 1) * 3) {
seed_cap_grow();
if (!_seed_cap) return;
}
size_t i = seed_cap_hash(p) & _seed_cap_mask;
size_t first_free = (size_t)-1;
for (;;) {
char* e = _seed_cap[i].ptr;
if (e == p) { _seed_cap[i].cap = cap; return; } /* address reused */
if (e == SEED_CAP_TOMB && first_free == (size_t)-1) first_free = i;
if (!e) {
if (first_free != (size_t)-1) i = first_free; else _seed_cap_used++;
_seed_cap[i].ptr = p;
_seed_cap[i].cap = cap;
return;
}
i = (i + 1) & _seed_cap_mask;
}
}
/* Capacity of a registered byte buffer, or -1 when the pointer is not one. */
static int64_t seed_cap_get(const char* p) {
if (!p || !_seed_cap) return -1;
size_t i = seed_cap_hash(p) & _seed_cap_mask;
for (;;) {
char* e = _seed_cap[i].ptr;
if (!e) return -1;
if (e == (char*)p) return (int64_t)_seed_cap[i].cap;
i = (i + 1) & _seed_cap_mask;
}
}
static void seed_cap_drop(const char* p) {
if (!p || !_seed_cap) return;
size_t i = seed_cap_hash(p) & _seed_cap_mask;
for (;;) {
char* e = _seed_cap[i].ptr;
if (!e) return;
if (e == (char*)p) { _seed_cap[i].ptr = SEED_CAP_TOMB; return; }
i = (i + 1) & _seed_cap_mask;
}
}
/* Effective addressable length of a String: its buffer capacity when it is a
* byte buffer, otherwise strlen. */
static int64_t seed_addressable_len(const char* p) {
int64_t cap = seed_cap_get(p);
return cap >= 0 ? cap : (int64_t)strlen(p);
}
/* ── String primitives ───────────────────────────────────────────────────── */ /* ── String primitives ───────────────────────────────────────────────────── */
el_val_t __str_len(el_val_t s) { el_val_t __str_len(el_val_t s) {
@@ -316,7 +199,7 @@ el_val_t __str_len(el_val_t s) {
el_val_t __str_char_at(el_val_t s, el_val_t i) { el_val_t __str_char_at(el_val_t s, el_val_t i) {
const char* p = EL_CSTR(s); const char* p = EL_CSTR(s);
if (!p) return 0; if (!p) return 0;
int64_t len = seed_addressable_len(p); /* capacity for byte buffers */ int64_t len = (int64_t)strlen(p);
int64_t idx = (int64_t)i; int64_t idx = (int64_t)i;
if (idx < 0 || idx >= len) return 0; if (idx < 0 || idx >= len) return 0;
return (el_val_t)(unsigned char)p[idx]; return (el_val_t)(unsigned char)p[idx];
@@ -327,14 +210,13 @@ el_val_t __str_alloc(el_val_t n) {
if (sz < 0) sz = 0; if (sz < 0) sz = 0;
char* buf = seed_strbuf((size_t)sz); char* buf = seed_strbuf((size_t)sz);
memset(buf, 0, (size_t)sz + 1); memset(buf, 0, (size_t)sz + 1);
seed_cap_put(buf, (size_t)sz); /* this is a byte buffer of width sz */
return seed_wrap_str(buf); return seed_wrap_str(buf);
} }
el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c) { el_val_t __str_set_char(el_val_t s, el_val_t i, el_val_t c) {
char* p = (char*)(uintptr_t)s; char* p = (char*)(uintptr_t)s;
if (!p) return s; if (!p) return s;
int64_t len = seed_addressable_len(p); /* capacity for byte buffers */ int64_t len = (int64_t)strlen(p);
int64_t idx = (int64_t)i; int64_t idx = (int64_t)i;
if (idx < 0 || idx >= len) return s; if (idx < 0 || idx >= len) return s;
p[idx] = (char)(unsigned char)(int64_t)c; p[idx] = (char)(unsigned char)(int64_t)c;
@@ -524,15 +406,6 @@ el_val_t __fs_mkdir(el_val_t path) {
return 1; return 1;
} }
/* stderr counterpart of println. Flushed immediately: a disclosure line is only
* worth anything if it lands before the thing it discloses happens. */
void eprintln(el_val_t s) {
const char* p = EL_CSTR(s);
fputs(p ? p : "", stderr);
fputc('\n', stderr);
fflush(stderr);
}
el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n) { el_val_t __fs_write_bytes(el_val_t path, el_val_t bytes, el_val_t n) {
const char* p = EL_CSTR(path); const char* p = EL_CSTR(path);
const char* b = EL_CSTR(bytes); const char* b = EL_CSTR(bytes);
+7 -36
View File
@@ -1143,7 +1143,6 @@ The `program` block is where a concern of this shape is declared once and enforc
``` ```
program "engram" { program "engram" {
singleton: "engram" singleton: "engram"
guards: engram_resolve_data_dir()
env ENGRAM_BIND: String = ":8742" env ENGRAM_BIND: String = ":8742"
env GUIDE_PORT: Int = "8771" env GUIDE_PORT: Int = "8771"
env ENGRAM_API_KEY: String required env ENGRAM_API_KEY: String required
@@ -1156,50 +1155,24 @@ Grammar:
```ebnf ```ebnf
program_block = "program" string "{" { program_field } "}" ; program_block = "program" string "{" { program_field } "}" ;
program_field = singleton_field | guards_field | env_field ; program_field = singleton_field | env_field ;
singleton_field = "singleton" ":" string [ "," ] ; singleton_field = "singleton" ":" string [ "," ] ;
guards_field = "guards" ":" expr [ "," ] ;
env_field = "env" ident ":" type env_field = "env" ident ":" type
[ "=" string ] [ "required" ] [ "," ] ; [ "=" string ] [ "required" ] [ "," ] ;
``` ```
`singleton`, `guards` and `env` are **not** reserved words. They are read as identifier token values by the block's own parse loop, so they remain usable as ordinary identifiers everywhere else. `program` is the only keyword this section adds. `singleton` and `env` are **not** reserved words. They are read as identifier token values by the block's own parse loop, so they remain usable as ordinary identifiers everywhere else. `program` is the only keyword this section adds.
### 18.2 Process identity — `singleton` and `guards` ### 18.2 Process identity — `singleton`
`singleton: "id"` with `guards: <expr>` compiles to `el_singleton_acquire("id", <expr>)`, injected as the **first statement of `main()`**, before any user statement runs. `<expr>` evaluates to the path of the **state** the singleton protects. `singleton: "id"` compiles to an `el_singleton_acquire("id")` call injected as the **first statement of `main()`**, before any user statement runs.
**`guards:` is mandatory.** A `singleton:` without one is a compile error. This is not defensive strictness; it is the correction of a defect measured in this tree on 2026-08-16, and the rule the rest of this section exists to state: The runtime takes an exclusive non-blocking `flock` on `<dir>/el-singleton-<id>.lock`, where `<dir>` is `$EL_SINGLETON_DIR`, else `$TMPDIR`, else `/tmp`. On success it writes its pid and holds the descriptor open for the life of the process. On contention it **refuses to start**: it reports the holder's pid, names the lock file, and exits 1.
> **Guard the thing, not the name.** A lock that protects state must be keyed on the state. Two properties are deliberate:
Until that date the lock was `<dir>/el-singleton-<id>.lock` where `<dir>` was `$EL_SINGLETON_DIR`, else `$TMPDIR`, else `/tmp`. It was keyed on the program's **name** and on a temp directory, and it never consulted the state it claimed to protect — while its own refusal message read *"Refusing to start a second instance against the same state."* Measured, it failed in **both** directions: - **It is a lock, not a pidfile.** The kernel releases an `flock` when the owning process dies — including on `SIGKILL` and on crash. There is therefore no stale-lock state, and so no "delete the lock file to get unstuck" recovery ritual. Such a ritual would itself be a convention, which is the thing this section exists to remove.
| Situation | Correct answer | Name-keyed lock gave |
|---|---|---|
| same data dir, same `$TMPDIR` | refuse | refuse ✅ |
| same data dir, different `$TMPDIR` | refuse | **started** ❌ — the two-writer data-loss condition, defeated by one environment variable |
| different data dirs, same `$TMPDIR` | both start | **refused**, naming an unrelated pid ❌ |
| same dir spelled differently, different `$TMPDIR` | refuse | **started** ❌ |
Both failure directions are one error: the identity of a resource had been replaced by a label for it. The false negative is the dangerous one — a guard whose bypass is `TMPDIR=/tmp/other` is not a guard.
**The mechanism.** The lock file lives **inside the guarded directory**: `<state>/.el-singleton-<id>.lock`. The runtime takes an exclusive non-blocking `flock` on it, writes its pid, and holds the descriptor open for the life of the process.
That single placement decision is the whole fix, and it is why there is no hashing, no canonical-path registry, and no environment variable left to subvert:
- **Same directory** ⇒ same file ⇒ same inode ⇒ the `flock` contends. `$TMPDIR` is not in the key, so there is nothing to change to get past it. `$EL_SINGLETON_DIR` no longer exists.
- **Different directories** ⇒ different files ⇒ no contention. Two stores are two stores; they were never in conflict, and are no longer treated as if they were.
- **Different spellings of one directory** — trailing slash, `x/../x`, a symlink — resolve to the same inode during the kernel's own path walk, so they contend without this code comparing strings. Path canonicalisation happens only to make the diagnostic name one directory in one spelling; the *decision* never depends on it.
- **An unguardable state** — the directory is missing, or read-only — is a **refusal**, not a fallback. Starting unguarded against the store the guard exists to protect is the failure being removed.
**Why `guards:` is an expression and not a string.** The runtime cannot know, generically, which environment variable holds an arbitrary program's state; and a program whose state path already has an owner must not restate it. The engram's data dir is resolved by `engram_resolve_data_dir()`, which owns both the `$ENGRAM_DATA_DIR` read and the `$HOME/.neuron/engram` fallback (§18.4). Writing `guards: engram_resolve_data_dir()` points the guard at that owner. A `guards:` that took a string would force the path's default to be written down twice, and a guard that resolved the path its own way could end up locking a directory the program never writes to — the same two-owners defect §18.4 exists to prevent.
Three properties are deliberate:
- **It is a lock, not a pidfile.** The kernel releases an `flock` when the owning process dies — including on `SIGKILL` and on crash. There is therefore no stale-lock state, and so no "delete the lock file to get unstuck" recovery ritual. Such a ritual would itself be a convention, which is the thing this section exists to remove. (A lock file left behind inside a copied data directory — `cp -Rc` and friends — is inert: it carries no lock, only a stale pid string that the next holder overwrites.)
- **It reports the holder's pid.** "Already running" is not actionable. A pid is. This is the direct answer to the observed failure where a stale process survived a `pkill` and went on answering probes. - **It reports the holder's pid.** "Already running" is not actionable. A pid is. This is the direct answer to the observed failure where a stale process survived a `pkill` and went on answering probes.
- **The message is true.** It names the state it checked and the lock it failed to take, and it says "the same state" only because the lock it contended for is *in* that state. A diagnostic that asserts a check that did not happen is worse than no diagnostic: it is what let the name-keyed version read as correct for as long as it did.
Refusal is loud and total. It is not a warning, and the program does not continue degraded. This matters more than it looks: today a second engram whose `bind()` fails merely *returns* from `http_serve` — after it has already replayed the WAL and written boot-time backup files — and then exits **0**, indistinguishable from a clean run. `singleton` refuses before the first side effect. Refusal is loud and total. It is not a warning, and the program does not continue degraded. This matters more than it looks: today a second engram whose `bind()` fails merely *returns* from `http_serve` — after it has already replayed the WAL and written boot-time backup files — and then exits **0**, indistinguishable from a clean run. `singleton` refuses before the first side effect.
@@ -1221,8 +1194,6 @@ Some values look like configuration and are not. `ENGRAM_DATA_DIR` already has a
The rule: **a variable belongs in the program block when the block would be its only owner.** If a resolver already owns it, leave it there. The rule: **a variable belongs in the program block when the block would be its only owner.** If a resolver already owns it, leave it there.
This is also why `guards:` (§18.2) takes an expression: it lets the block *reference* the existing owner — `guards: engram_resolve_data_dir()` — rather than become a second one.
`HOME` is likewise not configuration. It is an environment fact, and stays a raw `env()` read. `HOME` is likewise not configuration. It is an environment fact, and stays a raw `env()` read.
--- ---
+61 -165
View File
@@ -1,184 +1,80 @@
# peripheral — Neuron's I/O organ, in El # peripheral — Neuron's I/O organ (own-core, local, consent-gated)
**El speaks.** The engram stores geometry and does not speak; the speaking The interface made physical. Two afferent senses in, one efferent voice out —
belongs to the language and its runtime. all reached the way the agentic surface reaches any tool.
Until this landed, the organ was a 939-line Swift program (`src/periph.swift`)
that shelled out to `afplay`. Neuron's mouth and ears were a separate binary
standing next to the language, and "speak" meant "ask that binary to speak."
That program is now **reference material, not the implementation.**
``` ```
SPEAKER (speak) efferent samples ──────────────► CoreAudio ──► the room MIC (hear) afferent device -> capture -> descriptor -> ingest -> geometry
MIC (hear) afferent device ──► samples ──► descriptor ──► engram CAMERA (see) afferent device -> capture -> descriptor -> ingest -> scene-geometry
CAMERA (see) afferent device ──► frame ──► descriptor ──► engram SPEAKER(speak) efferent render WAV -> PLAY ALOUD out the speaker
``` ```
## The split, and why it falls where it does Closes the conversational loop: **hear (mic) -> understand (engram) -> speak (speaker)**.
Exactly **two** things here are not El, and they are the two things El cannot
express as arithmetic:
| Not El (realizers) | Why |
|---|---|
| `lang/runtime/el_audio_darwin.m` | Handing a buffer to the DAC and waiting for it to drain. There is no way to say "the hardware has now played these samples" in El, and there should not be. |
| `lang/runtime/el_capture_darwin.m` | Asking the OS for samples off a microphone or frames off a camera, plus the TCC permission dance. |
**Everything else is El**, because everything else is arithmetic:
| In El | Where |
|---|---|
| WAV encode / decode (chunk-walking, JUNK/FLLR tolerant) | `src/organ_dsp.el`, `elp/src/speech.el` |
| LPC autocorrelation + Levinson-Durbin (order 16 @ 16 kHz) | `src/organ_dsp.el` |
| Formant extraction off the all-pole spectral envelope | `src/organ_dsp.el` |
| Source-filter resynthesis (glottal impulse train through the filter) | `src/organ_dsp.el` |
| Audio descriptor `[seconds, sr, ch, rms, peak, zcr, centroid, F0]` | `src/organ_dsp.el` |
| Voice descriptor `[F0, F1..F5, bandwidths]` | `src/organ_dsp.el` |
| Scene descriptor `[w, h, meanRGB, brightness, 3×3 luminance grid]` | `src/organ.el` |
| Consent, disclosure, the voice-from-engram fetch | `src/organ.el` |
| Barge-in, yield-or-hold, backchannel, resume | `src/organ_converse.el` |
| The command surface | `src/organ_cli.el` |
Both realizers are their **own translation units**, declared in
`lang/runtime/el_runtime.h`, and deliberately **not** patches to
`el_runtime.c`. Acquiring a device must not mean editing the middle of the
language — the same rule the realizer registry follows for modalities.
`lang/runtime/el_peripheral_null.c` provides the identical entry points
everywhere else, so El that speaks links on any platform and truthfully reports
having no speaker rather than going quietly silent.
## The voice comes from the engram
A voice is **geometry in the engram**, not a JSON file next to the code and
certainly not constants in a source file. The organ fetches it the way anything
retrieves a memory — it asks:
```el
let g: [Int] = organ_voice_fetch("will")
// [peripheral] VOICE: fetched 'will' FROM THE ENGRAM —
// f0=137 f0_end=116 kf=1269 f1=500 f2=2093 f3=3531
```
`organ_voice_fetch` issues an engram query and reads the geometry off the node
that comes back. Nothing opens a file. If the region is not in the graph it
returns **empty**, not a plausible default — a caller has to be able to tell
"this is how they sound" from "I never heard them."
The reverse direction is `ingest-voice`: an LPC voiceprint becomes a node, and
from then on the voice is a memory rather than a measurement someone wrote down.
## What the organ never does
**It never learns a word.** Pronunciation, vocabulary and phonemes belong to the
language faculty and are already built as ingested geometry — *the engram knows
how to pronounce*. The seam is `synth_codes(codes, voice, pmap)`: the codes and
the phoneme map arrive from the language side as geometry, and the organ's whole
job is turning them into samples and getting the samples out the speaker, plus
the same trip in reverse for the senses. There is no lexicon here and no
grapheme-to-phoneme rule, by design.
## Rails ## Rails
- **Own-core.** macOS-native only: AVFoundation (camera/mic), CoreAudio voice-
- **Own-core.** CoreAudio / AVFoundation / ImageIO — all ship with macOS. No processing (AEC), afplay (speaker), ImageIO/CoreGraphics (frames), hand-rolled
cloud, no model, no heavy dependency. There is **no network code in the organ DSP (WAV, LPC, formant synthesis). No cloud, no heavy deps.
at all**, by construction. - **Local-only.** Raw streams are written to `out/` and never egress. `.gitignore`
- **Local-only.** Raw streams stay on the machine. What leaves a capture is a keeps captured media out of git.
descriptor of a few dozen numbers. A 1920×1080 frame becomes 15 integers - **Consent-gated (two locks).** A Neuron-level grant (`grant`/`revoke`) *and* the
(~414,000× smaller); three seconds of audio becomes 8. OS TCC permission. Sensitive senses (camera/mic) fail closed without both.
- **Consent, two locks.** A Neuron-level grant **and** the OS TCC permission. - **Disclosed.** Every device touch prints a `[peripheral]` line on stderr.
Camera and mic **fail closed** without both. The speaker is disclosed but not
gated — you cannot secretly speak aloud, and gating it would mean Neuron needs
permission to answer.
- **Disclosed.** Every device touch prints a `[peripheral]` line on **stderr**
(via `eprintln`, flushed immediately), so a disclosure lands before the device
is touched and never contaminates the program's stdout.
## Build ## Build
```bash
./peripheral/build.sh /tmp/organ
``` ```
swiftc -O -o bin/periph src/periph.swift \
Concatenates the El modules, compiles with `elc`, links the two realizers. -framework AVFoundation -framework CoreMedia -framework Foundation \
Run it **from the repo root** or the `.psv` phoneme data will not resolve. -framework CoreGraphics -framework ImageIO -framework CoreImage
```
## Commands ## Commands
``` ```
organ grant|revoke <camera|mic> Neuron-level consent periph grant|revoke <camera|mic> # Neuron-level consent
organ status consent + device state periph status
organ speak <file.wav> play a WAV aloud (efferent) periph speak <file.wav> # SPEAK ALOUD (efferent)
organ tone [hz] [ms] synthesize and play — no file at all periph tone <out.wav> [hz] [sec] # own-core WAV synth
organ say <voice> <CODE> [CODE...] fetch voice FROM THE ENGRAM, render, speak periph listen <sec> <out.wav> # MIC capture (afferent), 16k mono
organ listen <sec> <out.wav> mic capture 16k mono (afferent) periph see <out.jpg> # CAMERA one frame (afferent)
organ see <out.jpg> one camera frame (afferent) periph feat-audio <wav> | feat-image <jpg> # capture -> compact descriptor
organ wav-info <file.wav> WAV geometry periph ingest-audio|ingest-image <file> <engramURL> # descriptor -> engram node (geometry)
organ feat-audio <file.wav> compact audio descriptor (8 numbers) periph voiceprint <voice.wav> # extract F0 + formants F1-F5
organ feat-image compact scene-geometry from the camera periph imitate <voice.wav> <out.wav> # speak back in that voice (LPC resynthesis)
organ voiceprint <voice.wav> F0 + formants F1-F5 (LPC) periph hear-imitate <sec> <out.wav> # MIC -> signature -> imitate -> SPEAK ALOUD
organ imitate <in.wav> <out.wav> LPC analysis-resynthesis periph converse <manifest.json> [--authority F] [--barge-at S[:backchannel|:bargein]] [--resume] [--live-mic]
organ hear-imitate <sec> <out.wav> mic -> signature -> imitate -> speak aloud
organ ingest-audio <file.wav> descriptor -> engram node (geometry)
organ ingest-voice <voice.wav> <n> voiceprint -> engram voice region
organ converse <manifest.json> [--authority PM] [--barge-at MS[:kind]] [--live-mic] [--resume]
``` ```
## Interruptibility ## The afferent metabolism
A capture is never shipped raw. It becomes a **compact descriptor** — the afferent
twin of the music instrument-signature:
- audio -> `[seconds, sr, ch, rms, peak, zcr, centroid, F0]` (~2400-6000x smaller)
- image -> `[w, h, meanRGB, brightness, 3x3 luminance grid]` (~400000x smaller)
- voice -> `[F0, F1..F5, bandwidths]` (11 numbers)
`converse` speaks an ordered, salience-tagged **meaning-plan** while listening: That descriptor is what the ingest organ (engram `POST /api/nodes`) turns into an
embedded node = geometry.
- **barge-in** — output stops at the sample, not at the end of the buffer. The ## Voice by imitation
realizer exposes `pause`/`resume` and reports `played_frames` (the real DAC `voiceprint`/`imitate` are own-core LPC (autocorrelation + Levinson-Durbin, order
position) precisely so this is possible. 16 @ 16 kHz), formant extraction from the LPC spectral envelope, and source-filter
- **yield-or-hold** — a decision, not a rule: `hold = salience·0.6 + resynthesis (glottal impulse train at F0 through the all-pole formant filter). A
progress·0.4`, and holding also requires that the interrupter not be voice is grabbed by ear as ~a dozen numbers and spoken back — **no training, no
high-authority. Otherwise yield, because the polite default is the right one. stolen voice.** Measured fidelity on real speech: resynthesized formants match the
- **backchannel** — "mm-hm" is brief and low-energy; resume seamlessly. source within 2-3%. The full phoneme->formant path for *novel* sentences is the
- **resumable** — on yield the remaining plan persists to `.resume.json`; speech faculty's seam (`elp` audio surface profile); this engine provides the
`--resume` picks the thread back up. An interruption should cost a turn, not formant synthesis primitive it renders through.
the content.
Live full-duplex uses `--live-mic` with the OS voice-processing unit (AEC) so ## Interruptibility (native turn-taking)
Neuron does not barge in on its own voice. `--barge-at` injects the event `converse` plays the utterance as an ordered, salience-tagged **meaning-plan**
deterministically for testing. while the mic listens (full-duplex, AEC on so it never barges in on its own voice):
- **barge-in**: user speech -> pause on the spot (sample-accurate), not "finish the buffer."
- **yield-or-hold**: a decision grounded in the current segment's salience + progress
+ the interrupter's authority — YIELD (stop) or HOLD ("hang on, let me finish").
- **backchannel** ("mm-hm"): brief/low -> keep going, resume seamlessly.
- **resumable**: on yield the remaining plan persists (`.resume.json`); `--resume`
picks the thread back up ("as I was saying").
## Measured against the Swift original Live full-duplex uses `--live-mic` (OS AEC). Injected `--barge-at` drives the
decision loop deterministically for testing.
Same input (`out/mic_room.wav`, 16 kHz mono, 48121 samples), Swift `periph` ```
vs the El organ: ```
| | Swift | El |
|---|---|---|
| seconds | 3.0075625 | 3.0076 |
| rms | 0.0047766496761 | 0.004777 |
| peak | 0.01806640625 | 0.018066 |
| zcr_hz | 416.28395087 | 416.2840 |
| centroid_hz | 727.60529169 | 727.6053 |
| f0_hz | 400 | 400.0000 |
| formants F1F5 | 1734.375 / 3343.75 / 3875 / 4359.375 / 4468.75 | identical |
| bandwidths B1B5 | 2000 / 2968.75 / 4203.125 / 4687.5 / 5000 | identical |
Agreement to every printed digit. `imitate` cannot match bit-for-bit because the
Swift excites unvoiced frames with `Double.random` — two Swift runs correlate
0.957 with **each other**; El correlates **0.958** with Swift. The port is as
close to the original as the original is to itself, and the deterministic prefix
is bit-identical.
## Honest status
- **Works:** speaker (CoreAudio, no `afplay`, no subprocess — verified: zero
`afplay`/Swift strings in the binary, no child process during playback), mic
capture, camera capture, all descriptors, LPC voiceprint, imitate,
hear-imitate, voice fetch/ingest against the engram, converse (yield, hold,
yield-to-authority, backchannel, resume — all exercised with real audio).
- **Coarse, and labelled so:** a fetched voice is one formant triple with no
coarticulation and no prosody. It is an impression, explicitly **not a
clone**, and `prov=COARSE` says so on the node.
- **Not verified here:** live `--live-mic` barge-in in a real room with a real
interrupter. The AEC path is implemented and the deterministic path is proven;
the acoustic behaviour is not something a headless run can establish.
- **Not in the engram yet:** the structured `Voice` / `VowelTarget` geometry
nodes live in the organ's own store and in snapshot files from earlier work,
but the **production engram does not carry them**. Getting them there is an
ingest, not a code change.
- `src/periph.swift` is kept as the reference the port was measured against.
-67
View File
@@ -1,67 +0,0 @@
#!/usr/bin/env bash
# build.sh — build the El organ.
#
# El has no import system on this path, so the modules are concatenated in
# dependency order (the same thing elp/tests/run.sh does) and handed to elc as
# one unit. The two device realizers are then linked in.
#
# MUST be run from the repo root, or the .psv phoneme geometry will not resolve
# and the render silently degrades.
set -uo pipefail
OUT="${1:-./peripheral/organ}"
REPO="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# Dependency order. The elp modules supply the render (synth_codes) and the
# phoneme-geometry read; the organ supplies everything else.
cat elp/src/voice-profile.el \
elp/src/accent.el \
elp/src/voice-ingest.el \
elp/src/speech-ingest.el \
elp/src/speech.el \
peripheral/src/organ.el \
peripheral/src/organ_dsp.el \
peripheral/src/organ_converse.el \
peripheral/src/organ_cli.el \
| grep -v '^import ' > "$WORK/organ.el"
cd "$REPO/lang"
./dist/platform/elc "$WORK/organ.el" > "$WORK/organ.c" || { echo "elc failed" >&2; exit 1; }
SSL_PREFIX="$(brew --prefix openssl@3 2>/dev/null || echo /usr/local)"
# The peripheral realizers are per-platform: Darwin gets the real devices,
# anything else gets el_peripheral_null.c and honestly reports having none.
case "$(uname)" in
Darwin)
# The Objective-C realizers are compiled SEPARATELY, with -fobjc-arc. The
# capture realizer is written against ARC (it holds AVFoundation objects);
# compiling it MRR silently changes its memory semantics, which on a device
# path shows up as a use-after-free under load rather than as an error here.
cc -std=c11 -fobjc-arc -O1 -I runtime -c runtime/el_audio_darwin.m -o "$WORK/el_audio.o" || exit 1
cc -std=c11 -fobjc-arc -O1 -I runtime -c runtime/el_capture_darwin.m -o "$WORK/el_capture.o" || exit 1
PERIPH_SRC="$WORK/el_audio.o $WORK/el_capture.o"
PERIPH_LIBS="-framework AudioToolbox -framework AVFoundation -framework CoreMedia
-framework CoreVideo -framework CoreGraphics -framework ImageIO
-framework Foundation"
;;
*)
PERIPH_SRC="runtime/el_peripheral_null.c"
PERIPH_LIBS=""
;;
esac
cc -O1 -I runtime -I"$SSL_PREFIX/include" -L"$SSL_PREFIX/lib" \
-o "$OUT" "$WORK/organ.c" \
runtime/el_runtime.c runtime/el_seed.c \
runtime/engram_cognition.c runtime/engram_geometry.c runtime/engram_reason.c \
runtime/engram_store.c runtime/engram_verify.c runtime/engram_vindex.c \
runtime/eg_cosine_batch.c runtime/eg_cosine_batch_strategy_cpu.c \
$PERIPH_SRC $PERIPH_LIBS \
-lcurl -lssl -lcrypto -lpthread -lm || { echo "link failed" >&2; exit 1; }
echo "built: $OUT"
-570
View File
@@ -1,570 +0,0 @@
// organ.el Neuron's I/O organ, in El.
//
// THE PRINCIPLE. El speaks. The engram stores geometry and does not speak.
// Before this file the organ was a 939-line Swift program standing next to the
// language (peripheral/src/periph.swift): Neuron's mouth and ears were a
// separate binary, and "speak" meant "shell out to that binary, which shells
// out to afplay." That is not a voice, it is a subprocess. The voice belongs to
// the language and its runtime.
//
// THE SPLIT. Exactly two things here are not El, and they are the two things El
// cannot express as arithmetic:
//
// the speaker handing a buffer to the DAC and waiting for it to drain
// the capture asking the OS for samples off a mic or frames off a camera
//
// Those live in lang/runtime/el_audio_darwin.m and el_capture_darwin.m, as
// their own translation units, declared in el_runtime.h. Everything ELSE that
// the Swift did WAV encode and decode, LPC autocorrelation, Levinson-Durbin,
// formant extraction off the all-pole envelope, source-filter resynthesis, the
// compact descriptors, the converse yield-or-hold decision is arithmetic, and
// arithmetic is El's. See organ_dsp.el for that half.
//
// WHERE THE VOICE COMES FROM. Not from this file, and not from a JSON manifest
// on disk. A voice is GEOMETRY IN THE ENGRAM, and the organ goes and gets it by
// asking the engram, the same way anything else asks the engram for anything:
// a query against the graph, then read the numbers off the node that comes
// back. organ_voice_fetch is that. The previous path, load_voice("...json"),
// parsed a file which quietly made the voice a build artifact instead of a
// memory. If the region is not in the graph, the honest answer is an empty
// result, not a default voice.
//
// WHAT THE ORGAN NEVER DOES. It never learns a word. Pronunciation, vocabulary
// and phonemes are the language faculty's, already built as ingested geometry
// "the engram knows how to pronounce." The seam is synth_codes(codes, voice,
// pmap): the codes and the phoneme map arrive from the language side as
// geometry, and the organ's whole job is turning them into samples and getting
// the samples out the speaker, plus the same trip in reverse for the senses.
//
// RAILS, all non-negotiable:
// own-core CoreAudio / AVFoundation / ImageIO, all shipped with the OS.
// No cloud, no model, no heavy dependency. There is no network
// call anywhere in the organ, by construction.
// local-only raw streams stay on the machine. What leaves a capture is a
// DESCRIPTOR of a few dozen numbers, never the stream.
// consent two locks on the sensitive senses: a Neuron-level grant AND
// the OS TCC permission. Camera and mic FAIL CLOSED without
// both. The speaker is disclosed but not gated (see below).
// disclosed every device touch prints a [peripheral] line on stderr.
// Nothing here is ever silent about being a device.
// Disclosure
//
// stderr, not stdout: a program that announces "I am opening the microphone" on
// stdout has corrupted its own output. And flushed immediately, so the line is
// on the terminal BEFORE the device is touched a disclosure that arrives
// after the fact is a log, not a disclosure.
fn organ_disclose(msg: String) -> Bool {
eprintln(" [peripheral] " + msg)
return true
}
// Consent, the Neuron-level lock
//
// The OS has its own lock (TCC) and it is not enough on its own: TCC grants the
// TERMINAL access to the microphone, once, more or less forever. That says the
// user trusts the app. It does not say the user consents to THIS program
// listening THIS time. So Neuron keeps its own grant, revocable, on the same
// footing and both must be open for a sensitive sense to work.
//
// Stored next to the organ rather than in the engram deliberately: consent must
// be inspectable and revocable without a running graph, and a permission that
// can only be revoked by the system it governs is not a permission.
fn organ_consent_path() -> String {
let home: String = env("PERIPH_HOME")
if str_eq(home, "") {
return "peripheral/.consent.json"
}
return home + "/.consent.json"
}
fn organ_consent_granted(device: String) -> Bool {
let raw: String = fs_read(organ_consent_path())
if str_eq(raw, "") {
return false
}
// A device is granted only on an explicit true. Anything unparseable,
// missing or malformed reads as NOT granted the failure direction for a
// permission file is always closed.
let key: String = "\"" + device + "\""
let at: Int = str_index_of(raw, key)
if at < 0 {
return false
}
let tail: String = str_slice(raw, at, str_len(raw))
let t: Int = str_index_of(tail, "true")
let f: Int = str_index_of(tail, "false")
if t < 0 {
return false
}
if f < 0 {
return true
}
// whichever token appears first after the key is this device's value
if t < f {
return true
}
return false
}
fn organ_consent_write(camera: Bool, mic: Bool) -> Bool {
let c: String = "false"
if camera {
c = "true"
}
let m: String = "false"
if mic {
m = "true"
}
return fs_write(organ_consent_path(), "{\"camera\": " + c + ", \"mic\": " + m + "}\n")
}
fn organ_grant(device: String) -> Bool {
let cam: Bool = organ_consent_granted("camera")
let mic: Bool = organ_consent_granted("mic")
if str_eq(device, "camera") {
cam = true
}
if str_eq(device, "mic") {
mic = true
}
let ok: Bool = organ_consent_write(cam, mic)
organ_disclose("granted '" + device + "' (Neuron-level) — raw stream stays local, never egresses.")
return ok
}
fn organ_revoke(device: String) -> Bool {
let cam: Bool = organ_consent_granted("camera")
let mic: Bool = organ_consent_granted("mic")
if str_eq(device, "camera") {
cam = false
}
if str_eq(device, "mic") {
mic = false
}
let ok: Bool = organ_consent_write(cam, mic)
organ_disclose("revoked '" + device + "' (Neuron-level).")
return ok
}
fn organ_consent_status() -> String {
let cam: String = "denied"
if organ_consent_granted("camera") {
cam = "granted"
}
let mic: String = "denied"
if organ_consent_granted("mic") {
mic = "granted"
}
return "camera=" + cam + " mic=" + mic
}
// Both locks, in order, with a disclosure for each outcome. Returns false and
// says exactly which lock is shut a refusal that does not say why is
// indistinguishable from a bug.
fn organ_may_listen() -> Bool {
if organ_consent_granted("mic") == false {
organ_disclose("CONSENT DENIED for 'mic' (Neuron-level). Run: organ grant mic")
return false
}
if mic_available() == 0 {
organ_disclose("CONSENT DENIED for 'mic' (OS/TCC), or no input device. Grant microphone access to this terminal in System Settings > Privacy.")
return false
}
organ_disclose("consent OK (Neuron + OS) for 'mic' — local only, never egresses.")
return true
}
fn organ_may_see() -> Bool {
if organ_consent_granted("camera") == false {
organ_disclose("CONSENT DENIED for 'camera' (Neuron-level). Run: organ grant camera")
return false
}
if camera_available() == 0 {
organ_disclose("CONSENT DENIED for 'camera' (OS/TCC), or no capture device. Grant camera access to this terminal in System Settings > Privacy.")
return false
}
organ_disclose("consent OK (Neuron + OS) for 'camera' — local only, never egresses.")
return true
}
// SPEAKER (efferent)
//
// Not consent-gated, and that is a deliberate asymmetry rather than an
// oversight. The microphone and camera take information OFF the user without
// them necessarily knowing; the speaker puts information INTO a room the user
// is in, audibly, which is self-disclosing by its nature you cannot secretly
// speak aloud. So the speaker is DISCLOSED (every utterance announces itself on
// stderr) but not gated. Gating it would mean Neuron needs permission to answer.
fn organ_speak_samples(samples: [Int], sr: Int) -> Bool {
let n: Int = native_list_len(samples)
if n <= 0 {
organ_disclose("SPEAKER: nothing to say (0 samples) — not touching the device.")
return false
}
if speaker_available() == 0 {
organ_disclose("SPEAKER: no audio output on this build (" + speaker_name() + ") — cannot speak.")
return false
}
let secs: Int = n * 1000 / sr
organ_disclose("SPEAKER: playing " + int_to_str(n) + " samples (" + int_to_str(secs) + " ms @ " + int_to_str(sr) + " Hz) ALOUD via " + speaker_name() + " (efferent).")
let ok: Int = speaker_play_pcm16(samples, sr)
if ok == 1 {
organ_disclose("SPEAKER: done — Neuron spoke aloud.")
return true
}
organ_disclose("SPEAKER: playback FAILED.")
return false
}
fn organ_speak_wav(path: String) -> Bool {
if speaker_available() == 0 {
organ_disclose("SPEAKER: no audio output on this build — cannot speak.")
return false
}
if fs_exists(path) == false {
organ_disclose("SPEAKER: no such file: " + path)
return false
}
organ_disclose("SPEAKER: playing '" + path + "' ALOUD via " + speaker_name() + " (efferent).")
let ok: Int = speaker_play_wav(path)
if ok == 1 {
organ_disclose("SPEAKER: done — Neuron spoke aloud.")
return true
}
organ_disclose("SPEAKER: playback FAILED.")
return false
}
// The voice, fetched FROM THE ENGRAM
//
// This is the part that matters most and is easiest to get subtly wrong. A
// voice is not a constant in code and it is not a JSON file next to the code
// it is a region of the graph, put there by having heard someone, and the organ
// retrieves it the way anything retrieves a memory: by asking.
//
// The node content is the geometry, in the engram's own flat key=value form:
// voice will | f0=137 f0_end=116 kf=1269 f1=500 f2=2093 f3=3531 ...
// so the read is: query the graph, take the returned node, pull the numbers off
// it. Nothing here opens a file.
//
// Returns [f0, f0_end, kf, f1, f2, f3], or an EMPTY list when the region is not
// in the graph. Empty is the honest answer a caller that gets no voice must
// not be handed a plausible default and left unable to tell the difference
// between "this is how they sound" and "I never heard them."
// Read an unsigned integer that follows `key` in `s`. Stops at the first
// non-digit, returns 0 when the key is absent.
fn organ_int_after(s: String, key: String) -> Int {
let at: Int = str_index_of(s, key)
if at < 0 {
return 0
}
let i: Int = at + str_len(key)
let n: Int = str_len(s)
let v: Int = 0
let seen: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
if c < 48 {
i = n
} else {
if c > 57 {
i = n
} else {
v = v * 10 + (c - 48)
seen = seen + 1
i = i + 1
}
}
}
if seen == 0 {
return 0
}
return v
}
// Ask the engram for a named voice region and read its geometry back.
fn organ_voice_fetch(name: String) -> [Int] {
let out: [Int] = native_list_empty()
let marker: String = "voice " + name + " |"
// The graph is asked by MEANING, not by id or by path.
let hits: String = engram_search_json("voice " + name + " f0 formants", 12)
let at: Int = str_index_of(hits, marker)
if at < 0 {
// Fall back to a scan of the resident graph before giving up: search is
// geometric and a small graph may not rank the region first.
let scan: String = engram_scan_nodes_json(500, 0)
at = str_index_of(scan, marker)
if at < 0 {
organ_disclose("VOICE: no region for '" + name + "' in the engram — nothing to speak with.")
return out
}
hits = scan
}
let win: String = str_slice(hits, at, at + 240)
out = native_list_append(out, organ_int_after(win, "f0="))
out = native_list_append(out, organ_int_after(win, "f0_end="))
out = native_list_append(out, organ_int_after(win, "kf="))
out = native_list_append(out, organ_int_after(win, "f1="))
out = native_list_append(out, organ_int_after(win, "f2="))
out = native_list_append(out, organ_int_after(win, "f3="))
organ_disclose("VOICE: fetched '" + name + "' FROM THE ENGRAM — f0=" + int_to_str(native_list_get(out, 0)) + " f0_end=" + int_to_str(native_list_get(out, 1)) + " kf=" + int_to_str(native_list_get(out, 2)) + " f1=" + int_to_str(native_list_get(out, 3)) + " f2=" + int_to_str(native_list_get(out, 4)) + " f3=" + int_to_str(native_list_get(out, 5)))
return out
}
// Put a measured voice INTO the engram as geometry. This is the afferent end of
// the same wire: a voiceprint (organ_dsp.el's LPC analysis) becomes a node, and
// from then on the voice is a memory rather than a measurement someone happened
// to write down. `prov` carries the honesty: COARSE means one formant triple, no
// coarticulation, no prosody an impression, explicitly not a clone.
fn organ_voice_ingest(name: String, f0: Int, f0_end: Int, kf: Int, f1: Int, f2: Int, f3: Int, src: String, prov: String) -> String {
let hub: String = engram_node("voice-signature-set " + name + " grounding=measured src=" + src, "VoiceSet", 90)
let body: String = "voice " + name + " | f0=" + int_to_str(f0) + " f0_end=" + int_to_str(f0_end) + " kf=" + int_to_str(kf) + " f1=" + int_to_str(f1) + " f2=" + int_to_str(f2) + " f3=" + int_to_str(f3) + " grounding=measured src=" + src + " prov=" + prov
let vid: String = engram_node(body, "Voice", 90)
engram_connect(hub, vid, 90, "has-signature")
organ_disclose("VOICE: ingested '" + name + "' into the engram as geometry (node " + vid + ").")
return vid
}
// Turn the fetched geometry into the voice slot-map the render consumes. Kept
// separate from the fetch so the organ never invents a voice: if the fetch came
// back empty this returns empty too, and the caller has to deal with it.
//
// The slot-map is built here rather than by calling the render's own
// constructor, so the organ carries NO dependency on the language faculty's
// modules it only has to agree with them about a wire format, which is the
// looser and more honest coupling. (The layout is the same key/value [String]
// convention lang_get / surface_get / voice_get all read.)
fn organ_voice_profile(name: String, g: [Int]) -> [String] {
let r: [String] = native_list_empty()
if native_list_len(g) < 6 {
return r
}
r = native_list_append(r, "name")
r = native_list_append(r, name)
r = native_list_append(r, "f0")
r = native_list_append(r, int_to_str(native_list_get(g, 0)))
r = native_list_append(r, "f0_end")
r = native_list_append(r, int_to_str(native_list_get(g, 1)))
r = native_list_append(r, "kf")
r = native_list_append(r, int_to_str(native_list_get(g, 2)))
r = native_list_append(r, "dur")
r = native_list_append(r, "1000")
r = native_list_append(r, "tilt")
r = native_list_append(r, "1000")
r = native_list_append(r, "breath")
r = native_list_append(r, "8")
return r
}
// Scene geometry (afferent, camera)
//
// The image half of the afferent metabolism, and the same principle as the
// audio descriptor: a frame is never handed on raw. The realizer returns a
// small pixel grid; THIS computes the descriptor, in El, because averaging
// pixels is arithmetic and arithmetic is not a device concern.
//
// Returns 15 numbers [w, h, meanR, meanG, meanB, brightness_pm, and a 3x3
// luminance grid] standing in for a multi-megapixel frame. The 3x3 grid is
// the smallest thing that still says WHERE the light is, which is most of what
// makes a scene comparable to another scene; a single brightness average would
// make a lamp on the left indistinguishable from a lamp on the right.
//
// Luminance is Rec. 601 (0.299R + 0.587G + 0.114B), in integer per-mille, so
// the descriptor is reproducible rather than subject to float drift.
fn organ_image_descriptor() -> [Int] {
let out: [Int] = native_list_empty()
let frame: Any = camera_capture_rgb()
if frame == 0 {
return out
}
let w: Int = el_map_get(frame, "width")
let h: Int = el_map_get(frame, "height")
let gw: Int = el_map_get(frame, "grid_w")
let gh: Int = el_map_get(frame, "grid_h")
let px: [Int] = el_map_get(frame, "pixels")
let np: Int = native_list_len(px)
if np < 3 {
return out
}
let count: Int = np / 3
let rsum: Int = 0
let gsum: Int = 0
let bsum: Int = 0
// 3x3 accumulators, row-major
let cell: [Int] = native_list_empty()
let cn: [Int] = native_list_empty()
let z: Int = 0
while z < 9 {
cell = native_list_append(cell, 0)
cn = native_list_append(cn, 0)
z = z + 1
}
// El has no list-set, so the cells are summed into parallel scalars and
// reassembled nine explicit accumulators would be worse to read than one
// pass per cell over a grid this small.
let c0: Int = 0
let c1: Int = 0
let c2: Int = 0
let c3: Int = 0
let c4: Int = 0
let c5: Int = 0
let c6: Int = 0
let c7: Int = 0
let c8: Int = 0
let n0: Int = 0
let n1: Int = 0
let n2: Int = 0
let n3: Int = 0
let n4: Int = 0
let n5: Int = 0
let n6: Int = 0
let n7: Int = 0
let n8: Int = 0
let i: Int = 0
while i < count {
let r: Int = native_list_get(px, i * 3)
let g: Int = native_list_get(px, i * 3 + 1)
let b: Int = native_list_get(px, i * 3 + 2)
rsum = rsum + r
gsum = gsum + g
bsum = bsum + b
let lum: Int = (299 * r + 587 * g + 114 * b) / 1000
let x: Int = i - (i / gw) * gw
let y: Int = i / gw
let cx: Int = x * 3 / gw
let cy: Int = y * 3 / gh
if cx > 2 {
cx = 2
}
if cy > 2 {
cy = 2
}
let idx: Int = cy * 3 + cx
if idx == 0 {
c0 = c0 + lum
n0 = n0 + 1
}
if idx == 1 {
c1 = c1 + lum
n1 = n1 + 1
}
if idx == 2 {
c2 = c2 + lum
n2 = n2 + 1
}
if idx == 3 {
c3 = c3 + lum
n3 = n3 + 1
}
if idx == 4 {
c4 = c4 + lum
n4 = n4 + 1
}
if idx == 5 {
c5 = c5 + lum
n5 = n5 + 1
}
if idx == 6 {
c6 = c6 + lum
n6 = n6 + 1
}
if idx == 7 {
c7 = c7 + lum
n7 = n7 + 1
}
if idx == 8 {
c8 = c8 + lum
n8 = n8 + 1
}
i = i + 1
}
let rA: Int = rsum / count
let gA: Int = gsum / count
let bA: Int = bsum / count
let bright: Int = (299 * rA + 587 * gA + 114 * bA) / 255
out = native_list_append(out, w)
out = native_list_append(out, h)
out = native_list_append(out, rA)
out = native_list_append(out, gA)
out = native_list_append(out, bA)
out = native_list_append(out, bright)
if n0 < 1 {
n0 = 1
}
if n1 < 1 {
n1 = 1
}
if n2 < 1 {
n2 = 1
}
if n3 < 1 {
n3 = 1
}
if n4 < 1 {
n4 = 1
}
if n5 < 1 {
n5 = 1
}
if n6 < 1 {
n6 = 1
}
if n7 < 1 {
n7 = 1
}
if n8 < 1 {
n8 = 1
}
out = native_list_append(out, c0 / n0)
out = native_list_append(out, c1 / n1)
out = native_list_append(out, c2 / n2)
out = native_list_append(out, c3 / n3)
out = native_list_append(out, c4 / n4)
out = native_list_append(out, c5 / n5)
out = native_list_append(out, c6 / n6)
out = native_list_append(out, c7 / n7)
out = native_list_append(out, c8 / n8)
organ_disclose("FEAT(image): 15-number scene-geometry vs " + int_to_str(w * h * 3) + " pixel-channels — the descriptor travels, the frame does not.")
return out
}
// Own-core tone
//
// The smallest possible proof that the organ owns its medium end to end: a sine
// with a gentle attack and release, computed here, played by us, no file and no
// library anywhere in the path.
fn organ_tone(hz: Int, ms: Int, sr: Int) -> [Int] {
let n: Int = sr * ms / 1000
let out: [Int] = native_list_empty()
let two_pi: Float = 6.283185307
let srf: Float = int_to_float(sr)
let hzf: Float = int_to_float(hz)
let i: Int = 0
// 20 ms of ramp at each end; a square-edged tone clicks, and a click is the
// organ announcing that it does not understand envelopes.
let ramp: Int = sr / 50
if ramp < 1 {
ramp = 1
}
while i < n {
let t: Float = int_to_float(i) / srf
let s: Float = math_sin(two_pi * hzf * t)
let env: Int = 32767
if i < ramp {
env = 32767 * i / ramp
}
let tail: Int = n - i
if tail < ramp {
env = 32767 * tail / ramp
}
let v: Int = float_to_int(s * 9000.0) * env / 32767
out = native_list_append(out, v)
i = i + 1
}
return out
}
-459
View File
@@ -1,459 +0,0 @@
// organ_cli.el the organ's command surface. main() lives here.
//
// One binary, the same verbs the Swift program had, and nothing behind them
// except El and two thin device realizers. This file is the proof surface: if
// `organ speak` makes a sound and no Swift binary is in the process tree, the
// claim in organ.el's header is true.
//
// Verbs, and what each one demonstrates:
//
// grant/revoke/status the Neuron-level consent lock, inspectable
// speak <wav> efferent audio out of El's own speaker
// tone <hz> <ms> own-core synthesis: computed in El, played by El,
// never touching the disk
// say <name> <codes...> fetch a VOICE FROM THE ENGRAM and render through it
// listen <sec> <out> afferent mic capture, consent-gated, fails closed
// see <out.jpg> afferent one camera frame, same two locks
// wav-info <wav> WAV geometry, parsed in El
// feat-audio <wav> capture -> compact descriptor (8 numbers)
// feat-image <jpg> frame -> compact scene-geometry
// voiceprint <wav> F0 + formants F1-F5 by LPC, in El
// imitate <in> <out> LPC analysis-resynthesis, in El
// hear-imitate <sec> the closed loop: hear a voice, take its signature,
// speak back in it
// ingest-audio <wav> descriptor -> engram node (the capture becomes geometry)
// ingest-voice <wav> <n> voiceprint -> engram voice region (how a voice is learned)
// converse <manifest> full-duplex interruptible utterance
//
// The descriptors are the point of the afferent half. A capture is NEVER handed
// on raw: a three-second recording is ~48,000 samples and what leaves this
// process is eight numbers. That is both the privacy rail (the stream stays
// local because only its shape travels) and the reason the engram can hold a
// perception at all geometry is storable, a waveform is not.
fn cli_usage() -> Bool {
println("organ — Neuron's I/O organ, native El (own-core, local, consent-gated)")
println(" grant|revoke <camera|mic> Neuron-level consent")
println(" status consent + device state")
println(" speak <file.wav> play a WAV aloud (efferent)")
println(" tone [hz] [ms] synthesize and play, no file at all")
println(" say <voice> <CODE> [CODE...] fetch voice FROM THE ENGRAM, render, speak")
println(" listen <sec> <out.wav> mic capture 16k mono (afferent)")
println(" see <out.jpg> one camera frame (afferent)")
println(" wav-info <file.wav> WAV geometry")
println(" feat-audio <file.wav> compact audio descriptor (8 numbers)")
println(" feat-image compact scene-geometry from the camera")
println(" voiceprint <voice.wav> F0 + formants F1-F5 (LPC)")
println(" imitate <in.wav> <out.wav> LPC analysis-resynthesis")
println(" hear-imitate <sec> <out.wav> mic -> signature -> imitate -> speak aloud")
println(" ingest-audio <file.wav> descriptor -> engram node (geometry)")
println(" ingest-voice <voice.wav> <n> voiceprint -> engram voice region")
println(" converse <manifest.json> [--authority PM] [--barge-at MS[:kind]] [--live-mic] [--resume]")
return true
}
// The engram the organ reads and writes. Its own store, never production's.
fn cli_engram_dir() -> String {
let d: String = env("ORGAN_ENGRAM")
if str_eq(d, "") {
return "peripheral/.engram"
}
return d
}
fn cli_open_engram() -> Bool {
let dir: String = cli_engram_dir()
fs_mkdir(dir)
let ok: Int = engram_store_boot(dir)
if ok == 1 {
return true
}
return false
}
// formatting helpers
fn cli_f(v: Float, dec: Int) -> String {
return format_float(v, dec)
}
// the descriptor, printed and ingested
//
// [seconds, sr, ch, rms, peak, zcr, centroid, f0] the same eight numbers the
// Swift produced, computed in El, and the compression ratio is the headline:
// a few dozen bytes standing in for a few hundred kilobytes.
fn cli_audio_descriptor_text(v: [Float], path: String) -> String {
let secs: Float = native_list_get(v, 0)
let sr: Float = native_list_get(v, 1)
let ch: Float = native_list_get(v, 2)
let rms: Float = native_list_get(v, 3)
let peak: Float = native_list_get(v, 4)
let zcr: Float = native_list_get(v, 5)
let cen: Float = native_list_get(v, 6)
let f0: Float = native_list_get(v, 7)
return "Heard sound (afferent, mic): " + cli_f(secs, 2) + "s at " + cli_f(sr, 0) + "Hz. RMS energy " + cli_f(rms, 4) + ", peak " + cli_f(peak, 4) + ", zero-crossing rate " + cli_f(zcr, 0) + "Hz, spectral centroid " + cli_f(cen, 0) + "Hz, estimated voice pitch F0 " + cli_f(f0, 0) + "Hz. Compact voice/sound signature (8 numbers) — phonetic geometry seed."
}
fn cli_feat_audio(path: String) -> Bool {
let v: [Float] = dsp_compute_audio(path)
if native_list_len(v) < 8 {
println("{\"ok\": false, \"op\": \"feat-audio\", \"error\": \"cannot read PCM\"}")
return false
}
organ_disclose("FEAT(audio): 8-number signature vs " + int_to_str(float_to_int(native_list_get(v, 0) * native_list_get(v, 1))) + " raw samples.")
println("{\"ok\": true, \"op\": \"feat-audio\", \"file\": \"" + path + "\", \"seconds\": " + cli_f(native_list_get(v, 0), 4) + ", \"sample_rate\": " + cli_f(native_list_get(v, 1), 0) + ", \"channels\": " + cli_f(native_list_get(v, 2), 0) + ", \"rms\": " + cli_f(native_list_get(v, 3), 6) + ", \"peak\": " + cli_f(native_list_get(v, 4), 6) + ", \"zcr_hz\": " + cli_f(native_list_get(v, 5), 4) + ", \"centroid_hz\": " + cli_f(native_list_get(v, 6), 4) + ", \"f0_hz\": " + cli_f(native_list_get(v, 7), 4) + "}")
return true
}
fn cli_voiceprint(path: String) -> Bool {
let v: [Float] = dsp_voiceprint(path)
if native_list_len(v) < 4 {
println("{\"ok\": false, \"op\": \"voiceprint\", \"error\": \"cannot read speech\"}")
return false
}
let nf: Int = float_to_int(native_list_get(v, 3))
let fs: String = ""
let bs: String = ""
let i: Int = 0
while i < nf {
if i > 0 {
fs = fs + ", "
bs = bs + ", "
}
fs = fs + cli_f(native_list_get(v, 4 + i * 2), 3)
bs = bs + cli_f(native_list_get(v, 5 + i * 2), 3)
i = i + 1
}
println("{\"ok\": true, \"op\": \"voiceprint\", \"file\": \"" + path + "\", \"f0_hz\": " + cli_f(native_list_get(v, 0), 4) + ", \"f0_range\": [" + cli_f(native_list_get(v, 1), 4) + ", " + cli_f(native_list_get(v, 2), 4) + "], \"formants_hz\": [" + fs + "], \"bandwidths_hz\": [" + bs + "]}")
return true
}
// main
fn main() {
let a: [String] = args()
let n: Int = native_list_len(a)
if n < 1 {
cli_usage()
return
}
let cmd: String = native_list_get(a, 0)
// ---- consent -----------------------------------------------------------
if str_eq(cmd, "grant") {
if n < 2 {
println("grant needs a device")
return
}
organ_grant(native_list_get(a, 1))
println("{\"ok\": true, \"op\": \"grant\", \"consent\": \"" + organ_consent_status() + "\"}")
return
}
if str_eq(cmd, "revoke") {
if n < 2 {
println("revoke needs a device")
return
}
organ_revoke(native_list_get(a, 1))
println("{\"ok\": true, \"op\": \"revoke\", \"consent\": \"" + organ_consent_status() + "\"}")
return
}
if str_eq(cmd, "status") {
println("{\"ok\": true, \"op\": \"status\", \"consent\": \"" + organ_consent_status() + "\", \"speaker\": \"" + speaker_name() + "\", \"speaker_available\": " + int_to_str(speaker_available()) + ", \"mic_os_authorized\": " + int_to_str(mic_available()) + ", \"camera_os_authorized\": " + int_to_str(camera_available()) + "}")
return
}
// ---- efferent ----------------------------------------------------------
if str_eq(cmd, "speak") {
if n < 2 {
println("speak needs a wav")
return
}
let ok: Bool = organ_speak_wav(native_list_get(a, 1))
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"speak\", \"played_aloud\": " + bool_to_str(ok) + "}")
return
}
if str_eq(cmd, "tone") {
let hz: Int = 220
let ms: Int = 1000
if n >= 2 {
hz = str_to_int(native_list_get(a, 1))
}
if n >= 3 {
ms = str_to_int(native_list_get(a, 2))
}
let s: [Int] = organ_tone(hz, ms, 16000)
let ok: Bool = organ_speak_samples(s, 16000)
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"tone\", \"hz\": " + int_to_str(hz) + ", \"ms\": " + int_to_str(ms) + ", \"samples\": " + int_to_str(native_list_len(s)) + ", \"file\": null}")
return
}
// ---- the voice, from the engram ----------------------------------------
if str_eq(cmd, "say") {
if n < 3 {
println("say needs <voice> <CODE> [CODE...]")
return
}
cli_open_engram()
let vname: String = native_list_get(a, 1)
let g: [Int] = organ_voice_fetch(vname)
if native_list_len(g) < 6 {
println("{\"ok\": false, \"op\": \"say\", \"error\": \"no voice region '" + vname + "' in the engram\"}")
return
}
// Codes and the phoneme map come from the LANGUAGE side. The organ does
// not know what a word is and never looks one up.
let pmap: [String] = ingest_phonetics("elp/data/phonetics.psv")
let codes: [String] = native_list_empty()
let i: Int = 2
while i < n {
codes = native_list_append(codes, native_list_get(a, i))
i = i + 1
}
let voice: [String] = organ_voice_profile(vname, g)
let s: [Int] = synth_codes(codes, voice, pmap)
let ok: Bool = organ_speak_samples(s, 16000)
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"say\", \"voice\": \"" + vname + "\", \"f0\": " + int_to_str(native_list_get(g, 0)) + ", \"kf\": " + int_to_str(native_list_get(g, 2)) + ", \"codes\": " + int_to_str(native_list_len(codes)) + ", \"samples\": " + int_to_str(native_list_len(s)) + "}")
return
}
// ---- afferent ----------------------------------------------------------
if str_eq(cmd, "listen") {
if n < 3 {
println("listen needs <sec> <out.wav>")
return
}
let secs: Int = str_to_int(native_list_get(a, 1))
let out: String = native_list_get(a, 2)
if organ_may_listen() == false {
println("{\"ok\": false, \"op\": \"listen\", \"error\": \"consent denied (fails closed)\"}")
return
}
organ_disclose("MIC: capturing " + int_to_str(secs) + "s (16 kHz mono, LOCAL, never egresses).")
let s: [Int] = mic_capture_pcm16(secs, 16000)
let got: Int = native_list_len(s)
if got <= 0 {
println("{\"ok\": false, \"op\": \"listen\", \"error\": \"capture returned nothing\"}")
return
}
let ok: Bool = write_wav(s, 16000, out)
organ_disclose("MIC: captured " + int_to_str(got) + " frames — ready to hand to the ingest organ.")
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"listen\", \"file\": \"" + out + "\", \"frames\": " + int_to_str(got) + ", \"sample_rate\": 16000}")
return
}
if str_eq(cmd, "see") {
if n < 2 {
println("see needs an out path")
return
}
if organ_may_see() == false {
println("{\"ok\": false, \"op\": \"see\", \"error\": \"consent denied (fails closed)\"}")
return
}
organ_disclose("CAMERA: capturing one frame (LOCAL, never egresses).")
let ok: Int = camera_capture_jpeg(native_list_get(a, 1))
println("{\"ok\": " + int_to_str(ok) + ", \"op\": \"see\", \"file\": \"" + native_list_get(a, 1) + "\"}")
return
}
// ---- descriptors -------------------------------------------------------
if str_eq(cmd, "wav-info") {
if n < 2 {
println("wav-info needs a wav")
return
}
let p: String = native_list_get(a, 1)
let w: [Float] = dsp_read_wav(p)
if dsp_wav_n(w) <= 0 {
println("{\"ok\": false, \"op\": \"wav-info\"}")
return
}
println("{\"ok\": true, \"op\": \"wav-info\", \"sample_rate\": " + int_to_str(dsp_wav_sr(w)) + ", \"channels\": " + int_to_str(dsp_wav_ch(w)) + ", \"frames\": " + int_to_str(dsp_wav_n(w)) + "}")
return
}
if str_eq(cmd, "feat-audio") {
if n < 2 {
println("feat-audio needs a wav")
return
}
cli_feat_audio(native_list_get(a, 1))
return
}
if str_eq(cmd, "feat-image") {
if organ_may_see() == false {
println("{\"ok\": false, \"op\": \"feat-image\", \"error\": \"consent denied (fails closed)\"}")
return
}
let f: [Int] = organ_image_descriptor()
if native_list_len(f) < 15 {
println("{\"ok\": false, \"op\": \"feat-image\", \"error\": \"no frame\"}")
return
}
let grid: String = ""
let i: Int = 6
while i < 15 {
if i > 6 {
grid = grid + ", "
}
grid = grid + int_to_str(native_list_get(f, i))
i = i + 1
}
println("{\"ok\": true, \"op\": \"feat-image\", \"width\": " + int_to_str(native_list_get(f, 0)) + ", \"height\": " + int_to_str(native_list_get(f, 1)) + ", \"mean_rgb\": [" + int_to_str(native_list_get(f, 2)) + ", " + int_to_str(native_list_get(f, 3)) + ", " + int_to_str(native_list_get(f, 4)) + "], \"brightness_pm\": " + int_to_str(native_list_get(f, 5)) + ", \"luma_grid\": [" + grid + "]}")
return
}
if str_eq(cmd, "voiceprint") {
if n < 2 {
println("voiceprint needs a wav")
return
}
cli_voiceprint(native_list_get(a, 1))
return
}
if str_eq(cmd, "imitate") {
if n < 3 {
println("imitate needs <in.wav> <out.wav>")
return
}
let s: [Int] = dsp_imitate(native_list_get(a, 1))
if native_list_len(s) <= 0 {
println("{\"ok\": false, \"op\": \"imitate\"}")
return
}
let ok: Bool = write_wav(s, 16000, native_list_get(a, 2))
organ_disclose("IMITATE: rebuilt the voice from its own LPC signature (own-core, no training, no stolen voice).")
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"imitate\", \"out\": \"" + native_list_get(a, 2) + "\", \"samples\": " + int_to_str(native_list_len(s)) + ", \"method\": \"LPC analysis-resynthesis\"}")
return
}
if str_eq(cmd, "hear-imitate") {
if n < 3 {
println("hear-imitate needs <sec> <out.wav>")
return
}
let secs: Int = str_to_int(native_list_get(a, 1))
let out: String = native_list_get(a, 2)
if organ_may_listen() == false {
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"consent denied (fails closed)\"}")
return
}
let heard: String = out + ".heard.wav"
organ_disclose("HEAR-IMITATE: open the ear, listen " + int_to_str(secs) + "s, take the voice, speak it back.")
let s: [Int] = mic_capture_pcm16(secs, 16000)
if native_list_len(s) <= 0 {
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"capture returned nothing\"}")
return
}
write_wav(s, 16000, heard)
let re: [Int] = dsp_imitate(heard)
if native_list_len(re) <= 0 {
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"could not model the voice\"}")
return
}
write_wav(re, 16000, out)
let ok: Bool = organ_speak_samples(re, 16000)
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"hear-imitate\", \"heard\": \"" + heard + "\", \"out\": \"" + out + "\", \"spoke_aloud\": " + bool_to_str(ok) + "}")
return
}
// ---- the afferent wire: descriptor -> geometry --------------------------
if str_eq(cmd, "ingest-audio") {
if n < 2 {
println("ingest-audio needs a wav")
return
}
let p: String = native_list_get(a, 1)
let v: [Float] = dsp_compute_audio(p)
if native_list_len(v) < 8 {
println("{\"ok\": false, \"op\": \"ingest-audio\"}")
return
}
cli_open_engram()
let content: String = cli_audio_descriptor_text(v, p)
let id: String = engram_node(content, "Observation", 70)
engram_store_checkpoint()
organ_disclose("INGEST: the capture is now GEOMETRY in the engram (node " + id + ") — the descriptor travelled, the stream did not.")
println("{\"ok\": true, \"op\": \"ingest-audio\", \"node_id\": \"" + id + "\", \"content\": \"" + content + "\"}")
return
}
if str_eq(cmd, "ingest-voice") {
if n < 3 {
println("ingest-voice needs <voice.wav> <name>")
return
}
let p: String = native_list_get(a, 1)
let name: String = native_list_get(a, 2)
let v: [Float] = dsp_voiceprint(p)
if native_list_len(v) < 10 {
println("{\"ok\": false, \"op\": \"ingest-voice\", \"error\": \"no voiced frames\"}")
return
}
cli_open_engram()
let f0: Int = float_to_int(native_list_get(v, 0))
let f1: Int = float_to_int(native_list_get(v, 4))
let f2: Int = float_to_int(native_list_get(v, 6))
let f3: Int = float_to_int(native_list_get(v, 8))
// kf is the vocal-tract scale: this speaker's F1 against the nominal
// /AA/ F1 of 730 Hz. One number standing for a tract length.
let kf: Int = 1000 * f1 / 730
let f0e: Int = f0 * 85 / 100
let id: String = organ_voice_ingest(name, f0, f0e, kf, f1, f2, f3, "el-organ-lpc-voiceprint", "COARSE")
engram_store_checkpoint()
println("{\"ok\": true, \"op\": \"ingest-voice\", \"node_id\": \"" + id + "\", \"name\": \"" + name + "\", \"f0\": " + int_to_str(f0) + ", \"kf\": " + int_to_str(kf) + ", \"f1\": " + int_to_str(f1) + ", \"f2\": " + int_to_str(f2) + ", \"f3\": " + int_to_str(f3) + "}")
return
}
// ---- converse ----------------------------------------------------------
if str_eq(cmd, "converse") {
if n < 2 {
println("converse needs a manifest")
return
}
let mf: String = native_list_get(a, 1)
let authority: Int = 500
let barge: Int = 0 - 1
let kind: String = "bargein"
let live: Bool = false
let resume: Bool = false
let i: Int = 2
while i < n {
let f: String = native_list_get(a, i)
if str_eq(f, "--authority") {
if i + 1 < n {
authority = str_to_int(native_list_get(a, i + 1))
i = i + 1
}
}
if str_eq(f, "--barge-at") {
if i + 1 < n {
let spec: String = native_list_get(a, i + 1)
let c: Int = str_index_of(spec, ":")
if c < 0 {
barge = str_to_int(spec)
} else {
barge = str_to_int(str_slice(spec, 0, c))
kind = str_slice(spec, c + 1, str_len(spec))
}
i = i + 1
}
}
if str_eq(f, "--live-mic") {
live = true
}
if str_eq(f, "--resume") {
resume = true
}
i = i + 1
}
let plan: [String] = conv_load_manifest(mf)
if resume {
plan = conv_load_resume()
organ_disclose("CONVERSE: resuming — \"as I was saying...\" (" + int_to_str(plan_count(plan)) + " segments left).")
} else {
organ_disclose("CONVERSE: utterance = \"" + conv_utterance(mf) + "\" (" + int_to_str(plan_count(plan)) + " segments).")
}
let stopped: Int = conv_run(plan, authority, barge, kind, live)
println("{\"ok\": true, \"op\": \"converse\", \"stopped_at\": " + int_to_str(stopped) + ", \"complete\": " + bool_to_str(stopped < 0) + "}")
return
}
cli_usage()
}
-454
View File
@@ -1,454 +0,0 @@
// organ_converse.el full-duplex, interruptible speech. The turn-taking organ.
//
// WHAT THIS IS FOR. A system that plays an utterance to completion and only
// then listens is not conversational, it is a loudspeaker with a queue. Being
// interruptible is not a feature bolted onto speech; it is most of what makes
// speech social. So the utterance is not a blob of audio it is an ordered,
// SALIENCE-TAGGED MEANING-PLAN, and the organ speaks it while listening, decides
// what to do when interrupted, and can pick the thread back up afterwards.
//
// THREE THINGS HAVE TO BE TRUE, and each one is a place naive implementations
// go wrong:
//
// Barge-in is AT THE SAMPLE. When the mic hears speech, output stops on the
// spot not at the end of the current buffer, not at the end of the segment.
// A listener experiences even a fifth of a second of continued talking as
// being talked over. This is why the speaker realizer has pause/resume and
// reports played_frames: "finish the buffer" is not barge-in.
//
// Yield-or-hold is a DECISION, not a rule. Stopping every time anyone makes a
// noise is its own failure it means Neuron can never finish a sentence that
// matters. So the choice is grounded: how salient is what I am mid-saying,
// how close am I to done, and how much authority does the interrupter have.
// Holding the floor is justified when what I am saying matters AND finishing
// is cheap AND the interrupter is not high-priority. Otherwise yield, because
// the polite default is the right default.
//
// A backchannel is NOT an interruption. "mm-hm" means keep going. Treating it
// as a barge-in makes the system stop every three seconds during ordinary
// listening behaviour, which is worse than not listening at all. It is
// distinguished by being brief and low-energy: sample again shortly after
// onset, and if the speech already died away it was a backchannel.
//
// AND THE UTTERANCE SURVIVES. On yield, the remaining plan is persisted, so
// Neuron can resume "as I was saying" instead of losing the thought. An
// interruption should cost a turn, not the content.
//
// The AEC rail: the microphone runs with the OS voice-processing unit enabled
// so it does not hear our own speaker. Without it Neuron barges in on its own
// voice on the first syllable and the whole loop is unusable in a real room.
//
// Note what is NOT here: nothing about words. A segment carries a `text` field
// purely as a label for disclosure. The organ speaks pre-rendered audio and
// never inspects language that is the language faculty's, and the seam holds.
// The meaning-plan
//
// Stored as a flat [String] with stride 3 file, salience-per-mille, text
// because El has no record type and parallel lists drift out of step under
// editing. Salience is an integer per-mille rather than a Float so the decision
// arithmetic stays exact and reproducible; a turn-taking decision that varies
// with floating-point rounding is not one you can debug.
fn plan_new() -> [String] {
return native_list_empty()
}
fn plan_add(plan: [String], file: String, salience_pm: Int, text: String) -> [String] {
let p: [String] = plan
p = native_list_append(p, file)
p = native_list_append(p, int_to_str(salience_pm))
p = native_list_append(p, text)
return p
}
fn plan_count(plan: [String]) -> Int {
return native_list_len(plan) / 3
}
fn plan_file(plan: [String], i: Int) -> String {
return native_list_get(plan, i * 3)
}
fn plan_salience(plan: [String], i: Int) -> Int {
return str_to_int(native_list_get(plan, i * 3 + 1))
}
fn plan_text(plan: [String], i: Int) -> String {
return native_list_get(plan, i * 3 + 2)
}
// Manifest
//
// {"utterance": "...", "segments": [{"file":..., "salience":0.9, "text":"..."}]}
// Salience arrives as a 0..1 float in the manifest and is converted once, here,
// at the edge the same discipline the runtime uses for wire encodings.
fn conv_salience_pm(raw: String) -> Int {
// "0.85" -> 850. Parsed by hand rather than through a float so a manifest
// typo degrades to a visible number instead of a silent 0.0.
let dot: Int = str_index_of(raw, ".")
if dot < 0 {
let whole: Int = str_to_int(raw)
return whole * 1000
}
let ip: Int = str_to_int(str_slice(raw, 0, dot))
let frac: String = str_slice(raw, dot + 1, str_len(raw))
let pm: Int = 0
let scale: Int = 100
let i: Int = 0
while i < 3 {
let d: Int = 0
if i < str_len(frac) {
let c: Int = str_char_code(frac, i)
if c >= 48 {
if c <= 57 {
d = c - 48
}
}
}
pm = pm + d * scale
scale = scale / 10
i = i + 1
}
return ip * 1000 + pm
}
fn conv_load_manifest(path: String) -> [String] {
let plan: [String] = plan_new()
let raw: String = fs_read(path)
if str_eq(raw, "") {
organ_disclose("CONVERSE: cannot read manifest " + path)
return plan
}
let segs: String = json_get_raw(raw, "segments")
let n: Int = json_array_len(segs)
let i: Int = 0
while i < n {
let seg: String = json_array_get(segs, i)
let file: String = json_get_string(seg, "file")
let text: String = json_get_string(seg, "text")
let sal: String = json_get_raw(seg, "salience")
let pm: Int = conv_salience_pm(sal)
if pm <= 0 {
pm = 500
}
plan = plan_add(plan, file, pm, text)
i = i + 1
}
return plan
}
fn conv_utterance(path: String) -> String {
let raw: String = fs_read(path)
return json_get_string(raw, "utterance")
}
// The decision
//
// Returns: 0 = backchannel, carry on seamlessly
// 1 = hold the floor ("hang on, let me finish this thought")
// 2 = yield (stop, let them in)
//
// All arguments are per-mille integers. Holding requires BOTH that the material
// is worth finishing AND that the interrupter is not high-authority either
// condition alone is not enough, because "what I'm saying is important" is
// exactly the reasoning that produces a system nobody can get a word in against.
fn conv_decide(salience_pm: Int, progress_pm: Int, authority_pm: Int, is_backchannel: Bool) -> Int {
if is_backchannel {
return 0
}
let hold_score: Int = (salience_pm * 6 + progress_pm * 4) / 10
if hold_score >= 600 {
if authority_pm < 800 {
return 1
}
}
return 2
}
// Resume
//
// The remaining plan, written where a later run can find it. This is what turns
// an interruption into a pause rather than a loss.
fn conv_resume_path() -> String {
let home: String = env("PERIPH_HOME")
if str_eq(home, "") {
return "peripheral/.resume.json"
}
return home + "/.resume.json"
}
// Minimal JSON string escaping. Written here rather than reached for from the
// runtime because the organ needs exactly two escapes and no dependency: a
// segment label containing a quote or a backslash must not be able to produce a
// resume file that fails to parse and silently loses the thread.
fn conv_escape(s: String) -> String {
let n: Int = str_len(s)
let out: String = ""
let i: Int = 0
while i < n {
let c: Int = str_char_code(s, i)
if c == 34 {
out = out + "\\\""
} else {
if c == 92 {
out = out + "\\\\"
} else {
if c >= 32 {
out = out + str_slice(s, i, i + 1)
}
}
}
i = i + 1
}
return out
}
fn conv_persist_resume(plan: [String], start_at: Int, reason: String) -> Bool {
let n: Int = plan_count(plan)
let body: String = "{\"resume_from\": " + int_to_str(start_at) + ", \"reason\": \"" + reason + "\", \"segments\": ["
let i: Int = start_at
let first: Bool = true
while i < n {
if first == false {
body = body + ", "
}
body = body + "{\"file\": \"" + plan_file(plan, i) + "\", \"salience\": " + int_to_str(plan_salience(plan, i)) + ", \"text\": \"" + conv_escape(plan_text(plan, i)) + "\"}"
first = false
i = i + 1
}
body = body + "]}\n"
let ok: Bool = fs_write(conv_resume_path(), body)
organ_disclose("CONVERSE: meaning-plan persisted (" + int_to_str(n - start_at) + " segments remain) — Neuron can resume the thread.")
return ok
}
fn conv_clear_resume() -> Bool {
return fs_write(conv_resume_path(), "")
}
// Read a persisted plan back. Salience is already per-mille here (we wrote it),
// so it is NOT re-scaled the manifest and the resume file are different
// formats on purpose, and conflating them silently divides every salience by a
// thousand.
fn conv_load_resume() -> [String] {
let plan: [String] = plan_new()
let raw: String = fs_read(conv_resume_path())
if str_eq(raw, "") {
return plan
}
let segs: String = json_get_raw(raw, "segments")
let n: Int = json_array_len(segs)
let i: Int = 0
while i < n {
let seg: String = json_array_get(segs, i)
plan = plan_add(plan, json_get_string(seg, "file"), json_get_int(seg, "salience"), json_get_string(seg, "text"))
i = i + 1
}
return plan
}
// The loop
//
// live_mic : open the microphone with AEC and let real speech drive barge-in.
// barge_ms : if >= 0, inject a barge event at that offset into the utterance
// instead. Deterministic, so the decision paths can be exercised
// without a room and a person the same reason periph.swift has it.
// kind : "backchannel" or "bargein", for the injected case.
// authority : interrupter authority, per-mille.
//
// Returns the index the utterance stopped at, or -1 if it completed.
fn conv_run(plan: [String], authority_pm: Int, barge_ms: Int, kind: String, live_mic: Bool) -> Int {
let n: Int = plan_count(plan)
if n <= 0 {
organ_disclose("CONVERSE: nothing to say.")
return 0 - 1
}
if speaker_available() == 0 {
organ_disclose("CONVERSE: no speaker on this build — cannot hold a conversation.")
return 0 - 1
}
let mic_live: Bool = false
if live_mic {
if organ_may_listen() {
let m: Int = mic_monitor_start()
if m == 1 {
organ_disclose("CONVERSE: full-duplex — mic listening WHILE speaking, AEC on (won't self-interrupt).")
mic_live = true
}
if m == 2 {
organ_disclose("CONVERSE: full-duplex — mic listening, but AEC UNAVAILABLE; raising the VAD floor so we do not barge in on ourselves.")
mic_live = true
}
if m == 0 {
organ_disclose("CONVERSE: could not open the mic monitor — falling back to injected events.")
}
}
}
if mic_live == false {
organ_disclose("CONVERSE: deterministic mode (live mic off).")
}
// Without AEC the mic hears the speaker, so the threshold has to sit above
// our own output. This is a mitigation and not a fix: the honest note is
// that barge-in is markedly less sensitive in this mode.
let vad_pm: Int = 20
if mic_live {
if mic_monitor_start() == 2 {
vad_pm = 60
}
}
let elapsed_ms: Int = 0
let prior_ms: Int = 0
let handled: Bool = false
// An injected barge is ONE event, not a condition that stays true. Without
// this the deadline re-fires on every poll after a backchannel resume, and
// the utterance live-locks: paused, resumed, paused again, forever.
let injected_fired: Bool = false
let i: Int = 0
while i < n {
let file: String = plan_file(plan, i)
let sal: Int = plan_salience(plan, i)
let frames: Int = wav_frames(file)
let rate: Int = wav_rate(file)
if frames <= 0 {
organ_disclose("CONVERSE: missing or unreadable segment '" + file + "', skipping.")
i = i + 1
} else {
let dur_ms: Int = frames * 1000 / rate
organ_disclose("CONVERSE: speaking segment " + int_to_str(i + 1) + "/" + int_to_str(n) + " (salience " + int_to_str(sal) + "/1000) — \"" + plan_text(plan, i) + "\"")
let started: Int = speaker_play_wav_async(file)
if started == 0 {
organ_disclose("CONVERSE: could not start playback for '" + file + "'.")
i = i + 1
} else {
let seg_ms: Int = 0
let done: Bool = false
let interrupted: Bool = false
let speech_ticks: Int = 0
while done == false {
sleep_ms(10)
seg_ms = seg_ms + 10
if speaker_playing() == 0 {
done = true
} else {
// The tick counter is an approximation each pass costs
// more than the sleep it asked for. The DAC position is
// the truth, so drive the injected deadline off THAT and
// an injected barge lands where it was asked to land.
let pos_ms: Int = speaker_played_frames() * 1000 / rate
elapsed_ms = prior_ms + pos_ms
// --- onset detection: real speech, or an injected event ---
let onset: Bool = false
if mic_live {
let rms: Float = mic_monitor_rms()
let rms_pm: Int = float_to_int(rms * 1000.0)
if rms_pm > vad_pm {
speech_ticks = speech_ticks + 1
} else {
speech_ticks = 0
}
// ~60ms of continuous voice: short enough to feel
// instant, long enough that a door closing is not a turn.
if speech_ticks >= 3 {
if handled == false {
onset = true
}
}
}
if barge_ms >= 0 {
if injected_fired == false {
if elapsed_ms >= barge_ms {
onset = true
injected_fired = true
}
}
}
if onset {
handled = true
// (1) BARGE-IN pause on the spot.
speaker_pause()
let played: Int = speaker_played_frames()
let at_ms: Int = played * 1000 / rate
let progress_pm: Int = at_ms * 1000 / dur_ms
if progress_pm > 1000 {
progress_pm = 1000
}
organ_disclose("CONVERSE: << user speech at " + int_to_str(at_ms) + "ms into segment " + int_to_str(i + 1) + " — PAUSED instantly >>")
// (2) backchannel or real barge-in?
let is_bc: Bool = false
if barge_ms >= 0 {
if str_eq(kind, "backchannel") {
is_bc = true
}
} else {
// Live: look again ~250ms after onset. If the
// energy has already collapsed it was "mm-hm".
sleep_ms(250)
let r2: Float = mic_monitor_rms()
if float_to_int(r2 * 1000.0) < 15 {
is_bc = true
}
}
// (3) yield, hold, or carry on
let d: Int = conv_decide(sal, progress_pm, authority_pm, is_bc)
if d == 0 {
organ_disclose("CONVERSE: read as BACKCHANNEL (\"mm-hm\") — keep going, resume seamlessly.")
handled = false
speech_ticks = 0
speaker_resume()
}
if d == 1 {
organ_disclose("CONVERSE: HOLD the floor — \"hang on, let me finish this thought.\" (salience " + int_to_str(sal) + ", progress " + int_to_str(progress_pm) + ")")
speaker_resume()
// Finish THIS segment, then yield the remainder:
// holding is a request for a moment, not a claim
// on the rest of the conversation.
while speaker_playing() == 1 {
sleep_ms(20)
}
speaker_stop()
conv_persist_resume(plan, i + 1, "held-then-yield")
if mic_live {
mic_monitor_stop()
}
return i + 1
}
if d == 2 {
organ_disclose("CONVERSE: YIELD — stop, let them in. Remembering where I was (resumable).")
speaker_stop()
conv_persist_resume(plan, i, "yield")
if mic_live {
mic_monitor_stop()
}
return i
}
}
}
}
if interrupted == false {
prior_ms = prior_ms + dur_ms
i = i + 1
}
}
}
}
conv_clear_resume()
organ_disclose("CONVERSE: utterance complete (uninterrupted).")
if mic_live {
mic_monitor_stop()
}
return 0 - 1
}
File diff suppressed because it is too large Load Diff