# peripheral — Neuron's I/O organ, in El **El speaks.** The engram stores geometry and does not speak; the speaking belongs to the language and its runtime. Until this landed, the organ was a 939-line Swift program (`src/periph.swift`) that shelled out to `afplay`. Neuron's mouth and ears were a separate binary standing next to the language, and "speak" meant "ask that binary to speak." That program is now **reference material, not the implementation.** ``` SPEAKER (speak) efferent samples ──────────────► CoreAudio ──► the room MIC (hear) afferent device ──► samples ──► descriptor ──► engram CAMERA (see) afferent device ──► frame ──► descriptor ──► engram ``` ## The split, and why it falls where it does Exactly **two** things here are not El, and they are the two things El cannot express as arithmetic: | Not El (realizers) | Why | |---|---| | `lang/runtime/el_audio_darwin.m` | Handing a buffer to the DAC and waiting for it to drain. There is no way to say "the hardware has now played these samples" in El, and there should not be. | | `lang/runtime/el_capture_darwin.m` | Asking the OS for samples off a microphone or frames off a camera, plus the TCC permission dance. | **Everything else is El**, because everything else is arithmetic: | In El | Where | |---|---| | WAV encode / decode (chunk-walking, JUNK/FLLR tolerant) | `src/organ_dsp.el`, `elp/src/speech.el` | | LPC autocorrelation + Levinson-Durbin (order 16 @ 16 kHz) | `src/organ_dsp.el` | | Formant extraction off the all-pole spectral envelope | `src/organ_dsp.el` | | Source-filter resynthesis (glottal impulse train through the filter) | `src/organ_dsp.el` | | Audio descriptor `[seconds, sr, ch, rms, peak, zcr, centroid, F0]` | `src/organ_dsp.el` | | Voice descriptor `[F0, F1..F5, bandwidths]` | `src/organ_dsp.el` | | Scene descriptor `[w, h, meanRGB, brightness, 3×3 luminance grid]` | `src/organ.el` | | Consent, disclosure, the voice-from-engram fetch | `src/organ.el` | | Barge-in, yield-or-hold, backchannel, resume | `src/organ_converse.el` | | The command surface | `src/organ_cli.el` | Both realizers are their **own translation units**, declared in `lang/runtime/el_runtime.h`, and deliberately **not** patches to `el_runtime.c`. Acquiring a device must not mean editing the middle of the language — the same rule the realizer registry follows for modalities. `lang/runtime/el_peripheral_null.c` provides the identical entry points everywhere else, so El that speaks links on any platform and truthfully reports having no speaker rather than going quietly silent. ## The voice comes from the engram A voice is **geometry in the engram**, not a JSON file next to the code and certainly not constants in a source file. The organ fetches it the way anything retrieves a memory — it asks: ```el let g: [Int] = organ_voice_fetch("will") // [peripheral] VOICE: fetched 'will' FROM THE ENGRAM — // f0=137 f0_end=116 kf=1269 f1=500 f2=2093 f3=3531 ``` `organ_voice_fetch` issues an engram query and reads the geometry off the node that comes back. Nothing opens a file. If the region is not in the graph it returns **empty**, not a plausible default — a caller has to be able to tell "this is how they sound" from "I never heard them." The reverse direction is `ingest-voice`: an LPC voiceprint becomes a node, and from then on the voice is a memory rather than a measurement someone wrote down. ## What the organ never does **It never learns a word.** Pronunciation, vocabulary and phonemes belong to the language faculty and are already built as ingested geometry — *the engram knows how to pronounce*. The seam is `synth_codes(codes, voice, pmap)`: the codes and the phoneme map arrive from the language side as geometry, and the organ's whole job is turning them into samples and getting the samples out the speaker, plus the same trip in reverse for the senses. There is no lexicon here and no grapheme-to-phoneme rule, by design. ## Rails - **Own-core.** CoreAudio / AVFoundation / ImageIO — all ship with macOS. No cloud, no model, no heavy dependency. There is **no network code in the organ at all**, by construction. - **Local-only.** Raw streams stay on the machine. What leaves a capture is a descriptor of a few dozen numbers. A 1920×1080 frame becomes 15 integers (~414,000× smaller); three seconds of audio becomes 8. - **Consent, two locks.** A Neuron-level grant **and** the OS TCC permission. Camera and mic **fail closed** without both. The speaker is disclosed but not gated — you cannot secretly speak aloud, and gating it would mean Neuron needs permission to answer. - **Disclosed.** Every device touch prints a `[peripheral]` line on **stderr** (via `eprintln`, flushed immediately), so a disclosure lands before the device is touched and never contaminates the program's stdout. ## Build ```bash ./peripheral/build.sh /tmp/organ ``` Concatenates the El modules, compiles with `elc`, links the two realizers. Run it **from the repo root** or the `.psv` phoneme data will not resolve. ## Commands ``` organ grant|revoke Neuron-level consent organ status consent + device state organ speak play a WAV aloud (efferent) organ tone [hz] [ms] synthesize and play — no file at all organ say [CODE...] fetch voice FROM THE ENGRAM, render, speak organ listen mic capture 16k mono (afferent) organ see one camera frame (afferent) organ wav-info WAV geometry organ feat-audio compact audio descriptor (8 numbers) organ feat-image compact scene-geometry from the camera organ voiceprint F0 + formants F1-F5 (LPC) organ imitate LPC analysis-resynthesis organ hear-imitate mic -> signature -> imitate -> speak aloud organ ingest-audio descriptor -> engram node (geometry) organ ingest-voice voiceprint -> engram voice region organ converse [--authority PM] [--barge-at MS[:kind]] [--live-mic] [--resume] ``` ## Interruptibility `converse` speaks an ordered, salience-tagged **meaning-plan** while listening: - **barge-in** — output stops at the sample, not at the end of the buffer. The realizer exposes `pause`/`resume` and reports `played_frames` (the real DAC position) precisely so this is possible. - **yield-or-hold** — a decision, not a rule: `hold = salience·0.6 + progress·0.4`, and holding also requires that the interrupter not be high-authority. Otherwise yield, because the polite default is the right one. - **backchannel** — "mm-hm" is brief and low-energy; resume seamlessly. - **resumable** — on yield the remaining plan persists to `.resume.json`; `--resume` picks the thread back up. An interruption should cost a turn, not the content. Live full-duplex uses `--live-mic` with the OS voice-processing unit (AEC) so Neuron does not barge in on its own voice. `--barge-at` injects the event deterministically for testing. ## Measured against the Swift original Same input (`out/mic_room.wav`, 16 kHz mono, 48121 samples), Swift `periph` vs the El organ: | | Swift | El | |---|---|---| | seconds | 3.0075625 | 3.0076 | | rms | 0.0047766496761 | 0.004777 | | peak | 0.01806640625 | 0.018066 | | zcr_hz | 416.28395087 | 416.2840 | | centroid_hz | 727.60529169 | 727.6053 | | f0_hz | 400 | 400.0000 | | formants F1–F5 | 1734.375 / 3343.75 / 3875 / 4359.375 / 4468.75 | identical | | bandwidths B1–B5 | 2000 / 2968.75 / 4203.125 / 4687.5 / 5000 | identical | Agreement to every printed digit. `imitate` cannot match bit-for-bit because the Swift excites unvoiced frames with `Double.random` — two Swift runs correlate 0.957 with **each other**; El correlates **0.958** with Swift. The port is as close to the original as the original is to itself, and the deterministic prefix is bit-identical. ## Honest status - **Works:** speaker (CoreAudio, no `afplay`, no subprocess — verified: zero `afplay`/Swift strings in the binary, no child process during playback), mic capture, camera capture, all descriptors, LPC voiceprint, imitate, hear-imitate, voice fetch/ingest against the engram, converse (yield, hold, yield-to-authority, backchannel, resume — all exercised with real audio). - **Coarse, and labelled so:** a fetched voice is one formant triple with no coarticulation and no prosody. It is an impression, explicitly **not a clone**, and `prov=COARSE` says so on the node. - **Not verified here:** live `--live-mic` barge-in in a real room with a real interrupter. The AEC path is implemented and the deterministic path is proven; the acoustic behaviour is not something a headless run can establish. - **Not in the engram yet:** the structured `Voice` / `VowelTarget` geometry nodes live in the organ's own store and in snapshot files from earlier work, but the **production engram does not carry them**. Getting them there is an ingest, not a code change. - `src/periph.swift` is kept as the reference the port was measured against.