Neuron
5503e1d9a4
El could turn meaning into samples and could not make a sound. Every path from those samples to the air ran outside the language, through a 939-line Swift program that shelled out to afplay, so the voice was not a capability of El or of Neuron but a separate binary standing next to them. Two things land here. The speaker. el_audio_darwin.m is a CoreAudio AudioQueue realizer in its own translation unit, declared in el_runtime.h, deliberately not a patch to el_runtime.c — acquiring a device must not mean editing the middle of the language, the same rule the realizer registry follows for modalities. It takes samples straight out of memory, so nothing is written to disk and no process is spawned between the intent to speak and the sound. The async half (play/stop/playing/played_frames) exists because barge-in means stopping on the spot, and a blocking play cannot be interrupted. el_peripheral_null.c is the same entry points everywhere else, so El that speaks links anywhere and truthfully reports having no speaker. The voice. organ_voice_fetch asks the engram for a voice region by query and reads the geometry off the node that comes back. A voice is not a JSON file next to the code; it is a memory, and the organ retrieves it the way anything retrieves a memory. An absent region returns empty rather than a plausible default, because a caller must be able to tell 'this is how they sound' from 'I never heard them'. Underneath both: __str_set_char bounds-checked writes against strlen(), which is 0 for the zero-filled buffer __str_alloc hands back, so every write was rejected and every El-authored WAV in this repo was 55,244 bytes of silence that reported ok=true. Byte buffers now carry their capacity in a side table; text keeps the exact strlen behaviour it had. This is why nobody noticed El was mute. Measured: voice fetched from the engram reads f0=137 f0_end=116 kf=1269 f1=500 f2=2093 f3=3531, matching the 30s LPC measurement; render is 20160 samples at 16 kHz; both the rendered utterance and an own-core tone played aloud through CoreAudio with no Swift and no afplay in the chain.
peripheral — Neuron's I/O organ (own-core, local, consent-gated)
The interface made physical. Two afferent senses in, one efferent voice out — all reached the way the agentic surface reaches any tool.
MIC (hear) afferent device -> capture -> descriptor -> ingest -> geometry
CAMERA (see) afferent device -> capture -> descriptor -> ingest -> scene-geometry
SPEAKER(speak) efferent render WAV -> PLAY ALOUD out the speaker
Closes the conversational loop: hear (mic) -> understand (engram) -> speak (speaker).
Rails
- Own-core. macOS-native only: AVFoundation (camera/mic), CoreAudio voice- processing (AEC), afplay (speaker), ImageIO/CoreGraphics (frames), hand-rolled DSP (WAV, LPC, formant synthesis). No cloud, no heavy deps.
- Local-only. Raw streams are written to
out/and never egress..gitignorekeeps captured media out of git. - Consent-gated (two locks). A Neuron-level grant (
grant/revoke) and the OS TCC permission. Sensitive senses (camera/mic) fail closed without both. - Disclosed. Every device touch prints a
[peripheral]line on stderr.
Build
swiftc -O -o bin/periph src/periph.swift \
-framework AVFoundation -framework CoreMedia -framework Foundation \
-framework CoreGraphics -framework ImageIO -framework CoreImage
Commands
periph grant|revoke <camera|mic> # Neuron-level consent
periph status
periph speak <file.wav> # SPEAK ALOUD (efferent)
periph tone <out.wav> [hz] [sec] # own-core WAV synth
periph listen <sec> <out.wav> # MIC capture (afferent), 16k mono
periph see <out.jpg> # CAMERA one frame (afferent)
periph feat-audio <wav> | feat-image <jpg> # capture -> compact descriptor
periph ingest-audio|ingest-image <file> <engramURL> # descriptor -> engram node (geometry)
periph voiceprint <voice.wav> # extract F0 + formants F1-F5
periph imitate <voice.wav> <out.wav> # speak back in that voice (LPC resynthesis)
periph hear-imitate <sec> <out.wav> # MIC -> signature -> imitate -> SPEAK ALOUD
periph converse <manifest.json> [--authority F] [--barge-at S[:backchannel|:bargein]] [--resume] [--live-mic]
The afferent metabolism
A capture is never shipped raw. It becomes a compact descriptor — the afferent twin of the music instrument-signature:
- audio ->
[seconds, sr, ch, rms, peak, zcr, centroid, F0](~2400-6000x smaller) - image ->
[w, h, meanRGB, brightness, 3x3 luminance grid](~400000x smaller) - voice ->
[F0, F1..F5, bandwidths](11 numbers)
That descriptor is what the ingest organ (engram POST /api/nodes) turns into an
embedded node = geometry.
Voice by imitation
voiceprint/imitate are own-core LPC (autocorrelation + Levinson-Durbin, order
16 @ 16 kHz), formant extraction from the LPC spectral envelope, and source-filter
resynthesis (glottal impulse train at F0 through the all-pole formant filter). A
voice is grabbed by ear as ~a dozen numbers and spoken back — no training, no
stolen voice. Measured fidelity on real speech: resynthesized formants match the
source within 2-3%. The full phoneme->formant path for novel sentences is the
speech faculty's seam (elp audio surface profile); this engine provides the
formant synthesis primitive it renders through.
Interruptibility (native turn-taking)
converse plays the utterance as an ordered, salience-tagged meaning-plan
while the mic listens (full-duplex, AEC on so it never barges in on its own voice):
- barge-in: user speech -> pause on the spot (sample-accurate), not "finish the buffer."
- yield-or-hold: a decision grounded in the current segment's salience + progress
- the interrupter's authority — YIELD (stop) or HOLD ("hang on, let me finish").
- backchannel ("mm-hm"): brief/low -> keep going, resume seamlessly.
- resumable: on yield the remaining plan persists (
.resume.json);--resumepicks the thread back up ("as I was saying").
Live full-duplex uses --live-mic (OS AEC). Injected --barge-at drives the
decision loop deterministically for testing.