organ: the rest of the peripheral moves into El
El SDK CI - dev / build-and-test (pull_request) Failing after 4m18s
El SDK CI - dev / build-and-test (pull_request) Failing after 4m18s
The speaker and the voice-fetch landed in the previous commit. This is the remainder of the 939-line Swift program, ported, and the line it draws is between DEVICE and ARITHMETIC rather than between languages. Two things stay realizers, because they are the two things El cannot express as arithmetic: handing a buffer to the DAC and waiting for it to drain (el_audio_darwin.m), and asking the OS for samples off a mic or frames off a camera (el_capture_darwin.m). Both are their own translation units declared in el_runtime.h, never patches to el_runtime.c. Everything else is El. WAV decode, LPC autocorrelation, Levinson-Durbin at order 16, formant extraction off the all-pole envelope, source-filter resynthesis, and the three descriptors are organ_dsp.el. Consent, disclosure and the scene descriptor are organ.el. Barge-in, yield-or-hold, backchannel and resume are organ_converse.el. The organ never learns a word. Codes and phoneme geometry arrive from the language side; the organ turns them into samples and gets the samples out the speaker, and runs the same trip in reverse for the senses. No lexicon, no grapheme-to-phoneme, by design. Barge-in needed pause/resume and a real DAC position rather than a tick counter, because "finish the buffer" is not barge-in and a queue holding three buffers is a third of a second wrong about where it is. An injected barge also had to fire once rather than stay true, which is otherwise a livelock the moment a backchannel resumes. Measured against the Swift on out/mic_room.wav: seconds, rms, peak, zcr, centroid and F0 agree to every printed digit; formants F1-F5 and bandwidths B1-B5 are identical. imitate cannot match bit-for-bit because the Swift excites unvoiced frames with Double.random — two Swift runs correlate 0.957 with each other and El correlates 0.958 with Swift, so the port is as close to the original as the original is to itself. Verified end to end: consent fails closed on both locks, real mic capture (16000 frames), real camera frame (1920x1080 -> 15 numbers), voiceprint, imitate, hear-imitate, a voice learned by ear and fetched back out of the engram, and all five converse paths with real audio. The binary contains zero afplay/Swift strings and spawns no child process while speaking.
This commit is contained in:
Neuron
+195
-3
@@ -333,12 +333,204 @@ fn organ_voice_ingest(name: String, f0: Int, f0_end: Int, kf: Int, f1: Int, f2:
|
||||
// Turn the fetched geometry into the voice slot-map the render consumes. Kept
|
||||
// separate from the fetch so the organ never invents a voice: if the fetch came
|
||||
// back empty this returns empty too, and the caller has to deal with it.
|
||||
//
|
||||
// The slot-map is built here rather than by calling the render's own
|
||||
// constructor, so the organ carries NO dependency on the language faculty's
|
||||
// modules — it only has to agree with them about a wire format, which is the
|
||||
// looser and more honest coupling. (The layout is the same key/value [String]
|
||||
// convention lang_get / surface_get / voice_get all read.)
|
||||
fn organ_voice_profile(name: String, g: [Int]) -> [String] {
|
||||
let empty: [String] = native_list_empty()
|
||||
let r: [String] = native_list_empty()
|
||||
if native_list_len(g) < 6 {
|
||||
return empty
|
||||
return r
|
||||
}
|
||||
return voice_new(name, native_list_get(g, 0), native_list_get(g, 1), native_list_get(g, 2), 1000, 1000, 8)
|
||||
r = native_list_append(r, "name")
|
||||
r = native_list_append(r, name)
|
||||
r = native_list_append(r, "f0")
|
||||
r = native_list_append(r, int_to_str(native_list_get(g, 0)))
|
||||
r = native_list_append(r, "f0_end")
|
||||
r = native_list_append(r, int_to_str(native_list_get(g, 1)))
|
||||
r = native_list_append(r, "kf")
|
||||
r = native_list_append(r, int_to_str(native_list_get(g, 2)))
|
||||
r = native_list_append(r, "dur")
|
||||
r = native_list_append(r, "1000")
|
||||
r = native_list_append(r, "tilt")
|
||||
r = native_list_append(r, "1000")
|
||||
r = native_list_append(r, "breath")
|
||||
r = native_list_append(r, "8")
|
||||
return r
|
||||
}
|
||||
|
||||
// ── Scene geometry (afferent, camera) ────────────────────────────────────────
|
||||
//
|
||||
// The image half of the afferent metabolism, and the same principle as the
|
||||
// audio descriptor: a frame is never handed on raw. The realizer returns a
|
||||
// small pixel grid; THIS computes the descriptor, in El, because averaging
|
||||
// pixels is arithmetic and arithmetic is not a device concern.
|
||||
//
|
||||
// Returns 15 numbers — [w, h, meanR, meanG, meanB, brightness_pm, and a 3x3
|
||||
// luminance grid] — standing in for a multi-megapixel frame. The 3x3 grid is
|
||||
// the smallest thing that still says WHERE the light is, which is most of what
|
||||
// makes a scene comparable to another scene; a single brightness average would
|
||||
// make a lamp on the left indistinguishable from a lamp on the right.
|
||||
//
|
||||
// Luminance is Rec. 601 (0.299R + 0.587G + 0.114B), in integer per-mille, so
|
||||
// the descriptor is reproducible rather than subject to float drift.
|
||||
fn organ_image_descriptor() -> [Int] {
|
||||
let out: [Int] = native_list_empty()
|
||||
let frame: Any = camera_capture_rgb()
|
||||
if frame == 0 {
|
||||
return out
|
||||
}
|
||||
let w: Int = el_map_get(frame, "width")
|
||||
let h: Int = el_map_get(frame, "height")
|
||||
let gw: Int = el_map_get(frame, "grid_w")
|
||||
let gh: Int = el_map_get(frame, "grid_h")
|
||||
let px: [Int] = el_map_get(frame, "pixels")
|
||||
let np: Int = native_list_len(px)
|
||||
if np < 3 {
|
||||
return out
|
||||
}
|
||||
let count: Int = np / 3
|
||||
let rsum: Int = 0
|
||||
let gsum: Int = 0
|
||||
let bsum: Int = 0
|
||||
// 3x3 accumulators, row-major
|
||||
let cell: [Int] = native_list_empty()
|
||||
let cn: [Int] = native_list_empty()
|
||||
let z: Int = 0
|
||||
while z < 9 {
|
||||
cell = native_list_append(cell, 0)
|
||||
cn = native_list_append(cn, 0)
|
||||
z = z + 1
|
||||
}
|
||||
// El has no list-set, so the cells are summed into parallel scalars and
|
||||
// reassembled — nine explicit accumulators would be worse to read than one
|
||||
// pass per cell over a grid this small.
|
||||
let c0: Int = 0
|
||||
let c1: Int = 0
|
||||
let c2: Int = 0
|
||||
let c3: Int = 0
|
||||
let c4: Int = 0
|
||||
let c5: Int = 0
|
||||
let c6: Int = 0
|
||||
let c7: Int = 0
|
||||
let c8: Int = 0
|
||||
let n0: Int = 0
|
||||
let n1: Int = 0
|
||||
let n2: Int = 0
|
||||
let n3: Int = 0
|
||||
let n4: Int = 0
|
||||
let n5: Int = 0
|
||||
let n6: Int = 0
|
||||
let n7: Int = 0
|
||||
let n8: Int = 0
|
||||
let i: Int = 0
|
||||
while i < count {
|
||||
let r: Int = native_list_get(px, i * 3)
|
||||
let g: Int = native_list_get(px, i * 3 + 1)
|
||||
let b: Int = native_list_get(px, i * 3 + 2)
|
||||
rsum = rsum + r
|
||||
gsum = gsum + g
|
||||
bsum = bsum + b
|
||||
let lum: Int = (299 * r + 587 * g + 114 * b) / 1000
|
||||
let x: Int = i - (i / gw) * gw
|
||||
let y: Int = i / gw
|
||||
let cx: Int = x * 3 / gw
|
||||
let cy: Int = y * 3 / gh
|
||||
if cx > 2 {
|
||||
cx = 2
|
||||
}
|
||||
if cy > 2 {
|
||||
cy = 2
|
||||
}
|
||||
let idx: Int = cy * 3 + cx
|
||||
if idx == 0 {
|
||||
c0 = c0 + lum
|
||||
n0 = n0 + 1
|
||||
}
|
||||
if idx == 1 {
|
||||
c1 = c1 + lum
|
||||
n1 = n1 + 1
|
||||
}
|
||||
if idx == 2 {
|
||||
c2 = c2 + lum
|
||||
n2 = n2 + 1
|
||||
}
|
||||
if idx == 3 {
|
||||
c3 = c3 + lum
|
||||
n3 = n3 + 1
|
||||
}
|
||||
if idx == 4 {
|
||||
c4 = c4 + lum
|
||||
n4 = n4 + 1
|
||||
}
|
||||
if idx == 5 {
|
||||
c5 = c5 + lum
|
||||
n5 = n5 + 1
|
||||
}
|
||||
if idx == 6 {
|
||||
c6 = c6 + lum
|
||||
n6 = n6 + 1
|
||||
}
|
||||
if idx == 7 {
|
||||
c7 = c7 + lum
|
||||
n7 = n7 + 1
|
||||
}
|
||||
if idx == 8 {
|
||||
c8 = c8 + lum
|
||||
n8 = n8 + 1
|
||||
}
|
||||
i = i + 1
|
||||
}
|
||||
let rA: Int = rsum / count
|
||||
let gA: Int = gsum / count
|
||||
let bA: Int = bsum / count
|
||||
let bright: Int = (299 * rA + 587 * gA + 114 * bA) / 255
|
||||
out = native_list_append(out, w)
|
||||
out = native_list_append(out, h)
|
||||
out = native_list_append(out, rA)
|
||||
out = native_list_append(out, gA)
|
||||
out = native_list_append(out, bA)
|
||||
out = native_list_append(out, bright)
|
||||
if n0 < 1 {
|
||||
n0 = 1
|
||||
}
|
||||
if n1 < 1 {
|
||||
n1 = 1
|
||||
}
|
||||
if n2 < 1 {
|
||||
n2 = 1
|
||||
}
|
||||
if n3 < 1 {
|
||||
n3 = 1
|
||||
}
|
||||
if n4 < 1 {
|
||||
n4 = 1
|
||||
}
|
||||
if n5 < 1 {
|
||||
n5 = 1
|
||||
}
|
||||
if n6 < 1 {
|
||||
n6 = 1
|
||||
}
|
||||
if n7 < 1 {
|
||||
n7 = 1
|
||||
}
|
||||
if n8 < 1 {
|
||||
n8 = 1
|
||||
}
|
||||
out = native_list_append(out, c0 / n0)
|
||||
out = native_list_append(out, c1 / n1)
|
||||
out = native_list_append(out, c2 / n2)
|
||||
out = native_list_append(out, c3 / n3)
|
||||
out = native_list_append(out, c4 / n4)
|
||||
out = native_list_append(out, c5 / n5)
|
||||
out = native_list_append(out, c6 / n6)
|
||||
out = native_list_append(out, c7 / n7)
|
||||
out = native_list_append(out, c8 / n8)
|
||||
organ_disclose("FEAT(image): 15-number scene-geometry vs " + int_to_str(w * h * 3) + " pixel-channels — the descriptor travels, the frame does not.")
|
||||
return out
|
||||
}
|
||||
|
||||
// ── Own-core tone ────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -0,0 +1,459 @@
|
||||
// organ_cli.el — the organ's command surface. main() lives here.
|
||||
//
|
||||
// One binary, the same verbs the Swift program had, and nothing behind them
|
||||
// except El and two thin device realizers. This file is the proof surface: if
|
||||
// `organ speak` makes a sound and no Swift binary is in the process tree, the
|
||||
// claim in organ.el's header is true.
|
||||
//
|
||||
// Verbs, and what each one demonstrates:
|
||||
//
|
||||
// grant/revoke/status the Neuron-level consent lock, inspectable
|
||||
// speak <wav> efferent — audio out of El's own speaker
|
||||
// tone <hz> <ms> own-core synthesis: computed in El, played by El,
|
||||
// never touching the disk
|
||||
// say <name> <codes...> fetch a VOICE FROM THE ENGRAM and render through it
|
||||
// listen <sec> <out> afferent — mic capture, consent-gated, fails closed
|
||||
// see <out.jpg> afferent — one camera frame, same two locks
|
||||
// wav-info <wav> WAV geometry, parsed in El
|
||||
// feat-audio <wav> capture -> compact descriptor (8 numbers)
|
||||
// feat-image <jpg> frame -> compact scene-geometry
|
||||
// voiceprint <wav> F0 + formants F1-F5 by LPC, in El
|
||||
// imitate <in> <out> LPC analysis-resynthesis, in El
|
||||
// hear-imitate <sec> the closed loop: hear a voice, take its signature,
|
||||
// speak back in it
|
||||
// ingest-audio <wav> descriptor -> engram node (the capture becomes geometry)
|
||||
// ingest-voice <wav> <n> voiceprint -> engram voice region (how a voice is learned)
|
||||
// converse <manifest> full-duplex interruptible utterance
|
||||
//
|
||||
// The descriptors are the point of the afferent half. A capture is NEVER handed
|
||||
// on raw: a three-second recording is ~48,000 samples and what leaves this
|
||||
// process is eight numbers. That is both the privacy rail (the stream stays
|
||||
// local because only its shape travels) and the reason the engram can hold a
|
||||
// perception at all — geometry is storable, a waveform is not.
|
||||
|
||||
fn cli_usage() -> Bool {
|
||||
println("organ — Neuron's I/O organ, native El (own-core, local, consent-gated)")
|
||||
println(" grant|revoke <camera|mic> Neuron-level consent")
|
||||
println(" status consent + device state")
|
||||
println(" speak <file.wav> play a WAV aloud (efferent)")
|
||||
println(" tone [hz] [ms] synthesize and play, no file at all")
|
||||
println(" say <voice> <CODE> [CODE...] fetch voice FROM THE ENGRAM, render, speak")
|
||||
println(" listen <sec> <out.wav> mic capture 16k mono (afferent)")
|
||||
println(" see <out.jpg> one camera frame (afferent)")
|
||||
println(" wav-info <file.wav> WAV geometry")
|
||||
println(" feat-audio <file.wav> compact audio descriptor (8 numbers)")
|
||||
println(" feat-image compact scene-geometry from the camera")
|
||||
println(" voiceprint <voice.wav> F0 + formants F1-F5 (LPC)")
|
||||
println(" imitate <in.wav> <out.wav> LPC analysis-resynthesis")
|
||||
println(" hear-imitate <sec> <out.wav> mic -> signature -> imitate -> speak aloud")
|
||||
println(" ingest-audio <file.wav> descriptor -> engram node (geometry)")
|
||||
println(" ingest-voice <voice.wav> <n> voiceprint -> engram voice region")
|
||||
println(" converse <manifest.json> [--authority PM] [--barge-at MS[:kind]] [--live-mic] [--resume]")
|
||||
return true
|
||||
}
|
||||
|
||||
// The engram the organ reads and writes. Its own store, never production's.
|
||||
fn cli_engram_dir() -> String {
|
||||
let d: String = env("ORGAN_ENGRAM")
|
||||
if str_eq(d, "") {
|
||||
return "peripheral/.engram"
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
fn cli_open_engram() -> Bool {
|
||||
let dir: String = cli_engram_dir()
|
||||
fs_mkdir(dir)
|
||||
let ok: Int = engram_store_boot(dir)
|
||||
if ok == 1 {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ── formatting helpers ───────────────────────────────────────────────────────
|
||||
|
||||
fn cli_f(v: Float, dec: Int) -> String {
|
||||
return format_float(v, dec)
|
||||
}
|
||||
|
||||
// ── the descriptor, printed and ingested ─────────────────────────────────────
|
||||
//
|
||||
// [seconds, sr, ch, rms, peak, zcr, centroid, f0] — the same eight numbers the
|
||||
// Swift produced, computed in El, and the compression ratio is the headline:
|
||||
// a few dozen bytes standing in for a few hundred kilobytes.
|
||||
fn cli_audio_descriptor_text(v: [Float], path: String) -> String {
|
||||
let secs: Float = native_list_get(v, 0)
|
||||
let sr: Float = native_list_get(v, 1)
|
||||
let ch: Float = native_list_get(v, 2)
|
||||
let rms: Float = native_list_get(v, 3)
|
||||
let peak: Float = native_list_get(v, 4)
|
||||
let zcr: Float = native_list_get(v, 5)
|
||||
let cen: Float = native_list_get(v, 6)
|
||||
let f0: Float = native_list_get(v, 7)
|
||||
return "Heard sound (afferent, mic): " + cli_f(secs, 2) + "s at " + cli_f(sr, 0) + "Hz. RMS energy " + cli_f(rms, 4) + ", peak " + cli_f(peak, 4) + ", zero-crossing rate " + cli_f(zcr, 0) + "Hz, spectral centroid " + cli_f(cen, 0) + "Hz, estimated voice pitch F0 " + cli_f(f0, 0) + "Hz. Compact voice/sound signature (8 numbers) — phonetic geometry seed."
|
||||
}
|
||||
|
||||
fn cli_feat_audio(path: String) -> Bool {
|
||||
let v: [Float] = dsp_compute_audio(path)
|
||||
if native_list_len(v) < 8 {
|
||||
println("{\"ok\": false, \"op\": \"feat-audio\", \"error\": \"cannot read PCM\"}")
|
||||
return false
|
||||
}
|
||||
organ_disclose("FEAT(audio): 8-number signature vs " + int_to_str(float_to_int(native_list_get(v, 0) * native_list_get(v, 1))) + " raw samples.")
|
||||
println("{\"ok\": true, \"op\": \"feat-audio\", \"file\": \"" + path + "\", \"seconds\": " + cli_f(native_list_get(v, 0), 4) + ", \"sample_rate\": " + cli_f(native_list_get(v, 1), 0) + ", \"channels\": " + cli_f(native_list_get(v, 2), 0) + ", \"rms\": " + cli_f(native_list_get(v, 3), 6) + ", \"peak\": " + cli_f(native_list_get(v, 4), 6) + ", \"zcr_hz\": " + cli_f(native_list_get(v, 5), 4) + ", \"centroid_hz\": " + cli_f(native_list_get(v, 6), 4) + ", \"f0_hz\": " + cli_f(native_list_get(v, 7), 4) + "}")
|
||||
return true
|
||||
}
|
||||
|
||||
fn cli_voiceprint(path: String) -> Bool {
|
||||
let v: [Float] = dsp_voiceprint(path)
|
||||
if native_list_len(v) < 4 {
|
||||
println("{\"ok\": false, \"op\": \"voiceprint\", \"error\": \"cannot read speech\"}")
|
||||
return false
|
||||
}
|
||||
let nf: Int = float_to_int(native_list_get(v, 3))
|
||||
let fs: String = ""
|
||||
let bs: String = ""
|
||||
let i: Int = 0
|
||||
while i < nf {
|
||||
if i > 0 {
|
||||
fs = fs + ", "
|
||||
bs = bs + ", "
|
||||
}
|
||||
fs = fs + cli_f(native_list_get(v, 4 + i * 2), 3)
|
||||
bs = bs + cli_f(native_list_get(v, 5 + i * 2), 3)
|
||||
i = i + 1
|
||||
}
|
||||
println("{\"ok\": true, \"op\": \"voiceprint\", \"file\": \"" + path + "\", \"f0_hz\": " + cli_f(native_list_get(v, 0), 4) + ", \"f0_range\": [" + cli_f(native_list_get(v, 1), 4) + ", " + cli_f(native_list_get(v, 2), 4) + "], \"formants_hz\": [" + fs + "], \"bandwidths_hz\": [" + bs + "]}")
|
||||
return true
|
||||
}
|
||||
|
||||
// ── main ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
fn main() {
|
||||
let a: [String] = args()
|
||||
let n: Int = native_list_len(a)
|
||||
if n < 1 {
|
||||
cli_usage()
|
||||
return
|
||||
}
|
||||
let cmd: String = native_list_get(a, 0)
|
||||
|
||||
// ---- consent -----------------------------------------------------------
|
||||
if str_eq(cmd, "grant") {
|
||||
if n < 2 {
|
||||
println("grant needs a device")
|
||||
return
|
||||
}
|
||||
organ_grant(native_list_get(a, 1))
|
||||
println("{\"ok\": true, \"op\": \"grant\", \"consent\": \"" + organ_consent_status() + "\"}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "revoke") {
|
||||
if n < 2 {
|
||||
println("revoke needs a device")
|
||||
return
|
||||
}
|
||||
organ_revoke(native_list_get(a, 1))
|
||||
println("{\"ok\": true, \"op\": \"revoke\", \"consent\": \"" + organ_consent_status() + "\"}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "status") {
|
||||
println("{\"ok\": true, \"op\": \"status\", \"consent\": \"" + organ_consent_status() + "\", \"speaker\": \"" + speaker_name() + "\", \"speaker_available\": " + int_to_str(speaker_available()) + ", \"mic_os_authorized\": " + int_to_str(mic_available()) + ", \"camera_os_authorized\": " + int_to_str(camera_available()) + "}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- efferent ----------------------------------------------------------
|
||||
if str_eq(cmd, "speak") {
|
||||
if n < 2 {
|
||||
println("speak needs a wav")
|
||||
return
|
||||
}
|
||||
let ok: Bool = organ_speak_wav(native_list_get(a, 1))
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"speak\", \"played_aloud\": " + bool_to_str(ok) + "}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "tone") {
|
||||
let hz: Int = 220
|
||||
let ms: Int = 1000
|
||||
if n >= 2 {
|
||||
hz = str_to_int(native_list_get(a, 1))
|
||||
}
|
||||
if n >= 3 {
|
||||
ms = str_to_int(native_list_get(a, 2))
|
||||
}
|
||||
let s: [Int] = organ_tone(hz, ms, 16000)
|
||||
let ok: Bool = organ_speak_samples(s, 16000)
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"tone\", \"hz\": " + int_to_str(hz) + ", \"ms\": " + int_to_str(ms) + ", \"samples\": " + int_to_str(native_list_len(s)) + ", \"file\": null}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- the voice, from the engram ----------------------------------------
|
||||
if str_eq(cmd, "say") {
|
||||
if n < 3 {
|
||||
println("say needs <voice> <CODE> [CODE...]")
|
||||
return
|
||||
}
|
||||
cli_open_engram()
|
||||
let vname: String = native_list_get(a, 1)
|
||||
let g: [Int] = organ_voice_fetch(vname)
|
||||
if native_list_len(g) < 6 {
|
||||
println("{\"ok\": false, \"op\": \"say\", \"error\": \"no voice region '" + vname + "' in the engram\"}")
|
||||
return
|
||||
}
|
||||
// Codes and the phoneme map come from the LANGUAGE side. The organ does
|
||||
// not know what a word is and never looks one up.
|
||||
let pmap: [String] = ingest_phonetics("elp/data/phonetics.psv")
|
||||
let codes: [String] = native_list_empty()
|
||||
let i: Int = 2
|
||||
while i < n {
|
||||
codes = native_list_append(codes, native_list_get(a, i))
|
||||
i = i + 1
|
||||
}
|
||||
let voice: [String] = organ_voice_profile(vname, g)
|
||||
let s: [Int] = synth_codes(codes, voice, pmap)
|
||||
let ok: Bool = organ_speak_samples(s, 16000)
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"say\", \"voice\": \"" + vname + "\", \"f0\": " + int_to_str(native_list_get(g, 0)) + ", \"kf\": " + int_to_str(native_list_get(g, 2)) + ", \"codes\": " + int_to_str(native_list_len(codes)) + ", \"samples\": " + int_to_str(native_list_len(s)) + "}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- afferent ----------------------------------------------------------
|
||||
if str_eq(cmd, "listen") {
|
||||
if n < 3 {
|
||||
println("listen needs <sec> <out.wav>")
|
||||
return
|
||||
}
|
||||
let secs: Int = str_to_int(native_list_get(a, 1))
|
||||
let out: String = native_list_get(a, 2)
|
||||
if organ_may_listen() == false {
|
||||
println("{\"ok\": false, \"op\": \"listen\", \"error\": \"consent denied (fails closed)\"}")
|
||||
return
|
||||
}
|
||||
organ_disclose("MIC: capturing " + int_to_str(secs) + "s (16 kHz mono, LOCAL, never egresses).")
|
||||
let s: [Int] = mic_capture_pcm16(secs, 16000)
|
||||
let got: Int = native_list_len(s)
|
||||
if got <= 0 {
|
||||
println("{\"ok\": false, \"op\": \"listen\", \"error\": \"capture returned nothing\"}")
|
||||
return
|
||||
}
|
||||
let ok: Bool = write_wav(s, 16000, out)
|
||||
organ_disclose("MIC: captured " + int_to_str(got) + " frames — ready to hand to the ingest organ.")
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"listen\", \"file\": \"" + out + "\", \"frames\": " + int_to_str(got) + ", \"sample_rate\": 16000}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "see") {
|
||||
if n < 2 {
|
||||
println("see needs an out path")
|
||||
return
|
||||
}
|
||||
if organ_may_see() == false {
|
||||
println("{\"ok\": false, \"op\": \"see\", \"error\": \"consent denied (fails closed)\"}")
|
||||
return
|
||||
}
|
||||
organ_disclose("CAMERA: capturing one frame (LOCAL, never egresses).")
|
||||
let ok: Int = camera_capture_jpeg(native_list_get(a, 1))
|
||||
println("{\"ok\": " + int_to_str(ok) + ", \"op\": \"see\", \"file\": \"" + native_list_get(a, 1) + "\"}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- descriptors -------------------------------------------------------
|
||||
if str_eq(cmd, "wav-info") {
|
||||
if n < 2 {
|
||||
println("wav-info needs a wav")
|
||||
return
|
||||
}
|
||||
let p: String = native_list_get(a, 1)
|
||||
let w: [Float] = dsp_read_wav(p)
|
||||
if dsp_wav_n(w) <= 0 {
|
||||
println("{\"ok\": false, \"op\": \"wav-info\"}")
|
||||
return
|
||||
}
|
||||
println("{\"ok\": true, \"op\": \"wav-info\", \"sample_rate\": " + int_to_str(dsp_wav_sr(w)) + ", \"channels\": " + int_to_str(dsp_wav_ch(w)) + ", \"frames\": " + int_to_str(dsp_wav_n(w)) + "}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "feat-audio") {
|
||||
if n < 2 {
|
||||
println("feat-audio needs a wav")
|
||||
return
|
||||
}
|
||||
cli_feat_audio(native_list_get(a, 1))
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "feat-image") {
|
||||
if organ_may_see() == false {
|
||||
println("{\"ok\": false, \"op\": \"feat-image\", \"error\": \"consent denied (fails closed)\"}")
|
||||
return
|
||||
}
|
||||
let f: [Int] = organ_image_descriptor()
|
||||
if native_list_len(f) < 15 {
|
||||
println("{\"ok\": false, \"op\": \"feat-image\", \"error\": \"no frame\"}")
|
||||
return
|
||||
}
|
||||
let grid: String = ""
|
||||
let i: Int = 6
|
||||
while i < 15 {
|
||||
if i > 6 {
|
||||
grid = grid + ", "
|
||||
}
|
||||
grid = grid + int_to_str(native_list_get(f, i))
|
||||
i = i + 1
|
||||
}
|
||||
println("{\"ok\": true, \"op\": \"feat-image\", \"width\": " + int_to_str(native_list_get(f, 0)) + ", \"height\": " + int_to_str(native_list_get(f, 1)) + ", \"mean_rgb\": [" + int_to_str(native_list_get(f, 2)) + ", " + int_to_str(native_list_get(f, 3)) + ", " + int_to_str(native_list_get(f, 4)) + "], \"brightness_pm\": " + int_to_str(native_list_get(f, 5)) + ", \"luma_grid\": [" + grid + "]}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "voiceprint") {
|
||||
if n < 2 {
|
||||
println("voiceprint needs a wav")
|
||||
return
|
||||
}
|
||||
cli_voiceprint(native_list_get(a, 1))
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "imitate") {
|
||||
if n < 3 {
|
||||
println("imitate needs <in.wav> <out.wav>")
|
||||
return
|
||||
}
|
||||
let s: [Int] = dsp_imitate(native_list_get(a, 1))
|
||||
if native_list_len(s) <= 0 {
|
||||
println("{\"ok\": false, \"op\": \"imitate\"}")
|
||||
return
|
||||
}
|
||||
let ok: Bool = write_wav(s, 16000, native_list_get(a, 2))
|
||||
organ_disclose("IMITATE: rebuilt the voice from its own LPC signature (own-core, no training, no stolen voice).")
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"imitate\", \"out\": \"" + native_list_get(a, 2) + "\", \"samples\": " + int_to_str(native_list_len(s)) + ", \"method\": \"LPC analysis-resynthesis\"}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "hear-imitate") {
|
||||
if n < 3 {
|
||||
println("hear-imitate needs <sec> <out.wav>")
|
||||
return
|
||||
}
|
||||
let secs: Int = str_to_int(native_list_get(a, 1))
|
||||
let out: String = native_list_get(a, 2)
|
||||
if organ_may_listen() == false {
|
||||
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"consent denied (fails closed)\"}")
|
||||
return
|
||||
}
|
||||
let heard: String = out + ".heard.wav"
|
||||
organ_disclose("HEAR-IMITATE: open the ear, listen " + int_to_str(secs) + "s, take the voice, speak it back.")
|
||||
let s: [Int] = mic_capture_pcm16(secs, 16000)
|
||||
if native_list_len(s) <= 0 {
|
||||
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"capture returned nothing\"}")
|
||||
return
|
||||
}
|
||||
write_wav(s, 16000, heard)
|
||||
let re: [Int] = dsp_imitate(heard)
|
||||
if native_list_len(re) <= 0 {
|
||||
println("{\"ok\": false, \"op\": \"hear-imitate\", \"error\": \"could not model the voice\"}")
|
||||
return
|
||||
}
|
||||
write_wav(re, 16000, out)
|
||||
let ok: Bool = organ_speak_samples(re, 16000)
|
||||
println("{\"ok\": " + bool_to_str(ok) + ", \"op\": \"hear-imitate\", \"heard\": \"" + heard + "\", \"out\": \"" + out + "\", \"spoke_aloud\": " + bool_to_str(ok) + "}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- the afferent wire: descriptor -> geometry --------------------------
|
||||
if str_eq(cmd, "ingest-audio") {
|
||||
if n < 2 {
|
||||
println("ingest-audio needs a wav")
|
||||
return
|
||||
}
|
||||
let p: String = native_list_get(a, 1)
|
||||
let v: [Float] = dsp_compute_audio(p)
|
||||
if native_list_len(v) < 8 {
|
||||
println("{\"ok\": false, \"op\": \"ingest-audio\"}")
|
||||
return
|
||||
}
|
||||
cli_open_engram()
|
||||
let content: String = cli_audio_descriptor_text(v, p)
|
||||
let id: String = engram_node(content, "Observation", 70)
|
||||
engram_store_checkpoint()
|
||||
organ_disclose("INGEST: the capture is now GEOMETRY in the engram (node " + id + ") — the descriptor travelled, the stream did not.")
|
||||
println("{\"ok\": true, \"op\": \"ingest-audio\", \"node_id\": \"" + id + "\", \"content\": \"" + content + "\"}")
|
||||
return
|
||||
}
|
||||
if str_eq(cmd, "ingest-voice") {
|
||||
if n < 3 {
|
||||
println("ingest-voice needs <voice.wav> <name>")
|
||||
return
|
||||
}
|
||||
let p: String = native_list_get(a, 1)
|
||||
let name: String = native_list_get(a, 2)
|
||||
let v: [Float] = dsp_voiceprint(p)
|
||||
if native_list_len(v) < 10 {
|
||||
println("{\"ok\": false, \"op\": \"ingest-voice\", \"error\": \"no voiced frames\"}")
|
||||
return
|
||||
}
|
||||
cli_open_engram()
|
||||
let f0: Int = float_to_int(native_list_get(v, 0))
|
||||
let f1: Int = float_to_int(native_list_get(v, 4))
|
||||
let f2: Int = float_to_int(native_list_get(v, 6))
|
||||
let f3: Int = float_to_int(native_list_get(v, 8))
|
||||
// kf is the vocal-tract scale: this speaker's F1 against the nominal
|
||||
// /AA/ F1 of 730 Hz. One number standing for a tract length.
|
||||
let kf: Int = 1000 * f1 / 730
|
||||
let f0e: Int = f0 * 85 / 100
|
||||
let id: String = organ_voice_ingest(name, f0, f0e, kf, f1, f2, f3, "el-organ-lpc-voiceprint", "COARSE")
|
||||
engram_store_checkpoint()
|
||||
println("{\"ok\": true, \"op\": \"ingest-voice\", \"node_id\": \"" + id + "\", \"name\": \"" + name + "\", \"f0\": " + int_to_str(f0) + ", \"kf\": " + int_to_str(kf) + ", \"f1\": " + int_to_str(f1) + ", \"f2\": " + int_to_str(f2) + ", \"f3\": " + int_to_str(f3) + "}")
|
||||
return
|
||||
}
|
||||
|
||||
// ---- converse ----------------------------------------------------------
|
||||
if str_eq(cmd, "converse") {
|
||||
if n < 2 {
|
||||
println("converse needs a manifest")
|
||||
return
|
||||
}
|
||||
let mf: String = native_list_get(a, 1)
|
||||
let authority: Int = 500
|
||||
let barge: Int = 0 - 1
|
||||
let kind: String = "bargein"
|
||||
let live: Bool = false
|
||||
let resume: Bool = false
|
||||
let i: Int = 2
|
||||
while i < n {
|
||||
let f: String = native_list_get(a, i)
|
||||
if str_eq(f, "--authority") {
|
||||
if i + 1 < n {
|
||||
authority = str_to_int(native_list_get(a, i + 1))
|
||||
i = i + 1
|
||||
}
|
||||
}
|
||||
if str_eq(f, "--barge-at") {
|
||||
if i + 1 < n {
|
||||
let spec: String = native_list_get(a, i + 1)
|
||||
let c: Int = str_index_of(spec, ":")
|
||||
if c < 0 {
|
||||
barge = str_to_int(spec)
|
||||
} else {
|
||||
barge = str_to_int(str_slice(spec, 0, c))
|
||||
kind = str_slice(spec, c + 1, str_len(spec))
|
||||
}
|
||||
i = i + 1
|
||||
}
|
||||
}
|
||||
if str_eq(f, "--live-mic") {
|
||||
live = true
|
||||
}
|
||||
if str_eq(f, "--resume") {
|
||||
resume = true
|
||||
}
|
||||
i = i + 1
|
||||
}
|
||||
let plan: [String] = conv_load_manifest(mf)
|
||||
if resume {
|
||||
plan = conv_load_resume()
|
||||
organ_disclose("CONVERSE: resuming — \"as I was saying...\" (" + int_to_str(plan_count(plan)) + " segments left).")
|
||||
} else {
|
||||
organ_disclose("CONVERSE: utterance = \"" + conv_utterance(mf) + "\" (" + int_to_str(plan_count(plan)) + " segments).")
|
||||
}
|
||||
let stopped: Int = conv_run(plan, authority, barge, kind, live)
|
||||
println("{\"ok\": true, \"op\": \"converse\", \"stopped_at\": " + int_to_str(stopped) + ", \"complete\": " + bool_to_str(stopped < 0) + "}")
|
||||
return
|
||||
}
|
||||
|
||||
cli_usage()
|
||||
}
|
||||
@@ -0,0 +1,454 @@
|
||||
// organ_converse.el — full-duplex, interruptible speech. The turn-taking organ.
|
||||
//
|
||||
// WHAT THIS IS FOR. A system that plays an utterance to completion and only
|
||||
// then listens is not conversational, it is a loudspeaker with a queue. Being
|
||||
// interruptible is not a feature bolted onto speech; it is most of what makes
|
||||
// speech social. So the utterance is not a blob of audio — it is an ordered,
|
||||
// SALIENCE-TAGGED MEANING-PLAN, and the organ speaks it while listening, decides
|
||||
// what to do when interrupted, and can pick the thread back up afterwards.
|
||||
//
|
||||
// THREE THINGS HAVE TO BE TRUE, and each one is a place naive implementations
|
||||
// go wrong:
|
||||
//
|
||||
// Barge-in is AT THE SAMPLE. When the mic hears speech, output stops on the
|
||||
// spot — not at the end of the current buffer, not at the end of the segment.
|
||||
// A listener experiences even a fifth of a second of continued talking as
|
||||
// being talked over. This is why the speaker realizer has pause/resume and
|
||||
// reports played_frames: "finish the buffer" is not barge-in.
|
||||
//
|
||||
// Yield-or-hold is a DECISION, not a rule. Stopping every time anyone makes a
|
||||
// noise is its own failure — it means Neuron can never finish a sentence that
|
||||
// matters. So the choice is grounded: how salient is what I am mid-saying,
|
||||
// how close am I to done, and how much authority does the interrupter have.
|
||||
// Holding the floor is justified when what I am saying matters AND finishing
|
||||
// is cheap AND the interrupter is not high-priority. Otherwise yield, because
|
||||
// the polite default is the right default.
|
||||
//
|
||||
// A backchannel is NOT an interruption. "mm-hm" means keep going. Treating it
|
||||
// as a barge-in makes the system stop every three seconds during ordinary
|
||||
// listening behaviour, which is worse than not listening at all. It is
|
||||
// distinguished by being brief and low-energy: sample again shortly after
|
||||
// onset, and if the speech already died away it was a backchannel.
|
||||
//
|
||||
// AND THE UTTERANCE SURVIVES. On yield, the remaining plan is persisted, so
|
||||
// Neuron can resume — "as I was saying" — instead of losing the thought. An
|
||||
// interruption should cost a turn, not the content.
|
||||
//
|
||||
// The AEC rail: the microphone runs with the OS voice-processing unit enabled
|
||||
// so it does not hear our own speaker. Without it Neuron barges in on its own
|
||||
// voice on the first syllable and the whole loop is unusable in a real room.
|
||||
//
|
||||
// Note what is NOT here: nothing about words. A segment carries a `text` field
|
||||
// purely as a label for disclosure. The organ speaks pre-rendered audio and
|
||||
// never inspects language — that is the language faculty's, and the seam holds.
|
||||
|
||||
// ── The meaning-plan ─────────────────────────────────────────────────────────
|
||||
//
|
||||
// Stored as a flat [String] with stride 3 — file, salience-per-mille, text —
|
||||
// because El has no record type and parallel lists drift out of step under
|
||||
// editing. Salience is an integer per-mille rather than a Float so the decision
|
||||
// arithmetic stays exact and reproducible; a turn-taking decision that varies
|
||||
// with floating-point rounding is not one you can debug.
|
||||
|
||||
fn plan_new() -> [String] {
|
||||
return native_list_empty()
|
||||
}
|
||||
|
||||
fn plan_add(plan: [String], file: String, salience_pm: Int, text: String) -> [String] {
|
||||
let p: [String] = plan
|
||||
p = native_list_append(p, file)
|
||||
p = native_list_append(p, int_to_str(salience_pm))
|
||||
p = native_list_append(p, text)
|
||||
return p
|
||||
}
|
||||
|
||||
fn plan_count(plan: [String]) -> Int {
|
||||
return native_list_len(plan) / 3
|
||||
}
|
||||
|
||||
fn plan_file(plan: [String], i: Int) -> String {
|
||||
return native_list_get(plan, i * 3)
|
||||
}
|
||||
|
||||
fn plan_salience(plan: [String], i: Int) -> Int {
|
||||
return str_to_int(native_list_get(plan, i * 3 + 1))
|
||||
}
|
||||
|
||||
fn plan_text(plan: [String], i: Int) -> String {
|
||||
return native_list_get(plan, i * 3 + 2)
|
||||
}
|
||||
|
||||
// ── Manifest ─────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// {"utterance": "...", "segments": [{"file":..., "salience":0.9, "text":"..."}]}
|
||||
// Salience arrives as a 0..1 float in the manifest and is converted once, here,
|
||||
// at the edge — the same discipline the runtime uses for wire encodings.
|
||||
|
||||
fn conv_salience_pm(raw: String) -> Int {
|
||||
// "0.85" -> 850. Parsed by hand rather than through a float so a manifest
|
||||
// typo degrades to a visible number instead of a silent 0.0.
|
||||
let dot: Int = str_index_of(raw, ".")
|
||||
if dot < 0 {
|
||||
let whole: Int = str_to_int(raw)
|
||||
return whole * 1000
|
||||
}
|
||||
let ip: Int = str_to_int(str_slice(raw, 0, dot))
|
||||
let frac: String = str_slice(raw, dot + 1, str_len(raw))
|
||||
let pm: Int = 0
|
||||
let scale: Int = 100
|
||||
let i: Int = 0
|
||||
while i < 3 {
|
||||
let d: Int = 0
|
||||
if i < str_len(frac) {
|
||||
let c: Int = str_char_code(frac, i)
|
||||
if c >= 48 {
|
||||
if c <= 57 {
|
||||
d = c - 48
|
||||
}
|
||||
}
|
||||
}
|
||||
pm = pm + d * scale
|
||||
scale = scale / 10
|
||||
i = i + 1
|
||||
}
|
||||
return ip * 1000 + pm
|
||||
}
|
||||
|
||||
fn conv_load_manifest(path: String) -> [String] {
|
||||
let plan: [String] = plan_new()
|
||||
let raw: String = fs_read(path)
|
||||
if str_eq(raw, "") {
|
||||
organ_disclose("CONVERSE: cannot read manifest " + path)
|
||||
return plan
|
||||
}
|
||||
let segs: String = json_get_raw(raw, "segments")
|
||||
let n: Int = json_array_len(segs)
|
||||
let i: Int = 0
|
||||
while i < n {
|
||||
let seg: String = json_array_get(segs, i)
|
||||
let file: String = json_get_string(seg, "file")
|
||||
let text: String = json_get_string(seg, "text")
|
||||
let sal: String = json_get_raw(seg, "salience")
|
||||
let pm: Int = conv_salience_pm(sal)
|
||||
if pm <= 0 {
|
||||
pm = 500
|
||||
}
|
||||
plan = plan_add(plan, file, pm, text)
|
||||
i = i + 1
|
||||
}
|
||||
return plan
|
||||
}
|
||||
|
||||
fn conv_utterance(path: String) -> String {
|
||||
let raw: String = fs_read(path)
|
||||
return json_get_string(raw, "utterance")
|
||||
}
|
||||
|
||||
// ── The decision ─────────────────────────────────────────────────────────────
|
||||
//
|
||||
// Returns: 0 = backchannel, carry on seamlessly
|
||||
// 1 = hold the floor ("hang on, let me finish this thought")
|
||||
// 2 = yield (stop, let them in)
|
||||
//
|
||||
// All arguments are per-mille integers. Holding requires BOTH that the material
|
||||
// is worth finishing AND that the interrupter is not high-authority — either
|
||||
// condition alone is not enough, because "what I'm saying is important" is
|
||||
// exactly the reasoning that produces a system nobody can get a word in against.
|
||||
fn conv_decide(salience_pm: Int, progress_pm: Int, authority_pm: Int, is_backchannel: Bool) -> Int {
|
||||
if is_backchannel {
|
||||
return 0
|
||||
}
|
||||
let hold_score: Int = (salience_pm * 6 + progress_pm * 4) / 10
|
||||
if hold_score >= 600 {
|
||||
if authority_pm < 800 {
|
||||
return 1
|
||||
}
|
||||
}
|
||||
return 2
|
||||
}
|
||||
|
||||
// ── Resume ───────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// The remaining plan, written where a later run can find it. This is what turns
|
||||
// an interruption into a pause rather than a loss.
|
||||
|
||||
fn conv_resume_path() -> String {
|
||||
let home: String = env("PERIPH_HOME")
|
||||
if str_eq(home, "") {
|
||||
return "peripheral/.resume.json"
|
||||
}
|
||||
return home + "/.resume.json"
|
||||
}
|
||||
|
||||
// Minimal JSON string escaping. Written here rather than reached for from the
|
||||
// runtime because the organ needs exactly two escapes and no dependency: a
|
||||
// segment label containing a quote or a backslash must not be able to produce a
|
||||
// resume file that fails to parse and silently loses the thread.
|
||||
fn conv_escape(s: String) -> String {
|
||||
let n: Int = str_len(s)
|
||||
let out: String = ""
|
||||
let i: Int = 0
|
||||
while i < n {
|
||||
let c: Int = str_char_code(s, i)
|
||||
if c == 34 {
|
||||
out = out + "\\\""
|
||||
} else {
|
||||
if c == 92 {
|
||||
out = out + "\\\\"
|
||||
} else {
|
||||
if c >= 32 {
|
||||
out = out + str_slice(s, i, i + 1)
|
||||
}
|
||||
}
|
||||
}
|
||||
i = i + 1
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
fn conv_persist_resume(plan: [String], start_at: Int, reason: String) -> Bool {
|
||||
let n: Int = plan_count(plan)
|
||||
let body: String = "{\"resume_from\": " + int_to_str(start_at) + ", \"reason\": \"" + reason + "\", \"segments\": ["
|
||||
let i: Int = start_at
|
||||
let first: Bool = true
|
||||
while i < n {
|
||||
if first == false {
|
||||
body = body + ", "
|
||||
}
|
||||
body = body + "{\"file\": \"" + plan_file(plan, i) + "\", \"salience\": " + int_to_str(plan_salience(plan, i)) + ", \"text\": \"" + conv_escape(plan_text(plan, i)) + "\"}"
|
||||
first = false
|
||||
i = i + 1
|
||||
}
|
||||
body = body + "]}\n"
|
||||
let ok: Bool = fs_write(conv_resume_path(), body)
|
||||
organ_disclose("CONVERSE: meaning-plan persisted (" + int_to_str(n - start_at) + " segments remain) — Neuron can resume the thread.")
|
||||
return ok
|
||||
}
|
||||
|
||||
fn conv_clear_resume() -> Bool {
|
||||
return fs_write(conv_resume_path(), "")
|
||||
}
|
||||
|
||||
// Read a persisted plan back. Salience is already per-mille here (we wrote it),
|
||||
// so it is NOT re-scaled — the manifest and the resume file are different
|
||||
// formats on purpose, and conflating them silently divides every salience by a
|
||||
// thousand.
|
||||
fn conv_load_resume() -> [String] {
|
||||
let plan: [String] = plan_new()
|
||||
let raw: String = fs_read(conv_resume_path())
|
||||
if str_eq(raw, "") {
|
||||
return plan
|
||||
}
|
||||
let segs: String = json_get_raw(raw, "segments")
|
||||
let n: Int = json_array_len(segs)
|
||||
let i: Int = 0
|
||||
while i < n {
|
||||
let seg: String = json_array_get(segs, i)
|
||||
plan = plan_add(plan, json_get_string(seg, "file"), json_get_int(seg, "salience"), json_get_string(seg, "text"))
|
||||
i = i + 1
|
||||
}
|
||||
return plan
|
||||
}
|
||||
|
||||
// ── The loop ─────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// live_mic : open the microphone with AEC and let real speech drive barge-in.
|
||||
// barge_ms : if >= 0, inject a barge event at that offset into the utterance
|
||||
// instead. Deterministic, so the decision paths can be exercised
|
||||
// without a room and a person — the same reason periph.swift has it.
|
||||
// kind : "backchannel" or "bargein", for the injected case.
|
||||
// authority : interrupter authority, per-mille.
|
||||
//
|
||||
// Returns the index the utterance stopped at, or -1 if it completed.
|
||||
|
||||
fn conv_run(plan: [String], authority_pm: Int, barge_ms: Int, kind: String, live_mic: Bool) -> Int {
|
||||
let n: Int = plan_count(plan)
|
||||
if n <= 0 {
|
||||
organ_disclose("CONVERSE: nothing to say.")
|
||||
return 0 - 1
|
||||
}
|
||||
if speaker_available() == 0 {
|
||||
organ_disclose("CONVERSE: no speaker on this build — cannot hold a conversation.")
|
||||
return 0 - 1
|
||||
}
|
||||
|
||||
let mic_live: Bool = false
|
||||
if live_mic {
|
||||
if organ_may_listen() {
|
||||
let m: Int = mic_monitor_start()
|
||||
if m == 1 {
|
||||
organ_disclose("CONVERSE: full-duplex — mic listening WHILE speaking, AEC on (won't self-interrupt).")
|
||||
mic_live = true
|
||||
}
|
||||
if m == 2 {
|
||||
organ_disclose("CONVERSE: full-duplex — mic listening, but AEC UNAVAILABLE; raising the VAD floor so we do not barge in on ourselves.")
|
||||
mic_live = true
|
||||
}
|
||||
if m == 0 {
|
||||
organ_disclose("CONVERSE: could not open the mic monitor — falling back to injected events.")
|
||||
}
|
||||
}
|
||||
}
|
||||
if mic_live == false {
|
||||
organ_disclose("CONVERSE: deterministic mode (live mic off).")
|
||||
}
|
||||
|
||||
// Without AEC the mic hears the speaker, so the threshold has to sit above
|
||||
// our own output. This is a mitigation and not a fix: the honest note is
|
||||
// that barge-in is markedly less sensitive in this mode.
|
||||
let vad_pm: Int = 20
|
||||
if mic_live {
|
||||
if mic_monitor_start() == 2 {
|
||||
vad_pm = 60
|
||||
}
|
||||
}
|
||||
|
||||
let elapsed_ms: Int = 0
|
||||
let prior_ms: Int = 0
|
||||
let handled: Bool = false
|
||||
// An injected barge is ONE event, not a condition that stays true. Without
|
||||
// this the deadline re-fires on every poll after a backchannel resume, and
|
||||
// the utterance live-locks: paused, resumed, paused again, forever.
|
||||
let injected_fired: Bool = false
|
||||
let i: Int = 0
|
||||
|
||||
while i < n {
|
||||
let file: String = plan_file(plan, i)
|
||||
let sal: Int = plan_salience(plan, i)
|
||||
let frames: Int = wav_frames(file)
|
||||
let rate: Int = wav_rate(file)
|
||||
if frames <= 0 {
|
||||
organ_disclose("CONVERSE: missing or unreadable segment '" + file + "', skipping.")
|
||||
i = i + 1
|
||||
} else {
|
||||
let dur_ms: Int = frames * 1000 / rate
|
||||
organ_disclose("CONVERSE: speaking segment " + int_to_str(i + 1) + "/" + int_to_str(n) + " (salience " + int_to_str(sal) + "/1000) — \"" + plan_text(plan, i) + "\"")
|
||||
let started: Int = speaker_play_wav_async(file)
|
||||
if started == 0 {
|
||||
organ_disclose("CONVERSE: could not start playback for '" + file + "'.")
|
||||
i = i + 1
|
||||
} else {
|
||||
let seg_ms: Int = 0
|
||||
let done: Bool = false
|
||||
let interrupted: Bool = false
|
||||
let speech_ticks: Int = 0
|
||||
|
||||
while done == false {
|
||||
sleep_ms(10)
|
||||
seg_ms = seg_ms + 10
|
||||
|
||||
if speaker_playing() == 0 {
|
||||
done = true
|
||||
} else {
|
||||
// The tick counter is an approximation — each pass costs
|
||||
// more than the sleep it asked for. The DAC position is
|
||||
// the truth, so drive the injected deadline off THAT and
|
||||
// an injected barge lands where it was asked to land.
|
||||
let pos_ms: Int = speaker_played_frames() * 1000 / rate
|
||||
elapsed_ms = prior_ms + pos_ms
|
||||
// --- onset detection: real speech, or an injected event ---
|
||||
let onset: Bool = false
|
||||
if mic_live {
|
||||
let rms: Float = mic_monitor_rms()
|
||||
let rms_pm: Int = float_to_int(rms * 1000.0)
|
||||
if rms_pm > vad_pm {
|
||||
speech_ticks = speech_ticks + 1
|
||||
} else {
|
||||
speech_ticks = 0
|
||||
}
|
||||
// ~60ms of continuous voice: short enough to feel
|
||||
// instant, long enough that a door closing is not a turn.
|
||||
if speech_ticks >= 3 {
|
||||
if handled == false {
|
||||
onset = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if barge_ms >= 0 {
|
||||
if injected_fired == false {
|
||||
if elapsed_ms >= barge_ms {
|
||||
onset = true
|
||||
injected_fired = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if onset {
|
||||
handled = true
|
||||
// (1) BARGE-IN — pause on the spot.
|
||||
speaker_pause()
|
||||
let played: Int = speaker_played_frames()
|
||||
let at_ms: Int = played * 1000 / rate
|
||||
let progress_pm: Int = at_ms * 1000 / dur_ms
|
||||
if progress_pm > 1000 {
|
||||
progress_pm = 1000
|
||||
}
|
||||
organ_disclose("CONVERSE: << user speech at " + int_to_str(at_ms) + "ms into segment " + int_to_str(i + 1) + " — PAUSED instantly >>")
|
||||
|
||||
// (2) backchannel or real barge-in?
|
||||
let is_bc: Bool = false
|
||||
if barge_ms >= 0 {
|
||||
if str_eq(kind, "backchannel") {
|
||||
is_bc = true
|
||||
}
|
||||
} else {
|
||||
// Live: look again ~250ms after onset. If the
|
||||
// energy has already collapsed it was "mm-hm".
|
||||
sleep_ms(250)
|
||||
let r2: Float = mic_monitor_rms()
|
||||
if float_to_int(r2 * 1000.0) < 15 {
|
||||
is_bc = true
|
||||
}
|
||||
}
|
||||
|
||||
// (3) yield, hold, or carry on
|
||||
let d: Int = conv_decide(sal, progress_pm, authority_pm, is_bc)
|
||||
if d == 0 {
|
||||
organ_disclose("CONVERSE: read as BACKCHANNEL (\"mm-hm\") — keep going, resume seamlessly.")
|
||||
handled = false
|
||||
speech_ticks = 0
|
||||
speaker_resume()
|
||||
}
|
||||
if d == 1 {
|
||||
organ_disclose("CONVERSE: HOLD the floor — \"hang on, let me finish this thought.\" (salience " + int_to_str(sal) + ", progress " + int_to_str(progress_pm) + ")")
|
||||
speaker_resume()
|
||||
// Finish THIS segment, then yield the remainder:
|
||||
// holding is a request for a moment, not a claim
|
||||
// on the rest of the conversation.
|
||||
while speaker_playing() == 1 {
|
||||
sleep_ms(20)
|
||||
}
|
||||
speaker_stop()
|
||||
conv_persist_resume(plan, i + 1, "held-then-yield")
|
||||
if mic_live {
|
||||
mic_monitor_stop()
|
||||
}
|
||||
return i + 1
|
||||
}
|
||||
if d == 2 {
|
||||
organ_disclose("CONVERSE: YIELD — stop, let them in. Remembering where I was (resumable).")
|
||||
speaker_stop()
|
||||
conv_persist_resume(plan, i, "yield")
|
||||
if mic_live {
|
||||
mic_monitor_stop()
|
||||
}
|
||||
return i
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if interrupted == false {
|
||||
prior_ms = prior_ms + dur_ms
|
||||
i = i + 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
conv_clear_resume()
|
||||
organ_disclose("CONVERSE: utterance complete (uninterrupted).")
|
||||
if mic_live {
|
||||
mic_monitor_stop()
|
||||
}
|
||||
return 0 - 1
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user