neuron-technologies/neuron
3
0
Fork 0
Code Issues 44 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

feat(engine): port Anthropic server-side web_search into the agentic loop #108

Closed
tim.lingo wants to merge 0 commits from feat/soul-native-web-search-20260804 into main
pull from: feat/soul-native-web-search-20260804
merge into: :main
Conversation 1 Commits - Files Changed -
+265 -23
tim.lingo commented 2026-08-05 03:59:02 +00:00
Member
Copy Link
Copy Source

Root cause

The product promises web search; the engine never attached the tool. agentic_tools_with_web() (chat.el:1418) was defined but reachable only through a legacy tools_variant field nothing writes. The intended design existed and was lost: commit 8eea1d9 (2026-06-09, Tim-approved) made native web_search built-in with no user-facing toggle, and tests/test_agentic_tools.el §2 still asserts agentic_tools_all() contains it — an assertion main currently fails. The call site disappeared when agentic_tools_all() (connector tools, PR #19) replaced agentic_tools_with_web() in handle_chat_agentic.

Separately, pause_turn was absent everywhere — zero occurrences in the shipped binary. Server-side search pauses turns; without handling, the loop treats the partial text as final and the user gets a silently truncated answer.

The change

Re-authors soul-webfix-20260711.patch in El. That patch is a diff against generated C at month-old offsets, so nothing was applied as a patch — every hunk was re-authored. Its last two hunks are an unrelated /api/safety-contact implementation, deliberately not ported (the route already exists, is safety-critical, and passes the contract gate as-is).

One file, chat.el, +256/−22:

  • web_search_tool_json() (new) — one source of truth for the tool block; version from state key web_search_tool_version.
  • strip_client_web_search() (new) — removes any client tool named web_search before attaching the server one, so they can never collide on the name. No-op on our literal set (it ships web_get); exists for third-party MCP connector tools.
  • agentic_tools_all() — attaches the server tool unconditionally. One edit covers handle_chat_agentic, handle_dharma_room_turn_agentic, and agentic_resume (via the stored bridge blob).
  • agentic_loop() — pause_turn handling; final_text accumulates across resume cycles instead of overwriting; server_tool_use accounting into tools_used; iteration cap 8→12; max_tokens 4096→16384; container-id carry-forward; version-drift fallback; API error head logged instead of swallowed; tools_used gated on is_tool_turn so a truncated tool block isn't reported as work done.

The stopgap conflict — surfaced, not papered over

web_search_20260209 (dynamic filtering) is hard-incompatible with ADR 0005. Dynamic filtering uses server-side programmatic tool calling; the API refuses the pair. Verified live on both claude-opus-4-8 and claude-sonnet-4-5:

HTTP 400 invalid_request_error
tool_choice.disable_parallel_tool_use: true cannot be used with programmatic tool calling

Neither feature is dropped. Dropping the stopgap resurrects neuron#78 bug b (killed runs 3/3 in ADR 0005's own A/B); web_search_20250305 is compatible with the stopgap and returns real results. So the default is the basic variant. What we give up is dynamic filtering — a token-efficiency/quality nicety, not the capability.

Removal trigger: flip to _20260209 when ADR 0005 retires. One config write, no recompile. The fallback also fires on programmatic tool calling, so a premature flip self-heals loudly instead of dying.

Bug this port exposed (fixed here)

json_get is a first-match scanner, and a cited text block serialises citations first:

{"citations":[{"type":"web_search_result_location",...}],"type":"text","text":"The current temperature is 100°F"}

so json_get(block,"type") returned web_search_result_location, "text" never matched, and every citation-bearing block was dropped — exactly the blocks carrying the searched facts. Users got The current temperature is , with .. Only text blocks carry citations, so its presence identifies the type. Reply length on the same question: 79 → 360 chars.

How to test

Sandbox: throwaway HOME/engram, dead axon + ISE, NEURON_PORT set explicitly (~/neuron_soul_smokebuild.sh exports SOUL_PORT, which the soul ignores — it reads NEURON_PORT, default 7770, the live port; that latent bug is still open).

  • POST /api/chat {"agentic":true} asking for current weather → tools_used:["web_search"], live figures (86°F Bentonville, RealFeel 98°, isolated thunderstorms), 6s.
  • scripts/verify-soul-contract.sh <bin> <port> under bash ≥ 4.3 → GATE: PASS, 27/27 routes, immutability PASS. (macOS /bin/bash 3.2 fails on local -n; use /opt/homebrew/bin/bash.)
  • Stopgap intact: 3-file mission → ["read_file","read_file","read_file"], 4 iterations, correct answer, no 400.
  • No truncation: 3 searches in one turn, 3,486 chars, end sentinel present.

Honest gap: pause_turn is compiled in but not exercised by a live pause — max_uses:5 caps the server loop below the pause threshold. Code-complete, unexercised.

Needs Will

  1. Ratify unconditional activation vs a request flag.
  2. Ratify the version default, and own the flip when ADR 0005 retires.
  3. Regenerate dist/soul.c — deliberately not regenerated here. The local elc (2026-06-05) predates your last regen (e610a41); regenerating with an older compiler risks unrelated codegen drift. Source only.
  4. Committed .elh headers are stale vs the .el sources (regen adds bounded_persona_floor, affective_context_prefix, run_command_guard, classify_tool_risk; removes some from safety.elh). Left untouched so this PR is one change — worth a separate resync.

Note this branch is based on hotfix/parallel-tool-stopgap-20260803, so it carries that commit too (the stopgap is kept intact, per ADR 0005).

Artifacts, full proof transcript, and the build recipe: _engine-websearch-20260804/README.md.

⚠️ A separate safety finding is in _engine-websearch-20260804/SAFETY-STOP.md — the dead plain-chat route was not wired; doing so would remove the only enforcing output gate in the codebase. Not part of this PR.

🤖 Generated with Claude Code

## Root cause The product promises web search; the engine never attached the tool. `agentic_tools_with_web()` (`chat.el:1418`) was defined but reachable only through a legacy `tools_variant` field nothing writes. The intended design existed and was lost: commit `8eea1d9` (2026-06-09, **Tim-approved**) made native web_search built-in with no user-facing toggle, and `tests/test_agentic_tools.el` §2 still asserts `agentic_tools_all()` contains it — **an assertion `main` currently fails**. The call site disappeared when `agentic_tools_all()` (connector tools, PR #19) replaced `agentic_tools_with_web()` in `handle_chat_agentic`. Separately, `pause_turn` was absent everywhere — **zero occurrences in the shipped binary**. Server-side search pauses turns; without handling, the loop treats the partial text as final and the user gets a silently truncated answer. ## The change Re-authors `soul-webfix-20260711.patch` in El. That patch is a diff against **generated C** at month-old offsets, so nothing was applied as a patch — every hunk was re-authored. Its last two hunks are an unrelated `/api/safety-contact` implementation, **deliberately not ported** (the route already exists, is safety-critical, and passes the contract gate as-is). One file, `chat.el`, +256/−22: - **`web_search_tool_json()`** (new) — one source of truth for the tool block; version from state key `web_search_tool_version`. - **`strip_client_web_search()`** (new) — removes any *client* tool named `web_search` before attaching the server one, so they can never collide on the name. No-op on our literal set (it ships `web_get`); exists for third-party MCP connector tools. - **`agentic_tools_all()`** — attaches the server tool unconditionally. One edit covers `handle_chat_agentic`, `handle_dharma_room_turn_agentic`, and `agentic_resume` (via the stored bridge blob). - **`agentic_loop()`** — `pause_turn` handling; `final_text` accumulates across resume cycles instead of overwriting; `server_tool_use` accounting into `tools_used`; iteration cap 8→12; `max_tokens` 4096→16384; container-id carry-forward; version-drift fallback; API error head logged instead of swallowed; `tools_used` gated on `is_tool_turn` so a truncated tool block isn't reported as work done. ## The stopgap conflict — surfaced, not papered over `web_search_20260209` (dynamic filtering) is **hard-incompatible** with ADR 0005. Dynamic filtering uses server-side programmatic tool calling; the API refuses the pair. Verified live on both `claude-opus-4-8` and `claude-sonnet-4-5`: ``` HTTP 400 invalid_request_error tool_choice.disable_parallel_tool_use: true cannot be used with programmatic tool calling ``` Neither feature is dropped. Dropping the stopgap resurrects neuron#78 bug b (killed runs **3/3** in ADR 0005's own A/B); `web_search_20250305` **is** compatible with the stopgap and returns real results. So the default is the basic variant. What we give up is dynamic filtering — a token-efficiency/quality nicety, not the capability. **Removal trigger:** flip to `_20260209` when ADR 0005 retires. One config write, no recompile. The fallback also fires on `programmatic tool calling`, so a premature flip self-heals loudly instead of dying. ## Bug this port exposed (fixed here) `json_get` is a first-match scanner, and a cited text block serialises `citations` **first**: ```json {"citations":[{"type":"web_search_result_location",...}],"type":"text","text":"The current temperature is 100°F"} ``` so `json_get(block,"type")` returned `web_search_result_location`, `"text"` never matched, and **every citation-bearing block was dropped** — exactly the blocks carrying the searched facts. Users got `The current temperature is , with .`. Only text blocks carry `citations`, so its presence identifies the type. Reply length on the same question: **79 → 360 chars**. ## How to test Sandbox: throwaway HOME/engram, dead axon + ISE, **`NEURON_PORT` set explicitly** (`~/neuron_soul_smokebuild.sh` exports `SOUL_PORT`, which the soul ignores — it reads `NEURON_PORT`, default **7770, the live port**; that latent bug is still open). - `POST /api/chat {"agentic":true}` asking for current weather → `tools_used:["web_search"]`, live figures (86°F Bentonville, RealFeel 98°, isolated thunderstorms), 6s. - `scripts/verify-soul-contract.sh <bin> <port>` under **bash ≥ 4.3** → **GATE: PASS**, 27/27 routes, immutability PASS. (macOS `/bin/bash` 3.2 fails on `local -n`; use `/opt/homebrew/bin/bash`.) - Stopgap intact: 3-file mission → `["read_file","read_file","read_file"]`, 4 iterations, correct answer, no 400. - No truncation: 3 searches in one turn, 3,486 chars, end sentinel present. **Honest gap:** `pause_turn` is compiled in but **not exercised by a live pause** — `max_uses:5` caps the server loop below the pause threshold. Code-complete, unexercised. ## Needs Will 1. Ratify **unconditional activation** vs a request flag. 2. Ratify the **version default**, and own the flip when ADR 0005 retires. 3. **Regenerate `dist/soul.c`** — deliberately not regenerated here. The local `elc` (2026-06-05) predates your last regen (`e610a41`); regenerating with an older compiler risks unrelated codegen drift. Source only. 4. Committed **`.elh` headers are stale** vs the `.el` sources (regen adds `bounded_persona_floor`, `affective_context_prefix`, `run_command_guard`, `classify_tool_risk`; removes some from `safety.elh`). Left untouched so this PR is one change — worth a separate resync. Note this branch is based on `hotfix/parallel-tool-stopgap-20260803`, so it carries that commit too (the stopgap is kept intact, per ADR 0005). Artifacts, full proof transcript, and the build recipe: `_engine-websearch-20260804/README.md`. ⚠️ **A separate safety finding is in `_engine-websearch-20260804/SAFETY-STOP.md`** — the dead plain-chat route was **not** wired; doing so would remove the only enforcing output gate in the codebase. Not part of this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
tim.lingo added 2 commits 2026-08-05 03:59:03 +00:00
fix(engine): send tool_choice.disable_parallel_tool_use on agentic loop (STOPGAP) 710761e2d5 
The agentic loop keeps only the FIRST tool_use block per round (chat.el:2281,
"Capture first tool_use block only"). Anthropic lets a model emit several
tool_use blocks in one message and requires a tool_result for every one, so a
parallel-tool turn is answered once, the rest are dropped, and the next request
dies with:

  tool_use ids were found without tool_result blocks immediately after

(neuron#78 quotes this as "tool_use ids found without tool_result"; the above is
the API's actual wording - recorded so the next person's grep matches.)

This constrains the wire to match what the loop can assemble:
  "tool_choice":{"type":"auto","disable_parallel_tool_use":true}

STOPGAP - AND THE DURABLE FIX ALREADY EXISTS. A correct multi-tool loop is
already in Will's EL runtime, in C, and the soul does not call it. Verified on
el:origin/main lang/el-compiler/runtime/el_runtime.c: llm_register_tool:9616,
llm_build_tool_results:9743 - which walks EVERY content block, emits one
tool_result per tool_use, and sets is_error for an unregistered tool -
llm_call_agentic:9817 calling it at :9918, iteration cap 10 at :9847. Will's
commit 12d5e77 (2026-04-30). grep for llm_call_agentic/llm_register_tool across
every neuron/*.el returns nothing; dist/soul.c has zero references. chat.el
hand-rolls its own single-tool loop instead, and that is the one that breaks.

The durable fix is therefore to register the soul's tools via llm_register_tool
and call llm_call_agentic - deleting a loop, not writing one. See ADR 0005.

Our own approved spec called this seven weeks ago:
docs/research/agentic-tool-approval-design.md (2026-06-12, "Approved for build"),
line 20 on the defect, line 30 on the goal ("Execute all tool_use blocks in a
turn (one result per block)").

Two edits, because dist/soul.c cannot be regenerated here (Will's gated elc/elb
toolchain is not on this machine):
  (a) chat.el:2255 - source of truth, so a later regen carries the fix.
      One edit covers all three routes: agentic_loop is called from chat.el:2152
      (/api/chat agentic), :2676 (dharma room) and :2496 (agentic_resume).
  (b) dist/soul.c:28173 - generated form, hand-spliced. Line 27624 is the
      non-agentic/OpenAI-compat req_body (no tools) and was left untouched.
Prior art reused rather than reinvented: soul-narrated-runs-20260713.patch
(27,824 bytes) line 78 spliced this same string into the same concat chain on
2026-07-13.

Deliberately NOT ported from that patch, having read it: max_tokens is not
changed by it (16384 sits on both sides of the hunk; our main's 4096 is a
separate output-truncation concern), and its pause_turn pairing fix - same
defect class - is unreachable today because no server-side web_search is wired
(agentic_tools_with_web at chat.el:1418 is never called), so it is untestable
and logged instead.

Proven E2E on a scratch profile and port 7791, never the live chain. A/B against
a pristine origin/main control built from the same vendored runtime: fixed
completed the mission (tools_used read_file x3, 4 iterations, correct answer);
control failed 3/3. Direct API probe confirmed the mechanism - without the field
the model emits 3 parallel tool_use blocks and replaying the unfixed loop's next
turn returns HTTP 400; with it, exactly 1 block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(engine): port Anthropic server-side web_search into the agentic loop
Neuron Soul CI / build (pull_request) Failing after 10m45s
Details
Neuron Soul CI / deploy (pull_request) Has been skipped
Details
62af5649fe 
Re-authors soul-webfix-20260711.patch in El (the patch is a diff against
generated C at month-old offsets, so nothing was applied as a patch). Its
last two hunks — an unrelated /api/safety-contact implementation — were
deliberately not ported; that route already exists and is safety-critical.

Activation restores existing design, it does not invent a mechanism:
commit 8eea1d9 (2026-06-09, Tim-approved) made native web_search built-in
with no user-facing toggle, and tests/test_agentic_tools.el section 2 still
asserts agentic_tools_all() contains it — an assertion main currently fails.
The call site was lost when agentic_tools_all() (connector tools, PR #19)
replaced agentic_tools_with_web(). Attaching in agentic_tools_all() covers
handle_chat_agentic, handle_dharma_room_turn_agentic and agentic_resume.

pause_turn handling is included and was genuinely missing: the shipped
binary has zero occurrences of it. Without it a paused server-side search
returns only the text written so far and the loop treats it as final —
a silently truncated answer. final_text now accumulates across resume
cycles rather than overwriting (overwriting would discard everything
written before the pause).

Default tool version is web_search_20250305, NOT the newer _20260209, and
that is a measured choice: _20260209's dynamic filtering uses server-side
programmatic tool calling, which the API refuses to combine with ADR 0005's
stopgap —

  HTTP 400 invalid_request_error
  tool_choice.disable_parallel_tool_use: true cannot be used with
  programmatic tool calling

Dropping the stopgap would resurrect neuron#78 bug b (killed runs 3/3 in
ADR 0005's own A/B). The basic variant is compatible with the stopgap and
returns real results, so neither feature is dropped. Version lives in state
key web_search_tool_version; flipping it once the stopgap retires is a
config write, no recompile. The fallback also fires on "programmatic tool
calling" so a premature flip self-heals loudly instead of dying.

Also fixes a real bug this port exposed: json_get is a first-match scanner
and a cited text block serialises citations FIRST, so json_get(block,"type")
returned the nested citation's type and every citation-bearing block — the
ones carrying the searched facts — was silently dropped from the reply.
Reply length on the same question: 79 -> 360 chars.

Other loop changes: server_tool_use accounting into tools_used, iteration
cap 8->12 for pause/resume cycles, max_tokens 4096->16384, container-id
carry-forward, API error head logged instead of swallowed, tools_used gated
on is_tool_turn so a truncated tool block is not reported as work done.

dist/soul.c is deliberately NOT regenerated — the local elc predates Will's
last regen (e610a41) and regenerating with an older compiler risks unrelated
codegen drift. Source only; regen is Will's.

E2E-VERIFIED on a sandbox soul (scratch HOME/engram, dead axon+ISE, explicit
NEURON_PORT): live Bentonville weather with tools_used ["web_search"]; the
27-route contract gate PASSes; the parallel-tool stopgap still completes a
3-file mission with no 400; a 3-search 3,486-char answer kept its end
sentinel. Honest gap: pause_turn is compiled in but not exercised by a live
pause (max_uses:5 caps the server loop below the pause threshold).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tim.lingo commented 2026-08-06 02:19:53 +00:00
Author
Member
Copy Link
Copy Source

Superseded by #109 — verified by ancestry: #109's head 635f6fe contains this branch's head 62af564, so merging #109 lands the web_search port and plain-chat generation together. Please review #109.

**Superseded by #109** — verified by ancestry: #109's head `635f6fe` contains this branch's head `62af564`, so merging #109 lands the web_search port and plain-chat generation together. Please review **#109**.
tim.lingo closed this pull request 2026-08-06 02:20:01 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
BETA-CRITICAL
On the beta critical path — must resolve before ship
 blocks-public-beta
Must be fixed before any public beta
 HELD
Blocked pending an external decision (counsel / dual sign-off)
 ORTHOGONAL
Parallel workstream (testing/hardening) — safe to assign off critical path
 P0
Drop-everything severity
 POST-BETA
Important but not blocking the beta ship
 security
Security vulnerability
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
andre.almeida (Andre Almeida) tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#108
Delete Branch "feat/soul-native-web-search-20260804"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?