fix(runtime): pass model through to the LLM API (+ UTF-8 JSON escaping) #53

2026-06-10T01:35:11Z

tim.lingo commented

2026-06-10 01:35:11 +00:00

Why

On Tim's machine the Anthropic billing dashboard shows nearly all spend as Sonnet 4.5, even though the Neuron app has Opus 4.8 selected. Root cause is in the el runtime, not config.

llm_call_system(model, system, user) and llm_call(model, prompt) accepted a model argument and threw it away - they called llm_chain_call(system, user) with no model, and the legacy ANTHROPIC_API_KEY fallback passed NULL to llm_provider_request, so every non-agentic chat was pinned to LLM_DEFAULT_MODEL (claude-sonnet-4-5). The chat response still echoed the selected model as a label, so the UI looked right while the request billed Sonnet.

Changes

1. Model passthrough (el_runtime.c)

llm_chain_call now takes model_pref and threads it through.
Provider-chain entries still honor their own NEURON_LLM_N_MODEL override and fall back to the requested model otherwise.
The legacy Anthropic path uses the requested model instead of NULL.
NULL/empty preserves the prior default, so behavior is unchanged when no model is supplied. Agentic path already passed model through and is untouched.

2. UTF-8 JSON escaping (el_runtime.c, separate commit)

jb_emit_escaped now validates multi-byte UTF-8 continuation bytes, passes valid sequences through, and escapes orphaned/invalid start bytes as \u00xx. This was found uncommitted in the working tree and is committed here so it is reviewable; it addresses the garbled/binary content we have been seeing in graph nodes.

Testing

Not built locally - this checkout has no elc/cc toolchain. Needs a stage build + CI. Suggested check: with no NEURON_LLM_* providers and a real SOUL_LLM_MODEL=claude-opus-4-8, confirm the request to api.anthropic.com carries claude-opus-4-8 and bills Opus.

Note

Duplicate runtime copies exist (runtime/legacy/, releases/v1.0.0-…/, and neuron-web/runtime/) and carry the same model bug; out of scope here, flagging for follow-up.

🤖 Generated with Claude Code

## Why On Tim's machine the Anthropic billing dashboard shows nearly all spend as **Sonnet 4.5**, even though the Neuron app has **Opus 4.8** selected. Root cause is in the el runtime, not config. `llm_call_system(model, system, user)` and `llm_call(model, prompt)` accepted a `model` argument and **threw it away** - they called `llm_chain_call(system, user)` with no model, and the legacy `ANTHROPIC_API_KEY` fallback passed `NULL` to `llm_provider_request`, so every non-agentic chat was pinned to `LLM_DEFAULT_MODEL` (`claude-sonnet-4-5`). The chat response still echoed the *selected* model as a label, so the UI looked right while the request billed Sonnet. ## Changes **1. Model passthrough** (`el_runtime.c`) - `llm_chain_call` now takes `model_pref` and threads it through. - Provider-chain entries still honor their own `NEURON_LLM_N_MODEL` override and fall back to the requested model otherwise. - The legacy Anthropic path uses the requested model instead of `NULL`. - NULL/empty preserves the prior default, so behavior is unchanged when no model is supplied. Agentic path already passed model through and is untouched. **2. UTF-8 JSON escaping** (`el_runtime.c`, separate commit) - `jb_emit_escaped` now validates multi-byte UTF-8 continuation bytes, passes valid sequences through, and escapes orphaned/invalid start bytes as `\u00xx`. This was found uncommitted in the working tree and is committed here so it is reviewable; it addresses the garbled/binary content we have been seeing in graph nodes. ## Testing Not built locally - this checkout has no elc/cc toolchain. Needs a stage build + CI. Suggested check: with no `NEURON_LLM_*` providers and a real `SOUL_LLM_MODEL=claude-opus-4-8`, confirm the request to api.anthropic.com carries `claude-opus-4-8` and bills Opus. ## Note Duplicate runtime copies exist (`runtime/legacy/`, `releases/v1.0.0-…/`, and `neuron-web/runtime/`) and carry the same model bug; out of scope here, flagging for follow-up. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

tim.lingo added 3 commits 2026-06-10 01:35:12 +00:00

Merge pull request 'promote: stage → main (all elb linker fixes + ci-base rebuild)' (#42 ) from stage into main

El SDK Release / build-and-release (push) Successful in 6m28s

Details

2ed6b26dde

promote: stage → main (all elb linker fixes + ci-base rebuild)

fix(runtime): handle multi-byte UTF-8 in JSON string escaping 2b8062c55f

Validate UTF-8 continuation bytes in jb_emit_escaped; pass valid
sequences through and escape orphaned/invalid start bytes as \u00xx.
Pre-existing change found uncommitted in the working tree; committed
here so it is reviewable rather than lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(runtime): pass model through to the LLM API instead of dropping it

El SDK CI - stage / build-and-test (pull_request) Failing after 12s

Details

dbf2c659d9

llm_call_system / llm_call accepted a model argument and discarded it:
they called llm_chain_call(system, user) with no model, and the legacy
ANTHROPIC_API_KEY fallback passed NULL to llm_provider_request, so every
non-agentic chat was pinned to LLM_DEFAULT_MODEL (claude-sonnet-4-5)
regardless of the caller's selection.

Thread model_pref through llm_chain_call: provider-chain entries still
honor their own NEURON_LLM_N_MODEL override and fall back to the
requested model otherwise; the legacy Anthropic path now uses the
requested model. NULL/empty preserves prior default behavior.

Effect: the soul's model selection (state soul_model / SOUL_LLM_MODEL,
e.g. claude-opus-4-8) now reaches api.anthropic.com. Previously the
chat response echoed the selected model in its label while the request
billed Sonnet 4.5.

Not built locally (no elc/cc toolchain on this checkout); needs stage CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

will.anderson approved these changes 2026-06-10 22:01:30 +00:00

will.anderson left a comment

Approved. Building.

will.anderson merged commit b83ecf52f9 into stage

2026-06-10 22:01:52 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: neuron-technologies/el#53