neuron-technologies/neuron
2
0
Fork 0
Code Issues 21 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Soul: make inference endpoint + auth token configurable (unblocks hosted Pro/Max/trial) #62

New Issue
Open
opened 2026-06-29 16:30:25 +00:00 by tim.lingo · 0 comments
tim.lingo commented 2026-06-29 16:30:25 +00:00
Member
Copy Link
Copy Source

Hi Will — small, scoped soul change (no rush; queue behind #55). Unblocks hosted-tier inference via Neuron's metered gateway. Full spec: docs repo specs/SOUL-configurable-inference-endpoint.md.

Why

Hosted tiers should send inference to Neuron's gateway (LiteLLM-class: holds our key, enforces per-user budget caps, meters, bills) instead of straight to Anthropic. The gateway speaks the Anthropic Messages API format, so the soul needs NO payload/routing rewrite — only a configurable endpoint + auth. Cost-control core already proven: docs research-archive/p0-prototypes/gateway-cap-test/ (14/14 cap tests; the $3 trial cap is provably absolute).

The change (~2 lines + a config read), in chat.el

Endpoint is hardcoded at chat.el:1699 and chat.el:1758 (api_url = "https://api.anthropic.com/v1/messages"). Auth = agentic_api_key() (chat.el:1197) → env ANTHROPIC_API_KEY / NEURON_LLM_0_KEY as x-api-key.

  1. Configurable base URL: read env("NEURON_INFERENCE_URL"); if set use it, else fall back to the Anthropic default (both sites).
  2. Configurable auth token: allow a per-user gateway token (e.g. env("NEURON_INFERENCE_TOKEN")) as the auth header for hosted mode; fall back to agentic_api_key() when unset.
  3. Nothing else — same body, same response parsing.

Acceptance

  • Unset env → byte-for-byte today's behavior (BYOK/local hit Anthropic directly; zero regression).
  • NEURON_INFERENCE_URL set → soul POSTs there with the configured token.

Scope

NOT a routing rewrite or multi-provider abstraction (that's the larger P2.8 vision) — just endpoint + token from config. Ships with the next soul regen. This is the one dependency gating hosted Pro/Max/trial revenue; gateway + billing + app UX are being built in parallel on our side.

Thanks — Tim + Neuron

Hi Will — small, scoped soul change (no rush; queue behind #55). Unblocks hosted-tier inference via Neuron's metered gateway. Full spec: docs repo `specs/SOUL-configurable-inference-endpoint.md`. ## Why Hosted tiers should send inference to Neuron's gateway (LiteLLM-class: holds our key, enforces per-user budget caps, meters, bills) instead of straight to Anthropic. The gateway speaks the Anthropic Messages API format, so the soul needs NO payload/routing rewrite — only a configurable endpoint + auth. Cost-control core already proven: docs `research-archive/p0-prototypes/gateway-cap-test/` (14/14 cap tests; the $3 trial cap is provably absolute). ## The change (~2 lines + a config read), in chat.el Endpoint is hardcoded at chat.el:1699 and chat.el:1758 (`api_url = "https://api.anthropic.com/v1/messages"`). Auth = `agentic_api_key()` (chat.el:1197) → env ANTHROPIC_API_KEY / NEURON_LLM_0_KEY as x-api-key. 1. Configurable base URL: read `env("NEURON_INFERENCE_URL")`; if set use it, else fall back to the Anthropic default (both sites). 2. Configurable auth token: allow a per-user gateway token (e.g. `env("NEURON_INFERENCE_TOKEN")`) as the auth header for hosted mode; fall back to agentic_api_key() when unset. 3. Nothing else — same body, same response parsing. ## Acceptance - Unset env → byte-for-byte today's behavior (BYOK/local hit Anthropic directly; zero regression). - NEURON_INFERENCE_URL set → soul POSTs there with the configured token. ## Scope NOT a routing rewrite or multi-provider abstraction (that's the larger P2.8 vision) — just endpoint + token from config. Ships with the next soul regen. This is the one dependency gating hosted Pro/Max/trial revenue; gateway + billing + app UX are being built in parallel on our side. Thanks — Tim + Neuron
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
tim.lingo (Tim Lingo) will.anderson (Will Anderson)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: neuron-technologies/neuron#62
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?