Commit Graph

440 Commits

Author SHA1 Message Date
Neuron CI f525437c77 ci(daemon): deploy main neuron-daemon@b2bed846 2026-04-26 16:03:29 +00:00
Will Anderson baf147543c ci(daemon): deploy main neuron-daemon@c190e6e5 2026-04-26 10:37:29 -05:00
Will Anderson 714a7c7a8d launch: deploy daemon StatefulSet, fix neuron-rest security context, add daemon Argo CD app
- Add neuron-daemon Argo CD Application (apps/neuron-daemon.yaml) pointing to
  k8s/neuron-technologies/daemon/ on main — daemon will deploy once image is built
- Add daemon k8s manifests: StatefulSet (2 replicas, imagePullPolicy: Always),
  Service (ClusterIP :7749, session affinity), ExternalSecret (Anthropic API key from Vault)
- Fix neuron-rest runAsNonRoot: remove constraint since v0.15.3 image runs as root,
  clears the CreateContainerConfigError on the 11h-old stale pod
- Add NEURON_DAEMON_WS_URL env var to marketing deployment for chat UI connectivity
- Add neuron.neurontechnologies.ai Cloudflare DNS record and tunnel ingress rule (Terraform)
2026-04-26 10:13:48 -05:00
Will Anderson 655ab38558 Deploy neuron-mcp:a837369 — VBD marketplace architecture 2026-04-26 08:51:07 -05:00
Will Anderson 6041ac3b25 Deploy neuron-mcp:514db58 — STATELESS, built from current main 2026-04-26 08:33:03 -05:00
Will Anderson 623a6828e8 Deploy neuron-mcp:05d978f — STATELESS from main branch 2026-04-26 08:24:55 -05:00
Will Anderson d804d71040 Remove sticky session from MCP IngressRoute — STATELESS is sessionless 2026-04-26 08:21:16 -05:00
Will Anderson 5f1c9adaa6 Deploy neuron-mcp:1b26e3e — STATELESS HTTP transport
Removes STREAMABLE session state. No sticky sessions needed.
Works across all pods, survives restarts, scales cleanly.
2026-04-26 08:18:07 -05:00
Will Anderson 08ef13c40d Add sticky session affinity to MCP IngressRoute
STREAMABLE transport holds session state in-memory per-pod. With HPA
scaling to 2+ replicas and round-robin routing, requests hit different
pods and fail with "Session not found". Pin each client to one pod via
a secure HttpOnly cookie.
2026-04-26 08:07:23 -05:00
Will Anderson 20397e9831 Restore neuron-mcp:97bd19ba (rollback to v0.15.3 was incompatible) 2026-04-26 04:28:19 -05:00
Will Anderson 5fa8cf45d5 Rollback neuron-mcp to v0.15.3 to restore MCP connectivity 2026-04-26 04:20:57 -05:00
Will Anderson 83b8d0768e Add Traefik IngressRoute for neuron.neurontechnologies.ai MCP endpoint 2026-04-26 03:52:24 -05:00
Will Anderson 6ac4faabc5 Deploy neuron-mcp:97bd19ba — Postgres marketplace datasource 2026-04-26 03:27:46 -05:00
Will Anderson 2550786965 Revert CI to host Docker socket, remove buildkitd
BuildKit rootless failed on k3s (mount propagation), privileged mode
fixed that but buildkitd gRPC is incompatible with the Docker REST API
that forgejo-runner needs to manage job containers. Net security change
vs. the original socket approach was zero (privileged ≈ socket access).

Remove buildkitd.yaml entirely. Restore docker-sock hostPath mount on
both runners. Builds work again.
2026-04-26 02:50:27 -05:00
Will Anderson e18d953ff9 Wire marketplace Postgres datasource for neuron-mcp
Add MARKETPLACE_DB_URL/USER to ConfigMap and MARKETPLACE_DB_PASSWORD
ExternalSecret (sourced from secret/legion-db in Vault). Remove the
SQLite subPath volume mount and fix-data-ownership initContainer from
the blue deployment — marketplace storage is now in Postgres.
2026-04-26 02:33:26 -05:00
Will Anderson 952e781550 Pre-create neuron-marketplace.db in initContainer for subPath mount
subPath mounts require the source file to exist on the PVC before the
main container starts. Add 'touch /data/neuron-marketplace.db' to the
initContainer so the subPath mount can bind the file at /app/neuron-marketplace.db.
2026-04-26 02:06:02 -05:00
Will Anderson 702faca691 Mount neuron-marketplace.db from PVC via subPath at /app/neuron-marketplace.db
App hard-codes relative path 'neuron-marketplace.db' from working dir /app.
Mount the PVC file at /app/neuron-marketplace.db via subPath so the app can
write to it as UID 1000. PVC persists the file across pod restarts.
Also keep NEURON_MARKETPLACE_DB_PATH in ConfigMap for future app versions.
2026-04-26 02:03:09 -05:00
Will Anderson a6b0dc9211 Force pod restart to pick up NEURON_MARKETPLACE_DB_PATH configmap change
Add config-hash annotation to template to cycle the ReplicaSet and
ensure pods restart with the updated ConfigMap env var.
2026-04-26 01:52:19 -05:00
Will Anderson ddaab28638 Fix marketplace DB: revert workingDir, add NEURON_MARKETPLACE_DB_PATH
workingDir: /data broke the JVM (app.jar is a relative path in /app).
Add NEURON_MARKETPLACE_DB_PATH=/data/neuron-marketplace.db env var
following the same pattern as NEURON_DB_PATH for the core DB.
2026-04-26 01:49:48 -05:00
Will Anderson 26ed92ca38 Set workingDir: /data so relative DB paths resolve to PVC
neuron-marketplace.db was opening relative to /app (image root,
not writable by UID 1000). Set workingDir to /data so all relative
file opens land on the writable PVC volume.
2026-04-26 01:47:02 -05:00
Will Anderson 754cd72df5 Fix SQLite READONLY: chown /data to UID 1000 via initContainer
PVC was written as root before runAsUser: 1000 was added; SQLite
refused writes with SQLITE_READONLY. initContainer runs as root to
chown the volume, then the app runs as UID 1000 as required.
2026-04-26 01:43:37 -05:00
Will Anderson 01245a6278 Fix Neuron MCP: runAsUser + move to neuron.neurontechnologies.ai
Pod was crash-looping: image has no USER directive so kubelet rejected it
with runAsNonRoot. Add runAsUser: 1000 at both pod and container level.

MCP paths moved from neurontechnologies.ai (now a GCP Cloud Run A record,
unreachable on Legion) to private subdomain neuron.neurontechnologies.ai.
Cloudflare tunnel and DNS CNAME updated out-of-band.
2026-04-26 01:35:20 -05:00
Will Anderson ffac2161bc Use TCP readiness probe for BuildKit — exec fails on k3s privileged containers 2026-04-26 01:23:52 -05:00
Will Anderson 6e67f62ffe Switch BuildKit to privileged mode — rootless fails on k3s mount propagation 2026-04-26 01:20:45 -05:00
Will Anderson e138c45d51 Fix BuildKit rootless: allow privilege escalation for newuidmap/newgidmap 2026-04-26 01:17:41 -05:00
Will Anderson 74dd054a36 feat(swarm): add swarm self-improvement loop infrastructure
Adds complete k8s manifests, ArgoCD app, and Terraform namespace
resources for the Neuron swarm self-improvement loop system.

Each variant (alpha, beta, gamma) gets its own isolated namespace,
PVC, MCP/REST deployments, ExternalSecrets from Vault, RBAC for CI,
and a SQLite clone Job template for session startup.
2026-04-26 00:58:43 -05:00
Will Anderson 93358505fc Harden prod: security, autoscaling, observability, BuildKit CI
Security:
- Drop ALL capabilities, enforce non-root, RuntimeDefault seccomp on
  neuron-mcp, neuron-rest, neuron-marketing pods
- Add startup probes (150s window for JVM) so liveness doesn't fire early
- Replace docker-sock hostPath with BuildKit rootless TCP endpoint
  (moby/buildkit:v0.19.0-rootless) — removes node root access from CI
- Document full ESO AppRole migration path in cluster-secret-store.yaml

Autoscaling & availability:
- HPAs on mcp (1–6), rest (1–4), marketing (2–8) at 65–70% CPU
- PodDisruptionBudgets (minAvailable: 1) on all three services
- NetworkPolicy: default-deny-all in neuron-prod, explicit allow rules
  for Traefik ingress, intra-namespace, and egress to DNS/platform/vault

Observability:
- ServiceMonitors for mcp, rest, marketing (cross-namespace enabled in
  kube-prometheus-stack with serviceMonitorSelectorNilUsesHelmValues:false)
- PrometheusRules: high error rate, high latency, crash loops, replica
  shortage, Postgres down/connections, backup failure, backup staleness

Chart version pinning:
- kube-prometheus-stack, loki, tempo, redis, alloy, postgres — all pinned
  to major-version ranges to block silent breaking upgrades

Backup hardening:
- restic:latest → restic:0.17.3 (deterministic image)
- Weekly backup-verify CronJob: restores latest snapshot and validates
  SQL dump structure (≥5 CREATE TABLE, pg_dump header check)

ArgoCD:
- neuron-prod AppProject: scopes deploys to neuron-prod + platform ns,
  blacklists ClusterRole/ClusterRoleBinding/Namespace creation,
  automated sync window 2–6am UTC, manual always allowed
2026-04-25 22:54:18 -05:00
Will Anderson 7eeff54a11 Wire GCS backup to neuron-db-backup-prod bucket (neuron-494301)
Bucket created, SA key stored in Vault at secret/gcs.
CronJob ExternalSecret updated to pull from secret/gcs.
Hourly restic backup now runs to both R2 and GCS.
2026-04-25 17:51:57 -05:00
Will Anderson 8d97bbd802 Merge branch 'main' of https://git.neuralplatform.ai/will/infrastructure 2026-04-25 17:50:32 -05:00
Will Anderson 2d0ce77518 Scale Docuseal up to 1 replica 2026-04-25 15:59:03 -05:00
Will Anderson a37deca724 Add GCS backup bucket + dual-destination hourly backup (R2 + GCS)
Provision Google Cloud Storage bucket for neuron prod DB backups via Terraform.
Create dedicated backup service account with objectAdmin on the bucket.
Update neuron-prod backup CronJob to run restic against both R2 and GCS hourly —
R2 as primary, GCS as secondary, independent credentials and repositories.
2026-04-25 15:23:51 -05:00
Neuron CI 8de866a8b9 ci(neuron-prod): update rest+license to v0.15.3 2026-04-25 20:06:27 +00:00
Neuron CI df85598df2 ci(neuron-prod): blue-green flip to blue@v0.15.3 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.15.3) 2026-04-25 20:06:23 +00:00
Neuron CI 950f00586f ci(neuron-prod): update rest+license to v0.15.2 2026-04-25 19:53:53 +00:00
Neuron CI 35e346d94c ci(neuron-prod): blue-green flip to green@v0.15.2 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.15.2) 2026-04-25 19:53:50 +00:00
Neuron CI 8e5320c7b3 ci(neuron-prod): update rest+license to v0.15.1 2026-04-25 19:39:31 +00:00
Neuron CI 9b057411db ci(neuron-prod): blue-green flip to blue@v0.15.1 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.15.1) 2026-04-25 19:39:29 +00:00
Neuron CI 14f0663e28 ci(neuron-prod): update rest+license to v0.15.0 2026-04-25 19:20:40 +00:00
Neuron CI 8b56eb5290 ci(neuron-prod): blue-green flip to green@v0.15.0 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.15.0) 2026-04-25 19:20:35 +00:00
Neuron CI 39050bfde4 ci(neuron-prod): update rest+license to v0.14.12 2026-04-25 10:18:30 +00:00
Neuron CI 6b782f04e9 ci(neuron-prod): blue-green flip to blue@v0.14.12 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.14.12) 2026-04-25 10:18:28 +00:00
Neuron CI 5c9ea05631 ci(neuron-prod): update rest+license to v0.14.11 2026-04-25 10:08:15 +00:00
Neuron CI acf9f07dbd ci(neuron-prod): blue-green flip to green@v0.14.11 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.14.11) 2026-04-25 10:08:12 +00:00
Neuron CI 491d00fd1a ci(neuron-prod): update rest+license to v0.14.10 2026-04-25 10:02:23 +00:00
Neuron CI 663abd5188 ci(neuron-prod): blue-green flip to blue@v0.14.10 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.14.10) 2026-04-25 10:02:20 +00:00
Neuron CI bdd2033d78 ci(neuron-prod): update rest+license to v0.14.9 2026-04-25 09:47:48 +00:00
Neuron CI fed551185e ci(neuron-prod): blue-green flip to green@v0.14.9 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.14.9) 2026-04-25 09:47:46 +00:00
Neuron CI 2793799532 ci(neuron-prod): update rest+license to v0.14.8 2026-04-25 09:14:24 +00:00
Neuron CI 8d02a1164b ci(neuron-prod): blue-green flip to blue@v0.14.8 (registry.neuralplatform.ai/neuron-technologies/neuron-mcp:v0.14.8) 2026-04-25 09:14:22 +00:00
Neuron CI 1ee6dc2804 ci(neuron-prod): update rest+license to v0.14.7 2026-04-25 09:08:20 +00:00