Will Anderson
283e335e0a
deploy engram-server to neuron-prod: dharma storage backend
...
- Add engram-server Deployment, Service, PVC to neuron-prod
- engram-server is the graph memory substrate for DHARMA registry
- Add allow-engram-ingress NetworkPolicy (ingress from neuron-prod)
- ENGRAM_URL in Vault updated to http://engram-server.neuron-prod.svc.cluster.local:8742
- Image: registry.neuralplatform.ai/neuron-technologies/engram-server:latest
compiled from foundation/engram dist/engram.c + el_runtime
2026-05-04 11:28:45 -05:00
Will Anderson
f2f20c1f2c
remove missing soma manifests from prod kustomization
...
Soma moved to GCP Cloud Run. The soma-*.yaml files were deleted but
the kustomization still referenced them, blocking all Argo CD syncs
for neuron-prod (including the new allow-daemon-ingress NetworkPolicy).
2026-05-04 11:11:18 -05:00
Will Anderson
88d7799911
deploy soma inference gateway with neuron model on ai.neurontechnologies.ai
2026-04-28 09:31:46 -05:00
Will Anderson
83b8d0768e
Add Traefik IngressRoute for neuron.neurontechnologies.ai MCP endpoint
2026-04-26 03:52:24 -05:00
Will Anderson
93358505fc
Harden prod: security, autoscaling, observability, BuildKit CI
...
Security:
- Drop ALL capabilities, enforce non-root, RuntimeDefault seccomp on
neuron-mcp, neuron-rest, neuron-marketing pods
- Add startup probes (150s window for JVM) so liveness doesn't fire early
- Replace docker-sock hostPath with BuildKit rootless TCP endpoint
(moby/buildkit:v0.19.0-rootless) — removes node root access from CI
- Document full ESO AppRole migration path in cluster-secret-store.yaml
Autoscaling & availability:
- HPAs on mcp (1–6), rest (1–4), marketing (2–8) at 65–70% CPU
- PodDisruptionBudgets (minAvailable: 1) on all three services
- NetworkPolicy: default-deny-all in neuron-prod, explicit allow rules
for Traefik ingress, intra-namespace, and egress to DNS/platform/vault
Observability:
- ServiceMonitors for mcp, rest, marketing (cross-namespace enabled in
kube-prometheus-stack with serviceMonitorSelectorNilUsesHelmValues:false)
- PrometheusRules: high error rate, high latency, crash loops, replica
shortage, Postgres down/connections, backup failure, backup staleness
Chart version pinning:
- kube-prometheus-stack, loki, tempo, redis, alloy, postgres — all pinned
to major-version ranges to block silent breaking upgrades
Backup hardening:
- restic:latest → restic:0.17.3 (deterministic image)
- Weekly backup-verify CronJob: restores latest snapshot and validates
SQL dump structure (≥5 CREATE TABLE, pg_dump header check)
ArgoCD:
- neuron-prod AppProject: scopes deploys to neuron-prod + platform ns,
blacklists ClusterRole/ClusterRoleBinding/Namespace creation,
automated sync window 2–6am UTC, manual always allowed
2026-04-25 22:54:18 -05:00
Will Anderson
a6421a5f98
feat(api): expose api.neurontechnologies.ai + add Stripe and R2 secrets
...
- Add api.neurontechnologies.ai DNS CNAME → CF tunnel
- Add tunnel ingress rule for api.neurontechnologies.ai → Traefik
- Create rest-ingressroute.yaml: api.neurontechnologies.ai → neuron-rest:8081
- Add STRIPE_SECRET_KEY to neuron-prod ExternalSecret (from marketing-test Vault)
2026-04-25 01:25:39 -05:00
Will Anderson
8eb88a3116
feat(runpod): add inference pod templates, nginx LB, and provisioner script
...
Infrastructure readiness for RunPod inference workloads:
- runpod-inference.yaml: ConfigMap with pod creation payloads for RTX 4090,
A40 (single+dual), and custom templates
- runpod-lb-configmap.yaml: nginx least-conn load balancer for inference
endpoint distribution (Deployment + ClusterIP Service)
- runpod-provision.sh: bash provisioner script — reads RUNPOD_API_KEY/HF_TOKEN,
creates pods via GraphQL, polls until RUNNING, outputs endpoint URLs.
Does NOT spin up any pods (dry-run flag available).
2026-04-25 01:20:55 -05:00
Will Anderson
1d527a9365
chore(neuron-prod): add green deployment to kustomization
...
Green manifest was committed but missing from kustomization.yaml so Argo CD
never created the resource. Wires it in so the blue/green slot swap can work.
2026-04-24 22:22:20 -05:00
Will Anderson
95b4669839
feat(neuron-prod): add IngressRoute for license API at /api/v1/* paths
2026-04-24 15:12:37 -05:00
Will Anderson
59aa0f0dfb
deploy neuron-license service to prod
...
Adds license deployment, service, ingress route (/license), and
Unkey + admin token secrets via ExternalSecret. Image built and
pushed by CI on merge of neuron PR #30 .
2026-04-24 01:42:07 -05:00
Will Anderson
5085840a79
Add neuron prod DB backup and tighten actuator surface
...
- Add hourly restic backup CronJob to neuron-prod namespace
- Backs up /data/neuron-prod.db to Cloudflare R2 (legion-neuron-backup bucket)
- Retention: 24 hourly, 7 daily, 4 weekly — <$0.01/month at current DB size
- ExternalSecret pulls restic password + R2 creds from Vault secret/r2
- Actuator exposure per environment:
- prod: health,info only / show-details=never (no internals exposed)
- stage: health,info,metrics,loggers / show-details=always (pre-prod visibility)
- dev: full (health,info,metrics,env,beans,loggers,mappings) / show-details=always
2026-04-23 21:43:20 -05:00
Will Anderson
1b8ea63541
feat(neuron): add stage and prod k8s manifests + Argo CD apps
...
Completes the three-env setup (dev was added previously). Each env has
deployments, services, ingress, PVC, ConfigMap, and ExternalSecret pulling
from Vault. Prod has larger resource limits and 10Gi PVC. Vault secret at
secret/neuron-technologies/prod seeded separately.
2026-04-17 09:43:23 -05:00