Commit Graph

24 Commits

Author SHA1 Message Date
Will Anderson 246368c132 vault: GCE Raft HA cluster replacing defunct Legion k3s deployment
Legion is offline. Vault needs a GCP-native home. This commit provisions
a 3-node Vault HA cluster on GCE e2-small VMs (us-central1 a/b/c) backed
by Raft consensus with GCP KMS auto-unseal already in place.

- vault-nodes.tf: 3 VMs with separate pd-ssd data disks (prevent_destroy),
  IAP-only SSH firewall, Raft peer firewall (8201), global HTTPS LB for
  vault.neuralplatform.ai with Google-managed cert, HTTP health checks
  against /v1/sys/health
- vault/startup.sh: idempotent boot script — installs Vault from HashiCorp
  APT repo, mounts/formats the data disk, writes Raft config with dynamic
  retry_join peers resolved via gcloud metadata, starts vault.service
- vault-auth-gcp.tf: GCP Secret Manager secret for init script; outputs
  the service account emails needed for GCP auth roles
- vault/configure-vault-auth.sh: post-init script to enable gcp auth
  method, write marketing-policy and soma-policy, bind roles to Cloud Run
  service account identities

After apply: set vault_lb_ip output as A record for vault.neuralplatform.ai
in Cloudflare, then run vault operator init on vault-node-1 to bootstrap.
2026-05-04 10:17:12 -05:00
Will Anderson 89564ccec7 bring unmanaged GCP resources under Terraform
vault-kms.tf: KMS keyring/key for Vault auto-unseal + vault-unseal SA
and IAM bindings — all imported from existing GCP resources.
prevent_destroy=true on the crypto key to protect against accidental deletion.

unmanaged-secrets.tf: 8 secrets that were referenced by live Cloud Run
services but had no TF resource — anthropic-api-key, resend-api-key,
supabase-service-key, docuseal-webhook-token, stripe-price-professional,
stripe-price-founding, stripe-price-family-child, gitea-runner-token.

service-account.tf: 4 manually-created SAs — neuron-workspace-dwd,
docuseal-storage, analytics-reader, harmonic-search-console.
2026-05-04 09:57:33 -05:00
Will Anderson 7e092d4686 fix TF drift: add DOCUSEAL_WEBHOOK_TOKEN and TOGETHER_API_KEY env vars
DOCUSEAL_WEBHOOK_TOKEN was present on all live marketing services but
missing from cloud-run.tf and cloud-run-stage.tf — next apply would
have stripped it and broken DocuSeal webhook validation.

TOGETHER_API_KEY (soma-together-api-key) was present on soma-prod-us
but missing from cloud-run-soma.tf — next apply would have broken the
Together.ai inference routing path.
2026-05-04 09:44:46 -05:00
Will Anderson b3609d6401 fold in dev-env + ci-runner provisioning fixups
cloud-run.tf + cloud-run-stage.tf: small alignment edits from the
dev-env agent's work to match the actual deployed Cloud Run shape.

runners/startup.sh: 10-line additions from the gitea-runner agent
during initial provisioning - environment setup adjustments
discovered when the runner came up the first time.

stripe-billing.tf: prior-session work that hadn't been committed,
folding it in now to clean the working tree before further changes.
2026-05-02 13:21:32 -05:00
Will Anderson cd2c22c295 add marketing-dev environment - team-internal auto-deploy zone
dev.neurontechnologies.ai mirrors stage in shape but looser:
broken builds OK, auto-deploys on every push to dev branch (once
CI lands), dev Supabase project (isolated), test Stripe keys,
RESEND_DRY_RUN=1 so transactional email never actually sends.

Locked behind Cloudflare Access with team policy: anyone with
@neurontechnologies.ai email plus named external collaborators
via the access app's emails list.

Initial image is marketing:fix-gallery-render-1236 (matches stage
at the time of this commit). Future deploys land via CI workflow.
2026-05-02 13:13:58 -05:00
Will Anderson 40c6a31784 add sandbox.neurontechnologies.ai - internal-only experimentation env
Locked down to email_domain == neurontechnologies.ai via Cloudflare
Access. Dedicated SA with zero prod IAM. Isolated Supabase project
(not a schema in prod). No outbound email path. /api/share and /said
return 410 Gone in sandbox so the public-artifact surface stays
strictly in prod.

Stub deploy ready. Real soul build pipeline lands separately.
2026-05-02 12:53:16 -05:00
Will Anderson 0c32964ead ci: add gitea actions runner on GCP with WIF-backed deploy SA
A single e2-standard-4 GCE VM in us-central1-a runs act_runner,
registered to git.neuralplatform.ai. Workload Identity Federation
binds the runner to the existing neuron-ci-pusher SA (artifactregistry.writer
+ run.developer + serviceAccountUser on the runtime SAs). No long-lived
JSON keys live on the runner; the GCP_SA_KEY secret remains as a fallback
in case the Gitea OIDC issuer isn't reachable from oauth2.googleapis.com.

The runner VM has its own minimal-scope SA (gitea-runner-vm) that can
only read the registration token from Secret Manager — splitting boot
identity from deploy identity so a runner compromise doesn't grant push.

SSH is IAP-only (no public ingress on :22). Reference workflow lives at
products/web/.gitea/workflows/deploy.yaml and takes manual gcloud out of
the loop: push to main triggers build, push to AR, parallel deploy to
all 3 marketing prod regions, traffic flip, smoke check.
2026-05-02 12:45:25 -05:00
Will Anderson 7ee6099139 dns: document email auth records and rationale for out-of-band management
SPF / DKIM / DMARC for neurontechnologies.ai are managed via the Cloudflare
API rather than Terraform because the cloudflare_record provider rewrites
name="@" on apply, which forces replacement of the apex SPF record and
opens a propagation gap. A duplicate (malformed-quoted) apex SPF that was
causing Gmail to fail SPF was also removed via the API; the surviving
record is the canonical one and is documented here for traceability.
2026-05-01 22:49:32 -05:00
Will Anderson ec2d5794ec prod: fix Cloud Run config for v1.0 (port 8080, env vars, secrets) 2026-05-01 18:13:00 -05:00
Will Anderson d9ac977fef Wire Stripe billing secrets and env vars into Soma Cloud Run
- stripe-billing.tf: Secret Manager resources for all billing IDs:
  meters (input/output), plan prices (free/professional/founding),
  overage prices per plan per token type
- cloud-run-soma.tf: inject all 13 Stripe env vars into the Soma service
  via Secret Manager secret_key_ref; add billing secrets to depends_on
2026-04-29 04:34:13 -05:00
Will Anderson e5c9fd1287 Add Anthropic API key secret and env var to Soma Cloud Run
Provisions soma-anthropic-api-key in Secret Manager and injects it as
ANTHROPIC_API_KEY into the Cloud Run service. Also sets ANTHROPIC_MODEL
to claude-sonnet-4-5. Soma's chat handler now uses this to route the
neuron model to Anthropic instead of spinning up GPU VMs.
2026-04-28 21:29:27 -05:00
Will Anderson c3a90ac484 Add SOMA_KEY_SECRET to Secret Manager and wire it into Cloud Run
Creates soma-key-secret in GCP Secret Manager for HMAC key signing, and
adds SOMA_KEY_SECRET + SOMA_BASE_DOMAIN env vars to the soma-prod-us Cloud
Run service so stateless API key validation works in production.
2026-04-28 11:39:33 -05:00
Will Anderson 1a8e039ecf Rename inference endpoint to neuron.neurontechnologies.ai, add wildcard DNS for customer orgs 2026-04-28 11:15:11 -05:00
Will Anderson 9e2646701c Grant soma compute permissions for GCE inference nodes
- Add roles/compute.instanceAdmin.v1 for soma SA (provision/terminate GCE VMs)
- Add roles/iam.serviceAccountUser for soma SA (assign SA to inference VMs)
- Add firewall rule: allow TCP 8000 inbound to soma-inference-node tagged VMs
- Add soma-node-secret to Secret Manager (GCE startup script auth)
- Inject SOMA_NODE_SECRET, GCP_PROJECT_ID, GCP_ZONE, SOMA_PUBLIC_URL into
  the soma Cloud Run service via env vars
2026-04-28 10:48:32 -05:00
Will Anderson f2b025a433 Deploy soma to GCP Cloud Run at ai.neurontechnologies.ai
- Add cloud-run-soma.tf: soma-prod-us Cloud Run service in us-central1,
  neuron-soma-sa service account, soma Artifact Registry repo, Secret
  Manager secrets for HF token and operator key, serverless NEG, backend
  service, SSL cert
- Add dns-gcp.tf: Cloudflare A record for ai.neurontechnologies.ai pointing
  to GCP LB IP; Cloudflare provider added to main.tf/variables.tf
- Update load-balancer.tf: soma host rule + path matcher, soma SSL cert
  added to HTTPS proxy
- Update outputs.tf: soma service URL and artifact registry URL outputs
- Remove legion soma k8s manifests (Legion is gone)
- Update AGENTS.md to reflect GCP as primary production environment
2026-04-28 10:15:15 -05:00
Will Anderson 55a38bb693 fix: update Gitea URL to new GCP cluster instance
Legion hardware died April 27, 2026. New GCP k3s instance at 104.197.5.199.
Gitea now at gitea.git.svc.cluster.local:3000 within cluster.
2026-04-27 16:20:46 -05:00
Will Anderson fdbbfdb0ad iam: grant ci-pusher serviceAccountUser on all Cloud Run runtime SAs 2026-04-26 11:18:50 -05:00
Will Anderson e22dce6b5a iam: grant ci-pusher run.developer for Cloud Run deploys 2026-04-26 10:53:21 -05:00
Will Anderson 5b16aebdcb Remove stage environment from GCP — staging is local only 2026-04-26 01:09:44 -05:00
Will Anderson 53f2ad3c16 Fix Cloud Run probe ports and LB timeout_sec
- accounts: add ACCOUNTS_PORT=8080 env var (service defaults to 7753)
- api: add SERVER_PORT=8080 env var, change probe path to /actuator/health
- LB backends: remove timeout_sec (not supported for serverless NEG backends)
2026-04-26 00:28:33 -05:00
Will Anderson 36cfd3738d Fix Cloud Run: remove reserved PORT env, increase startup probe tolerance 2026-04-25 23:55:33 -05:00
Will Anderson bb583e3ccb Fix HCL syntax errors in accounts and api Cloud Run definitions 2026-04-25 22:54:18 -05:00
Will Anderson d4c65d5857 Expand GCP infra: accounts + API services, Cloud SQL, Artifact Registry
Architecture: intelligence stays on Legion; only compiled artifacts cross
to GCP. Source code and Neuron's knowledge base never leave the system.

Artifact Registry:
- neuron-marketing, neuron-accounts, neuron-api repos in us-central1
- Keep-last-10 cleanup policy; ci-pusher SA with writer access
- Legion CI runners authenticate via GCP_SA_KEY Gitea secret

Cloud SQL (cloud-sql.tf):
- postgres-15 on db-g1-small, us-central1 (scale up to REGIONAL HA at 1k users)
- Point-in-time recovery, 14-day backup retention
- Accounts DB + user; password generated and stored in Secret Manager
- JWT signing key in Secret Manager (shared by accounts + api)
- Cloud Run connects via built-in Auth Proxy (Unix socket volume mount)

Accounts Cloud Run (cloud-run-accounts.tf):
- 3 regions (us-central1, europe-west1, asia-northeast1), min:1 max:50
- Cloud SQL proxy volume mount; secrets via Secret Manager
- Stripe + JWT env vars; health probe on /health

API Cloud Run (cloud-run-api.tf):
- 3 regions, min:1 max:100, cpu_idle=false (always-hot)
- Validates JWTs from accounts service; no direct DB connection
- License admin token from Secret Manager

Load balancer (host-based routing):
- Same global anycast IP for all three services
- URL map routes by Host: neurontechnologies.ai→marketing,
  api.neurontechnologies.ai→api, accounts.neurontechnologies.ai→accounts
- New managed SSL certs for api.* and accounts.* added to HTTPS proxy
- Cloud Armor (WAF + rate limit) applied to all backends

Service accounts + IAM:
- neuron-accounts-sa: secretmanager.secretAccessor + cloudsql.client
- neuron-api-sa: secretmanager.secretAccessor
- allUsers invoker on all prod Cloud Run services (LB health checks)

bootstrap.sh:
- One-shot setup: pulls Stripe secrets from Vault → Secret Manager,
  creates CI SA JSON key, prints DNS + next-step instructions
2026-04-25 22:54:18 -05:00
Will Anderson 91bfc42989 ops: scale down all non-critical services — dedicate resources to Neuron
Scaled to replicas: 0: fornax (coordinator/ui/workers), jellyfin via media,
overseerr, plex, radarr, sonarr, bazarr, prowlarr, flaresolverr, ollama,
harmonic-wordpress, docuseal, listmonk, memos, plane, redpanda, wp-coordinator
2026-04-24 20:07:41 -05:00