will.anderson/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5334881d3fa7ca5b600adada28bd31b1241ca72f
infrastructure/Agents.md
T

8.8 KiB
Raw Blame History

Agent System Reference

Quick reference for AI agents working in Will's infrastructure. Read this before doing any infrastructure work.

Machines

Machine Role IP SSH
Local Mac Dev workstation 192.168.68.55 —
Legion Ubuntu 24.04, k3s server 192.168.68.77 ssh legion

Legion runs all production services: k3s, Postgres, Redis, Ollama (GPU), Docker Registry, GitHub Actions runner, Vault, Gitea, AdGuard, Neuron.

Secrets — Always Use Vault

Vault is the source of truth for all credentials. Never hardcode secrets. Never ask the user for a secret value.

CLI access (on local Mac)

# Load all secrets into env
set -a; source ~/Secrets/credentials/infrastructure.env; set +a

# Or with direnv (auto-loads in legion/ directory):
cd ~/Development/infrastructure/servers/servers/legion/
# direnv auto-sources infrastructure.env

# Friendly alias (after sourcing infrastructure.env):
secret civitai          # → civitai API key
secret slack-bot-token  # → Slack bot token
secret cf-key           # → Cloudflare API key

# Direct vault lookup:
vault kv get -field=api_key secret/ai          # CivitAI
vault kv get -field=bot_token secret/slack      # Slack
vault kv get -field=api_key secret/cloudflare   # Cloudflare

Full path+field table is in ~/Secrets/credentials/infrastructure.env — look at the _v() calls.

Vault connection

VAULT_ADDR=https://vault.neuralplatform.ai
VAULT_TOKEN=$(cat ~/Secrets/tokens/vault-root-token)

If Vault is unreachable (CF tunnel down), port-forward:

ssh legion -L 8200:localhost:8200  # in background
export VAULT_ADDR=http://localhost:8200

How secrets flow to pods

Vault → TF_VAR_* (via infrastructure.env) → kubernetes_secret (Terraform) → pod env via secretKeyRef

Secrets in Terraform:

  • neuron-secrets (neuron ns): GITHUB_WEBHOOK_SECRET, GITEA_WEBHOOK_SECRET, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET
  • cloudflared-secret (neuron ns): TUNNEL_TOKEN
  • github-runner-secret (ci ns): ACCESS_TOKEN
  • gitea-db (git ns): password

Infrastructure Management

The split: Terraform vs Argo CD

Layer Tool What it owns
Infrastructure Terraform Namespaces, PVCs, ConfigMaps, k8s Secrets, Ingresses, Helm releases
Applications Argo CD Deployments, Services

Never modify k8s resources directly with kubectl. Always go through Terraform or Argo CD (Git push).

Running Terraform

cd ~/Development/infrastructure/servers/servers/legion/
direnv exec . terraform plan    # load secrets + plan
direnv exec . terraform apply   # apply

State is stored in Cloudflare R2 bucket legion-terraform-state.

Argo CD

Root app legion-apps watches servers/legion/apps/ in the will/infrastructure Gitea repo. Push to main → Argo CD syncs within ~30 seconds.

App manifests: ~/Development/infrastructure/servers/servers/legion/apps/*.yaml

Argo CD UI: https://argocd.neuralplatform.ai

Repo layout

~/Development/
  infrastructure/                      ← this repo (local only, no GitHub remote)
    Agents.md                          ← you are here
    servers/                           ← nested repo (will/infrastructure on Gitea)
      servers/legion/
        *.tf                           ← Terraform (infrastructure layer)
        apps/*.yaml                    ← Argo CD manifests (app layer)
  neural-platform/
    neuron/                            ← github: harmonic-framework/neuron, gitea: neural-platform/neuron
  harmonic-framework/
    harmonic-framework.com/            ← github: harmonic-framework/harmonic-framework.com
  projects/
    personal/
      prism/                           ← github: harmonic-framework/prism, gitea: neural-platform/prism
    clients/
      ilih.life/                       ← github: harmonic-framework/ilih.life, gitea: will/ilih.life

Services & Domains

Family (nook.family — via Cloudflare tunnel)

Service URL Notes
AdGuard https://dns.nook.family DNS + ad blocking (DoH: https://dns.nook.family/dns-query)

Platform (neuralplatform.ai — via Cloudflare tunnel)

Service URL Notes
Argo CD https://argocd.neuralplatform.ai GitOps UI
Vault https://vault.neuralplatform.ai Secrets
Gitea https://git.neuralplatform.ai Git server
Ollama https://ollama.neuralplatform.ai LLM API
Neuron MCP https://neuron.neuralplatform.ai MCP server (port 8001)
Axon webhooks https://axon.neuralplatform.ai Webhook hub (port 3847)
npm registry https://npm.neuralplatform.ai Verdaccio
PyPI registry https://pypi.neuralplatform.ai devpi
Docker Registry https://registry.neuralplatform.ai Push images here
Registry UI https://docker.neuralplatform.ai Docker registry browser

NodePort services (direct to Legion IP)

Service Port Notes
Gitea SSH 30022 git@192.168.68.77:30022
Ollama API 31434 http://192.168.68.77:31434

Access Patterns

SSH

ssh legion                        # Ubuntu 24.04, user: will
ssh legion "kubectl get pods -A"  # run kubectl remotely

kubectl (local)

export KUBECONFIG=~/.kube/legion-config
kubectl get pods -A
kubectl logs -n neuron deployment/neuron

Gitea API

CF Access blocks direct calls from Mac. Always use Legion cluster IP via SSH:

TOKEN=$(vault kv get -field=api_token secret/gitea)
ssh legion "curl -s -H 'Authorization: token $TOKEN' http://10.43.1.53:3000/api/v1/repos/search?limit=50"

Key Paths

What Where
This repo ~/Development/infrastructure/
Infrastructure (Terraform + Argo CD) ~/Development/infrastructure/servers/
Neural Platform projects ~/Development/neural-platform/
Harmonic Framework projects ~/Development/harmonic-framework/
Personal projects ~/Development/projects/personal/
Client projects ~/Development/projects/clients/
Knowledge base ~/Knowledge/
Secrets & tokens ~/Secrets/
Infrastructure env ~/Secrets/credentials/infrastructure.env
kubeconfig ~/.kube/legion-config
Agent directives ~/.claude/projects/-Users-will/memory/directives.md
Agent memory index ~/.claude/projects/-Users-will/memory/MEMORY.md

GitHub / Gitea

GitHub (github.com/harmonic-framework) — open-source, CI runners, public-facing

Gitea (git.neuralplatform.ai) — private/infra, three orgs:

  • will — personal infra (will/infrastructure)
  • neural-platform — platform projects (neural-platform/neuron, neural-platform/prism)
  • harmonic-framework — org site/assets (harmonic-framework/harmonic-framework.com)

Runner labels: self-hosted,linux,x64,legion

Namespaces (k8s)

Namespace What
dns AdGuard
git Gitea
neuron Neuron + cloudflared
ollama Ollama
ci GitHub runner
packages Verdaccio + devpi
registry Docker registry + UI
platform Postgres, Redis
monitoring Prometheus, Grafana, Loki, Tempo, Alloy
vault HashiCorp Vault
argocd Argo CD
cert-manager cert-manager

Common Operations

Deploy a config change to an app

  1. Edit servers/servers/legion/apps/<app>.yaml
  2. git commit && git push (from servers/)
  3. Argo CD syncs in ~30s

Add a new secret to a pod

  1. Add kubernetes_secret resource to the relevant .tf file (value from var.X)
  2. Add variable "X" to variables.tf (sensitive = true)
  3. Add TF_VAR_X=$(_v path field) to infrastructure.env
  4. Store the secret: vault kv put secret/path field=value
  5. direnv exec . terraform apply
  6. Reference secretKeyRef: name: <secret-name> in the app manifest

Add a new Helm service (stays in Terraform)

Add helm_release resource to appropriate .tf file, apply.

Rotate a secret

  1. Update value in Vault: vault kv patch secret/path field=newvalue
  2. Re-run direnv exec . terraform apply (updates kubernetes_secret)
  3. Restart the affected pod: kubectl rollout restart deployment/<name> -n <ns>

Network

  • Subnet: 192.168.68.x (TP-Link Deco, router mode)
  • DHCP DNS: 192.168.68.77 (AdGuard) + 1.1.1.1 fallback
  • AdGuard provides *.nook.family resolution → 192.168.68.77
  • Cloudflare tunnel ID: 54bc9b05-3953-47a2-9c3e-adecdcc53d51
  • systemd-resolved is disabled on Legion (conflicts with AdGuard port 53)

What NOT To Do

  • kubectl apply directly — use Terraform or push to Git
  • kubectl edit — use Terraform or Git
  • Hardcode any secret value — always use var.X → Vault
  • Commit terraform.tfstate — state is in R2, never local
  • Call Gitea API from Mac directly — CF Access blocks it, use SSH to Legion
Reference in New Issue View Git Blame Copy Permalink