fix: gitea runner workingDir, README updates, monitoring domain
- Fix act-runner workingDir: /data on both init and main containers so .runner registration file is found at daemon startup - Add servers/legion/README.md with full service catalog and deploy docs - Update root README.md to be an index pointing to key docs - Fix monitoring.tf comment: grafana.neuralplatform.ai (not nook.family) - Fix AGENTS.md router model to TP-Link Deco BE5000
This commit is contained in:
@@ -1,16 +1,20 @@
|
|||||||
# infrastructure
|
# infrastructure
|
||||||
|
|
||||||
Personal infrastructure-as-code — servers, networking, DNS, and config.
|
Personal infrastructure-as-code for Legion (k3s home server) and supporting tooling.
|
||||||
|
|
||||||
## Structure
|
| Doc | What it covers |
|
||||||
|
|-----|---------------|
|
||||||
|
| [`AGENTS.md`](AGENTS.md) | Full reference: machines, secrets, services, domains, namespaces, common operations |
|
||||||
|
| [`servers/legion/README.md`](servers/legion/README.md) | What's running on Legion and how deployment works |
|
||||||
|
| [`servers/legion/RUNBOOK.md`](servers/legion/RUNBOOK.md) | Disaster recovery — full restore from scratch |
|
||||||
|
|
||||||
- `servers/legion/` — Legion k3s cluster (Terraform)
|
## Quick start
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd servers/legion
|
cd servers/legion
|
||||||
direnv allow # loads credentials from ~/Secrets/credentials/infrastructure.env
|
direnv allow
|
||||||
terraform plan
|
terraform plan
|
||||||
terraform apply
|
terraform apply
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Push to `servers/` to deploy app changes via Argo CD.
|
||||||
|
|||||||
@@ -0,0 +1,121 @@
|
|||||||
|
# Legion
|
||||||
|
|
||||||
|
Ubuntu 24.04 home server running k3s. Hosts all personal and platform infrastructure.
|
||||||
|
|
||||||
|
**LAN IP:** `192.168.68.77`
|
||||||
|
**SSH:** `ssh legion`
|
||||||
|
**Kubeconfig:** `~/.kube/legion-config`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Services
|
||||||
|
|
||||||
|
### Git & Code
|
||||||
|
| Service | URL | What it does |
|
||||||
|
|---------|-----|--------------|
|
||||||
|
| Gitea | https://git.neuralplatform.ai | Self-hosted Git — all repos live here |
|
||||||
|
| GitHub Actions Runner | — | Runs CI jobs from harmonic-framework org |
|
||||||
|
| Verdaccio | https://npm.neuralplatform.ai | Private npm registry |
|
||||||
|
|
||||||
|
### Platform
|
||||||
|
| Service | URL | What it does |
|
||||||
|
|---------|-----|--------------|
|
||||||
|
| Argo CD | https://argocd.neuralplatform.ai | Deploys everything in `apps/` from Gitea |
|
||||||
|
| Vault | https://vault.neuralplatform.ai | Secret storage |
|
||||||
|
| Docker Registry | https://registry.neuralplatform.ai | Private container image registry |
|
||||||
|
|
||||||
|
### AI & Apps
|
||||||
|
| Service | URL | What it does |
|
||||||
|
|---------|-----|--------------|
|
||||||
|
| Neuron | internal | MCP server + webhook handler for Axon events |
|
||||||
|
| Ollama | http://ollama.legion.local | Local LLM inference (GPU) |
|
||||||
|
|
||||||
|
### Networking & DNS
|
||||||
|
| Service | URL | What it does |
|
||||||
|
|---------|-----|--------------|
|
||||||
|
| AdGuard Home | https://dns.nook.family | LAN DNS with ad blocking |
|
||||||
|
| DoT | `dot.nook.family:853` | DNS-over-TLS for mobile devices (Android Private DNS) |
|
||||||
|
| Traefik | — | Ingress controller, terminates TLS for all services |
|
||||||
|
| Cloudflare Tunnel | — | Exposes public services without opening ports |
|
||||||
|
|
||||||
|
### Observability
|
||||||
|
| Service | URL | What it does |
|
||||||
|
|---------|-----|--------------|
|
||||||
|
| Grafana | https://grafana.neuralplatform.ai | Dashboards, log search, trace viewer |
|
||||||
|
| Prometheus | internal | Metrics collection and alerting |
|
||||||
|
| Alertmanager | internal | Routes alerts to #infrastructure-alerts in Slack |
|
||||||
|
| Loki | internal | Log aggregation (fed by Alloy) |
|
||||||
|
| Tempo | internal | Distributed tracing backend |
|
||||||
|
| Alloy | internal | Collects pod logs → Loki, forwards OTLP traces → Tempo |
|
||||||
|
|
||||||
|
### Data
|
||||||
|
| Service | What it does |
|
||||||
|
|---------|--------------|
|
||||||
|
| Postgres | Shared database (Gitea, Kong, others) |
|
||||||
|
| Redis | Shared cache |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How Deployment Works
|
||||||
|
|
||||||
|
- **Infrastructure** (namespaces, secrets, Helm charts, PVCs): managed by Terraform in this directory
|
||||||
|
- **Apps** (Deployments, Services, ConfigMaps): live in `apps/`, deployed by Argo CD watching this repo
|
||||||
|
- Never apply changes with `kubectl` directly — use Terraform or push to `apps/`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Apply infra changes
|
||||||
|
cd servers/legion/
|
||||||
|
direnv exec . terraform apply
|
||||||
|
|
||||||
|
# Deploy app changes — just push to Gitea
|
||||||
|
git push # Argo CD auto-syncs within ~30s
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Networking
|
||||||
|
|
||||||
|
**Router:** TP-Link Deco BE5000 mesh network, router mode. Subnet `192.168.68.x`, DHCP DNS points to AdGuard (`192.168.68.77`).
|
||||||
|
|
||||||
|
External traffic reaches Legion two ways:
|
||||||
|
|
||||||
|
1. **Cloudflare Tunnel** — public services (`*.neuralplatform.ai`) route through a tunnel pod in k8s. No ports open on the router.
|
||||||
|
2. **Direct port forward** — DoT on port 853 (TCP) is forwarded from the router to `192.168.68.77:853`. Required because Cloudflare can't proxy raw TCP.
|
||||||
|
|
||||||
|
LAN devices use AdGuard as their DNS server (`192.168.68.77`). AdGuard rewrites `*.nook.family` and `*.neuralplatform.ai` to the LAN IP so local traffic stays local.
|
||||||
|
|
||||||
|
ddclient keeps the `dot.nook.family` A record updated if the home IP changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## GPU
|
||||||
|
|
||||||
|
Legion has an NVIDIA GPU. k3s is configured to use it as the default container runtime. Ollama uses it for local inference.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl describe node legion | grep nvidia.com/gpu
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backups
|
||||||
|
|
||||||
|
Nightly at 2am (cron job in `git` namespace):
|
||||||
|
- Dumps all Postgres databases
|
||||||
|
- Backs up Gitea data volume + DB dump via restic → R2 bucket `legion-gitea-backup`
|
||||||
|
- Retains 30 daily / 12 weekly / 6 monthly snapshots
|
||||||
|
|
||||||
|
`~/Secrets` on the Mac is encrypted with age and uploaded to R2 bucket `legion-secrets-backup` nightly.
|
||||||
|
|
||||||
|
See `RUNBOOK.md` for full disaster recovery procedure.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Files
|
||||||
|
|
||||||
|
| File | What it does |
|
||||||
|
|------|--------------|
|
||||||
|
| `bootstrap.sh` | Provisions a fresh Ubuntu 24.04 machine from scratch |
|
||||||
|
| `RUNBOOK.md` | Step-by-step disaster recovery |
|
||||||
|
| `variables.tf` | All configurable values and secrets |
|
||||||
|
| `apps/` | Kubernetes manifests deployed by Argo CD |
|
||||||
@@ -18,6 +18,7 @@ spec:
|
|||||||
initContainers:
|
initContainers:
|
||||||
- name: register
|
- name: register
|
||||||
image: gitea/act_runner:latest
|
image: gitea/act_runner:latest
|
||||||
|
workingDir: /data
|
||||||
command: ["/bin/sh", "-c"]
|
command: ["/bin/sh", "-c"]
|
||||||
args:
|
args:
|
||||||
- |
|
- |
|
||||||
@@ -36,6 +37,7 @@ spec:
|
|||||||
containers:
|
containers:
|
||||||
- name: runner
|
- name: runner
|
||||||
image: gitea/act_runner:latest
|
image: gitea/act_runner:latest
|
||||||
|
workingDir: /data
|
||||||
command: ["act_runner", "daemon"]
|
command: ["act_runner", "daemon"]
|
||||||
envFrom:
|
envFrom:
|
||||||
- secretRef:
|
- secretRef:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
# Monitoring stack — Prometheus, Grafana, Loki, Tempo, Alloy, Alertmanager
|
# Monitoring stack — Prometheus, Grafana, Loki, Tempo, Alloy, Alertmanager
|
||||||
# Grafana UI: https://grafana.nook.family (admin / var.grafana_admin_password)
|
# Grafana UI: https://grafana.neuralplatform.ai (admin / var.grafana_admin_password)
|
||||||
#
|
#
|
||||||
# TODO: Migrate Tempo to tempo-distributed once R2 is wired as trace storage backend
|
# TODO: Migrate Tempo to tempo-distributed once R2 is wired as trace storage backend
|
||||||
# TODO: Replace loki.useTestSchema=true with proper schema_config for production
|
# TODO: Replace loki.useTestSchema=true with proper schema_config for production
|
||||||
|
|||||||
Reference in New Issue
Block a user