Platform¶
Blackrack Labs runs on a hybrid Kubernetes cluster: a managed cloud control plane plus workers in AWS, and a Proxmox virtual machine in Bogotá used as the low-cost self-hosted tier. Everything — agents, gateways, apps, databases, observability — runs on this cluster as code.
The five pillars¶
- LLM gateway — LiteLLM exposes every model behind one OpenAI-compatible endpoint; the smart router turns provider selection into policy.
- Provider Broker — the scoped capability gateway through which every agent reaches search, voice, media, and browser tools without holding provider credentials.
- Identity — Authentik is the single sign-on for every human-facing service.
- Secrets — Infisical is the source of truth for credentials, projected into Kubernetes and rotated without touching agents.
- Storage & data — managed PostgreSQL (RDS) for stateful services, Longhorn for cluster-local volumes, S3 for backups.
Design rules that shape everything¶
- Agents never hold upstream provider keys. Capabilities arrive through the broker with a scoped token; model calls arrive through LiteLLM with a per-tenant virtual key. (Security model)
- Providers are interchangeable. A model alias is a policy, not a dependency. Tiers, fallbacks, and failover drills are exercised continuously.
- Live state must be reproducible from git. A change is not done until it is in the manifests, the sealed secrets, and the changelog.
- Fail closed. Missing config, missing keys, or unknown hosts mean the request is denied — never passed through.
Cluster at a glance¶
| Layer | Technology | Role |
|---|---|---|
| Control plane | k3s on AWS EC2 | Kubernetes control plane + storage node |
| Workers (cloud) | AWS EC2 (arm64 and x86) | Infra services, gateways, data nodes |
| Worker (self-hosted) | Proxmox VM (Bogotá) | Low-cost tenant gateway tier |
| Networking | Caddy ingress, Tailscale mesh | Public entry, zero-trust admin access |
| Storage | Longhorn (replicated), RDS PostgreSQL, S3 | Cluster volumes, relational data, backups |
| Provisioning | Terraform + GitHub Actions | Infrastructure as code, deploy pipelines |
The cluster runs services in dedicated namespaces: the agent platform, the self-hosted app suite, observability, and the CI/CD control plane.