Skip to content

Security model

Security is the platform's founding constraint — the company promise is "your server, your data" — so the engineering reflects it at every layer.

1. Credential isolation

The single most important rule: agent gateways never hold upstream provider credentials.

Credential isolation — tenants, platform, upstream providers
Tenants get tokens; providers get keys; the two never meet. Upstream credentials exist only inside LiteLLM and the Provider Broker.
  • Model keys live only in the LiteLLM deployment and in Infisical.
  • Capability keys (search, STT, TTS, voice, browser, image) live only in the Provider Broker deployment and in Infisical.
  • Tenants authenticate with a per-tenant token bound to a slug, with an explicit scope list and daily budgets. A leaked token is a leak of one tenant's quota, not of the platform's providers.
  • Automated audits scan cluster secrets and running pod environments for forbidden provider-key patterns and fail the pipeline on drift.

2. Secrets management

  • Infisical is the single source of truth for secrets; a Kubernetes operator projects them into the cluster as Secrets.
  • Secrets in git are always SealedSecrets (encrypted manifests) — never plaintext — and every live patch is re-sealed and committed in the same session.
  • Rotation is routine: change the value in Infisical, the operator rolls the Secret, the deployment restarts, smoke tests verify, the sealed manifest is updated.

3. The Provider Broker as a security boundary

The broker fails closed on every axis:

  • Missing auth config → no capability requests are accepted.
  • Missing provider key → the provider-specific endpoint fails closed.
  • Brokered services (MCP proxies) must be explicitly registered and their hosts allowlisted; agents can never point the broker at arbitrary URLs.
  • Incoming Authorization, Cookie, and provider-key-style headers are never forwarded to upstream services.
  • Request/response byte caps fail closed and are audited.

4. Application-layer defenses

  • SSRF guards on scrape/search paths: http/https only, no embedded credentials, DNS-resolved addresses must be public, and provider-reported final URLs are re-checked so a redirect into the private network is rejected. Egress is additionally constrained by NetworkPolicies.
  • Signed webhooks: voice callbacks verify Telnyx Ed25519 signatures over timestamp|body with the public key, and are idempotent by event id.
  • Payment-data handling: the broker rejects any payload carrying card data (Luhn-validated), so payment flows stay human-only in a live browser session; CVV mentions are blocked unconditionally.
  • No sensitive logging: the broker records provider, capability, tenant, units, latency and status — never raw query text, transcripts, TTS text, phone-message bodies, or provider keys.

5. Zero-trust networking

  • Tailscale mesh for all admin access: no public SSH, no open admin ports; services that don't need public entry are reachable only on the mesh.
  • Public entry goes through Caddy with automatic TLS; admin UIs sit behind Authentik SSO with a mandatory one-step login (no "choose a login method" screens).
  • NetworkPolicies restrict east-west traffic: tenants can reach the proxy and the broker — and little else.

6. Supply chain

  • Runtime images are built in CI, pushed to GHCR with immutable tags and digests, and deployed only through reviewed manifests pinned to those artifacts.
  • Secret scanning (gitleaks) runs in CI and locally; the repository refuses commits that trip it.
  • Every runtime change is a reviewed PR — no direct pushes to the main branch, no hand-edited live state that isn't mirrored to git in the same session.