Engineering highlights¶
What an engineer would notice walking through the platform — the design decisions that are harder to copy than to describe.
1. Providers as interchangeable policy¶
60+ model aliases behind one endpoint; four stable tiers; weighted multi-key capacity groups; PAYG mirrors for subscription tiers; permanent failover drills that force real provider errors in production so the fallback machinery is proven continuously. Changing a provider is a config PR, not a fleet rollout.
2. Capabilities without credentials¶
The Provider Broker gives tenants search, scrape, STT, TTS, voice calls, browser automation, image generation, and MCP services — with per-tenant tokens, scopes, budgets, and cost caps, and upstream keys that never leave the broker. Tenant isolation by token instead of by network.
3. The smart router¶
Deterministic keyword rules → cheap LLM classifier (1.5 s timeout) → <1 ms fail-open heuristic scorer → session-pinned tier. Cost and latency as routing policy, with a documented answer to "which model should answer this?"
4. Security as an engineering discipline¶
Credential isolation with automated audits, SealedSecrets + gitleaks, SSRF-guarded scraping with redirect re-checks, Ed25519-verified webhooks, Luhn-gated card-data rejection, zero-trust Tailscale networking, and NetworkPolicy east-west restrictions. "Your server, your data" is enforced in code, not claimed in marketing.
5. A fleet operated like production software¶
Immutable images with digest pinning, canary-first rollouts, versioned agent baselines mounted read-only, durable sagas with verified postconditions, placement audits, health cron jobs, backup restore drills, and a mission-control panel. The lab treats agents as deployable services, not scripts.
6. Cost discipline with real numbers¶
Self-hosted free rungs (SearXNG, Crawl4AI) first in every provider ladder; the cheapest tier that answers as the router default; per-tenant budgets in the broker; spend telemetry in LiteLLM and Langfuse; AWS right-sizing and a hybrid Proxmox tier. The platform's cost model is a design input, not an afterthought.
7. One database, twelve logical databases¶
A single managed PostgreSQL instance with one logical DB per service — centralized backups, point-in-time recovery, no per-app database operations — proving that a small platform can carry a whole suite of stateful services without a fleet of database pods.
8. Developer productivity as a product¶
Every coding harness (Hermes, Codex, Claude Code, OpenCode, Pi) points at the same gateway with virtual keys; the public MIT-licensed agent-dev-kit extends the same discipline (layers, adversarial review, injection defenses, planted-bug evals) to any developer's machine.