Architecture

The AIOps architecture you actually need — full control, end to end. Your tools, your clouds, your decisions, start to finish.

What it is, and why we built it

OpenAgentics is an agentic platform for IT operations that runs entirely on your infrastructure — not hidden behind a SaaS, not locked into any platform or vendor. AIOps means handing real operational power to an agent; the only sane way to do that is to own every link in the chain: the box it runs on, the identities it uses, the models it calls, and the tools it's allowed to touch. So that's what we built.

We were tired of "AI for ops" products that are really someone else's cloud holding your data, your credentials, and your roadmap. The architecture below is the alternative: every decision — where it runs, which provider, which model, which tool, whether an action even happens — stays yours.

The rest of this page is the how: the services, the data plane, and the systems inside the agent core. OpenAgentics is a small set of cooperating services running in Docker Compose (or Helm on Kubernetes), all wired to a shared Postgres, Milvus, and Redis. Drag a node, scroll to zoom, grab the board to pan — and flip between the software services and the infrastructure they run on.

The agent core, at a glance

Eight systems do the real work inside the agent. Here's the one-line version of each — every one is unpacked, with an animation, further down.

Services

Data plane

• Postgres — Sessions, messages, users, workflow definitions and versions, prompt templates, the tamper-evident audit log, integrations. (pgvector for vector columns.)
• Milvus — Vector store for per-user memory, MCP tool descriptions, model-capability profiles, and chat-RAG documents.
• Redis — Session cache, rate limits, offloaded large tool results, pub/sub for cache invalidation.
• Ollama — Local embeddings and self-hosted chat models (GPU-accelerated where available).
• MinIO / S3 · etcd — Object store and coordination for the Milvus vector-store profile.

The agent loop

A chat turn or a workflow node runs through the same pipeline:

1. Resolve — the smart model router picks a provider based on policy, request shape, and user preference.
2. Compose — the system prompt is assembled per request from the static template for your RBAC role, with environment and context variables injected, fit to a token budget.
3. Remember — the unified context layer pulls relevant memories from chat and flows.
4. Tools — semantic dispatch through mcp-proxy selects relevant MCP servers; tool calls run with per-user, credential-scoped tokens.
5. Compact — if the session is large, history is windowed/summarized and oversized tool output is offloaded — silently, never failing the turn.
6. Synthesize — final response synthesis; raw JSON is blocked from reaching users.
7. Audit — every step is recorded with a per-event SHA-256 checksum (tamper-evident).

The rest of this page goes one level deeper into each of those systems.

Model routing & tuning

The SmartModelRouter in openagentic-api decides which model serves each request. It discovers models from every configured provider on startup and builds a capability profile for each — so only models from providers you've actually configured are routable, and routing reasons over real capabilities rather than guessing from model names.

• Structural analysis. Each request is scored on shape, not keywords: does it carry tools, how many, does it include images, roughly how many tokens, how many distinct clouds it touches.
• Capability floors. Agentic and high-stakes task types must clear a function-calling-accuracy floor and a context-window floor; cheap models are gated out of work they'd botch, and a top capability gate reserves the hardest tasks for the strongest models.
• Cost vs. capability. Surviving models are scored on a weighted blend of cost, latency, tool-calling quality, and capability headroom — so simple chat lands on something cheap and complex work escalates.
• Preference & feedback. Per-user model preferences and aggregate response feedback nudge the score.
• Failover. The provider layer enforces per-model daily cost caps and, on a retryable error, fails over to another provider that serves an equivalent model — remapping the request automatically.

Providers out of the box: Anthropic, OpenAI, Azure OpenAI, Azure AI Foundry, AWS Bedrock, Google Vertex, Ollama, and anything OpenAI-compatible. Model capability profiles are embedded into a Milvus collection so the router can reason about capability, not just a hard-coded table.

How we run it. The point of all this is that the floor is yours to set. In our own clusters we run gpt-oss-20b as the floor model — every task starts cheap and local — and the router escalates up through more capable models only as task complexity demands, choosing whatever model in your registry has the capabilities the agent needs to get the job done correctly. Set your own floor; the routing logic does the rest.

Semantic tool routing

A production deployment can expose hundreds of MCP tools — far more than you'd want to dump into a model's context every turn. So tools are routed by meaning. Each tool's name and description is embedded into a Milvus collection, and at request time the agent's intent is embedded and matched by cosine similarity to return only the handful of relevant tools.

• Cloud-aware. A multi-cloud question won't collapse onto one provider — the matcher guarantees diversity across the AWS / Azure / GCP tools it detects.
• Optional rerank. A reranking pass can re-order the top candidates before the final top-k slice.
• Self-healing index. The API re-indexes whenever the connected tool set changes (it hashes the catalogue and re-embeds on a diff), so a newly added tool is reachable as soon as it's indexed.

Embeddings & unified memory

One embedding abstraction serves the whole platform — memory recall, tool search, and chat-RAG all embed through the same service, with the same provider and the same vector dimension. That's deliberate: embed with one provider and search with another and you silently get zero results, so the abstraction makes the provider and dimension a single choice.

• Self-hosted by default. Ollama with nomic-embed-text (768-dim) — no data leaves the box.
• Or a cloud provider. Azure (text-embedding-3-large, 3072-dim), AWS Bedrock Titan, Google Vertex — same code path, one config switch.
• Unified, cross-mode memory. Chat and flows write to per-user vector collections, so the system can recall what you did last week and surface it when it's relevant — no copy-pasting context between tools.

Context & sources compaction

Long, tool-heavy sessions blow past a model's context window and get expensive to re-ship every turn. Compaction keeps a session inside the window without losing the thread, and it runs silently — it is best-effort and never fails a user's turn.

• History windowing. As usage climbs (soft pass around two-thirds full, a harder pass higher up), older messages are windowed and summarized into a compact note, while recent messages, tool-call/result pairs, and anything flagged important (errors, credentials, conclusions) are preserved intact.
• Per-mode budgets. The window is split into a system-prompt / tools / history / response budget that differs for chat vs. flows, and the oldest history is compacted first when the history slice overflows.
• Source offload. Oversized tool output — a giant cloud API dump — isn't re-injected every turn. It's offloaded to Redis (RBAC-scoped, with a TTL) and replaced in the conversation with a compact handle the model can query on demand via a query_data tool.

Every compaction and offload is recorded as a metric, so an operator can see exactly how much context was freed and when.

Prompt composition

The system prompt has two layers — a static base keyed to your role and dynamic values injected per context — so behavior is governed, versioned data, not code.

• Static, one per RBAC role. Each role has a single system prompt, stored in the database and admin-editable with version history. Edit the admin or member prompt once and every session on that role picks it up — no redeploy, no code change.
• Dynamic, per context. At request time the base is filled in with the variables that matter for the turn — environment values, user and workspace variables, and runtime context — injected into the static prompt so the same role prompt adapts to the situation instead of being rewritten for it.

Credentials, permissions & tool trust

Tools don't act as a shared root account. This is the real trust model (there is no first-party / verified / bring-your-own "tier ladder"; the T1/T2/T3 labels you may notice in the UI are display groupings and a model-capability hint, not a security boundary).

• Service-account credentials, scoped per tool. In the self-hosted build, cloud tool calls run with the service-account credentials you configure for each cloud (AWS, Azure, GCP) — provisioned and revocable in your own CSP. No standing god-account is handed to the agent.
• Permission gating. A Claude-Code-style permission service evaluates allow / deny / ask glob rules against each tool call, with modes from fully gated to auto-accept, plus human-in-the-loop approval where you want it.
• Least privilege at the boundary. Each tool only ever receives the credentials it needs for the call it's making.
• Per-user identity is enterprise. Running each action as the signed-in human — SSO plus on-behalf-of token exchange so every cloud log names the individual — is an Gnomus enterprise feature, not part of the self-hosted build.

Flows — the workflow engine

Flows runs as a standalone service. It's a schema-driven plugin engine written from scratch — not a fork of anything. It takes the best ideas from n8n, Flowise, and Langflow and rebuilds them specifically for AIOps. A flow is a node/edge graph; adding a new node type is defining its config schema + an executor and registering it.

• Compile, then run. Before execution, a flow is validated: duplicate IDs, dangling edges, cycle detection, topological sort, and unknown-node-type rejection — so a broken graph fails fast, not mid-run.
• 73 node types across triggers, control flow (condition / switch / loop / parallel / merge), AI (LLM, router, reasoning, guardrails, agents), data/RAG, and integrations (Slack, Teams, PagerDuty, ServiceNow, Jira, email).
• Reliability built in. Error edges route failures to error_handler nodes; per-node retries use exponential backoff with circuit breakers; fallbacks, per-node timeouts, and output assertions catch an agent that silently "fake-succeeds."
• Triggers. Manual, schedule (cron), chat message, file upload, and webhook.
• Versioning. Saves auto-snapshot; runs link back to the exact version that produced them.
• Governance. Every execution is tenant-scoped; secrets are scoped global / group / workflow with per-node-type, per-user, and per-group ACLs (and {{env.*}} pod-environment exfiltration is blocked); approval nodes gate runs on a required number of human approvers with a timeout.

Example flow

An incident-triage runbook that runs itself: a PagerDuty/Alertmanager page routes through a Kubernetes MCP, into an agent that diagnoses, out to Slack and the audit ledger. The red dashed edge is the error handler — self-healing, with a human gate where you want one.

What runs where

• Single-node: docker compose up -d on any Linux/macOS host with Docker. See Docker Compose.
• Kubernetes: helm install openagentic ./helm/openagentic — horizontal scaling, GPU, and air-gapped support. See Kubernetes.

For a managed deployment with SSO, multi-tenant, and enterprise controls, see AgenticWork.