Skip to content

AKKO Control Plane

AKKO integrates ~42 components (storage, catalog, compute, orchestration, BI, AI, RBAC). Exposing each one's native UI made the platform read like "a stack of open-source bricks" rather than "one product". The Control Plane fixes that: a thin FastAPI service that owns the product abstractions and proxies downstream to the right component.

Principle — the cockpit and external clients only talk to /api/v1/*. Each native component keeps its URL for direct operator use, but day-to-day product flows never need it.

What it exposes

Abstraction Endpoint What's behind
Dataset GET /api/v1/datasets OpenMetadata tables + OPA allow/deny + Trino FQN
Workspace GET /api/v1/workspaces
POST /api/v1/workspaces		 JupyterHub single-user server (spawn / stop / describe)
Pipeline GET /api/v1/pipelines Airflow DAGs + DagRun history (trigger coming Phase 1.5)
Agent POST /api/v1/agents/ask Forwards to ADEN with caller identity
Components GET /api/v1/components
GET /api/v1/components/drift		 Reconciler: every ComponentSpec vs live cluster state + drift + remediation plan (admin-only)

OpenAPI spec: /api/openapi.json · Swagger UI: /api/docs.

Architecture

flowchart LR
    Client[Cockpit React / CLI / SDK] --> Traefik[Traefik + oauth2-proxy]
    Traefik -->|X-Auth-Request-* headers| CP[akko-control-plane]
    CP --> OM[OpenMetadata]
    CP --> JH[JupyterHub]
    CP --> AF[Airflow 3]
    CP --> AD[ADEN]
    CP --> OPA[OPA]
    CP --> TR[Trino]

The Control Plane is stateless: no database, no background jobs. Identity comes from the X-Auth-Request-Email and X-Auth-Request-Groups headers injected by oauth2-proxy forward-auth. A NetworkPolicy limits ingress to cockpit + Traefik pods, so the headers cannot be forged by other workloads in the namespace.

What it is NOT

  1. Not a god service. No business logic duplicated from downstream components. If a call can be a 1-line proxy, it is. The role of the Control Plane is translation, not re-implementation.
  2. Not on the hot path. Trino query execution, JupyterHub WebSocket kernel traffic, Airflow task logs stream directly from their native ingress. The Control Plane is for product intent (list / create / describe), not sub-second data flow.
  3. Not the auth layer. oauth2-proxy + Keycloak do the JWT validation; the Control Plane only reads the trusted headers that land after.

Role of each abstraction

Dataset

A Dataset is the product-level view of a governed data asset. Each Dataset record composes:

  • the OpenMetadata entity (name, owner, description, tags, domain)
  • the Trino FQN the caller can paste into any notebook or BI client (catalog.schema.table)
  • the OPA allow/deny flag for the caller (arriving in Phase 1.2)
  • the PII / classification tags from OpenMetadata

The cockpit never shows iceberg.analytics.transactions as a raw path — it shows "Transactions dataset, owner @carol, PII tagged, you can read".

Workspace

A Workspace is the user's isolated working area. Behind the scenes it composes:

  • a JupyterHub single-user server (the notebook compute)
  • a Trino session scoped to the caller's identity (so OPA row-filter + column-mask apply)
  • a object storage prefix under akko-users/<user>/ for artefacts
  • metadata (name, description, created_at)

POST /api/v1/workspaces returns 501 until the JupyterHub admin-token wiring ships (Phase 1.4b). Read-only endpoints already work.

Pipeline

A Pipeline projects an Airflow DAG as a business pipeline:

  • the DAG run status (healthy / stale / failing)
  • the OpenLineage graph (upstream / downstream Datasets)
  • the last N DagRun states — at a glance, not one click into Airflow

Users never learn the word "DAG". Trigger / pause / resume land in Phase 1.5b.

Agent

Today, a single Agent: ADEN. POST /api/v1/agents/ask forwards a natural-language question to ADEN with the caller headers injected so ADEN's per-user OPA decisions see the right principal. Multiple Agents (SQL duel validator, auto-doc, auto-remediation) plug in here in Phase 3 without breaking the cockpit contract.

Reconciler — declared vs live

The abstractions above translate product intent downstream. The Reconciler answers the operator question the stack could not answer before: "what does AKKO DECLARE, and what is actually RUNNING?"

  • It loads every ComponentSpec (baked at /app/components, single source = helm/akko/components/*.yaml) and resolves the live state through the in-cluster Kubernetes API (pods matched by each spec's health.selector, read with the ServiceAccount token, no heavyweight client).
  • Each component is classified healthy / degraded / absent / unknown, and drift is flagged when a component declared defaultEnabled has no pods, or its pods are not all ready.
  • For each drifted component it also computes a remediation plan — the action reconciliation would take (deploy when declared-on-but-absent, stabilize when pods lag), with a suggested command. This plan is read-only: safe_to_auto is always false. The Control Plane plans, it does not apply — honest by construction. Guarded write-reconciliation (apply behind an explicit admin confirmation + audit) is a later, opt-in step.
  • Both GET /api/v1/components and GET /api/v1/components/drift are admin-only (403 otherwise). Read-only means a Role limited to get/list on pods, plus a NetworkPolicy egress to the apiserver (443 and 6443 — on k3s the ClusterIP 10.43.0.1:443 is DNAT'd to the host apiserver on 6443, and kube-router applies the rule on the post-DNAT port).

This inventory + drift + plan is surfaced in the cockpit at /#/x/platform (React island features/platform, admin-only): status counters, components grouped by layer, and the remediation plan inline under each drifted row. Still stateless — no database, the k8s read happens on demand.

Deployment

Part of the umbrella chart when enabled. Ships as its own sub-chart:

# helm/akko/charts/akko-control-plane/values.yaml
image:
  repository: akko-control-plane
  tag: "2026.06"
env:
  openmetadataUrl: "http://openmetadata:8585"
  trinoUrl: "http://akko-trino:8080"
  airflowUrl: "http://akko-api-server:8080"
  jupyterhubUrl: "http://proxy-public"
  jupyterhubHubUrl: "http://hub:8081"
  adenUrl: "http://akko-akko-aden:8000"
  opaUrl: "http://akko-akko-opa:8181"

Every URL is overridable for air-gapped / non-standard naming.

Roadmap

The Control Plane is Phase 1 of the broader product-over-tech plan documented in the AKKO planning repository (private). Timeline:

Phase Period Goal
1.1 (done) 2026-04-22 Service skeleton + Helm chart + 4 endpoint stubs
1.2 Sprint 42 OPA enrichment, projection polish, Trino FQN rewrite
1.3-1.5 Sprint 42 Full abstraction coverage (Dataset / Workspace / Pipeline mutating verbs)
2 Sprint 43-44 Cockpit canvas + Cmd+K built on top of the Control Plane
3 Sprint 45 Additional Agents (SQL duel, auto-remediation, auto-doc)

See also