Catalog Manager¶

AKKO Catalog Manager is the self-service backend + cockpit module that lets admins manage Trino catalogs at runtime. It pairs with Trino 481 dynamic catalog management (catalog.management=dynamic, catalog.store=file) so adding, editing or removing a catalog incurs zero downtime and no helm upgrade.

Rebuilt 2026-06 (spec catalog-onboarding-refonte-2026-06). The previous /api/admin/catalogs API and its 4 overlapping cockpit surfaces were retired after the cutover; the canonical API is /api/catalogs and the cockpit ships ONE surface: the catalog-onboarding module.

Supported connectors¶

The connector list is data-driven: the cockpit form renders whatever GET /api/catalogs/connectors returns (registry pattern, YAML-extensible via CONNECTOR_SPECS_DIR without a code change).

Category	Connectors
Relational (JDBC)	PostgreSQL, MySQL, SQL Server, Oracle, ClickHouse, Redshift
Lakehouse	Iceberg (REST / Polaris), Iceberg (HMS), Hive + Kerberos, Delta Lake
NoSQL / Streaming	MongoDB, Cassandra, Elasticsearch, Kafka
Escape hatch	Custom (`connector.name` + raw properties)

Architecture¶

Cockpit module pages/catalog-onboarding/ (MVC, single surface)
    │ HTTPS + SSO cookie
    ▼
nginx (akko-cockpit-react)  /api/catalog-manager/ → injects Bearer
    ▼
FastAPI host (akko-catalog-manager) + catalog_manager package
    │   1. verify JWT → require akko-admin role
    │   2. registry → family builder → properties (Strategy pattern)
    │   3. secrets → credential FILE on the shared catalog PVC
    │      (never in the SQL — CREATE CATALOG is fully logged by Trino)
    │   4. CREATE CATALOG via /v1/statement as svc-catalog-manager
    │      (OPA: catalog_admin machine identity, least privilege)
    │   5. saga with compensation: any failure rolls everything back
    │   6. structured correlated audit JSON (secrets redacted)
    ▼
Trino 481 — coordinator RW + workers RO on PVC /etc/trino/catalog
            platform catalogs snapshotted in .baseline/ (restore)

Key decisions:

Form ⟷ SQL bidirectional: the cockpit shows the exact CREATE CATALOG statement while you type, and parses your SQL edits back into the form. The displayed SQL is a projection — the submit sends {connector, name, config} and the backend regenerates the SQL, so a display drift can never execute something else.
No permissions tab: onboarding registers the catalog (admin-visible by default). WHO can read WHAT is managed in the Data Access component (platform RBAC ≠ data access).
Edit = DROP + CREATE plan: Trino has no ALTER CATALOG. The module shows the plan honestly and the backend rolls back to the previous definition if the re-CREATE fails. Secrets are never read back: re-enter them when editing.
Delete / restore: platform catalogs (provenance platform) are snapshotted in a baseline on the PVC. Deleting one shows a restore banner; POST /api/catalogs/restore re-creates the missing ones. Client catalogs require a typed-name confirmation (definitive).

Add a catalog (cockpit)¶

Log in as a user with the akko-admin role.
Sidebar → Sources de données (catalogs page).
＋ Ajouter une source → pick a connector chip, fill the fields (password has a reveal toggle) — or edit the SQL panel directly.
⚡ Tester la connexion runs a TCP pre-flight probe.
Créer la source — Trino validates the connector at CREATE CATALOG time; the catalog is live cluster-wide in ~5 s.

Add a catalog (API)¶

curl -X POST https://cockpit.my-company.example/api/cockpit/catalog-manager/api/catalogs \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "connector": "postgresql",
    "name": "client_pg_01",
    "config": {
      "host": "db.customer.example", "port": "5432",
      "database": "sales", "user": "trino_readonly",
      "password": "<redacted>"
    }
  }'
# → 201 {name, connector, provenance, sql}   (sql contains NO secret)

Catalog names are bare Trino SQL identifiers: ^[a-z][a-z0-9_]{0,40}$ (no hyphen), system and information_schema are reserved, and Trino has no IF NOT EXISTS — duplicates are rejected before the CREATE.

Full API¶

Method · path	Purpose
`GET /api/catalogs/connectors`	connector registry (drives the form)
`GET /api/catalogs/meta`	platform constants for the frontend SQL codec
`GET /api/catalogs`	list — live ∪ stored ∪ missing-baseline (`missing: true`)
`GET /api/catalogs/{name}`	detail, properties WITHOUT secrets (edit prefill)
`POST /api/catalogs/preview`	validation + generated SQL, zero side effect
`POST /api/catalogs`	create (saga, credential file, audit)
`PUT /api/catalogs/{name}`	update = DROP+CREATE with automatic rollback
`DELETE /api/catalogs/{name}`	idempotent; returns `restorable` + `leaky_connector`
`POST /api/catalogs/restore`	re-CREATE missing platform catalogs from baseline
`POST /api/catalogs/test`	TCP pre-flight probe (3 s timeout)
`GET /api/catalogs/{name}/status`	live `SHOW SCHEMAS` count
`POST /api/catalogs/sql`	expert mode — accepts ONLY `CREATE/DROP CATALOG`

Hive + Kerberos — how the keytab reaches Trino¶

Two distinct paths, both live-proven:

Platform path (chart-managed) — the federation catalogs shipped with the platform (e.g. the Cloudera demo): the keytab lives in a Kubernetes Secret mounted on the Trino pods by the chart, and krb5.conf plus the hadoop conf files are JVM/platform-level mounts. This is what runs the 6.3M-row Cloudera federation.

Self-service path (cockpit / API) — for catalogs created at runtime:

Sources de données → connector Hive → advanced options → enable Authentification Kerberos.
Fill service principal + client principal.
Paste the keytab as base64 in the Keytab (base64) field (base64 -w0 my.keytab), or send config.keytab_b64 through POST /api/catalogs.

Backend mechanics: the base64 is decoded to binary and written next to the credential files on the shared catalog PVC — the only path readable by the coordinator AND the workers (workers open their own metastore/HDFS connections). The generated SQL references that file path but never contains the keytab (Trino logs every CREATE CATALOG in full). The emitted property set mirrors the live-proven federation catalog: metastore KERBEROS auth + fs.hadoop.enabled=true + hive.config.resources (platform-mounted hadoop confs, env-driven) + kerberized HDFS principal/keytab, impersonation disabled.

Ticket lifecycle: Trino's JVM acquires and renews tickets from the keytab automatically — there is no ticket-expiry operation to run.

Failure modes (all verified against the real kerberized metastore)¶

Situation	Behaviour
Wrong client principal (keytab doesn't contain it)	CREATE rejected by Trino in ~1.5 s, saga rolls back, zero residue, error shown in the form
Kerberos enabled but no keytab provided	same fail-fast + rollback
Invalid base64 in keytab field	400 before anything reaches Trino
Metastore unreachable	TCP pre-flight probe reports it before submit
Valid keytab + principals	catalog live, schemas of the kerberized metastore browsable in the row drill-down

E2E coverage (tests/e2e/playwright/test_e2e_catalogs_page.py): test_26 creates a Hive+Kerberos catalog through the UI with the REAL demo keytab and proves the handshake via the schema drill-down; test_27 proves the bad-principal fail-fast + rollback path. The keytab is pulled from the cluster Secret at run time — never committed.

See demos/cloudera-simulation/bootstrap-akko-catalog.sh for the scripted API variant.

Security model¶

Writes require the akko-admin realm role (JWT verified against Keycloak JWKS with OIDC discovery).
JDBC passwords go to a credential file referenced by credential-provider.type=FILE — never into catalog properties, never into the (fully logged) CREATE CATALOG statement.
Trino-side, the service identity svc-catalog-manager is a machine identity with catalog_admin: true ONLY: it can manage catalog lifecycle but cannot read any table (NIST AC-6). It is seeded through akko-init.opaSeedPolicies so the Keycloak→OPA sync never drops it.
Audit: every operation emits correlated structured JSON with secrets redacted.

Troubleshooting¶

Symptom	Cause	Fix
422 `trino_rejected` `Access Denied: Cannot create catalog`	OPA lost the `svc-catalog-manager` seed	check `akko-init.opaSeedPolicies` is set, re-run the opa-sync Job
500 `Error loading catalogs on worker`	credential file not readable on workers	PVC must be mounted RO on workers; files 0644
coordinator CrashLoop after config change	`catalog.config-dir` / `catalog.store.file.path` set	NEITHER is valid on Trino 481 — the file store dir is always `/etc/trino/catalog` (launcher symlink)
deleted platform catalog missing	expected — restore it	banner → Restaurer depuis le baseline or `POST /api/catalogs/restore`