Air-Gapped Deployment

This guide explains how to deploy AKKO in an air-gapped (disconnected) environment with no internet access. All container images, Helm charts, and LLM models must be pre-staged in a private registry and local storage.

---

Prerequisites

Before starting, ensure you have:

Requirement	Description
Private container registry	Harbor, Nexus, GitLab Registry, or any OCI-compliant registry accessible from your air-gapped cluster
Helm chart archive	The AKKO Helm chart packaged as a .tgz file
Internet-connected staging machine	A workstation with Docker and Helm to pull and package everything
Transfer medium	USB drive, DVD, or a one-way data diode to move artifacts into the air-gapped network
Kubernetes cluster	A running cluster (k3s, kubeadm, RKE2, OpenShift) in the isolated network

---

1. Bundle Container Images

AKKO uses 12 custom images and 15+ community images. All must be saved on the staging machine and loaded into the private registry.

Custom Images (built from source)

Build all custom images on the staging machine:

cd /path/to/AKKO

# Build all custom images
bash helm/scripts/build-images.sh

This produces the following images:

Image	Tag	Source
akko-postgres	2026.04	docker/postgres/Dockerfile
akko-spark	2026.04	docker/spark/Dockerfile
akko-notebook	2026.04	docker/jupyterhub/Dockerfile.notebook
akko-cockpit	2026.04	branding/cockpit/Dockerfile
akko-trino	2026.04	docker/trino-ai-functions/Dockerfile
akko-ai-service	2026.04	docker/ai-service/Dockerfile
akko-mlflow	2026.04	docker/mlflow/Dockerfile
akko-airflow	2026.04	docker/airflow/Dockerfile
akko-dbt	2026.04	docker/dbt/Dockerfile
akko-mcp-trino	2026.04	docker/mcp-trino/Dockerfile
akko-mcp-openmetadata	2026.04	docker/mcp-openmetadata/Dockerfile

Community Images

Pull all community images used by the Helm chart:

# Core services
docker pull quay.io/keycloak/keycloak:26.2.4
docker pull apache/polaris:1.3.0
docker pull trinodb/trino:473
docker pull apache/airflow:3.0.1
docker pull apache/superset:4.1.2
docker pull ollama/ollama:0.6.2
docker pull ghcr.io/berriai/litellm:main-v1.65.4
docker pull openpolicyagent/opa:1.4.2
docker pull lldap/lldap:v0.6.1-alpine
docker pull docker.getcollate.io/openmetadata/server:1.6.5

# Infrastructure
docker pull traefik:v3.3.6
docker pull minio/minio:RELEASE.2025-02-18T16-25-55Z
docker pull minio/mc:RELEASE.2025-02-15T09-18-34Z
docker pull nginxinc/nginx-unprivileged:1.27.4-alpine
docker pull curlimages/curl:8.12.1

# Monitoring
docker pull quay.io/prometheus/prometheus:v3.2.1
docker pull grafana/grafana:11.5.2
docker pull grafana/loki:3.4.3
docker pull grafana/promtail:3.4.3
docker pull quay.io/prometheus/alertmanager:v0.28.1

# JupyterHub
docker pull quay.io/jupyterhub/k8s-hub:4.1.0

# OpenSearch (for OpenMetadata)
docker pull opensearchproject/opensearch:2.19.1

Version accuracy

The versions listed above are current as of 2026.04. Always cross-reference with helm/akko/values.yaml for the exact tags used in your AKKO version.

Save All Images to a Tarball

# Save all images into a single compressed archive
docker save \
  akko-postgres:2026.04 \
  akko-spark:2026.04 \
  akko-notebook:2026.04 \
  akko-jupyterhub:2026.04 \
  akko-cockpit:2026.04 \
  akko-airflow:2026.04 \
  akko-dbt:2026.04 \
  akko-mcp-trino:2026.04 \
  quay.io/keycloak/keycloak:26.2.4 \
  apache/polaris:1.3.0 \
  trinodb/trino:473 \
  apache/airflow:3.0.1 \
  apache/superset:4.1.2 \
  ollama/ollama:0.6.2 \
  ghcr.io/berriai/litellm:main-v1.65.4 \
  openpolicyagent/opa:1.4.2 \
  lldap/lldap:v0.6.1-alpine \
  docker.getcollate.io/openmetadata/server:1.6.5 \
  traefik:v3.3.6 \
  minio/minio:RELEASE.2025-02-18T16-25-55Z \
  minio/mc:RELEASE.2025-02-15T09-18-34Z \
  nginxinc/nginx-unprivileged:1.27.4-alpine \
  curlimages/curl:8.12.1 \
  quay.io/prometheus/prometheus:v3.2.1 \
  grafana/grafana:11.5.2 \
  grafana/loki:3.4.3 \
  grafana/promtail:3.4.3 \
  quay.io/prometheus/alertmanager:v0.28.1 \
  quay.io/jupyterhub/k8s-hub:4.1.0 \
  opensearchproject/opensearch:2.19.1 \
  | gzip > akko-images.tar.gz

The resulting archive is typically 15-20 GB depending on enabled services.

Transfer and Load Images

On the air-gapped network:

# Load all images from the archive
docker load < akko-images.tar.gz

# Tag and push each image to your private registry
REGISTRY="registry.internal.corp.com/akko"

# Custom images
for IMG in akko-postgres akko-spark akko-notebook akko-jupyterhub \
           akko-cockpit akko-airflow akko-dbt akko-mcp-trino; do
  docker tag ${IMG}:2026.04 ${REGISTRY}/${IMG}:2026.04
  docker push ${REGISTRY}/${IMG}:2026.04
done

# Community images — re-tag under your registry namespace
docker tag quay.io/keycloak/keycloak:26.2.4 ${REGISTRY}/keycloak:26.2.4
docker push ${REGISTRY}/keycloak:26.2.4

docker tag trinodb/trino:473 ${REGISTRY}/trino:473
docker push ${REGISTRY}/trino:473

# ... repeat for all community images

Scripting the re-tag

For production, create a script that reads image references from helm/akko/values.yaml and automates the tag-and-push process. This prevents human error and ensures no image is missed.

---

2. Package the Helm Chart

On the internet-connected staging machine:

cd /path/to/AKKO/helm/akko

# Update all sub-chart dependencies
helm dependency update .

# Package the umbrella chart into a .tgz
helm package . -d /tmp/akko-charts/
# Output: /tmp/akko-charts/akko-0.1.0.tgz (version from Chart.yaml)

Transfer the .tgz file to the air-gapped network.

On the air-gapped network, you can either:

• Push to a chart museum (e.g., Harbor's built-in chart repository):

  helm push /tmp/akko-charts/akko-0.1.0.tgz oci://registry.internal.corp.com/akko-charts
  

• Install directly from the .tgz:

  helm install akko /tmp/akko-charts/akko-0.1.0.tgz -n akko --create-namespace ...
  

---

3. Configure Image References

Create a values overlay that points all images to your private registry:

# values-airgapped.yaml
global:
  image:
    registry: "registry.internal.corp.com/akko/"
    pullPolicy: IfNotPresent
    pullSecrets:
      - name: registry-credentials

# --- Custom images ---
akko-postgres:
  image:
    repository: registry.internal.corp.com/akko/akko-postgres
    tag: "2026.04"

akko-postgres-data:
  image:
    repository: registry.internal.corp.com/akko/akko-postgres
    tag: "2026.04"

akko-keycloak:
  image:
    repository: registry.internal.corp.com/akko/keycloak
    tag: "26.2.4"

akko-polaris:
  image:
    repository: registry.internal.corp.com/akko/polaris
    tag: "1.3.0"

akko-spark:
  image:
    repository: registry.internal.corp.com/akko/akko-spark
    tag: "2026.04"

akko-cockpit:
  image:
    repository: registry.internal.corp.com/akko/akko-cockpit
    tag: "2026.04"

akko-ollama:
  image:
    repository: registry.internal.corp.com/akko/ollama
    tag: "0.6.2"

akko-litellm:
  image:
    repository: registry.internal.corp.com/akko/litellm
    tag: "main-v1.65.4"

akko-mlflow:
  image:
    repository: registry.internal.corp.com/akko/akko-mlflow
    tag: "2026.04"

akko-opa:
  image:
    repository: registry.internal.corp.com/akko/opa
    tag: "1.4.2"

akko-lldap:
  image:
    repository: registry.internal.corp.com/akko/lldap
    tag: "v0.6.1-alpine"

akko-docs:
  image:
    repository: registry.internal.corp.com/akko/nginx-unprivileged
    tag: "1.27.4-alpine"

jupyterhub:
  hub:
    image:
      name: registry.internal.corp.com/akko/k8s-hub
      tag: "4.1.0"
  singleuser:
    image:
      name: registry.internal.corp.com/akko/akko-notebook
      tag: "2026.04"

trino:
  image:
    repository: registry.internal.corp.com/akko/trino
    tag: "473"

airflow:
  defaultAirflowRepository: registry.internal.corp.com/akko/airflow
  defaultAirflowTag: "3.0.1"

superset:
  image:
    repository: registry.internal.corp.com/akko/superset
    tag: "4.1.2"

minio:
  image:
    repository: registry.internal.corp.com/akko/minio
    tag: "RELEASE.2025-02-18T16-25-55Z"
  mcImage:
    repository: registry.internal.corp.com/akko/mc
    tag: "RELEASE.2025-02-15T09-18-34Z"

Create the registry pull secret on the air-gapped cluster:

kubectl create namespace akko

kubectl create secret docker-registry registry-credentials \
  --docker-server=registry.internal.corp.com \
  --docker-username=<USERNAME> \
  --docker-password=<PASSWORD> \
  -n akko

---

4. Pre-Stage Ollama Models

Ollama models cannot be pulled from the internet in an air-gapped environment. You must pre-download them and load them via a PersistentVolumeClaim.

On the staging machine (internet-connected)

# Pull models on a machine with Ollama installed
ollama pull qwen2.5-coder:7b
ollama pull qwen2.5:3b
ollama pull nomic-embed-text

# The model files are stored in ~/.ollama/models/
# Archive them
tar czf ollama-models.tar.gz -C ~/.ollama models/

On the air-gapped cluster

1. Create a PVC for Ollama:

   kubectl apply -f - <<EOF
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: akko-ollama-data
     namespace: akko
   spec:
     accessModes: [ReadWriteOnce]
     resources:
       requests:
         storage: 20Gi
   EOF
   

2. Copy models into the PVC using a temporary pod:

   # Create a temporary pod that mounts the PVC
   kubectl run ollama-loader --image=busybox:1.36 -n akko \
     --overrides='{
       "spec": {
         "containers": [{
           "name": "loader",
           "image": "busybox:1.36",
           "command": ["sleep", "3600"],
           "volumeMounts": [{
             "name": "data",
             "mountPath": "/ollama"
           }]
         }],
         "volumes": [{
           "name": "data",
           "persistentVolumeClaim": {
             "claimName": "akko-ollama-data"
           }
         }]
       }
     }'
   
   # Wait for the pod to be ready
   kubectl wait --for=condition=Ready pod/ollama-loader -n akko --timeout=60s
   
   # Copy the models archive and extract
   kubectl cp ollama-models.tar.gz akko/ollama-loader:/tmp/
   kubectl exec -n akko ollama-loader -- tar xzf /tmp/ollama-models.tar.gz -C /ollama/
   
   # Cleanup
   kubectl delete pod ollama-loader -n akko
   

3. Disable automatic model pull in your values:

   # values-airgapped.yaml (add to existing)
   akko-ollama:
     modelPull:
       enabled: false     # Models are pre-staged in the PVC
     persistence:
       existingClaim: "akko-ollama-data"
   

---

5. Deploy

With all artifacts staged, deploy AKKO:

# Generate domain values (if not already done)
bash helm/scripts/generate-domain-values.sh akko.internal.corp.com
bash helm/scripts/generate-dev-secrets.sh

# Install
helm upgrade --install akko /tmp/akko-charts/akko-0.1.0.tgz \
  -n akko --create-namespace \
  --wait --wait-for-jobs --timeout 20m \
  -f helm/examples/values-dev.yaml \
  -f helm/examples/values-domain.yaml \
  -f values-airgapped.yaml \
  -f helm/examples/values-dev-secrets.yaml \
  --set-file akko-keycloak.realm.data=helm/examples/realm-domain.json

---

6. Verification Checklist

After deployment, verify:

• [ ] All pods are in Running or Completed state: kubectl get pods -n akko
• [ ] No ImagePullBackOff errors (would indicate a missing image in the registry)
• [ ] Cockpit loads at https://akko.internal.corp.com
• [ ] Keycloak login works at https://keycloak.internal.corp.com
• [ ] Ollama responds: kubectl exec -n akko deploy/akko-ollama -- ollama list
• [ ] JupyterHub spawns notebooks without pulling images from the internet
• [ ] Trino queries execute successfully (test with a simple SELECT 1)

---

Updating in an Air-Gapped Environment

To update AKKO:

1. On the staging machine, pull the new version and rebuild images
2. Save the delta images (only new or changed images)
3. Transfer and load into the private registry
4. Package the updated Helm chart
5. Run helm upgrade with the new chart and values

# On the staging machine
git pull origin main
bash helm/scripts/build-images.sh
helm dependency update helm/akko/
helm package helm/akko/ -d /tmp/akko-charts/

# On the air-gapped cluster (after transfer)
helm upgrade akko /tmp/akko-charts/akko-<NEW_VERSION>.tgz \
  -n akko \
  --wait --wait-for-jobs --timeout 20m \
  -f helm/examples/values-dev.yaml \
  -f helm/examples/values-domain.yaml \
  -f values-airgapped.yaml \
  -f helm/examples/values-dev-secrets.yaml \
  --set-file akko-keycloak.realm.data=helm/examples/realm-domain.json

---

Further Reading

• Enterprise Deployment Guide -- full production setup
• External Identity Provider -- connect corporate IdP
• Configuration Reference -- all Helm values
• Troubleshooting -- common issues and fixes