OpenSearch¶

Overview¶

OpenSearch provides full-text search and indexing for AKKO, primarily serving as the search backend for OpenMetadata. It indexes metadata entities (tables, pipelines, dashboards, users, tags) and powers the search bar, lineage graph queries, and data discovery features of the governance layer.

Architecture¶

  OpenMetadata Server ──→ OpenSearch (:9200)
       │                    (search index)
       │
       ▼
  PostgreSQL (metadata store)

OpenMetadata writes entity metadata to PostgreSQL for persistence and simultaneously indexes it into OpenSearch for fast full-text search. OpenSearch is a read-heavy component — all writes come from OpenMetadata's indexing pipeline.

Ports¶

Port	Purpose	Exposed
9200	REST API — search queries and index management	Internal only
9300	Node-to-node transport (unused in single-node mode)	Internal only

Internal Only

OpenSearch is never exposed outside the cluster. Only OpenMetadata communicates with it directly.

Configuration¶

Single-Node Mode¶

In development and small deployments, OpenSearch runs as a single-node cluster, which avoids the overhead of cluster coordination:

discovery.type: single-node

Security Plugin¶

The security plugin is disabled in development to simplify the setup. In production, enable it and configure TLS + authentication:

plugins.security.disabled: true    # dev only

Production Security

In production environments, enable the OpenSearch security plugin with TLS certificates and internal user authentication. The dev configuration with plugins.security.disabled: true is not suitable for production.

Memory¶

OpenSearch requires significant memory for indexing and search operations:

OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"

The default 512MB heap is sufficient for development. For production with large metadata catalogs (10,000+ entities), increase to 1-2GB.

Helm Chart¶

OpenSearch is deployed via the akko-opensearch custom sub-chart:

helm/akko/charts/akko-opensearch/
├── Chart.yaml
├── values.yaml
└── templates/
    ├── statefulset.yaml
    ├── service.yaml
    └── configmap.yaml

Key Values¶

values.yaml

akko-opensearch:
  image:
    repository: opensearchproject/opensearch
    tag: "2.19.1"
  replicas: 1
  config:
    discovery.type: single-node
    plugins.security.disabled: "true"
  resources:
    requests:
      memory: 512Mi
    limits:
      memory: 1Gi
  persistence:
    enabled: true
    size: 5Gi

Integration with OpenMetadata¶

OpenMetadata requires OpenSearch for its search functionality. The connection is configured in OpenMetadata's values:

openmetadata:
  config:
    elasticsearch:
      host: akko-opensearch
      port: 9200
      scheme: http

Elasticsearch-Compatible API

OpenMetadata uses the Elasticsearch client library, but OpenSearch maintains API compatibility. The configuration key is named elasticsearch even though the backend is OpenSearch.

Health Check¶

OpenSearch exposes a cluster health endpoint:

GET http://akko-opensearch:9200/_cluster/health

Returns a JSON response with status: "green" (or "yellow" in single-node mode, which is normal).

Troubleshooting¶

Common Issues

OpenMetadata search returns no results: Check that OpenSearch is running and the index exists. Run kubectl exec deploy/akko-opensearch -- curl -s localhost:9200/_cat/indices to list indices. If empty, trigger a reindex from OpenMetadata's admin UI (Settings > Event Publishers > Elasticsearch).
Pod OOMKilled: OpenSearch is memory-hungry. If the pod is killed with OOMKilled, increase the memory limit in values.yaml and ensure OPENSEARCH_JAVA_OPTS heap size matches (heap should be ~50% of the container memory limit).
Slow startup (>60s): OpenSearch performs index recovery on startup. If the readiness probe timeout is too short, the pod may restart in a loop. Increase initialDelaySeconds to 90s for the readiness probe.