Skip to content

ClueCDC v0.1.0

ClueCDC is a self-hosted control plane for configuring, operating, and observing change-data-capture pipelines built on Debezium, Apache Kafka, and Kafka Connect. CDC records stay in the data plane; the ClueCDC API does not relay row events.

This is an early 0.x release. APIs, configuration, migrations, and operational interfaces may evolve before v1.0.0.

Highlights

  • Manage reusable connections, source discovery, capture pipelines, and multiple independent deliveries from one UI and API.
  • Operate connector pause, resume, restart, task recovery, table additions, mappings, alerts, and audit history.
  • Store credentials encrypted and resolve them inside Kafka Connect without returning plaintext through public APIs.
  • Run locally with Docker Compose or use the documented Kubernetes baseline.

Supported Sources

  • PostgreSQL 13+ through Debezium PostgreSQL
  • MySQL 8 through Debezium MySQL

Supported Destinations

  • PostgreSQL through the Debezium JDBC sink
  • MySQL through the Debezium JDBC sink
  • AWS S3 through the Aiven S3 sink (gzip JSONL CDC envelopes)
  • MinIO through the S3-compatible Aiven sink (gzip JSONL CDC envelopes)

Architecture

Source database
  -> Debezium source connector
  -> Apache Kafka
  -> Kafka Connect delivery connector
  -> PostgreSQL, MySQL, AWS S3, or MinIO

ClueCDC is the management plane around this flow. It stores desired state, configures the data-plane services, and reports observed status.

Quick Start

git clone https://github.com/cluedata/cluecdc.git
cd cluecdc
cp .env.example .env
docker compose up -d --build --wait

Open http://localhost:3000. The example credentials are development-only; run python scripts/bootstrap.py to replace infrastructure and cryptographic examples with unique local values. The documented public development Admin is preserved for local access and must never be used in production.

Docker Images

The tag-triggered workflow publishes these multi-tag GHCR images:

  • ghcr.io/cluedata/cluecdc-web:0.1.0, :0.1, and :latest
  • ghcr.io/cluedata/cluecdc-api:0.1.0, :0.1, and :latest
  • ghcr.io/cluedata/cluecdc-connect:0.1.0, :0.1, and :latest

Pin 0.1.0 or an image digest in production. Immutable version tags must not be rebuilt with different contents.

Known Limitations

  • Compose is a loopback-only development topology, not a production HA setup.
  • Authentication is static token-based; SSO/OIDC, built-in TLS, Kafka ACL provisioning, and external-secret integration are not included.
  • The Kubernetes files are a baseline and require an external Kafka deployment, ingress/TLS, secret management, backups, and production sizing.
  • Connector actions cannot make external effects and metadata commits globally atomic; reconciliation relies on stable names and idempotent retries.
  • A development-only ESLint dependency currently includes the upstream unpatched braces advisory. Production npm dependencies scan clean at the high threshold; critical findings remain blocked across the complete tree.

Upgrade Notes

Fresh installations migrate automatically. A transactional compatibility bridge upgrades metadata databases at prototype revision 5e2d8a9f1c30; back up the metadata database and encryption key first. Other prototype revisions require a fresh metadata database.

Full Changelog

See CHANGELOG.md for the complete 0.1.0 change list.