ClickHouse

ClickHouse is an open-source column-oriented database management system. It can efficiently filter, aggregate, and query across large data sets.

GitLab uses ClickHouse as a secondary data store to enable advanced analytics features such as GitLab Duo, SDLC trends, and CI Analytics. GitLab only stores data that supports these features in ClickHouse.

You should use ClickHouse Cloud to connect ClickHouse to GitLab.

Alternatively, you can bring your own ClickHouse. For more information, see ClickHouse recommendations for GitLab Self-Managed.

Analytics available with ClickHouse

After you configure ClickHouse, you can use the following analytics features:

Supported ClickHouse versions

The supported ClickHouse version differs depending on your GitLab version:

• GitLab 17.7 and later supports ClickHouse 23.x. To use either ClickHouse 24.x or 25.x, use the workaround.
• GitLab 18.1 and later supports ClickHouse 23.x, 24.x, and 25.x.
• GitLab 18.8 and later supports ClickHouse 23.x, 24.x, 25.x, and the Replicated database engine.
  • Older clusters will require an additional permission (dictGet), see the snippet.
• GitLab 19.0 and later supports ClickHouse 25.x and 26.x up to and including 26.6. Support for ClickHouse 23.x and 24.x has been removed.
• ClickHouse 26.7 and later is not supported. For a temporary workaround, see ClickHouse 26.7 and later.

ClickHouse Cloud is always compatible with the latest stable GitLab release.

Set up ClickHouse

Choose your deployment type based on your operational requirements:

• ClickHouse Cloud (Recommended): Fully managed service with automatic upgrades, backups, and scaling.
• ClickHouse for GitLab Self-Managed (BYOC): Complete control over your infrastructure and configuration.

After setting up your ClickHouse instance:

1. Create the GitLab database and user.
2. Configure the GitLab connection.
3. Verify the connection.
4. Run ClickHouse migrations.
5. Enable ClickHouse for Analytics.

Set up ClickHouse Cloud

Prerequisites:

• Have a ClickHouse Cloud account.
• Enable network connectivity from your GitLab instance to ClickHouse Cloud.
• Be an administrator of your GitLab instance.

To set up ClickHouse Cloud:

1. Sign in to ClickHouse Cloud.
2. Select New Service.
3. Choose your service tier:
   • Development: For testing and development environments.
   • Production: For production workloads with high availability.
4. Select your cloud provider and region. Choose a region close to your GitLab instance for optimal performance.
5. Configure your service name and settings.
6. Select Create Service.
7. Once provisioned, note your connection details from the service dashboard:
   • Host
   • Port (8443 for HTTPS connections used by GitLab, or 9440 for native TCP with TLS used by clickhouse-client)
   • Username
   • Password

After you create your ClickHouse Cloud service, you then create the GitLab database and user.

Set up ClickHouse for GitLab Self-Managed (BYOC)

Prerequisites:

• Have a ClickHouse instance installed and running. If ClickHouse is not installed, see:
  • ClickHouse official installation guide.
  • ClickHouse recommendations for GitLab Self-Managed.
• Have a supported ClickHouse version.
• Enable network connectivity from your GitLab instance to ClickHouse.
• Be an Administrator for both ClickHouse and your GitLab instance.

Configure High Availability

For a multi-node, high-availability (HA) setup, GitLab supports the Replicated table engine in ClickHouse.

Prerequisites:

• Have a ClickHouse cluster with multiple nodes. A minimum of three nodes is recommended.
• Define a cluster in the remote_servers configuration section.
• Configure the following macros in your ClickHouse configuration:
  • cluster
  • shard
  • replica

When configuring the database for HA, you must run the statements with the ON CLUSTER clause.

For more information, see ClickHouse Replicated database engine documentation.

Configure Load balancer

The GitLab application communicates with the ClickHouse cluster through the HTTP/HTTPS interface. For HA deployments, use an HTTP proxy or load balancer to distribute requests across ClickHouse cluster nodes.

Recommended load balancer options:

• chproxy - ClickHouse-specific HTTP proxy with built-in caching and routing.
• HAProxy - General-purpose TCP/HTTP load balancer.
• NGINX - Web server with load balancing capabilities.
• Cloud provider load balancers (AWS Application Load Balancer, GCP Load Balancer, Azure Load Balancer).

Basic chproxy configuration example:

server:
  http:
    listen_addr: ":8080"

clusters:
  - name: "clickhouse_cluster"
    nodes: [
      "http://ch-node1:8123",
      "http://ch-node2:8123",
      "http://ch-node3:8123"
    ]

users:
  - name: "gitlab"
    password: "your_secure_password"
    to_cluster: "clickhouse_cluster"
    to_user: "gitlab"

When using a load balancer, configure GitLab to connect to the load balancer URL instead of individual ClickHouse nodes.

For more information, see chproxy documentation.

After you configure your ClickHouse for GitLab Self-Managed instance, create the GitLab database and user.

Verify ClickHouse installation

Before configuring the database, verify ClickHouse is installed and accessible:

1. Check ClickHouse is running:

   clickhouse-client --query "SELECT version()"

   If ClickHouse is running, you see the version number (for example, 24.3.1.12).

2. Verify you can connect with credentials:

   clickhouse-client --host your-clickhouse-host --port 9440 --secure --user default --password 'your-password'

   If you have not configured TLS yet, use port 9000 without the --secure flag for initial testing.

Create database and user

To create the necessary user and database objects:

1. Generate a secure password and save it.
2. Sign in to:
   • For ClickHouse Cloud, the ClickHouse SQL console.
   • For ClickHouse for GitLab Self-Managed, the clickhouse-client.
3. Run the following commands, replacing PASSWORD_HERE with the generated password.

CREATE DATABASE gitlab_clickhouse_main_production;
CREATE USER gitlab IDENTIFIED WITH sha256_password BY 'PASSWORD_HERE';
CREATE ROLE gitlab_app;
GRANT SELECT, INSERT, ALTER, CREATE, UPDATE, DROP, TRUNCATE, OPTIMIZE, dictGet ON gitlab_clickhouse_main_production.* TO gitlab_app;
GRANT SELECT ON information_schema.* TO gitlab_app;
GRANT gitlab_app TO gitlab;

CREATE DATABASE gitlab_clickhouse_main_production ON CLUSTER CLUSTER_NAME_HERE ENGINE = Replicated('/clickhouse/databases/{cluster}/gitlab_clickhouse_main_production', '{shard}', '{replica}');
CREATE USER gitlab IDENTIFIED WITH sha256_password BY 'PASSWORD_HERE' ON CLUSTER CLUSTER_NAME_HERE;
CREATE ROLE gitlab_app ON CLUSTER CLUSTER_NAME_HERE;
GRANT SELECT, INSERT, ALTER, CREATE, UPDATE, DROP, TRUNCATE, OPTIMIZE, dictGet ON gitlab_clickhouse_main_production.* TO gitlab_app ON CLUSTER CLUSTER_NAME_HERE;
GRANT SELECT ON information_schema.* TO gitlab_app ON CLUSTER CLUSTER_NAME_HERE;
GRANT gitlab_app TO gitlab ON CLUSTER CLUSTER_NAME_HERE;

Configure the GitLab connection