GitLab

We are splitting our database into Main and CI

GitLab.com split CI tables into their own Postgres database to relieve its main database.

GitHub did something similar with schema domains and separate MySQL clusters. See all our Ruby on Rails scalability stories.

Published
2022-06-02
Scale
Not stated in source
Techniques
vertical partitioning decomposition
Stack
PostgreSQL Ruby on Rails

Try this stack yourself

  • Railway · Deploy something like this on Railway: spin up PostgreSQL and Ruby on Rails plus your app in a few clicks. New accounts get $20 in credit (applied when you pick a paid plan).
    Railway isn't part of GitLab's setup in this story. It's just an easy place to try the same building blocks.

Some links above are affiliate links: if you sign up, How They Scaled may earn a commission at no extra cost to you. We only list tools that fit the stack in the story.

See also

GitLab architectureClickHouse vs Postgres for analytics at scalePostgreSQL vs MySQL at scaleSharding vs partitioning

The best scaling write-ups, weekly

One email a week. No spam.

Open subscribe page

Related stories

GitLab · 2021

Why we spent the last month eliminating PostgreSQL subtransactions

GitLab traced replica stalls to Postgres subtransaction overflow and removed subtransactions from its code.

query optimizationread replicasPostgreSQLRuby on Rails
Figma · 2023

The growing pains of database architecture

How Figma moved from one Postgres instance to multiple databases by vertically partitioning table groups.

vertical partitioningread replicasconnection poolingPostgreSQLAWS RDS
GitHub · 2021

Partitioning GitHub's relational databases to handle scale

GitHub defined schema domains and linted cross-domain queries so it could move table groups onto separate MySQL clusters.

vertical partitioningschema domainsonline migrationMySQLRuby on RailsVitess
PlanetScale · 2026

Designing a planned switchover for minimum disruption

When a Neki shard's primary has to step aside for a pod replacement, kernel upgrade or cell move, PlanetScale does a planned switchover instead of stopping Postgres the way Patroni does, so a failed switchover can quickly fall back to the old primary. Admin, Neki's control plane, first validates shard state, the target replica, logical failover slots and Postgres settings. Routers then proactively buffer primary-bound queries, the primary defaults new transactions to read-only, and the other replicas are disconnected to pin a final WAL position the target must replay. If the target can't catch up before a deadline, a config reload makes the old primary writable again. Buffered queries are replanned against the new primary, so apps see added latency instead of errors, and every acknowledged commit is on the target when it is promoted.

one primary and multiple replicas per shardsynchronous replication ANY 1 across eligible replicasplanned switchoverquery bufferingsynchronous replicationPostgreSQLNekiKubernetes