GitHub

Upgrading GitHub.com to MySQL 8.0

How GitHub upgraded its whole MySQL fleet to 8.0 with staged replicas and a working rollback path.

Published
2023-12-07
Scale
5.5 million QPS, 300+ TB, 50+ database clusters
Techniques
fleet upgrade replication rollback planning
Stack
MySQL

Try this stack yourself

  • Railway · Deploy something like this on Railway: spin up MySQL plus your app in a few clicks. New accounts get $20 in credit (applied when you pick a paid plan).
    Railway isn't part of GitHub's setup in this story. It's just an easy place to try the same building blocks.
  • Better Stack · Monitor uptime and logs with Better Stack, and get alerted fast if a migration or traffic spike goes sideways on your own system.
    Better Stack isn't part of GitHub's setup in this story. It's a tool for watching your own systems while you try ideas like these.

Some links above are affiliate links: if you sign up, How They Scaled may earn a commission at no extra cost to you. We only list tools that fit the stack in the story.

See also

GitHub architectureRedis vs MySQL at scalePostgreSQL vs MySQL at scale

The best scaling write-ups, weekly

One email a week. No spam.

Open subscribe page

Related stories

GitHub · 2021

Partitioning GitHub's relational databases to handle scale

GitHub defined schema domains and linted cross-domain queries so it could move table groups onto separate MySQL clusters.

vertical partitioningschema domainsonline migrationMySQLRuby on RailsVitess
Cloudflare · 2026

Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug

Hyperdrive, Cloudflare's distributed connection pooler for Workers, keeps warm database connections on its network and can cache common query results in the CDN cache. Taking its MySQL support from beta to GA meant finding bugs like utf8 being a 3-byte alias (fixed with utf8mb4) and COM_RESET_CONNECTION silently resetting the charset to the server default. The hardest was an empty-result bug seen by the highest-volume MySQL customer in about 0.001% of queries, only with caching on. Each Hyperdrive instance has a Client node next to the Worker and an Endpoint node near the database, and they share a cache when they run in the same data center. Because drivers like Node's mysql2 don't support CLIENT_DEPRECATE_EOF, the Client rewrites results into the Worker's capabilities, so the two nodes were caching different bytes for the same query and could read each other's entries. Caching one canonical encoding no matter which node stores it fixed the bug, and Client connection success reached 99.99%.

99.99% production availability targetbug hit ~0.001% of queriesconnection poolingcachingprotocol translationMySQLWorkersHyperdrive
Canva · 2026

Session revocations at scale

Canva keeps 12 hours of session revocations in memory on every gateway, but hundreds of pods each loading over a million revocations from MySQL on deploy caused a stampede on the database. A worker now packs revocations into sorted 16-byte records in 30-minute chunks on S3, using conditional PUTs for optimistic concurrency and ZooKeeper leader election to avoid conflicts, and reaches over 2,000 revocations per second. Gateways binary-search the downloaded bytes directly and poll with conditional GETs, which shrank the cache by 87.5% and cut the revocation database down to two read replicas kept for redundancy.

hundreds of millions of usershundreds of thousands of session checks/seccachingobject storageoptimistic concurrency controlMySQLAmazon S3ZooKeeper
Shopify · 2026

We replaced Redis with MySQL for inventory reservations—and it scaled

Shopify moved checkout inventory reservations from Redis to MySQL using one row per sellable unit, a bounded pool of 1,000 rows per item/location and SELECT ... FOR UPDATE SKIP LOCKED. The real ceiling was connections held by other checkout code: tagging SQL by business process and measuring hold time in ProxySQL found it, and the cleanup removed 50% of reads and 33% of transactions on the primary.

$5.1 million sales/minute (Black Friday 2025 peak)database migrationdual writesquery optimizationMySQLRedisProxySQL