Caching: 10 real scaling case studies
10 of the 94 engineering write-ups in our database are tagged with the technique "caching". They come from 9 companies (Cloudflare, Canva, Discord, Figma, GitHub and 4 more) and were published between 2013 and 2026. Stack that shows up most often in these stories: MySQL (3), Java (2), Kubernetes (2) and PostgreSQL (2). Often combined with connection pooling (2), read replicas (2) and workload isolation (2). 7 of the 10 state concrete scale numbers, for example "R2 p99 time to first byte roughly halved (hits and misses)" (Cloudflare). Each card below links to our short summary and the original post.
Origin Range Requests: Serving large files without range-request tuning
Cloudflare's old range-request caching sliced files into a per-zone configured range size, so a small video seek could pull 128 MiB from origin for R2, while small ranges caused floods of tiny origin requests and memory pressure, and concurrent requests in the same range blocked each other. Moving from nginx to its Pingora-based cache, it reused Pingora's fixed-size on-disk chunks as the unit: a coordinator serves cached bytes, attaches to in-flight downloads, fetches only missing chunk-aligned ranges from origin, and validates size and ETag before composing one 206 response. For R2, p99 time to first byte dropped by about half, and letting one response combine disk and upstream bytes cut tiered cache traffic about 6% week over week.
Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug
Hyperdrive, Cloudflare's distributed connection pooler for Workers, keeps warm database connections on its network and can cache common query results in the CDN cache. Taking its MySQL support from beta to GA meant finding bugs like utf8 being a 3-byte alias (fixed with utf8mb4) and COM_RESET_CONNECTION silently resetting the charset to the server default. The hardest was an empty-result bug seen by the highest-volume MySQL customer in about 0.001% of queries, only with caching on. Each Hyperdrive instance has a Client node next to the Worker and an Endpoint node near the database, and they share a cache when they run in the same data center. Because drivers like Node's mysql2 don't support CLIENT_DEPRECATE_EOF, the Client rewrites results into the Worker's capabilities, so the two nodes were caching different bytes for the same query and could read each other's entries. Caching one canonical encoding no matter which node stores it fixed the bug, and Client connection success reached 99.99%.
Building Git infrastructure for agent-scale development
GitHub explains why agent workloads strain Spokes, which keeps five durable copies of each repo on fileserver disks, so every added read replica slows writes. The new design coordinates only the ref update, moves compaction and GC to separate workers, and separates storage from compute: Azure Blob Storage holds the authoritative data while lightweight caching workers serve reads.
Session revocations at scale
Canva keeps 12 hours of session revocations in memory on every gateway, but hundreds of pods each loading over a million revocations from MySQL on deploy caused a stampede on the database. A worker now packs revocations into sorted 16-byte records in 30-minute chunks on S3, using conditional PUTs for optimistic concurrency and ZooKeeper leader election to avoid conflicts, and reaches over 2,000 revocations per second. Gateways binary-search the downloaded bytes directly and poll with conditional GETs, which shrank the cache by 87.5% and cut the revocation database down to two read replicas kept for redundancy.
Client-Side Load Balancing at a Million Requests Per Second
Zalando moved the internal fan-out of its Product Read API off the shared Skipper ingress into an in-process JVM load balancer that rebuilds the same xxHash64 consistent-hash ring. They added N-ring fade-in so new pods warm only the products they will keep, bounded load based on occupancy (Little's Law) weighted by latency, and a capped ring walk. Pods ran more evenly, the autoscaler target rose from 50% to 65% CPU, and the fleet shrank by over 25%.
Scaling PostgreSQL to power 800 million ChatGPT users
OpenAI runs ChatGPT on a single-primary Postgres with nearly 50 global read replicas. Reads are offloaded, write-heavy shardable workloads move to Cosmos DB, and PgBouncer, cache leasing and multi-layer rate limits keep the primary safe.
LiveGraph: real-time data fetching at Figma
Figma built LiveGraph, which tails the Postgres replication log so its UI can subscribe to live query results.
Why Discord is switching from Go to Rust
Discord rewrote its Read States service from Go to Rust to get rid of garbage-collection latency spikes in a large in-memory cache.
Stack Overflow: The Architecture - 2016 Edition
A tour of how Stack Overflow serves a huge site from a small number of powerful servers with heavy caching.
TAO: The power of the graph
TAO is Facebook's geo-distributed graph cache over sharded MySQL, serving the social graph at more than a billion reads per second.