Cloudflare architecture: how Cloudflare scaled (7 case studies)
The Cloudflare write-ups we track cover five systems: Pingora, a Rust proxy that replaced Nginx (the post cites 1 trillion requests a day); HTTP analytics rebuilt on ClickHouse in place of a Postgres/Citus setup (6M requests/sec); error-log analytics on ClickHouse; Quicksilver, which replicates Cloudflare's configuration key-value store to every edge server in seconds on LMDB; and D1, which runs SQLite queries at the edge while a separate control plane manages millions of databases.
We track 7 public engineering write-ups by Cloudflare, published between 2018 and 2026. Techniques that come up more than once: caching (2) and columnar storage (2). Stack mentioned most: ClickHouse (2), Go (2) and Workers (2). 6 of the 7 state concrete scale numbers, for example "R2 p99 time to first byte roughly halved (hits and misses)". Each card below links to our short summary and the original post.
Tech used in Cloudflare's scaling write-ups
Cloudflare scaling timeline: every story, summarized
- 2018-03-06
HTTP Analytics for 6M requests per second using ClickHouse
Cloudflare rebuilt its HTTP analytics pipeline on ClickHouse, replacing a Postgres/Citus setup.
- 2020-03-30
Introducing Quicksilver: Configuration Distribution at Internet Scale
Quicksilver replicates Cloudflare's config key-value store to every edge server in seconds, built on LMDB.
- 2022-09-02
Log analytics using ClickHouse
Cloudflare moved its error-log pipeline to ClickHouse and shares schema and query lessons.
- 2022-09-14
How we built Pingora, the proxy that connects Cloudflare to the Internet
Cloudflare replaced Nginx with Pingora, a Rust proxy that reuses connections better and needs less CPU and memory.
- 2026-10-13
Under the hood, off the hot path: how D1 scales to millions of databases
How Cloudflare's D1 keeps query execution at the edge while a separate control plane manages millions of SQLite databases.
- 2026-10-14
Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug
Hyperdrive, Cloudflare's distributed connection pooler for Workers, keeps warm database connections on its network and can cache common query results in the CDN cache. Taking its MySQL support from beta to GA meant finding bugs like utf8 being a 3-byte alias (fixed with utf8mb4) and COM_RESET_CONNECTION silently resetting the charset to the server default. The hardest was an empty-result bug seen by the highest-volume MySQL customer in about 0.001% of queries, only with caching on. Each Hyperdrive instance has a Client node next to the Worker and an Endpoint node near the database, and they share a cache when they run in the same data center. Because drivers like Node's mysql2 don't support CLIENT_DEPRECATE_EOF, the Client rewrites results into the Worker's capabilities, so the two nodes were caching different bytes for the same query and could read each other's entries. Caching one canonical encoding no matter which node stores it fixed the bug, and Client connection success reached 99.99%.
- 2026-10-23
Origin Range Requests: Serving large files without range-request tuning
Cloudflare's old range-request caching sliced files into a per-zone configured range size, so a small video seek could pull 128 MiB from origin for R2, while small ranges caused floods of tiny origin requests and memory pressure, and concurrent requests in the same range blocked each other. Moving from nginx to its Pingora-based cache, it reused Pingora's fixed-size on-disk chunks as the unit: a coordinator serves cached bytes, attaches to in-flight downloads, fetches only missing chunk-aligned ranges from origin, and validates size and ETag before composing one 206 response. For R2, p99 time to first byte dropped by about half, and letting one response combine disk and upstream bytes cut tiered cache traffic about 6% week over week.
FAQ
Why did Cloudflare replace Nginx with Pingora?
Cloudflare's 2022 post says it replaced Nginx with Pingora, a proxy written in Rust that reuses connections better and needs less CPU and memory. The post cites 1 trillion requests per day.
What database does Cloudflare use for analytics?
ClickHouse, in both analytics posts we track. In 2018 Cloudflare rebuilt its HTTP analytics pipeline on ClickHouse (with Kafka and Go), replacing a Postgres/Citus setup, at 6M requests per second. In 2022 it moved its error-log pipeline to ClickHouse too.
How does Cloudflare D1 scale to millions of databases?
Cloudflare's 2026 post explains that D1 is built on SQLite, Durable Objects and Workers. It keeps query execution at the edge while a separate control plane manages millions of databases off the hot path.