PlanetScale · 2026
Designing a planned switchover for minimum disruption
When a Neki shard's primary has to step aside for a pod replacement, kernel upgrade or cell move, PlanetScale does a planned switchover instead of stopping Postgres the way Patroni does, so a failed switchover can quickly fall back to the old primary. Admin, Neki's control plane, first validates shard state, the target replica, logical failover slots and Postgres settings. Routers then proactively buffer primary-bound queries, the primary defaults new transactions to read-only, and the other replicas are disconnected to pin a final WAL position the target must replay. If the target can't catch up before a deadline, a config reload makes the old primary writable again. Buffered queries are replanned against the new primary, so apps see added latency instead of errors, and every acknowledged commit is on the target when it is promoted.
one primary and multiple replicas per shardsynchronous replication ANY 1 across eligible replicasplanned switchoverquery bufferingsynchronous replicationPostgreSQLNekiKubernetes
Cloudflare · 2026
Origin Range Requests: Serving large files without range-request tuning
Cloudflare's old range-request caching sliced files into a per-zone configured range size, so a small video seek could pull 128 MiB from origin for R2, while small ranges caused floods of tiny origin requests and memory pressure, and concurrent requests in the same range blocked each other. Moving from nginx to its Pingora-based cache, it reused Pingora's fixed-size on-disk chunks as the unit: a coordinator serves cached bytes, attaches to in-flight downloads, fetches only missing chunk-aligned ranges from origin, and validates size and ETag before composing one 206 response. For R2, p99 time to first byte dropped by about half, and letting one response combine disk and upstream bytes cut tiered cache traffic about 6% week over week.
R2 p99 time to first byte roughly halved (hits and misses)~6% less tiered cache traffic for R2-like workloads, petabytes/daycachingrequest coalescingtiered storagePingoraR2Tiered Cache
PlanetScale · 2026
Shard design doesn't start from your tables
In Neki, PlanetScale's sharded Postgres, a shard group is the unit of co-location: tables in the same group that hash their shard key the same way (xxhash into a 64-bit keyspace split by key ranges) always land on the same shard. Sharding customers, orders, items and addresses by customer_id keeps single-customer queries and transactions on one shard, which matters because a transaction across two shards gets no atomic commit or shared snapshot. Lookup tables become reference tables copied to every shard, catalog tables get their own group keyed on lower(sku), range and modulo indexes support legacy schemes, and a global secondary index routes queries like orders by id without fanning out. Shard groups can be changed later with the Reshard workflow.
64-bit hashed keyspace split across shardscross-shard transactions commit per shard, no atomic commitshard groupsco-locationreference tablesPostgreSQLNekixxhash
Netflix · 2026
TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads
Netflix moved older data in its multi-petabyte TimeSeries Abstraction out of hot Cassandra storage in two steps. First, Kafka and Flink streamed events to S3 as Parquet files, which a daily job compacted into small, time-sorted row groups with Bloom filters cached in EVCache (p50 ~150 ms, p99 ~500 ms). The second design skips that streaming pipeline: a ReadOnlyCompactionStrategy aligns immutable SSTables to token ranges, and a custom pluggable Memtable (Cassandra 4.1) reads directly from existing Cassandra backups in S3. It now holds over 15 PB of compressed cold data, cut disk usage on affected clusters from 80% to 7%, and improved p90 latency by ~30%.
15+ PB compressed data in cold storagedisk usage cut from 80% to 7%tiered storagecompactioncost optimizationCassandraKafkaFlink