OpenAI · 2026
Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)
OpenAI's Habitat storage platform went from one shared Azure Cosmos DB container, to dedicated single-tenant containers that pushed capacity planning onto every product team, and back to a shared container by default once Azure added hot-key throttling, request-priority QoS and a new serverless mode where each physical partition scales to 10K RU/s and splits on RU usage. After failover and failback outages in late 2024, OpenAI and Azure now co-own testing and monitoring. Replicating small latency-critical datasets widely, a Valkey lookaside cache invalidated through the Cosmos DB change feed, and a Rust rewrite of the proxy cut global read p50 from about 120 ms to about 10 ms.
1B+ users across 40 regions500+ PB served by Habitatmulti-tenancyhot key throttlingCDC cache invalidationAzure Cosmos DBValkeyKafka
Datadog · 2025
Evolving our real-time timeseries storage again: Built in Rust for performance at scale
Datadog rebuilt its real-time timeseries database (RTDB) storage engine in Rust as Monocle after outgrowing Cassandra, Redis, MDBM, and a Go B+ tree plus a separate RocksDB path for DDSketch distributions. The design shards the keyspace early with a shard-per-core model (for example, 24 storage shards on a 32-core machine), ingesting from Kafka partitions over MPSC channels into per-shard LSM trees on single-threaded Tokio schedulers, with tiered compaction, a unified series cache, and permit- and cost-based query throttlers (including CoDel). At peak scale the rewrite delivered a 60x increase in ingestion performance and 5x faster queries, turning extreme outliers that used to take minutes into results that return in seconds.
60x increase in ingestion performance5x faster queries at peak scaleshard-per-coreLSM treetiered compactionRustTokioLSM tree