Netflix

Single-tenant Titus: VM-level isolation on a multi-tenant container platform

Titus, Netflix's Kubernetes-based container platform, launches millions of containers a week on tens of thousands of EC2 instances, packing many teams onto shared hosts. cgroups can't fence last-level cache, memory bandwidth or the global lock in HTB network shaping, so a small-packet flood from one container could starve its neighbors, and stateful streaming jobs like Flink would fall behind and need minutes to catch up. After ruling out microVMs (Firecracker, Kata), gVisor and moving teams to plain EC2, Netflix added an allowlisted single-tenant mode: the federation layer tags eligible tasks, the autoscaler launches the smallest instance that fits each one just in time, and anti-affinity, taints, no preemption and no repacking keep it alone on its host, all behind the same Titus API. Flink, streaming connectors and telemetry pipelines now report stable operation, and the shared fleet packs more densely because sensitive workloads no longer need anti-affinity spreading. The cost is slower launches, since every task waits for its own instance, and fixed host overhead that can take tens of percent of a small node.

Published
2026-10-22
Scale
millions of containers launched per week, tens of thousands of EC2 instances across multiple AWS regions, fixed host overhead: tens of percent of a small dedicated node, ~2% of the largest
Techniques
workload isolation multi-tenancy autoscaling right-sizing bin-packing
Stack
Titus Kubernetes AWS EC2 cgroups Flink

See also

Netflix architecture

The best scaling write-ups, weekly

One email a week. No spam.

Open subscribe page

Related stories

Zalando · 2026

From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale

Zalando Marketing Services replaced a 7-year-old homegrown in-memory join that matched Sponsored Products auction bids to clicks/views within 15 minutes. The old Java service read up to 200 MB/s, could not partition both streams cleanly, and lost up to 15 minutes of state on every deploy or rescale. After Interval Join burned 15-30% CPU on RocksDB seeks and held a ~70M-entry timer queue, they rewrote the match with KeyedCoProcessFunction (point lookups, RocksDB TTL expiry), incremental 3-minute checkpoints, and two-way state. A four-week shadow pipeline hit 99.9% enrichment parity, then a one-week A/B test; average pods fell from 20 to about 5, memory from ~320 GB to ~100 GB, and EC2 cost from about €80 to €30 per day, with enrichment up ~0.5%.

up to 200 MB/s on the old pipelineaverage pods cut from a constant 20 to ~5 (range 1-8)stream joinscheckpointingkeyed stateApache FlinkRocksDBKubernetes
OpenAI · 2026

Scaling PostgreSQL to power 800 million ChatGPT users

OpenAI runs ChatGPT on a single-primary Postgres with nearly 50 global read replicas. Reads are offloaded, write-heavy shardable workloads move to Cosmos DB, and PgBouncer, cache leasing and multi-layer rate limits keep the primary safe.

800 million usersmillions of QPSread replicasconnection poolingcachingPostgreSQLAzurePgBouncer
Netflix · 2026

TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads

Netflix moved older data in its multi-petabyte TimeSeries Abstraction out of hot Cassandra storage in two steps. First, Kafka and Flink streamed events to S3 as Parquet files, which a daily job compacted into small, time-sorted row groups with Bloom filters cached in EVCache (p50 ~150 ms, p99 ~500 ms). The second design skips that streaming pipeline: a ReadOnlyCompactionStrategy aligns immutable SSTables to token ranges, and a custom pluggable Memtable (Cassandra 4.1) reads directly from existing Cassandra backups in S3. It now holds over 15 PB of compressed cold data, cut disk usage on affected clusters from 80% to 7%, and improved p90 latency by ~30%.

15+ PB compressed data in cold storagedisk usage cut from 80% to 7%tiered storagecompactioncost optimizationCassandraKafkaFlink
Netflix · 2026

Behind the Scenes: Evolving Netflix's Ads Event Pipeline for Live, Part II

Netflix's Ads Event Publisher used to enrich each client ad event by looking up metadata the ad server had written to a globally replicated key-value registry. For live events with tens of millions of concurrent viewers, that meant must-succeed database writes on the ad-serving critical path, a store that couldn't safely be scaled mid-event, and late events whenever lookups slowed. Netflix replaced it with an Apache Flink job: the ad server logs its responses to Kafka, and a custom KeyedCoProcessFunction joins them to client events over a 60-minute window, which catches roughly 99.999% of events. An hourly Spark job orchestrated by Maestro recovers the rest from the data warehouse. Revenue-bearing events are de-duplicated in local Flink state keyed by a hashed event ID. Both pipelines ran side by side with automated audits, and a stable per-event key moved publishing to Flink a slice at a time, region by region. When they agreed on well over 99.99% of events, Netflix retired the legacy service and the registry. In a load test, one AWS region handled an ad break at hundreds of thousands of events per second.

hundreds of thousands of events/sec for one ad break in a single AWS region (load test)~99.999% of events joined within a 60-minute windowstreaming joinsdeduplicationshadow trafficFlinkKafkaSpark