Netflix · 2026
Behind the Scenes: Evolving Netflix's Ads Event Pipeline for Live, Part II
Netflix's Ads Event Publisher used to enrich each client ad event by looking up metadata the ad server had written to a globally replicated key-value registry. For live events with tens of millions of concurrent viewers, that meant must-succeed database writes on the ad-serving critical path, a store that couldn't safely be scaled mid-event, and late events whenever lookups slowed. Netflix replaced it with an Apache Flink job: the ad server logs its responses to Kafka, and a custom KeyedCoProcessFunction joins them to client events over a 60-minute window, which catches roughly 99.999% of events. An hourly Spark job orchestrated by Maestro recovers the rest from the data warehouse. Revenue-bearing events are de-duplicated in local Flink state keyed by a hashed event ID. Both pipelines ran side by side with automated audits, and a stable per-event key moved publishing to Flink a slice at a time, region by region. When they agreed on well over 99.99% of events, Netflix retired the legacy service and the registry. In a load test, one AWS region handled an ad break at hundreds of thousands of events per second.
hundreds of thousands of events/sec for one ad break in a single AWS region (load test)~99.999% of events joined within a 60-minute windowstreaming joinsdeduplicationshadow trafficFlinkKafkaSpark
Grab · 2026
Agent platform (Part 1): How we help Grab build and run AI agents at scale
Grab's internal AI agent work started as a Tech Infra support bot and grew into a company-wide platform: more than 500 services run on its agent framework (LLM-Kit), over 50 Model Context Protocol servers are registered on its remote MCP framework, and a single LLM gateway fronts every model call across the company, handling billions of tokens each month for production agents used daily by millions of merchants, drivers, and consumers. Part 1 covers how the original Go bot's reasoning plane and tool plane became reusable framework building blocks, and how problems that appeared around 500 agents—shared model control, safe tool reuse across teams, and evals after prompt changes—pushed the team toward the GrabGPT Gateway, remote MCP, and an evals platform covered in later parts.
500+ services on the internal agent framework50+ remote MCP serversagent frameworkLLM gatewayremote MCPLLM-KitGrabGPT GatewayModel Context Protocol
Figma · 2026
How Figma manages sandboxes at scale
Figma's AI products needed secure places to run untrusted code, so the team built Nimbus, a centralized sandbox platform that product teams call through a simple namespace config and GetOrCreate/Get/Delete APIs. Nimbus picks a cloud provider, applies networking and security policies (trusted-host allowlists plus read-only GET/HEAD/OPTIONS to the rest of the internet, credentials decrypted only on the way out to their intended destination), and runs sandboxd inside each box for process supervision and metrics. Warm pools of already-running sandboxes cut provisioning latency by about 90% versus cold starts, raised availability from 99% to 99.95%, and absorb minutes of provider outages while Postgres claim transactions and a Redis health-check queue keep the pool correctly sized and singly assigned.
tens of thousands of sandboxes across productswarm pools cut provisioning latency ~90%warm poolsmulti-provider failovernetwork policy isolationNimbusPostgresRedis