Scaling Git at OpenAI
OpenAI built Pando, a Git-compatible backend for its monorepo after Codex pushed concurrent branches, merges and fetches past the old system (reliability sometimes below 90%). Pando stores each push as an immutable pack in blob storage and records per-branch ref versions in a horizontally scalable metadata store, giving serializability per ref and causal consistency across the repo so unrelated branches do not contend. Instead of building unique packs per client, it reuses stored packs and serves them via packfile-uri through regional Express caches (over 99% hit rate), with LSM-style background compaction so fetches plan in logarithmic time. Deployed first as a read/write-through proxy, then cut over with a Git CLI wrapper; it now powers all monorepo development, cut median push latency by nearly 5x, and can deliver nearly a petabyte of Git data per day (57+ TB in a single hour).
- Published
- 2026-11-02
- Scale
- nearly a petabyte of Git data/day, median push latency cut nearly 5x (~80%), 57+ TB of Git packs in a single hour, packfile-uri cache hit rates over 99%, saturates 10 Gbps clone/fetch links on standard VMs
- Techniques
- per-ref causal consistency reusable packfiles storage/compute separation LSM-style compaction zero downtime migration
- Stack
- Git Pando Rust blob storage