Erasure coding: 2 real scaling case studies
2 of the 94 engineering write-ups in our database are tagged with the technique "erasure coding". They come from 1 company (Dropbox) and were published in 2019 (1 undated). Stack that shows up most often in these stories: Custom storage (2). 1 of the 2 state concrete scale numbers, for example "500 PB" (Dropbox). Each card below links to our short summary and the original post.
Dropbox's Magic Pocket cold tier uses erasure coding for rarely read data. For other ways to move cold data to cheaper storage, see tiered storage examples.
Dropbox · 2019
How we optimized Magic Pocket for cold storage
Dropbox added a cold tier to Magic Pocket that cuts replication overhead for rarely read data.
Dropbox · n.d.
Scaling to exabytes and beyond
Dropbox built Magic Pocket, its own exabyte-class block store, and moved user data off third-party cloud storage.
Get new erasure coding scaling stories weekly
One email a week. No spam.