How we improved push processing on GitHub
GitHub broke one huge push-processing job into smaller, independently retried jobs driven by Kafka events.
GitHub.com is a Ruby on Rails app. For how other teams scaled Rails, see Ruby on Rails at scale: GitHub, GitLab and Shopify.
- Published
- 2024-06-11
- Scale
- Not stated in source
- Techniques
- queues job decomposition
- Stack
- Ruby on Rails Kafka
Try this stack yourself
- Railway · Deploy something like this on Railway: spin up Ruby on Rails plus your app in a few clicks. New accounts get $20 in credit (applied when you pick a paid plan).
Railway isn't part of GitHub's setup in this story. It's just an easy place to try the same building blocks.
Some links above are affiliate links: if you sign up, How They Scaled may earn a commission at no extra cost to you. We only list tools that fit the stack in the story.