Meta
How Meta trains large language models at scale
Meta covers the hardware, networking and reliability work needed to train LLMs across very large GPU clusters.
- Published
- 2024-06-12
- Scale
- Not stated in source
- Techniques
- distributed training hardware reliability networking
- Stack
- GPUs PyTorch