DeepSpeed
Microsoft open source library
From Wikipedia, the free encyclopedia
DeepSpeed is an open source deep learning optimization library for PyTorch.[1]
| DeepSpeed | |
|---|---|
| |
| Original author | Microsoft Research |
| Developer | Microsoft |
| Initial release | May 18, 2020 |
| Stable release | v0.18.6
/ February 13, 2026 |
| Written in | Python, CUDA, C++ |
| Type | Software library |
| License | Apache License 2.0 |
| Website | deepspeed |
| Repository | github |
Library
The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware.[2][3] DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 1 trillion or more parameters.[4] Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under MIT License and available on GitHub.[5]
The team claimed to achieve up to a 6.2x throughput improvement, 2.8x faster convergence, and 4.6x less communication.[6]
