DeepSpeed's Next Talk Won't Just Be About ZeRO

PyTorch Day Japan 2026 has scheduled DeepSpeed's session for December 10 from 17:55 to 18:15 in 2F Hall. The speaker is DeepSpeed's Masahiro Tanaka, and the title is DeepSpeed at the Frontier: Community-Driven Optimizations for State-of-the-Art Training Performance.

DeepSpeed is a distributed training framework built on PyTorch, with more than 43,000 GitHub stars and over 1 million monthly PyPI downloads. It first became widely discussed thanks to ZeRO. In a reply, X user The SI Maximalist described ZeRO as sending eviction notices to redundant optimizer states to free up GPU memory for the model. The phrasing is crude, but it captures the point: first drive down memory usage.

This talk will cover a broader scope. Beyond ZeRO, DeepSpeed has added AutoEP, AutoSP, and AutoTP, corresponding to expert parallelism, sequence parallelism, and tensor parallelism, respectively. They can be combined with ZeRO and used with many Hugging Face models. AutoEP targets large-scale MoE training and brings optimized kernels and communication. DeepCompile, meanwhile, uses torch.compile to optimize communication scheduling and memory offload at the compiler level.

These efforts share one thing in common: many of the optimizations come from researchers and engineers at different organizations. DeepSpeed's open development also draws in experimental and research-oriented features. In the framework wars, launch-event-style new features tend to grab attention. The hard part is making existing models run faster with a configuration change without rewriting all the training code. DeepSpeed says that on frontier models, performance can compete with other advanced frameworks, with changes concentrated in configuration. That claim makes the session worth attending to hear the benchmarks and details.

The event is in Tokyo on December 10. Standard tickets cost 5,000 yen if you register by November 11, and 8,000 yen afterward, a difference of 3,000 yen. Academic tickets remain 5,000 yen. The session will be recorded and posted to the PyTorch Conference YouTube channel within two weeks.

Registration and schedule: PyTorch Day Japan 2026, full schedule. DeepSpeed session details: link.

If you remember only one thing: DeepSpeed's memory-saving label remains, but the focus of this talk is on community contributions and frontier training performance. ZeRO is the entry point; beyond it are parallel combinations, compiler optimizations, and communication scheduling. For people working on training, these details are more practically useful than a phrase like 'supports large-scale models'.