PyVideo.org · Linsong Chu

Contribute Media

Speaker: Linsong Chu

Image from A Distributed Stateful Dataloader for Large-Scale Pretraining

A Distributed Stateful Dataloader for Large-Scale Pretraining

Image from Maximizing Training Throughput Using Torch.Compile and FSDP

Maximizing Training Throughput Using Torch.Compile and FSDP

Image from Torchtitan: Large-Scale LLM Training Using Native PyTorch 3D Parallelism

Torchtitan: Large-Scale LLM Training Using Native PyTorch 3D Parallelism