Blog
Distributed Training

Distributed Training

June 26, 2025·Deependu
Deependu

Roadmap

  • Distributed Data Parallel (DDP)
  • FSDP (Fully Sharded Data Parallel)
  • Tensor Parallelism (TP)
  • Pipeline Parallelism
  • Device Mesh (Dtensor & DeviceMesh)
  • Remote Procedure Call (RPC) distributed training
  • Custom Extensions
Last updated on
Deep Q-Learning (DQN)Distributed Data Parallel (DDP)