DDP with Multiple gpus is not providing gains

rajamohan_reddy · June 30, 2022, 7:14am

Hi, I am using DDP with four GPUs in single machine, I don’t see improvements in terms of speed
can anyone help me with correct setting

With 4 GPUS

rajamohan_reddy · June 30, 2022, 7:16am

With Single GPU

Topic		Replies	Views
Testing Multi GPU training on a Single GPU DDP/GPU	1	2559	February 22, 2021
Why might speed stay the same when moving from 1 GPU to 8 GPUs (DDP)? DDP/GPU	2	1405	September 6, 2020
Reproduce one GPU score/loss using DDP - Disrepancy DDP/GPU	1	340	January 28, 2024
Set seed on DDP DDP/GPU	0	1564	June 11, 2021
DDP MultiGPU Training does not reduce training time DDP/GPU	3	1567	November 8, 2023