DDP with Multiple gpus is not providing gains

[rajamohan_reddy]

Hi, I am using DDP with four GPUs in single machine, I don’t see improvements in terms of speed
can anyone help me with correct setting

With 4 GPUS

[rajamohan_reddy]

With Single GPU