DDP with Multiple gpus is not providing gains

DDP/GPU
1

Hi, I am using DDP with four GPUs in single machine, I don’t see improvements in terms of speed
can anyone help me with correct setting

With 4 GPUS

image
image2188×268 16.1 KB

2

With Single GPU

image
image2188×314 23.3 KB