CUDA OOM while initializing DDP

[dedeswim]

UPDATE: I opened an issue here as the behavior is pretty weird and it could be a bug.