Default value of sync_batchnorm

Trainer
1

Hi,
I am just having a simple question here.
Why not the flag sync_batchnorm default be True?
If using multi-GPUs, aren’t the weights be different across processes if sync_batchnorm is False?
Is there case of not using sync_batchnorm in multi-GPU mode?

Thanks.

2

Quite a few papers in the past have posted results without using sync batch norm as True. So, the default is not to use it.