Hi, As the title,Dose batch norm need to be converted to SyncBatchNorm by using torch.nn.SyncBatchNorm.convert_sync_batchnorm(model) before fabric.setup(model,optimizer)?
If you want sync’d batch norm, then yes.
understand.thanks for your quick reply! I