I have a PL Callback. I would like to use the equivalent of dist.barrier in PL.
I saw that there is an equivalent in Fabric, but not sure if it’s the right way to use it.
I have this issue, because my code looks like that:
def on_validation_epoch_end(self): # it's a PL callback
if not self.time_to_compute_fid(): return
for _ in range(nb_fid_batch_per_gpu)
fakes = gen_fakes()
self.fid.update(fakes, real=False)
fid_value = self.fid.compute()
# log the value using wandb
self.fid.reset()
I would need a barrier before the compute at least.
What is the equivalent of dist.barrier?