What is the equivalent of dist barrier

implementation help
1

I have a PL Callback. I would like to use the equivalent of dist.barrier in PL.
I saw that there is an equivalent in Fabric, but not sure if it’s the right way to use it.

I have this issue, because my code looks like that:

    def on_validation_epoch_end(self): # it's a PL callback
        if not self.time_to_compute_fid(): return
        for _ in range(nb_fid_batch_per_gpu)
            fakes = gen_fakes()
            self.fid.update(fakes, real=False)
        fid_value = self.fid.compute()
        # log the value using wandb
        self.fid.reset()

I would need a barrier before the compute at least.

What is the equivalent of dist.barrier?

2

ok the solution is self.trainer.strategy.barrier() in PyTorch Lightning.

https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.strategies.Strategy.html#lightning.pytorch.strategies.Strategy.barrier

https://lightning.ai/docs/pytorch/stable/debug/debugging_advanced.html#debug-distributed-models