I think that the __init__ here follows the same distributed initialization logic as in nn.Module, since LightningModule subclasses nn.Module - that being said, I don’t think it should make much of a difference if you initialize your model in __init__ or __setup__.
What’s your use-case? Could you provide a MWE?