Hello everyone, I am training a relation extraction model, but my data is not as diverse as it could be (Several entities are too commonly associated with several relations). To overcome this, I want to apply masks to random entities every epoch. Importantly, I do want the masks to change every epoch. I wrote code to create the masks as part of the DataLoader, however the data is only masked once at the start of the training instead of every epoch.
Is there a way to apply random masks every epoch OR reload the DataLoader every epoch so that I can trigger my function in every epoch again?
Any help is massively appreciated, thank you!
In the latest version of Lightning Trainer, we have the option called reload_dataloaders_every_n_epochs
That means if you set
Trainer(reload_dataloaders_every_n_epochs=1) the dataloaders will reset (meaning being re-instantiated) every epoch and this should trigger your random mask generation again.