Thanks, can I ask what the bug is (or how to fix it, it’s my first pytorch lightning script). I went through the link to sent, I’m wondering if its’ that on_epoch should be set to false?
But thanks, good to know that it’s error-prone, I guess if you have any more info about this that would be great.
I have a data set, that I randomised the order of, and then randomly split into train, test and validation, and ran the script multiple times. And then I tried the script on multiple data sets and always found the same pattern but yes I understand that training and validaton could in theory be similar.