How to reload dataset every epoch

Since my dataset is too large for memory, I have to split it into multiple subsets like:

For every epoch, I want to choose one dir(‘sub_x’) and use it to create a dataset during training. Could anyone tell me how to implement it in lightning, especially in DDP setting?