The problem here seems to be that hydra is creating 2 run dirs, one for each process. It would seem to me that the solution would be to create your own run dirs. Using the following in your config, you can set the current dir as run dir and disable creating new subdirs:
hydra:
output_subdir: null # Disable saving of config files. We'll do that ourselves.
run:
dir: . # Set working dir to current directory
Inside PL, I create new logging dirs from the rank 0 process (there’s convenience funcs available in PL). For a concrete example, take a look at my repo: GitHub - Shreeyak/pytorch-lightning-segmentation-template: Semantic Segmentation on the LaPa dataset using Pytorch Lightning
This is a function to generate the path to the log dir (run dir): pytorch-lightning-segmentation-template/utils.py at 064e13ca0f7606af2928bb62dfc713ae7c23b277 · Shreeyak/pytorch-lightning-segmentation-template · GitHub
And it is created within a custom logging callback. You can modify it to create the dir within the main script (but only from rank 0 process).