Copying results from the work folder of a job in an automated fashion

1

I am running a training job and logging some metrics as well as the model with mlflow in the “work” folder. I now want to copy that mlflow run folder into the “main” mlflow directory of my studio in an automated fashion after the job finished.

My questions are the following:

  1. Is there a way to access the status of a job with the Lightning sdk? That way I could wait until the job finished and copy from my sdk script.
  2. I also tried to copy the mlflow run directory after running the training but it seems it cannot find the “work” folder from within the job.

Thanks in advance!