Is it possible to initiate downstream tasks with callbacks and then log the result?

implementation help
1

I’m working on a representation learning problem and the only way I really have of knowing if my model generalised is to forward the checkpoint to a few supervised learning problems and seeing the latents performance on each supervised task. Is it possible to asynchrnously submit a job after each epoch and then log the final result when the models are finished?

I appreciate this questions might be beyond the scope of pytorch lightning itself but it would be good to know if there’s a way to make the model “listen” for an external task to finish and log the outcome. Is it also possible to send a “hook” to the logger to an external process?

2

Hello, my apology for the late reply. We are slowly converging to deprecate this forum in favor of the GH build-in version… Could we kindly ask you to recreate your question there - Lightning Discussions