Found the issue. The gradients have to be explicitly turned off after the checkpointed model is loaded! lightning puts it in eval mode in the valid and test methods as noted here https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html but one has to do it explicitly in the above situation. For the sake of completeness:
model = FFNPL.load_from_checkpoint(s)
model.eval()
...