How to combine PTL arguments with ArgumentParser

implementation help
1

As lightning 2.0 has released, we are working on fixing the mlflow pytorch examples to be compatible with the latest release.

Based on the lightning release notes and the ptl examples, i have fixed couple of the examples which can be tracked via this WIP PR.

Until the previos releases, we used pl.Trainer.add_argparse_args for combing the lightning arguments with user defined ArgumentParser - ax_hpo_iris.py#L59

With the latest release pl.Trainer.add_argparse_args has been removed. I could not use LightningCLI as the model (ax_hpo_iris.py#L10) and the datamodule classes which i user are the child classes.

By doing

cli = LightningCLI(IrisClassification, IrisDataModule, run=False)

I get the following error

Traceback (most recent call last):
  File "ax_hpo_iris.py", line 79, in <module>
    model_training_hyperparameter_tuning(
  File "ax_hpo_iris.py", line 34, in model_training_hyperparameter_tuning
    train_evaluate(params=params, max_epochs=max_epochs)
  File "ax_hpo_iris.py", line 11, in train_evaluate
    cli = LightningCLI(IrisClassification, IrisDataModule, run=False)
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/cli.py", line 342, in __init__
    self.setup_parser(run, main_kwargs, subparser_kwargs)
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/cli.py", line 379, in setup_parser
    self._add_arguments(self.parser)
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/cli.py", line 412, in _add_arguments
    self.add_core_arguments_to_parser(parser)
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/cli.py", line 399, in add_core_arguments_to_parser
    parser.add_lightning_class_args(self._model_class, "model", subclass_mode=self.subclass_mode_model)
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/cli.py", line 143, in add_lightning_class_args
    raise MisconfigurationException(
lightning.fabric.utilities.exceptions.MisconfigurationException: Cannot add arguments from: <class 'iris.IrisClassification'>. You should provide either a callable or a subclass of: Trainer, LightningModule, LightningDataModule, or Callback.

Without using LightningCLI, is there any way to combine the PTL arguments with user’s argparser ?

To achieve something as below

python ax_hpo_iris.py --total_trails=3 --trainer.max_epochs=3

where total_trial is the argument from user’s argument parser and trainer.max_epochs is the default argument from lightning.

2

Hey

Without LightningCLI, you would basically extend the user’s argument parser with the relevant arguments you need for the Trainer. We describe it briefly here in our release notes:

import lightning as L
import argparse

parser = argparse.ArgumentParser()

# 1. Add the arguments you need
parser.add_argument("--accelerator", type=str, default="cuda")
args = parser.parse_args()

# 2. Pass them into the Trainer
trainer = L.Trainer(**vars(args))
...

It is basically the same as with the previous pl.Trainer.add_argparse_args, but now you have to write the parser.add_argument() statements with the arguments you want exposed.

3

Thanks for the reply @awaelchli .

pl.Trainer.add_argparse_args saved the job of writing parse.add_argument

We have so many useful variables (max_epochs, strategy, devices, accelerator, early stopping variables such as patience, monitor, modelcheckpoint callback variables and so on.

Adding parser.add_argument for all these variables, makes it hard to maintain.

And if the script contains user defined variable, I couldnt do L.Trainer(**vars(args)) .

For example

import lightning as L
import argparse

parser = argparse.ArgumentParser()

# 1. Add the arguments you need
parser.add_argument("--accelerator", type=str, default="cuda")
parser.add_argument("--total_trials", type=int, default=1)
args = parser.parse_args()

# 2. Pass them into the Trainer
trainer = L.Trainer(**vars(args))

Throws

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/anaconda3/envs/mlflow-8da05ac741d03eed5e60f74266642b39d460004d/lib/python3.8/site-packages/lightning/pytorch/utilities/argparse.py", line 69, in insert_env_defaults
    return fn(self, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'total_trials'

Until last release, it wasn’t the case. Even if args contains user defined variable, the trainer was getting instantiated.

    trainer = pl.Trainer.from_argparse_args(
        args, callbacks=[lr_logger, early_stopping, checkpoint_callback]
    )