Frequently asked questions for LightningCLI¶

What does CLI stand for?¶

CLI is short for command line interface. This means it is a tool intended to be run from a terminal, similar to commands like git.

What is a yaml config file?¶

A YAML is a standard for configuration files used to describe parameters for sections of a program. It is a common tool in engineering and has recently started to gain popularity in machine learning. An example of a YAML file is the following:

# file.yaml
car:
    max_speed:100
    max_passengers:2
plane:
    fuel_capacity: 50
class_3:
    option_1: 'x'
    option_2: 'y'

If you are unfamiliar with YAML, the short introduction at realpython.com#yaml-syntax might be a good starting point.

What is a subcommand?¶

A subcommand is what is the action the LightningCLI applies to the script:

python main.py [subcommand]

See the Potential subcommands with:

python main.py --help

which prints:

...

fit                 Runs the full optimization routine.
validate            Perform one evaluation epoch over the validation set.
test                Perform one evaluation epoch over the test set.
predict             Run inference on your data.
tune                Runs routines to tune hyperparameters before training.

use a subcommand as follows:

python main.py fit
python main.py test

What is the relation between LightningCLI and argparse?¶

LightningCLI makes use of jsonargparse which is an extension of argparse. Due to this, LightningCLI follows the same arguments style as many POSIX command line tools. Long options are prefixed with two dashes and its corresponding values are separated by space or an equal sign, as --option value or --option=value. Command line options are parsed from left to right, therefore if a setting appears multiple times, the value most to the right will override the previous ones.

What is the override order of LightningCLI?¶

The final configuration of CLIs implemented with LightningCLI can depend on default config files (if defined), environment variables (if enabled) and command line arguments. The override order between these is the following:

Defaults defined in the source code.
Existing default config files in the order defined in default_config_files, e.g. ~/.myapp.yaml.
Entire config environment variable, e.g. PL_FIT__CONFIG.
Individual argument environment variables, e.g. PL_FIT__SEED_EVERYTHING.
Command line arguments in order left to right (might include config files).

How do I troubleshoot a CLI?¶

The standard behavior for CLIs, when they fail, is to terminate the process with a non-zero exit code and a short message to hint the user about the cause. This is problematic while developing the CLI since there is no information to track down the root of the problem. To troubleshoot set the environment variable JSONARGPARSE_DEBUG to any value before running the CLI:

export JSONARGPARSE_DEBUG=true
python main.py fit

Note

When asking about problems and reporting issues, please set the JSONARGPARSE_DEBUG and include the stack trace in your description. With this, users are more likely to help identify the cause without needing to create a reproducible script.