LSFEnvironment¶

class pytorch_lightning.plugins.environments.LSFEnvironment[source]¶

An environment for running on clusters managed by the LSF resource manager.

It is expected that any execution using this ClusterEnvironment was executed using the Job Step Manager i.e. jsrun.

This plugin expects the following environment variables:

LSB_JOBID: The LSF assigned job ID
LSB_DJOB_RANKFILE: The OpenMPI compatible rank file for the LSF job
JSM_NAMESPACE_LOCAL_RANK: The node local rank for the task. This environment variable is set by jsrun
JSM_NAMESPACE_SIZE: The world size for the task. This environment variable is set by jsrun
JSM_NAMESPACE_RANK: The global rank for the task. This environment variable is set by jsrun

static detect()[source]¶

Returns True if the current process was launched using the jsrun command.

global_rank()[source]¶

The world size is read from the environment variable JSM_NAMESPACE_RANK.

local_rank()[source]¶

The local rank is read from the environment variable JSM_NAMESPACE_LOCAL_RANK.

node_rank()[source]¶

The node rank is determined by the position of the current hostname in the OpenMPI host rank file stored in LSB_DJOB_RANKFILE.

world_size()[source]¶

The world size is read from the environment variable JSM_NAMESPACE_SIZE.

property creates_processes_externally: bool¶

LSF creates subprocesses, i.e., PyTorch Lightning does not need to spawn them.

property main_address: str¶

The main address is read from an OpenMPI host rank file in the environment variable LSB_DJOB_RANKFILE.

property main_port: int¶

The main port is calculated from the LSF job ID.