---
title: Multi-Machine Training
description: Compatibility interface for multi-machine jobs.
canonical: 'https://lightning.ai/docs/sdk/stable/api/mmt'
---
# Multi-Machine Training {#multi-machine-training}

### `class lightning_sdk.MMT(name, teamspace=None, org=None, user=None, *, _fetch_job=True, _num_machines=2)` {#lightning_sdk.MMT}

Bases: [`Job`](/docs/sdk/2026.11.04/api/job.md#lightning_sdk.Job "lightning_sdk.job.Job")

Compatibility interface for multi-machine jobs.

Multi-machine functionality is implemented by [`lightning_sdk.job.Job`](/docs/sdk/2026.11.04/api/job.md#lightning_sdk.Job "lightning_sdk.job.Job"). `MMT` is deprecated; use `Job` instead.

Fetch already existing jobs.

Parameters:

*   **name** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")) – the name of the job.
    
*   **teamspace** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Teamspace`](/docs/sdk/2026.11.04/api/teamspace.md#lightning_sdk.Teamspace "lightning_sdk.teamspace.Teamspace"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – the teamspace the job is part of.
    
*   **org** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Organization`](/docs/sdk/2026.11.04/api/organization.md#lightning_sdk.Organization "lightning_sdk.organization.Organization"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – the name of the organization owning the `teamspace` in case it is owned by an org. Deprecated — pass the owner as part of `teamspace` instead, e.g. `teamspace="owner/teamspace"`.
    
*   **user** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`User`](/docs/sdk/2026.11.04/api/user.md#lightning_sdk.User "lightning_sdk.user.User"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – the name of the user owning the `teamspace` in case it is owned directly by a user instead of an org. Deprecated — pass the owner as part of `teamspace` instead, e.g. `teamspace="owner/teamspace"`.
    
*   **name** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"))
    
*   **teamspace** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_Teamspace_](/docs/sdk/2026.11.04/api/teamspace.md#lightning_sdk.Teamspace "lightning_sdk.Teamspace") _|_ _None_)
    
*   **org** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_Organization_](/docs/sdk/2026.11.04/api/organization.md#lightning_sdk.Organization "lightning_sdk.Organization") _|_ _None_)
    
*   **user** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_User_](/docs/sdk/2026.11.04/api/user.md#lightning_sdk.User "lightning_sdk.User") _|_ _None_)
    
*   **\_fetch\_job** ([_bool_](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)"))
    
*   **\_num\_machines** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)"))
    

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If the teamspace cannot be resolved from the provided arguments, or if the job is not found when `_fetch_job=True`.
    
*   [**PermissionError**](https://docs.python.org/3/builtins/exceptions.html#PermissionError "(in Python v3.15)") – If the user does not have access to jobs in the given teamspace.
    

#### `classmethod run(name, num_machines, machine, cloud=None, command=None, studio=None, image=None, teamspace=None, org=None, user=None, env=None, interruptible=False, image_credentials=None, cloud_account_auth=False, entrypoint=None, path_mappings=None, max_runtime=None, max_run_attempts=None, reuse_snapshot=True, placement_group_id=None, tags=None, keep_minutes=None)` {#lightning_sdk.MMT.run}

Run async workloads using a docker image or a compute environment from your studio.

Parameters:

*   **name** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")) – The name of the job. Needs to be unique within the teamspace.
    
*   **machine** (`Union`\[[`Machine`](/docs/sdk/2026.11.04/api/machine.md#lightning_sdk.Machine "lightning_sdk.machine.Machine"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – The machine type to run the job on.
    
*   **num\_machines** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – The number of machines to run on. Defaults to one.
    
*   **command** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – The command to run inside your job. Required if using a studio. Optional if using an image. If not provided for images, will run the container entrypoint and default command. Studio jobs run the command as a bash script with `set -e`, so the first command that fails ends the job with its exit code. That includes cleanup commands and `EXIT` traps that run after `exit 0`.
    
*   **studio** (`Union`\[[`Studio`](/docs/sdk/2026.11.04/api/studio.md#lightning_sdk.Studio "lightning_sdk.studio.Studio"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – The studio env to run the job with. Mutually exclusive with image. If both `studio` and `image` are left unset and this code is running inside a Studio (detected via the `LIGHTNING_CLOUD_SPACE_ID` env var), defaults to that Studio, provided its teamspace matches the resolved `teamspace`.
    
*   **image** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – The docker image to run the job with. Mutually exclusive with studio.
    
*   **teamspace** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Teamspace`](/docs/sdk/2026.11.04/api/teamspace.md#lightning_sdk.Teamspace "lightning_sdk.teamspace.Teamspace"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – The teamspace the job should be associated with. Defaults to the current teamspace. Accepts a bare name or an `owner/teamspace` slug.
    
*   **org** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Organization`](/docs/sdk/2026.11.04/api/organization.md#lightning_sdk.Organization "lightning_sdk.organization.Organization"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – The organization owning the teamspace, if any. Defaults to the current organization. Deprecated — pass the owner as part of `teamspace` instead, e.g. `teamspace="owner/teamspace"`.
    
*   **user** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`User`](/docs/sdk/2026.11.04/api/user.md#lightning_sdk.User "lightning_sdk.user.User"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – The user owning the teamspace, if any. Defaults to the current user. Deprecated — pass the owner as part of `teamspace` instead, e.g. `teamspace="owner/teamspace"`.
    
*   **cloud** (`Union`\[[`CloudProvider`](/docs/sdk/2026.11.04/api/machine.md#lightning_sdk.CloudProvider "lightning_sdk.machine.CloudProvider"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Cloud provider or cloud account to run the job on.
    
*   **env** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Dict`](https://docs.python.org/3/library/typing.html#typing.Dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – Environment variables to set inside the job.
    
*   **interruptible** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether the job should run on interruptible instances. Cheaper but can be preempted.
    
*   **image\_credentials** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Credentials secret name used to pull a private image.
    
*   **cloud\_account\_auth** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to authenticate with the cloud account to pull the image. Required if the registry is part of a cloud provider, such as ECR.
    
*   **entrypoint** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – The entrypoint of your docker container. Defaults to `sh -c`. Set to an empty string to use the image’s pre-defined entrypoint with a command. Only applicable when submitting docker jobs.
    
*   **path\_mappings** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Dict`](https://docs.python.org/3/library/typing.html#typing.Dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – Maps container paths to data-connection paths in the form `{"<CONTAINER_PATH>": "<CONNECTION_NAME>:<PATH>"}` or `{"<CONTAINER_PATH>": "<CONNECTION_NAME>"}` for the root of a connection. Only applicable when submitting docker jobs.
    
*   **max\_runtime** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – How long, in seconds, to reserve a DWS (Dynamic Workload Scheduler) machine, such as some top-end GCP GPUs or Lightning baremetal. On GCP the job is stopped when the reservation ends. On Lightning baremetal the job keeps running afterwards and the machine becomes interruptible. For a hard time limit, wrap your command, e.g. `timeout 7200 python train.py`. Has no effect on interruptible (spot) machines. `None` means no reservation is requested.
    
*   **max\_run\_attempts** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Max number of run attempts for this job. `None` or `0` means unset (backend default). `1` means a single attempt (no retries). `N > 1` allows up to `N` attempts. For multi-machine jobs this is set at the multi-machine job level (not the per-machine `JobSpec`). When greater than `1` for a multi-machine job, the `RECREATE_ALL_NODES` fault tolerance strategy is set automatically on the multi-machine job body.
    
*   **reuse\_snapshot** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to reuse a Studio snapshot when multiple jobs for the same Studio are submitted. Turning this off may result in longer startup times. Defaults to True.
    
*   **scratch\_disks** – Optional mapping of scratch-disk mount paths to their sizes in GiB.
    
*   **placement\_group\_id** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Optional placement group identifier for colocating the job.
    
*   **tags** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – Teamspace tag names to apply to the job. Tags that don’t exist in the teamspace yet are created, which requires permission to create tags. Names are normalised by the platform (lowercased, whitespace collapsed), so `job.tags` may read back slightly differently from what was passed in.
    
*   **keep\_minutes** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Minutes to keep the machine after the job stops. `None` or `0` releases the machine immediately.
    
*   **name** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"))
    
*   **num\_machines** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)"))
    
*   **machine** ([_Machine_](/docs/sdk/2026.11.04/api/machine.md#lightning_sdk.Machine "lightning_sdk.machine.Machine") _|_ [_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"))
    
*   **cloud** ([_CloudProvider_](/docs/sdk/2026.11.04/api/machine.md#lightning_sdk.CloudProvider "lightning_sdk.machine.CloudProvider") _|_ [_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **command** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **studio** ([_Studio_](/docs/sdk/2026.11.04/api/studio.md#lightning_sdk.Studio "lightning_sdk.studio.Studio") _|_ [_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **image** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **teamspace** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_Teamspace_](/docs/sdk/2026.11.04/api/teamspace.md#lightning_sdk.Teamspace "lightning_sdk.teamspace.Teamspace") _|_ _None_)
    
*   **org** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_Organization_](/docs/sdk/2026.11.04/api/organization.md#lightning_sdk.Organization "lightning_sdk.organization.Organization") _|_ _None_)
    
*   **user** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ [_User_](/docs/sdk/2026.11.04/api/user.md#lightning_sdk.User "lightning_sdk.user.User") _|_ _None_)
    
*   **env** ([_Dict_](https://docs.python.org/3/library/typing.html#typing.Dict "(in Python v3.15)")_\[_[_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")_,_ [_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")_\]_ _|_ _None_)
    
*   **interruptible** ([_bool_](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)"))
    
*   **image\_credentials** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **cloud\_account\_auth** ([_bool_](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)"))
    
*   **entrypoint** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **path\_mappings** ([_Dict_](https://docs.python.org/3/library/typing.html#typing.Dict "(in Python v3.15)")_\[_[_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")_,_ [_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")_\]_ _|_ _None_)
    
*   **max\_runtime** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)") _|_ _None_)
    
*   **max\_run\_attempts** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)") _|_ _None_)
    
*   **reuse\_snapshot** ([_bool_](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)"))
    
*   **placement\_group\_id** ([_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)") _|_ _None_)
    
*   **tags** ([_List_](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")_\[_[_str_](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")_\]_ _|_ _None_)
    
*   **keep\_minutes** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)") _|_ _None_)
    

Returns:

The newly submitted Job instance.

Return type:

[Job](/docs/sdk/2026.11.04/api/job.md#lightning_sdk.Job "lightning_sdk.Job")

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If required arguments are missing or mutually exclusive arguments are both provided.
    
*   [**RuntimeError**](https://docs.python.org/3/builtins/exceptions.html#RuntimeError "(in Python v3.15)") – If image and studio are both provided.

### `class lightning_sdk.mmt.MMTMachine(*args, **kwargs)` {#lightning_sdk.mmt.MMTMachine}

Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol "(in Python v3.15)")

A single machine in a multi-machine job.

---

**Navigation**

- Parent: [API Reference](/docs/sdk/2026.11.04/api.md)