Teamspace

class lightning_sdk.Teamspace(name=None, org=None, user=None)

Bases: object

A teamspace is a collection of Studios, Clusters, Members and an associated Budget.

Parameters:

name (Optional[str]) – the name of the teamspace, either a bare name or an owner/teamspace slug
org (Union[str, Organization, None]) – the owning organization. Deprecated — pass the owner as part of name instead, e.g. name="owner/teamspace".
user (Union[str, User, None]) – the owning user. Deprecated — pass the owner as part of name instead, e.g. name="owner/teamspace".
name (str | None)
org (str | Organization | None)
user (str | User | None)

Note

Either user or organization should be specified.

Note

Arguments will be automatically inferred from environment variables if possible, unless explicitly specified

bucket_credentials(connection)

Get short-lived credentials for the bucket behind a data connection.

The credentials expire, so call this again for a fresh set rather than caching them; BucketCredentials.expires_at is the deadline to refresh against. To hand them to a tool that refreshes on its own, see lightning connection credentials, which speaks AWS’s credential_process protocol.

Parameters:

connection (str) – Name of the data connection, as it appears in the teamspace drive.
connection (str)

Returns:

The credentials, with the region and endpoint to use them against.

Return type:

BucketCredentials

Raises:

ValueError – If the teamspace has no data connection with that name.

create_agent(name, api_key, base_url, model, org_id='', prompt_template='', description='', prompt_suggestions=None, file_uploads_enabled=None)

Create a new AI agent backed by a custom model endpoint.

Parameters:

name (str) – Display name for the agent.
api_key (str) – API key used to authenticate requests to the model endpoint.
base_url (str) – Base URL of the model endpoint (e.g. an OpenAI-compatible server).
model (str) – Model identifier to use (e.g. "gpt-4o").
org_id (Optional[str]) – Optional organization ID to associate with the agent. Defaults to "".
prompt_template (Optional[str]) – System prompt template shown to the model. Defaults to "".
description (Optional[str]) – Human-readable description of the agent. Defaults to "".
prompt_suggestions (Optional[List[str]]) – Suggested user prompts shown in the chat UI. Defaults to None.
file_uploads_enabled (Optional[bool]) – Whether the agent accepts file uploads from users. Defaults to None.
name (str)
api_key (str)
base_url (str)
model (str)
org_id (str | None)
prompt_template (str | None)
description (str | None)
prompt_suggestions (List[str] | None)
file_uploads_enabled (bool | None)

Returns:

The newly created agent.

Return type:

Agent

delete_model(name)

Delete a model from the model store.

Parameters:

name (str) – Name tag of the model to delete. Can optionally also contain a version tag separated by a colon, e.g. ‘entity/modelname:v1’.
name (str)

Return type:

None

delete_secret(key)

Delete a generic encrypted secret from the teamspace.

Parameters:

key (str) – Secret name. Must start with a letter or underscore and contain only alphanumeric characters and underscores.
key (str)

Raises:

ValueError – If the key is invalid or no generic secret with that name exists.

Return type:

None

download_file(remote_path, file_path=None, cloud_account=None)

Download a file from the Teamspace drive to a local path.

Parameters:

remote_path (str) – Path of the file inside the Teamspace drive.
file_path (Optional[str]) – Local destination path. Defaults to remote_path.
cloud_account (Optional[str]) – Cloud account to use. Defaults to None.
remote_path (str)
file_path (str | None)
cloud_account (str | None)

Return type:

None

download_folder(remote_path, target_path=None, cloud_account=None)

Download a directory from the Teamspace drive to a local path.

Parameters:

remote_path (str) – Path of the directory inside the Teamspace drive.
target_path (Optional[str]) – Local destination directory. Defaults to remote_path.
cloud_account (Optional[str]) – Cloud account to use. Defaults to None.
remote_path (str)
target_path (str | None)
cloud_account (str | None)

Return type:

None

download_model(name, download_dir=None, progress_bar=True)

Download a checkpoint from the model store.

Parameters:

name (str) – Name tag of the model to download. Can optionally also contain a version tag separated by a colon, e.g. ‘modelname:v1’.
download_dir (Optional[str]) – A path to directory where the model should be downloaded. Defaults to the current working directory.
progress_bar (bool) – Whether to show a progress bar for the download.
name (str)
download_dir (str | None)
progress_bar (bool)

Return type:

str

Returns:

The absolute path to the downloaded model file or folder.

Raises:

ValueError – If no model name is provided.
RuntimeError – If the model upload is not yet complete or no files were downloaded.

list_files(remote_path='', recursive=False, cloud_account=None)

List the entries of a directory in the Teamspace drive.

Parameters:

remote_path (str) – Path of the directory inside the Teamspace drive. Defaults to the drive root, which lists its top-level folders (studios, jobs, …).
recursive (bool) – When True, descend into subdirectories. Each returned path is then relative to remote_path.
cloud_account (Optional[str]) – Cloud account to use. Defaults to None.
remote_path (str)
recursive (bool)
cloud_account (str | None)

Returns:

One entry per file or folder, in the order the server returns them.

Return type:

List[FileEntry]

list_jobs(tags=None)

The standalone and multi-machine jobs in this teamspace, optionally filtered by tag.

Parameters:

tags (Optional[Sequence[str]]) – Only return jobs carrying at least one of these tags. None returns every job.
tags (Sequence[str] | None)

Returns:

The matching Jobs, standalone ones first.

Return type:

tuple[Job, …]

Raises:

PermissionError – If the authenticated user cannot access Jobs in this teamspace.
ValueError – If the teamspace has no tag by one of the given names.

list_machines(cloud_account=None, machine=None)

List available machines across cloud accounts.

Parameters:

cloud_account (Optional[str]) – The cloud account from which to list available machines. If None, uses LIGHTNING_CLUSTER_ID environment variable. If that’s also None, queries all global cloud accounts.
machine (Optional[str]) – Specific machine name to filter by. If provided, only returns that machine type. Must be a valid Machine enum value.
cloud_account (str | None)
machine (str | None)

Return type:

List[Machine]

Returns:

List of available machines, excluding out-of-capacity machines.

Raises:

RuntimeError – If no cloud account can be resolved.
ValueError – If the provided machine name is not a valid Machine value.

list_model_versions(name)

List all versions of a model in the model store.

Parameters:

name (str) – The model name to list versions for. Must not include a version tag (no colon).
name (str)

Returns:

All available versions of the specified model.

Return type:

List[V1ModelVersionArchive]

Raises:

ValueError – If name contains a colon (version tag).
PermissionError – If the authenticated user cannot access Models in this teamspace.

list_models()

List all models in the model store.

Returns:: All models stored in this teamspace’s model store.
Return type:: List[V1Model]

new_connection(name, source, connection_type, writable=True, cloud_account=None, region=None)

Add an existing data source to this Teamspace.

Parameters:

name (str) – the name under which your data will be available in this Teamspace
source (str) – the source spec of your data. Format depends on the type of data to connect. For EFS, this should be the filsystem id
connection_type (ConnectionType) – the kind of data to connect to this Teamspace
writable (bool) – whether to support write-back to this data source. If False, the data is connected as read-only
cloud_account (Optional[str]) – which cloud-account to connect to the data source to. If not specified, will retrieve the cloud-account that matches the required provider type starting with private cloud accounts and falling back to public if necessary.
region (Optional[str]) – which provider region this data is in. Required for some connection types only.
name (str)
source (str)
connection_type (ConnectionType)
writable (bool)
cloud_account (str | None)
region (str | None)

Raises:

ValueError – If region is not specified for EFS connections, or if the resolved cloud account is not found.

Return type:

None

new_folder(name, location=None, cloud_account=None)

Create a new folder in this Teamspace.

Parameters:

name (str) – The name of the folder. Folders will be accesible under /teamspace/folders/<name>
location (Optional[FolderLocation]) – The location of the folder. Defaults to cloud agnostic.
cloud_account (Optional[str]) – The cloud account to create the folder in. Not used for cloud agnostic folders.
name (str)
location (FolderLocation | None)
cloud_account (str | None)

Raises:

ValueError – If the specified cloud account cannot be found in this teamspace.

Return type:

None

set_secret(key, value, secret_type=SecretType.GENERIC)

Set an encrypted secret for the teamspace.

Parameters:

key (str) – Secret name. Must start with a letter or underscore and contain only alphanumeric characters and underscores.
value (str) – The secret value to store.
secret_type (Union[str, SecretType]) – The type of secret, either a SecretType member or its string value. Defaults to SecretType.GENERIC. Use SecretType.HF_TOKEN (or "hf_token") to tag the secret as a HuggingFace access token so it can be referenced via Deployment.start(hf_token_secret=...). The type is only applied when the secret is created; updating an existing secret leaves its type unchanged.
key (str)
value (str)
secret_type (str | SecretType)

Raises:

ValueError – If key contains invalid characters or secret_type is invalid.

Return type:

None

upload_file(file_path, remote_path=None, progress_bar=True, cloud_account=None)

Upload a file to the Teamspace drive.

Parameters:

file_path (Union[Path, str]) – Local path of the file to upload.
remote_path (Optional[str]) – Destination path inside the Teamspace drive. Defaults to the file’s basename.
progress_bar (bool) – Whether to display a progress bar. Defaults to True.
cloud_account (Optional[str]) – Cloud account to use for storage. Defaults to the teamspace default.
file_path (Path | str)
remote_path (str | None)
progress_bar (bool)
cloud_account (str | None)

Return type:

None

upload_folder(folder_path, remote_path=None, progress_bar=True, cloud_account=None)

Upload a local directory to the Teamspace drive.

Parameters:

folder_path (str) – Local directory to upload.
remote_path (Optional[str]) – Destination directory inside the Teamspace drive.
progress_bar (bool) – Whether to display a progress bar. Defaults to True.
cloud_account (Optional[str]) – Cloud account to use for storage. Defaults to the teamspace default.
folder_path (str)
remote_path (str | None)
progress_bar (bool)
cloud_account (str | None)

Raises:

ValueError – If folder_path is None.
NotADirectoryError – If folder_path is a file or does not exist.

Return type:

None

upload_model(path, name, version=None, cloud_account=None, progress_bar=True, metadata=None, experiment=None)

Upload a local checkpoint file to the model store.

If a previous upload for the same model was interrupted, it is automatically resumed: already-uploaded files are skipped and only the remaining files are uploaded under the same model version.

Parameters:

path (Union[str, Path, List[Union[str, Path]]]) – Path to the model file or folder to upload.
name (str) – Name tag of the model to upload.
version (Optional[str]) – Version tag of the model to upload. If not provided, the vX version will be used where X is running index.
cloud_account (Optional[str]) – The name of the cloud account to store the Model in. If not provided, the default cloud account for the Teamspace will be used.
progress_bar (bool) – Whether to show a progress bar for the upload.
metadata (Optional[dict]) – Metadata to attach to the model. Can be a dictionary.
experiment (Optional[Experiment]) – The experiment producing this model.
path (str | Path | List[str | Path])
name (str)
version (str | None)
cloud_account (str | None)
progress_bar (bool)
metadata (dict | None)
experiment (Experiment | None)

Returns:

An object describing the uploaded model (name, version, teamspace, cloud_account).

Return type:

UploadedModelInfo

Raises:

ValueError – If no path or name is provided, or if metadata is not a dict.
FileNotFoundError – If the path contains no files.
RuntimeError – If multiple paths resolve to conflicting relative paths.
TypeError – If metadata is not a dictionary.

property cloud_account_objs: List[V1ProjectClusterBinding]

All cloud accounts associated with that teamspace.

Returns:: The raw cloud account binding objects for this teamspace.
Return type:: List[V1ProjectClusterBinding]

property cloud_accounts: List[str]

All cloud accounts associated with that teamspace.

Returns:: The cloud account names for this teamspace.
Return type:: List[str]

property clusters: List[str]

All clusters associated with that teamspace.

Returns:: The cluster names for this teamspace. Deprecated — use cloud_accounts instead.
Return type:: List[str]

property default_cloud_account: str | None

The default cloud account for this teamspace.

Falls back to the owning organization’s preferred cloud account when the teamspace has none set.

Returns:: The preferred cloud account ID, or None if not configured.
Return type:: str | None

property id: str

The teamspace’s ID.

Returns:: The unique identifier of this teamspace.
Return type:: str

property jobs: Tuple[Job, ...]

All standalone and multi-machine jobs in this teamspace.

Returns:: Every Job belonging to this teamspace.
Return type:: tuple[Job, …]
Raises:: PermissionError – If the authenticated user cannot access Jobs in this teamspace.

property multi_machine_jobs: Tuple[MMT, ...]

All multi-machine training jobs in this teamspace.

Returns:: Every MMT job belonging to this teamspace.
Return type:: tuple[MMT, …]
Raises:: PermissionError – If the authenticated user cannot access Jobs in this teamspace.

property name: str

The teamspace’s name.

Returns:: The name of this teamspace.
Return type:: str

property owner: Owner

The teamspace’s owner.

Returns:: The owning Owner (user or org).
Return type:: Owner

property secrets: Dict[str, str]

All (encrypted) secrets for the teamspace.

Note

Once created, the secret values are encrypted and cannot be viewed here anymore.

Returns:: A mapping of secret names to their (encrypted) values.
Return type:: Dict[str, str]

property start_studios_on_interruptible: bool

Whether new Studios in this teamspace start on interruptible (spot) instances by default.

Returns:: True if Studios default to interruptible instances.
Return type:: bool

property studios: List[Studio]

All studios within that teamspace.

Returns:: Every Studio belonging to this teamspace.
Return type:: List[Studio]

property tags: Tuple[str, ...]

Every job tag defined in this teamspace.

Returns:: The tag names, as the platform stores them.
Return type:: tuple[str, …]

class lightning_sdk.BucketCredentials(access_key_id, secret_access_key, session_token=None, expires_at=None, region=None, endpoint=None)

Bases: object

Short-lived credentials for the bucket behind a data connection.

These expire — usually within the hour — so they are meant to be fetched when needed rather than held. expires_at is the deadline to refresh against, and is None only for connections backed by static keys, which do not expire.

region and endpoint are part of the answer, not decoration. Inside a Studio the default AWS profile can point at Lightning Storage, and botocore will apply that profile’s endpoint to any client built without one — which sends an AWS request to Cloudflare and gets it rejected as InvalidAccessKeyId. Pass these through rather than letting a client resolve its own.

Parameters:

access_key_id (str)
secret_access_key (str)
session_token (str | None)
expires_at (datetime | None)
region (str | None)
endpoint (str | None)

to_credential_process()

Render as the JSON payload AWS’s credential_process expects.

Any tool that understands that setting — the AWS CLI, boto3, rclone, s5cmd, the Go and Java SDKs — will re-run the process to refresh once Expiration passes, which is what makes the short lifetime survivable without per-tool support.

Expiration is omitted for credentials that never expire, which AWS reads as “cache indefinitely”.

Return type:: Dict[str, Any]
Returns:: The credential payload, ready to be serialised to stdout as JSON.

class lightning_sdk.FileEntry(path, is_dir, size=None, cloud_account=None, last_modified=None)

Bases: object

One entry in a teamspace drive listing.

Parameters:

path (str)
is_dir (bool)
size (int | None)
cloud_account (str | None)
last_modified (datetime | None)

path: Path relative to the folder that was listed.

is_dir: Whether the entry is a folder rather than a file.

size: Size in bytes, or None for folders.

cloud_account: Cloud account holding the file, when the server reports one.

last_modified: When the file was last written, when the server reports it.

property name: str: The entry’s own name, without any leading folders.

class lightning_sdk.SecretType(*values)

Bases: Enum

Type of an encrypted secret.

GENERIC secrets are exposed as environment variables. HF_TOKEN tags a secret as a HuggingFace access token so it can be referenced when deploying gated or private HuggingFace models (Deployment.start(hf_token_secret=...)).

GENERIC = 'generic'

HF_TOKEN = 'hf_token'

class lightning_sdk.ConnectionType(*values)

Bases: Enum

Type of external data source that can be connected to a Teamspace.

EFS = 'EFS'

FILESTORE = 'FILESTORE'

GCS = 'GCS'

S3 = 'S3'

class lightning_sdk.FolderLocation(*values)

Bases: Enum

Cloud provider location for a Teamspace folder.

AWS = 'AWS'

CLOUD_AGNOSTIC = 'CLOUD_AGNOSTIC'

GCP = 'GCP'

Was this page helpful?