Deployment

Deploy autoscaling inference APIs.

Usage:

lightning deployment [OPTIONS] COMMAND [ARGS]...

Commands

Command	Description
`create`	Create a deployment.
`delete`	Delete a deployment.
`env`	Manage Deployment environment variables.
`inspect`	Inspect a deployment as JSON.
`list`	List deployments in a teamspace.
`logs`	View the logs for a deployment.
`reload-weights`	Reload weights on a running BYOM deployment.
`secret`	Manage Deployment secret references.
`update`	Update a deployment.

lightning deployment create

Create a deployment.

Usage:

lightning deployment create [OPTIONS] [NAME]

Arguments

Name	Type	Required	Description
`NAME`	`text`	no	Optional deployment name. Lightning generates one if omitted.

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	no	`Sentinel.UNSET`	The deployment name. Overrides the optional NAME argument.
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--image`	`TEXT`	no	`Sentinel.UNSET`	Docker image to deploy. Mutually exclusive with –studio.
`--studio`	`TEXT`	no	`Sentinel.UNSET`	Studio to deploy from. Mutually exclusive with –image.
`--machine`	`[CPU_SMALL\|CPU\|CPU_X_2\|CPU_X_4\|CPU_X_8\|CPU_X_16\|DATA_PREP\|DATA_PREP_MAX\|DATA_PREP_ULTRA\|T4_SMALL\|T4\|T4_X_2\|T4_X_4\|T4_X_8\|L4\|L4_X_2\|L4_X_4\|L4_X_8\|L40S\|L40S_X_2\|L40S_X_4\|L40S_X_8\|RTXP_6000\|RTXP_6000_X_2\|RTXP_6000_X_4\|RTXP_6000_X_8\|A100\|A100_X_2\|A100_X_4\|A100_X_8\|H100\|H100_X_2\|H100_X_4\|H100_X_8\|H200\|H200_X_2\|H200_X_4\|H200_X_8\|B200\|B200_X_8]`	no	`'CPU'`	The machine type to run replicas on.
`--port`	`FLOAT`	no	`Sentinel.UNSET`	Port to expose. Can be repeated.
`--command`	`TEXT`	no	`Sentinel.UNSET`	Container command.
`--entrypoint`	`TEXT`	no	`Sentinel.UNSET`	Container entrypoint. Omit to use the image default.
`--replicas`	`INTEGER`	no	`Sentinel.UNSET`	Fixed number of replicas to start.
`--min-replicas`	`INTEGER`	no	`Sentinel.UNSET`	Minimum autoscaling replicas.
`--max-replicas`	`INTEGER`	no	`Sentinel.UNSET`	Maximum autoscaling replicas.
`--autoscale-metric`	`[CPU\|GPU\|RPM]`	no	`Sentinel.UNSET`	Autoscaling metric.
`--autoscale-threshold`	`FLOAT`	no	`Sentinel.UNSET`	Autoscaling threshold.
`--cloud`	`TEXT`	no	`Sentinel.UNSET`	Cloud provider or cloud account to run replicas on.
`--env`, `-e`	`TEXT`	no	`['']`	Environment variable in KEY=VALUE or JSON format.
`--secret`	`TEXT`	no	`Sentinel.UNSET`	Secret to expose as an environment variable. Use SECRET to inject under the secret name, or ENV_NAME=SECRET to inject under a different env var name.
`--interruptible`, `--no-interruptible`	—	no	—	Whether to use interruptible instances.
`--quantity`	`INTEGER`	no	`Sentinel.UNSET`	Number of machines per replica.
`--include-credentials`, `--no-include-credentials`	—	no	—	Inject SDK credentials.
`--custom-domain`	`TEXT`	no	`Sentinel.UNSET`	Custom domain to attach to the deployment.
`--basic-auth`	`TEXT`	no	`Sentinel.UNSET`	Require basic auth on the endpoint, in USERNAME:PASSWORD format.
`--token-auth`	`TEXT`	no	`Sentinel.UNSET`	Require bearer token auth on the endpoint.
`--path-mapping`, `--path_mapping`	`TEXT`	no	`['']`	Map a container path to a data connection path. Can be repeated.
`--path-mappings`, `--path_mappings`	`TEXT`	no	`''`	Comma-separated path mappings.
`--max-runtime`	`INTEGER`	no	`Sentinel.UNSET`	Maximum machine allocation duration in seconds.
`--model`	`TEXT`	no	`Sentinel.UNSET`	HuggingFace model id to serve. Mutually exclusive with –image/–studio.
`--hf-token-secret`	`TEXT`	no	`Sentinel.UNSET`	Name of the Lightning secret holding your HuggingFace token (gated/private models).
`--serving-image-variant`	`TEXT`	no	`Sentinel.UNSET`	vLLM serving image variant. Defaults to ‘stable’.
`--tensor-parallel-size`	`INTEGER`	no	`Sentinel.UNSET`	vLLM tensor-parallel size.
`--max-model-len`	`INTEGER`	no	`Sentinel.UNSET`	Maximum model context length.
`--gpu-memory-utilization`	`FLOAT`	no	`Sentinel.UNSET`	Fraction of GPU memory vLLM may use (0-1).
`--quantization`	`TEXT`	no	`Sentinel.UNSET`	Quantization method (e.g. fp8, awq).
`--dtype`	`TEXT`	no	`Sentinel.UNSET`	Model weight dtype (e.g. bfloat16).
`--vllm-arg`	`TEXT`	no	`Sentinel.UNSET`	Extra raw vLLM arg, repeatable (e.g. –vllm-arg –enable-chunked-prefill).
`--enable-weight-reload`, `--no-enable-weight-reload`	—	no	—	Enable hot weight reload (HuggingFace auto-detect + user-triggered) without redeploying.
`--ack`	`TEXT`	no	`Sentinel.UNSET`	Acknowledge a model validation warning by code (repeatable).
`--force`	—	no	—	Acknowledge all model validation warnings and deploy anyway.
`--dry-run`	—	no	—	Print the resolved model configuration without creating the deployment.
`--json`	—	no	—	Output the created deployment as JSON.

lightning deployment delete

Delete a deployment.

Usage:

lightning deployment delete [OPTIONS] NAME

Arguments

Name	Type	Required	Description
`NAME`	`text`	yes	Deployment name to delete.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--yes`, `-y`	—	no	—	Delete without prompting for confirmation.

lightning deployment env

Manage Deployment environment variables.

Usage:

lightning deployment env [OPTIONS] COMMAND [ARGS]...

Commands

Command	Description
`delete`	Delete one directly configured Deployment environment variable.
`list`	List directly configured Deployment environment variables.
`set`	Set one Deployment environment variable from KEY=VALUE.

lightning deployment env delete

Delete one directly configured Deployment environment variable.

Usage:

lightning deployment env delete [OPTIONS] KEY

Arguments

Name	Type	Required	Description
`KEY`	`text`	yes	No description provided.

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment env list

List directly configured Deployment environment variables.

Usage:

lightning deployment env list [OPTIONS]

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment env set

Set one Deployment environment variable from KEY=VALUE.

Usage:

lightning deployment env set [OPTIONS] ASSIGNMENT

Arguments

Name	Type	Required	Description
`ASSIGNMENT`	`text`	yes	No description provided.

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment inspect

Inspect a deployment as JSON.

Usage:

lightning deployment inspect [OPTIONS] NAME

Arguments

Name	Type	Required	Description
`NAME`	`text`	yes	Deployment name to inspect.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--jobs`	—	no	—	Include jobs for this deployment.
`--json`	—	no	—	Output as JSON (inspect always emits JSON).

lightning deployment list

List deployments in a teamspace.

Usage:

lightning deployment list [OPTIONS]

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--all`	—	no	—	List deployments in all teamspaces visible to the selected owner.
`--sort-by`	`[name\|teamspace\|state\|replicas\|ready\|pending\|failing\|machine\|source\|cloud-account]`	no	—	The attribute to sort deployments by.
`--json`	—	no	—	Output as JSON.

lightning deployment logs

View the logs for a deployment.

Usage:

lightning deployment logs [OPTIONS] [COMMAND] [ARGS]...

Commands

Command	Description
`download`	Download the complete logs for a deployment replica.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--job-id`	`TEXT`	no	`Sentinel.UNSET`	Specific deployment job ID. Can be repeated.
`--since`	`TEXT`	no	`Sentinel.UNSET`	Only include lines at or after this time (e.g. “2h”, RFC3339).
`--until`	`TEXT`	no	`Sentinel.UNSET`	Only include lines at or before this time (e.g. “30m”, RFC3339).
`--query`	`TEXT`	no	`Sentinel.UNSET`	Only include lines containing every whitespace-separated term.
`--severity`	`[error\|warning\|info\|debug]`	no	`Sentinel.UNSET`	Only include lines at or above this severity.
`--rank`	`INTEGER`	no	`Sentinel.UNSET`	Machine (0-based) to read within a multi-machine replica.
`--follow`, `-f`	—	no	—	Stream new log lines as they are produced.
`--tail`	`INTEGER`	no	`Sentinel.UNSET`	Only show the last N lines. Defaults to 100.
`--timestamps`	—	no	—	Prepend each line with its ISO-8601 timestamp.
`--json`	—	no	—	Output entries as a JSON array.
`--interactive`, `-i`	—	no	—	Launch the interactive TUI log viewer.

lightning deployment logs download

Download the complete logs for a deployment replica.

Usage:

lightning deployment logs download [OPTIONS] NAME

Arguments

Name	Type	Required	Description
`NAME`	`text`	yes	No description provided.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--job-id`	`TEXT`	no	`Sentinel.UNSET`	Specific deployment job ID. Can be repeated.
`--timestamps`	—	no	—	Prepend each line with its ISO-8601 timestamp.

lightning deployment reload-weights

Reload weights on a running BYOM deployment.

Usage:

lightning deployment reload-weights [OPTIONS] NAME

Arguments

Name	Type	Required	Description
`NAME`	`text`	yes	Deployment name whose weights should be reloaded.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).

lightning deployment secret

Manage Deployment secret references.

Usage:

lightning deployment secret [OPTIONS] COMMAND [ARGS]...

Commands

Command	Description
`delete`	Remove a teamspace secret reference by its injected environment variable name.
`list`	List teamspace secrets referenced by the Deployment.
`set`	Reference a teamspace secret.

lightning deployment secret delete

Remove a teamspace secret reference by its injected environment variable name.

Usage:

lightning deployment secret delete [OPTIONS] ENV_NAME

Arguments

Name	Type	Required	Description
`ENV_NAME`	`text`	yes	No description provided.

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment secret list

List teamspace secrets referenced by the Deployment.

Usage:

lightning deployment secret list [OPTIONS]

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment secret set

Reference a teamspace secret. Use SECRET, or ENV_NAME=SECRET to inject under a different env var name.

Usage:

lightning deployment secret set [OPTIONS] ASSIGNMENT

Arguments

Name	Type	Required	Description
`ASSIGNMENT`	`text`	yes	No description provided.

Options

Option	Type	Required	Default	Description
`--name`	`TEXT`	yes	`Sentinel.UNSET`	Deployment name.
`--teamspace`	`TEXT`	no	—	The teamspace to use, as a bare name or an ‘owner/teamspace’ slug. Defaults to the current teamspace.
`--org`	`TEXT`	no	—	The organization owning the teamspace (if any). Defaults to the current organization.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--user`	`TEXT`	no	—	The user owning the teamspace (if any). Defaults to the current user.(DEPRECATED: Pass the owner as part of –teamspace instead, e.g. ‘–teamspace owner/teamspace’. This option will be removed in a future release.)
`--json`	—	no	—	Output as JSON.

lightning deployment update

Update a deployment.

Usage:

lightning deployment update [OPTIONS] NAME

Arguments

Name	Type	Required	Description
`NAME`	`text`	yes	Deployment name to update.

Options

Option	Type	Required	Default	Description
`--teamspace`	`TEXT`	no	`Sentinel.UNSET`	Override default teamspace (format: owner/teamspace).
`--new-name`	`TEXT`	no	`Sentinel.UNSET`	Rename the deployment.
`--image`	`TEXT`	no	`Sentinel.UNSET`	New docker image.
`--machine`	`[CPU_SMALL\|CPU\|CPU_X_2\|CPU_X_4\|CPU_X_8\|CPU_X_16\|DATA_PREP\|DATA_PREP_MAX\|DATA_PREP_ULTRA\|T4_SMALL\|T4\|T4_X_2\|T4_X_4\|T4_X_8\|L4\|L4_X_2\|L4_X_4\|L4_X_8\|L40S\|L40S_X_2\|L40S_X_4\|L40S_X_8\|RTXP_6000\|RTXP_6000_X_2\|RTXP_6000_X_4\|RTXP_6000_X_8\|A100\|A100_X_2\|A100_X_4\|A100_X_8\|H100\|H100_X_2\|H100_X_4\|H100_X_8\|H200\|H200_X_2\|H200_X_4\|H200_X_8\|B200\|B200_X_8]`	no	`Sentinel.UNSET`	New machine type for replicas.
`--command`	`TEXT`	no	`Sentinel.UNSET`	New container command.
`--entrypoint`	`TEXT`	no	`Sentinel.UNSET`	New container entrypoint.
`--replicas`	`INTEGER`	no	`Sentinel.UNSET`	New fixed replica count.
`--min-replicas`	`INTEGER`	no	`Sentinel.UNSET`	New minimum autoscaling replicas.
`--max-replicas`	`INTEGER`	no	`Sentinel.UNSET`	New maximum autoscaling replicas.
`--port`	`FLOAT`	no	`Sentinel.UNSET`	Replacement exposed port. Can be repeated.
`--cloud`	`TEXT`	no	`Sentinel.UNSET`	New cloud provider or cloud account to run replicas on.
`--env`, `-e`	`TEXT`	no	`['']`	Replacement env var in KEY=VALUE or JSON format.
`--secret`	`TEXT`	no	`Sentinel.UNSET`	Replacement secret to expose as an environment variable. Use SECRET to inject under the secret name, or ENV_NAME=SECRET to inject under a different env var name.
`--interruptible`, `--no-interruptible`	—	no	—	Whether to use interruptible instances.
`--quantity`	`INTEGER`	no	`Sentinel.UNSET`	New number of machines per replica.
`--include-credentials`, `--no-include-credentials`	—	no	—	Inject SDK credentials.
`--custom-domain`	`TEXT`	no	`Sentinel.UNSET`	New custom domain.
`--basic-auth`	`TEXT`	no	`Sentinel.UNSET`	Require basic auth on the endpoint, in USERNAME:PASSWORD format.
`--token-auth`	`TEXT`	no	`Sentinel.UNSET`	Require bearer token auth on the endpoint.
`--path-mapping`, `--path_mapping`	`TEXT`	no	`['']`	Replacement path mapping. Can be repeated.
`--path-mappings`, `--path_mappings`	`TEXT`	no	`''`	Comma-separated replacement path mappings.
`--max-runtime`	`INTEGER`	no	`Sentinel.UNSET`	New maximum machine allocation duration in seconds.
`--max-surge`	`INTEGER`	no	`1`	Rolling update max surge.
`--max-unavailable`	`INTEGER`	no	`0`	Rolling update max unavailable.

Was this page helpful?