Shortcuts

Path

class lightning_app.storage.path.Path(*args, **kwargs)[source]

Bases: pathlib.PosixPath

A drop-in replacement for pathlib.Path for all paths in Lightning.

The Lightning Path works exactly the same as pathlib.Path but it also remembers in which LightningWork it was created. If the Path gets passed to a different LightningWork, the file or folder can then be easily accessed no matter where it is located in the other Work’s filesystem.

Parameters

Construct a PurePath from one or several strings and or existing PurePath objects. The strings and path objects are combined so as to yield a canonicalized path, which is incorporated into the new PurePath object.

_attach_queues(request_queue, response_queue)[source]

Attaches the queues for communication with the Storage Orchestrator.

Return type

None

_attach_work(work)[source]

Attach a LightningWork to this Path.

The first work to be attached becomes the origin, i.e., the Work that is meant to expose the file to other Work. Attaching a Work to a Path that already has an origin Work will make it a consumer. A consumer Work is a work that can access the file only by first transferring it via transfer().

Parameters

work (LightningWork) – LightningWork to be attached to this Path.

Return type

None

classmethod _from_parts(args, **_Path__unused)[source]

This gets called from the super class in pathlib.Path.__new__.

The Lightning Path overrides this to validate the instantiation in the case parts are passed in individually. In such a case we need to validate that all parts have the same origin and if not, an error is raised.

Return type

Path

_sanitize()[source]

Sanitize this Path so that it can be deep-copied.

Return type

None

exists()[source]

Check if the path exists locally or remotely.

If the path exists locally, this method immediately returns True, otherwise it will make a RPC call to the attached origin Work and check if the path exists remotely. If you strictly want to check local existence only, use exists_local() instead. If you strictly want to check existence on the remote (regardless of whether the file exists locally or not), use exists_remote().

Return type

bool

exists_local()[source]

Check if the path exists locally.

Return type

bool

exists_remote()[source]

Check if the path exists remotely on the attached orgin Work.

Raises

RuntimeError – If the path is not attached to any Work (origin undefined).

Return type

bool

classmethod from_dict(content)[source]

Instantiate a Path from a dictionary.

Return type

Path

relative_to(*other)[source]

Return the relative path to another path identified by the passed arguments. If the operation is not possible (because this is not a subpath of the other path), raise ValueError.

Return type

Path

to_dict()[source]

Serialize this Path to a dictionary.

Return type

dict

with_name(name)[source]

Return a new path with the file name changed.

Return type

Path

with_stem(stem)[source]

Return a new path with the stem changed.

Return type

Path

with_suffix(suffix)[source]

Return a new path with the file suffix changed. If the path has no suffix, add given suffix. If the given suffix is an empty string, remove the suffix from the path.

Return type

Path

property consumer_name: str[source]

The name of the LightningWork where this path is being accessed.

By default, this is the same as the origin_name.

Return type

str

property hash: Optional[str][source]

The hash of this Path uniquely identifies the file path and the associated origin Work.

Returns None if the origin is not defined, i.e., this Path did not yet get attached to a LightningWork.

Return type

Optional[str]

property origin_name: str[source]

The name of the LightningWork where this path was first created.

Attaching a Path to a LightningWork will automatically make it the origin.

Return type

str

property parent: lightning_app.storage.path.Path[source]

The logical parent of the path.

Return type

Path

property parents: Sequence[lightning_app.storage.path.Path][source]

A sequence of this path’s logical parents.

Return type

Sequence[Path]