Path

class lightning.app.storage.path.Path(*args, **kwargs)

Bases: PosixPath

A drop-in replacement for pathlib.Path for all paths in Lightning.

The Lightning Path works exactly the same as pathlib.Path but it also remembers in which LightningWork it was created. If the Path gets passed to a different LightningWork, the file or folder can then be easily accessed no matter where it is located in the other Work’s filesystem.

Parameters:

• *args – Accepts the same arguments as in pathlib.Path

• **kwargs – Accepts the same keyword arguments as in pathlib.Path

Construct a PurePath from one or several strings and or existing PurePath objects. The strings and path objects are combined so as to yield a canonicalized path, which is incorporated into the new PurePath object.

_attach_queues(request_queue, response_queue)

Attaches the queues for communication with the Storage Orchestrator.

Return type:

None

_attach_work(work)

Attach a LightningWork to this Path.

The first work to be attached becomes the origin, i.e., the Work that is meant to expose the file to other Work. Attaching a Work to a Path that already has an origin Work will make it a consumer. A consumer Work is a work that can access the file only by first transferring it via transfer().

Parameters:

work (LightningWork) – LightningWork to be attached to this Path.

Return type:

None

classmethod _from_parts(args, **_Path__unused)

This gets called from the super class in pathlib.Path.__new__.

The Lightning Path overrides this to validate the instantiation in the case parts are passed in individually. In such a case we need to validate that all parts have the same origin and if not, an error is raised.

Return type:

Path

_sanitize()

Sanitize this Path so that it can be deep-copied.

Return type:

None

exists()

Check if the path exists locally or remotely.

If the path exists locally, this method immediately returns True, otherwise it will make a RPC call to the attached origin Work and check if the path exists remotely. If you strictly want to check local existence only, use exists_local() instead. If you strictly want to check existence on the remote (regardless of whether the file exists locally or not), use exists_remote().

Return type:

bool

exists_local()

Check if the path exists locally.

Return type:

bool

exists_remote()

Check if the path exists remotely on the attached orgin Work.

Raises:

RuntimeError – If the path is not attached to any Work (origin undefined).

Return type:

bool

classmethod from_dict(content)

Instantiate a Path from a dictionary.

Return type:

Path

relative_to(*other)

Return the relative path to another path identified by the passed arguments. If the operation is not possible (because this is not a subpath of the other path), raise ValueError.

Return type:

Path

to_dict()

Serialize this Path to a dictionary.

Return type:

dict

with_name(name)

Return a new path with the file name changed.

Return type:

Path

with_stem(stem)

Return a new path with the stem changed.

Return type:

Path

with_suffix(suffix)

Return a new path with the file suffix changed. If the path has no suffix, add given suffix. If the given suffix is an empty string, remove the suffix from the path.

Return type:

Path

property consumer_name: str

The name of the LightningWork where this path is being accessed.

By default, this is the same as the origin_name.

property hash: Optional[str]

The hash of this Path uniquely identifies the file path and the associated origin Work.

Returns None if the origin is not defined, i.e., this Path did not yet get attached to a LightningWork.

property origin_name: str

The name of the LightningWork where this path was first created.

Attaching a Path to a LightningWork will automatically make it the origin.

property parent: Path

The logical parent of the path.

property parents: Sequence[Path]

A sequence of this path’s logical parents.