---
title: Remote filesystem and FSSPEC
description: >-
  PyTorch Lightning enables working with data from a variety of filesystems,
  including local filesystems and several cloud storage providers such as
canonical: 'https://lightning.ai/docs/pytorch/stable/reference/common/remote_fs'
---
# Remote filesystem and FSSPEC {#remote-filesystem-and-fsspec}

PyTorch Lightning enables working with data from a variety of filesystems, including local filesystems and several cloud storage providers such as
[S3](https://aws.amazon.com/s3/) on [AWS](https://aws.amazon.com/), [GCS](https://cloud.google.com/storage) on [Google Cloud](https://cloud.google.com/),
or [ADL](https://azure.microsoft.com/solutions/data-lake/) on [Azure](https://azure.microsoft.com/).

This applies to saving and writing checkpoints, as well as for logging.
Working with different filesystems can be accomplished by appending a protocol like "s3:/" to file paths for writing and reading data.

```python
# `default_root_dir` is the default path used for logs and checkpoints
trainer = Trainer(default_root_dir="s3://my_bucket/data/")
trainer.fit(model)
```

For logging, remote filesystem support depends on the particular logger integration being used. Consult [the documentation of the individual logger](/docs/pytorch/2.6.6/optional-api/api_references.md#loggers) for more details.

```python
from lightning.pytorch.loggers import TensorBoardLogger

logger = TensorBoardLogger(save_dir="s3://my_bucket/logs/")

trainer = Trainer(logger=logger)
trainer.fit(model)
```

Additionally, you could also resume training with a checkpoint stored at a remote filesystem.

```python
trainer = Trainer(default_root_dir=tmpdir, max_steps=3)
trainer.fit(model, ckpt_path="s3://my_bucket/ckpts/classifier.ckpt")
```

PyTorch Lightning uses [fsspec](https://filesystem-spec.readthedocs.io/) internally to handle all filesystem operations.

The most common filesystems supported by Lightning are:

* Local filesystem: `file://` - It's the default and doesn't need any protocol to be used. It's installed by default in Lightning.
* Amazon S3: `s3://` - Amazon S3 remote binary store, using the library [s3fs](https://s3fs.readthedocs.io/). Run `pip install fsspec[s3]` to install it.
* Google Cloud Storage: `gcs://` or `gs://` - Google Cloud Storage, using [gcsfs](https://gcsfs.readthedocs.io/en/stable/). Run `pip install fsspec[gcs]` to install it.
* Microsoft Azure Storage: `adl://`, `abfs://` or `az://` - Microsoft Azure Storage, using [adlfs](https://github.com/fsspec/adlfs). Run `pip install fsspec[adl]` to install it.
* Hadoop File System: `hdfs://` - Hadoop Distributed File System. This uses [PyArrow](https://arrow.apache.org/docs/python/) as the backend. Run `pip install fsspec[hdfs]` to install it.

You could learn more about the available filesystems with:

```python
from fsspec.registry import known_implementations

print(known_implementations)
```

You could also look into [CheckpointIO Plugin](/docs/pytorch/2.6.6/reference/common/checkpointing_expert.md) for more details on how to customize saving and loading checkpoints.

---

**Navigation**

- Parent: [Common](/docs/pytorch/2.6.6/reference/common.md)