Get Started
Blog
Docs
PyTorch Lightning Lightning Fabric TorchMetrics Lightning Flash Lightning Bolts
Previous Versions
GitHub
Lightning AI

Table of Contents

1.9.6

Get Started

Lightning in 15 minutes
Installation
Guide how to upgrade to the 2.0 version

Level Up

Basic skills
Intermediate skills
Advanced skills
Expert skills

Core API

LightningModule
Trainer
Fabric (Beta)

API Reference

accelerators
callbacks
cli
core
loggers
profiler
trainer
strategies
tuner
utilities

Common Workflows

Avoid overfitting
Build a Model
Configure hyperparameters from the CLI
Customize the progress bar
Deploy models into production
Effective Training Techniques
Find bottlenecks in your code
Manage experiments
Organize existing PyTorch into Lightning
Run on an on-prem cluster
Save and load model progress
Save memory with half-precision
Training over the internet
Train 1 trillion+ parameter models
Train on single or multiple GPUs
Train on single or multiple HPUs
Train on single or multiple IPUs
Train on single or multiple TPUs
Train on MPS
Use a pretrained model
Inject Custom Data Iterables
Use a pure PyTorch training loop

Glossary

Hands-on Examples

Tutorial 1: Introduction to PyTorch
Tutorial 2: Activation Functions
Tutorial 3: Initialization and Optimization
Tutorial 4: Inception, ResNet and DenseNet
Tutorial 5: Transformers and Multi-Head Attention
Tutorial 6: Basics of Graph Neural Networks
Tutorial 7: Deep Energy-Based Generative Models
Tutorial 8: Deep Autoencoders
Tutorial 9: Normalizing Flows for Image Modeling
Tutorial 10: Autoregressive Image Modeling
Tutorial 11: Vision Transformers
Tutorial 12: Meta-Learning - Learning to Learn
Tutorial 13: Self-Supervised Contrastive Learning with SimCLR
GPU and batched data augmentation with Kornia and PyTorch-Lightning
Barlow Twins Tutorial
PyTorch Lightning Basic GAN Tutorial
PyTorch Lightning CIFAR10 ~94% Baseline Tutorial
PyTorch Lightning DataModules
Fine-Tuning Scheduler
Introduction to Pytorch Lightning
TPU training with PyTorch Lightning
How to train a Deep Q Network
Finetune Transformers Models with PyTorch Lightning
Multi-agent Reinforcement Learning With WarpDrive
PyTorch Lightning 101 class
From PyTorch to PyTorch Lightning [Blog]
From PyTorch to PyTorch Lightning [Video]

Community

Contributor Covenant Code of Conduct
Contributing
How to Become a Core Contributor
Lightning Governance
Versioning Policy
Past PyTorch Lightning versions
Changelog

Docs >
SingleDeviceCollective

Shortcuts

SingleDeviceCollective

class lightning_fabric.plugins.collectives.SingleDeviceCollective[source]: Bases: lightning_fabric.plugins.collectives.collective.Collective

© Copyright Copyright (c) 2018-2023, Lightning AI et al...

Built with Sphinx using a theme provided by Read the Docs.

SingleDeviceCollective

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. Read PyTorch Lightning's Privacy Policy.

Get Started
Blog
Ecosystem

PyTorch Lightning
Lightning Fabric
TorchMetrics
Fabric

Github
Lightning.ai