Get Started
Blog
Docs
PyTorch Lightning Lightning Fabric TorchMetrics Lightning Flash Lightning Bolts
Previous Versions
GitHub
Lightning AI

Table of Contents

Version:04/20/2023

Get Started

Lightning in 15 minutes
Installation
Guide how to upgrade to the 2.0 version

Level Up

Basic skills
Intermediate skills
Advanced skills
Expert skills

Core API

LightningModule
Trainer

API Reference

accelerators
callbacks
cli
core
loggers
profiler
trainer
strategies
tuner
utilities

Common Workflows

Avoid overfitting
Build a Model
Configure hyperparameters from the CLI
Customize the progress bar
Deploy models into production
Effective Training Techniques
Find bottlenecks in your code
Manage experiments
Organize existing PyTorch into Lightning
Run on an on-prem cluster
Save and load model progress
Save memory with half-precision
Train 1 trillion+ parameter models
Train on single or multiple GPUs
Train on single or multiple HPUs
Train on single or multiple IPUs
Train on single or multiple TPUs
Train on MPS
Use a pretrained model
Complex data uses
Use a pure PyTorch training loop

Glossary

Hands-on Examples

Tutorial 1: Introduction to PyTorch
Tutorial 2: Activation Functions
Tutorial 3: Initialization and Optimization
Tutorial 4: Inception, ResNet and DenseNet
Tutorial 5: Transformers and Multi-Head Attention
Tutorial 6: Basics of Graph Neural Networks
Tutorial 7: Deep Energy-Based Generative Models
Tutorial 8: Deep Autoencoders
Tutorial 9: Normalizing Flows for Image Modeling
Tutorial 10: Autoregressive Image Modeling
Tutorial 11: Vision Transformers
Tutorial 12: Meta-Learning - Learning to Learn
Tutorial 13: Self-Supervised Contrastive Learning with SimCLR
GPU and batched data augmentation with Kornia and PyTorch-Lightning
Barlow Twins Tutorial
PyTorch Lightning Basic GAN Tutorial
PyTorch Lightning CIFAR10 ~94% Baseline Tutorial
PyTorch Lightning DataModules
Fine-Tuning Scheduler
Introduction to PyTorch Lightning
TPU training with PyTorch Lightning
How to train a Deep Q Network
Finetune Transformers Models with PyTorch Lightning
Multi-agent Reinforcement Learning With WarpDrive

Community

Contributor Covenant Code of Conduct
Contributing
How to Become a Core Contributor
Lightning Governance
Versioning Policy
Past PyTorch Lightning versions
Changelog

Docs >
accelerators >
distributed

Shortcuts

distributed

Utilities that can be used with distributed training.

Next

Previous

© Copyright Copyright (c) 2018-2023, Lightning AI et al...

Built with Sphinx using a theme provided by Read the Docs.

distributed

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. Read PyTorch Lightning's Privacy Policy.

Get Started
Blog
Ecosystem

PyTorch Lightning
Lightning Fabric
TorchMetrics
Fabric

Github
Lightning.ai

You are viewing an outdated version of PyTorch Lightning Docs

Click here to view the latest version→