PyTorch Lightning Tutorials

---

Tutorial 1: Introduction to PyTorch

This tutorial will give a short introduction to PyTorch basics, and get you setup for writing your own neural networks. This notebook is part of a lecture series on Deep...

GPU/TPU,UvA-DL-Course

Tutorial 2: Activation Functions

In this tutorial, we will take a closer look at (popular) activation functions and investigate their effect on optimization properties in neural networks. Activation functions...

GPU/TPU,UvA-DL-Course

Tutorial 3: Initialization and Optimization

In this tutorial, we will review techniques for optimization and initialization of neural networks. When increasing the depth of neural networks, there are various challenges...

Image,Initialization,Optimizers,GPU/TPU,UvA-DL-Course

Tutorial 4: Inception, ResNet and DenseNet

In this tutorial, we will implement and discuss variants of modern CNN architectures. There have been many different architectures been proposed over the past few years. Some...

Image,GPU/TPU,UvA-DL-Course

Tutorial 5: Transformers and Multi-Head Attention

In this tutorial, we will discuss one of the most impactful architectures of the last 2 years: the Transformer model. Since the paper Attention Is All You Need by Vaswani et...

Text,GPU/TPU,UvA-DL-Course

Tutorial 6: Basics of Graph Neural Networks

In this tutorial, we will discuss the application of neural networks on graphs. Graph Neural Networks (GNNs) have recently gained increasing popularity in both applications...

Graph,GPU/TPU,UvA-DL-Course

Tutorial 7: Deep Energy-Based Generative Models

In this tutorial, we will look at energy-based deep learning models, and focus on their application as generative models. Energy models have been a popular tool before the...

Image,GPU/TPU,UvA-DL-Course

Tutorial 8: Deep Autoencoders

In this tutorial, we will take a closer look at autoencoders (AE). Autoencoders are trained on encoding input data such as images into a smaller feature vector, and afterward,...

Image,GPU/TPU,UvA-DL-Course

Tutorial 9: Normalizing Flows for Image Modeling

In this tutorial, we will take a closer look at complex, deep normalizing flows. The most popular, current application of deep normalizing flows is to model datasets of...

Image,GPU/TPU,UvA-DL-Course

Tutorial 10: Autoregressive Image Modeling

In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute...

Image,GPU/TPU,UvA-DL-Course

Tutorial 11: Vision Transformers

In this tutorial, we will take a closer look at a recent new trend: Transformers for Computer Vision. Since [Alexey Dosovitskiy et...

Image,GPU/TPU,UvA-DL-Course

Tutorial 12: Meta-Learning - Learning to Learn

In this tutorial, we will discuss algorithms that learn models which can quickly adapt to new classes and/or tasks with few samples. This area of machine learning is called...

Few-shot-learning,MAML,ProtoNet,GPU/TPU,UvA-DL-Course

Tutorial 13: Self-Supervised Contrastive Learning with SimCLR

In this tutorial, we will take a closer look at self-supervised contrastive learning. Self-supervised learning, or also sometimes called unsupervised learning, describes the...

Image,Self-Supervised,Contrastive-Learning,GPU/TPU,UvA-DL-Course

GPU and batched data augmentation with Kornia and PyTorch-Lightning

In this tutorial we will show how to combine both Kornia.org and PyTorch Lightning to perform efficient data augmentation to train a simpple model using the GPU in batch mode...

Image,GPU/TPU,Lightning-Examples

Barlow Twins Tutorial

This notebook describes the self-supervised learning method Barlow Twins. Barlow Twins differs from other recently proposed algorithms as it doesn't fall under the category of...

Image,Self-Supervised,GPU/TPU,Lightning-Examples

PyTorch Lightning Basic GAN Tutorial

How to train a GAN! Main takeaways: 1. Generator and discriminator are arbitrary PyTorch modules. 2. training_step does both the generator and discriminator training.

Image,GPU/TPU,Lightning-Examples

PyTorch Lightning CIFAR10 ~94% Baseline Tutorial

Train a Resnet to 94% accuracy on Cifar10!

Image,GPU/TPU,Lightning-Examples

PyTorch Lightning DataModules

This notebook will walk you through how to start using Datamodules. With the release of `pytorch-lightning` version 0.9.0, we have included a new class called...

GPU/TPU,Lightning-Examples

Fine-Tuning Scheduler

This notebook introduces the [Fine-Tuning Scheduler](https://finetuning-scheduler.readthedocs.io/en/stable/index.html) extension and demonstrates the use of it to fine-tune a...

Fine-Tuning,GPU/TPU,Lightning-Examples

Introduction to Pytorch Lightning

In this notebook, we'll go over the basics of lightning by preparing models to train on the [MNIST Handwritten Digits dataset](https://en.wikipedia.org/wiki/MNIST_database).

Image,GPU/TPU,Lightning-Examples

TPU training with PyTorch Lightning

In this notebook, we'll train a model on TPUs. Updating one Trainer flag is all you need for that. The most up to documentation related to TPU training can be found...

Image,GPU/TPU,Lightning-Examples

How to train a Deep Q Network

Main takeaways: 1. RL has the same flow as previous models we have seen, with a few additions 2. Handle unsupervised learning by using an IterableDataset where the dataset...

RL,GPU/TPU,Lightning-Examples

Finetune Transformers Models with PyTorch Lightning

This notebook will use HuggingFace's `datasets` library to get data, which will be wrapped in a `LightningDataModule`. Then, we write a class to perform text classification on...

Text,GPU/TPU,Lightning-Examples

Multi-agent Reinforcement Learning With WarpDrive

This notebook introduces multi-agent reinforcement learning (MARL) with WarpDrive (Lan et al. https://arxiv.org/abs/2108.13976). WarpDrive is a flexible, lightweight, and...

Reinforcement-Learning,Multi-agent,GPU,GPU/TPU,Lightning-Examples