PyTorch Lightning TutorialsΒΆ All Tutorial 1: Introduction to PyTorch This tutorial will give a short introduction to PyTorch basics, and get you setup for writing your own neural networks. This notebook is part of a lecture series on Deep... GPU/TPU,UvA-DL-Course Tutorial 2: Activation Functions In this tutorial, we will take a closer look at (popular) activation functions and investigate their effect on optimization properties in neural networks. Activation functions... GPU/TPU,UvA-DL-Course Tutorial 3: Initialization and Optimization In this tutorial, we will review techniques for optimization and initialization of neural networks. When increasing the depth of neural networks, there are various challenges... Image,Initialization,Optimizers,GPU/TPU,UvA-DL-Course Tutorial 4: Inception, ResNet and DenseNet In this tutorial, we will implement and discuss variants of modern CNN architectures. There have been many different architectures been proposed over the past few years. Some... Image,GPU/TPU,UvA-DL-Course Tutorial 5: Transformers and Multi-Head Attention In this tutorial, we will discuss one of the most impactful architectures of the last 2 years: the Transformer model. Since the paper Attention Is All You Need by Vaswani et... Text,GPU/TPU,UvA-DL-Course Tutorial 6: Basics of Graph Neural Networks In this tutorial, we will discuss the application of neural networks on graphs. Graph Neural Networks (GNNs) have recently gained increasing popularity in both applications... Graph,GPU/TPU,UvA-DL-Course Tutorial 7: Deep Energy-Based Generative Models In this tutorial, we will look at energy-based deep learning models, and focus on their application as generative models. Energy models have been a popular tool before the... Image,GPU/TPU,UvA-DL-Course Tutorial 8: Deep Autoencoders In this tutorial, we will take a closer look at autoencoders (AE). Autoencoders are trained on encoding input data such as images into a smaller feature vector, and afterward,... Image,GPU/TPU,UvA-DL-Course Tutorial 9: Normalizing Flows for Image Modeling In this tutorial, we will take a closer look at complex, deep normalizing flows. The most popular, current application of deep normalizing flows is to model datasets of... Image,GPU/TPU,UvA-DL-Course Tutorial 10: Autoregressive Image Modeling In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute... Image,GPU/TPU,UvA-DL-Course Tutorial 11: Vision Transformers In this tutorial, we will take a closer look at a recent new trend: Transformers for Computer Vision. Since [Alexey Dosovitskiy et... Image,GPU/TPU,UvA-DL-Course Tutorial 12: Meta-Learning - Learning to Learn In this tutorial, we will discuss algorithms that learn models which can quickly adapt to new classes and/or tasks with few samples. This area of machine learning is called... Few-shot-learning,MAML,ProtoNet,GPU/TPU,UvA-DL-Course Tutorial 13: Self-Supervised Contrastive Learning with SimCLR In this tutorial, we will take a closer look at self-supervised contrastive learning. Self-supervised learning, or also sometimes called unsupervised learning, describes the... Image,Self-Supervised,Contrastive-Learning,GPU/TPU,UvA-DL-Course GPU and batched data augmentation with Kornia and PyTorch-Lightning In this tutorial we will show how to combine both Kornia and PyTorch Lightning to perform efficient data augmentation to train a simple model using the GPU in batch mode... Image,GPU/TPU,Lightning-Examples Barlow Twins Tutorial This notebook describes the self-supervised learning method Barlow Twins. Barlow Twins differs from other recently proposed algorithms as it doesn't fall under the category of... Image,Self-Supervised,GPU/TPU,Lightning-Examples PyTorch Lightning Basic GAN Tutorial How to train a GAN! Main takeaways: 1. Generator and discriminator are arbitrary PyTorch modules. 2. training_step does both the generator and discriminator training. Image,GPU/TPU,Lightning-Examples PyTorch Lightning CIFAR10 ~94% Baseline Tutorial Train a Resnet to 94% accuracy on Cifar10! Image,GPU/TPU,Lightning-Examples PyTorch Lightning DataModules This notebook will walk you through how to start using Datamodules. With the release of `pytorch-lightning` version 0.9.0, we have included a new class called... GPU/TPU,Lightning-Examples Fine-Tuning Scheduler This notebook introduces the [Fine-Tuning Scheduler](https://finetuning-scheduler.readthedocs.io/en/stable/index.html) extension and demonstrates the use of it to fine-tune a... Fine-Tuning,GPU/TPU,Lightning-Examples Introduction to PyTorch Lightning In this notebook, we'll go over the basics of lightning by preparing models to train on the [MNIST Handwritten Digits dataset](https://en.wikipedia.org/wiki/MNIST_database). Image,GPU/TPU,Lightning-Examples TPU training with PyTorch Lightning In this notebook, we'll train a model on TPUs. Updating one Trainer flag is all you need for that. The most up to documentation related to TPU training can be found... Image,GPU/TPU,Lightning-Examples How to train a Deep Q Network Main takeaways: 1. RL has the same flow as previous models we have seen, with a few additions 2. Handle unsupervised learning by using an IterableDataset where the dataset... RL,GPU/TPU,Lightning-Examples Finetune Transformers Models with PyTorch Lightning This notebook will use HuggingFace's `datasets` library to get data, which will be wrapped in a `LightningDataModule`. Then, we write a class to perform text classification on... Text,GPU/TPU,Lightning-Examples Multi-agent Reinforcement Learning With WarpDrive This notebook introduces multi-agent reinforcement learning (MARL) with WarpDrive (Lan et al. https://arxiv.org/abs/2108.13976). WarpDrive is a flexible, lightweight, and... Reinforcement-Learning,Multi-agent,GPU,GPU/TPU,Lightning-Examples