Deep Learning Fundamentals
- Deep Learning Fundamentals
- Unit 1Intro to ML and DL
- Unit 2Using Tensors w/ PyTorch
- Unit 3Model Training in PyTorch
- Unit 3.1Using Logistic Regression for Classification
- Unit 3.2The Logistic Regression Computation Graph
- Unit 3.3Model Training with Stochastic Gradient Descent
- Unit 3.4Automatic Differentiation in PyTorch
- Unit 3.5The PyTorch API
- Unit 3.6Training a Logistic Regression Model in PyTorch
- Unit 3.7 Feature Normalization
- Unit 3 ExercisesUnit 3 Exercies
- Unit 4Training Multilayer Neural Networks Overview
- Unit 4.1Logistic Regression for Multiple Classes
- Unit 4.2Multilayer Neural Networks
- Unit 4.3Training a Multilayer Neural Network in PyTorch
- Unit 4.4Defining Efficient Data Loaders
- Unit 4.5Multilayer Neural Networks for Regression
- Unit 4.6Speeding Up Model Training Using GPUs
- Unit 4 ExercisesUnit 4 Exercises
- Unit 5Organizing Your Code with Lightning
- Unit 5.1 Organizing Your Code with Lightning
- Unit 5.2Training a Multilayer Perceptron using the Lightning Trainer
- Unit 5.3Computing Metrics Efficiently with TorchMetrics
- Unit 5.4Making Code Reproducible
- Unit 5.5Organizing Your Data Loaders with Data Modules
- Unit 5.6The Benefits of Logging Your Model Training
- Unit 5.7Evaluating and Using Models on New Data
- Unit 5.8Add Functionality with Callbacks
- Unit 5 ExercisesUnit 5 Exercises
- Unit 6Essential Deep Learning Tips & Tricks
- Unit 6.1 Model Checkpointing and Early Stopping
- Unit 6.2Learning Rates and Learning Rate Schedulers
- Unit 6.3Using More Advanced Optimization Algorithms
- Unit 6.4Choosing Activation Functions
- Unit 6.5Automating The Hyperparameter Tuning Process
- Unit 6.6Improving Convergence with Batch Normalization
- Unit 6.7Reducing Overfitting With Dropout
- Unit 6.8Debugging Deep Neural Networks
- Unit 6 ExercisesUnit 6 Exercises
- Unit 7Getting Started with Computer Vision
- Unit 7.1Working With Images
- Unit 7.2How Convolutional Neural Networks Work
- Unit 7.3Convolutional Neural Network Architectures
- Unit 7.4Training Convolutional Neural Networks
- Unit 7.5Improving Predictions with Data Augmentation
- Unit 7.6Leveraging Pretrained Models with Transfer Learning
- Unit 7.7Using Unlabeled Data with Self-Supervised
- Unit 7 ExercisesUnit 7 Exercises
- Unit 8Natural Language Processing and Large Language Models
- Unit 8.1Working with Text Data
- Unit 8.2Training A Text Classifier Baseline
- Unit 8.3Introduction to Recurrent Neural Networks
- Unit 8.4From RNNs to the Transformer Architecture
- Unit 8.5Understanding Self-Attention
- Unit 8.6Large Language Models
- Unit 8.7A Large Language Model for Classification
- Unit 8 ExercisesUnit 8 Exercises
- Unit 9Techniques for Speeding Up Model Training
- Unit 10 The Finale: Our Next Steps After AI Model Training
Unit 4 Exercises
Exercise 1: Changing the Number of Layers
In this exercise, we are toying around with the multilayer perceptron architecture from Unit 4.3.
In Unit 4.3, we fit the following multilayer perceptron on the MNIST dataset:
class PyTorchMLP(torch.nn.Module):
def __init__(self, num_features, num_classes):
super().__init__()
self.all_layers = torch.nn.Sequential(
# 1st hidden layer
torch.nn.Linear(num_features, 50),
torch.nn.ReLU(),
# 2nd hidden layer
torch.nn.Linear(50, 25),
torch.nn.ReLU(),
# output layer
torch.nn.Linear(25, num_classes),
)
def forward(self, x):
x = torch.flatten(x, start_dim=1)
logits = self.all_layers(x)
return logits
This network had 40,785 parameters (see Quiz on how to calculate this number), and it achieved the following accuracy values:
- Train Acc 97.24%
- Val Acc 95.64%
- Test Acc 96.46%
Can you change the architecture to achieve the same (or better) performance with fewer parameters and only 1 hidden layer?
PS: You may also try to add additional layers, but as a rule of thumb, using more than two hidden layers in a multi-layer perceptron rarely improves the predictive performance.
You can use th notebook in this folder as a template: Unit 4 exercise 1
Exercise 2: Implementing a Custom Dataset Class for Fashion MNIST
In this exercise, we are going to train the multilayer perceptron from Unit 4.3 on a new dataset, Fashion MNIST, based on the PyTorch Dataset concepts introduced in Unit 4.4.
Fashion MNIST is a dataset with a same number of images and image dimension as MNIST. However, instead of handwritten digits, it contains low-resolution images of fashion objects.
Since the image format of MNIST and Fashion MNIST is identical, we can use the multilayer perceptron code from Unit 4.3 without any modification. The only adjustments we have to make is replacing the MNIST dataset code with a custom Dataset
class for Fashion MNIST.
To get started, download the GitHub folder https://github.com/zalandoresearch/fashion-mnist and place the data
subfolder next to the notebook. Then, implement the custom Dataset
class and train the multilayer perceptron. You should get at least >85% training accuracy.
Hint: You may use the following Dataset
code as a starter and fill out the missing blanks:
class MyDataset(Dataset):
def __init__(self, ..., transform=None):
self.transform = transform
# ...
def __getitem__(self, index):
# ...
img = torch.tensor(img).to(torch.float32)
img = img/255.
# ...
if self.transform is not None:
img = self.transform(img)
# ...
def __len__(self):
return self.labels.shape[0]
You can use th notebook in this folder as a template: Unit 4 exercise 2
PS: If you get stuck, please don’t hesitate to reach out for help via the forum!
Log in or create a free Lightning.ai account to access:
- Quizzes
- Completion badges
- Progress tracking
- Additional downloadable content
- Additional AI education resources
- Notifications when new units are released
- Free cloud computing credits
Watch Video 1
Unit 4 Exercises