Lightning AI Studios: Never set up a local environment again →

Log in or create a free Lightning.ai account to track your progress and access additional course materials  

4.3 Training a Multilayer Neural Network in PyTorch (PART 1-5)

References

Code

What we covered in this video lecture

In this series of coding videos, we trained our first multilayer perceptron in PyTorch.

First, we started with the XOR dataset as a warm-up exercise. Then, we moved to the MNIST handwritten digit classification dataset.

The MNIST dataset consists of 28×28 handwritten digits, which we reshaped into a vector format for our multilayer perceptron model. Since the MNIST dataset is relatively small and simple, we were able to achieve a relatively high prediction accuracy of 96%.

Additional resources if you want to learn more

If you are looking for simple but more challenging datasets, also check out EMNIST dataset, which extends the MNIST dataset with handwritten letters:

  • Gregory Cohen, Saeed Afshar, Jonathan Tapson, André van Schaik (2017). EMNIST: An Extension of MNIST to Handwritten Lettershttps://arxiv.org/abs/1702.05373

Another popular dataset is Fashion-MNIST, which has the same format as MNIST (28×28 grayscale images from 10 classes) but consists of fashion products:

  • Han Xiao, Kashif Rasul, Roland Vollgraf (2017). Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmshttps://arxiv.org/abs/1708.07747.

Log in or create a free Lightning.ai account to access:

  • Quizzes
  • Completion badges
  • Progress tracking
  • Additional downloadable content
  • Additional AI education resources
  • Notifications when new units are released
  • Free cloud computing credits

Quiz: 4.3 Training a Multilayer Perceptron in PyTorch

Suppose we implemented the following multilayer perceptron architecture for a 2-dimensional dataset with 3 classes:

class PyTorchMLP(torch.nn.Module):
    def __init__(self, num_features, num_classes):
        super().__init__()

        self.all_layers = torch.nn.Sequential(
            # 1st hidden layer
            torch.nn.Linear(num_features, 50),
            torch.nn.ReLU(),
            # 2nd hidden layer
            torch.nn.Linear(50, 25),
            torch.nn.ReLU(),
            # output layer
            torch.nn.Linear(25, num_classes),
        )

    def forward(self, x):
        x = torch.flatten(x, start_dim=1)
        logits = self.all_layers(x)
        return logits

How many parameters does this neural network approximately have?

Correct. The exact number is 1503. layer 1: 2*50+50, layer 2: 50*25+25, output layer: 25*3+3.

Incorrect. To calculate the number of parameters per layer, multiply the number of inputs and outputs, and add the bias units. Do this for all 3 layers.

Incorrect. To calculate the number of parameters per layer, multiply the number of inputs and outputs, and add the bias units. Do this for all 3 layers.

Incorrect. To calculate the number of parameters per layer, multiply the number of inputs and outputs, and add the bias units. Do this for all 3 layers.

Please answer all questions to proceed.
Watch Video 1

Unit 4.3

Videos