4.3 Training a Multilayer Neural Network in PyTorch (PART 1-5)

References

• The official MNIST website: http://yann.lecun.com/exdb/mnist/

Code

• Parts 1-2: XOR dataset, 4.3-mlp-pytorch-part1-2-xor
• Parts 3-5: MNIST dataset, 4.3-mlp-pytorch-part3-5-mnist

What we covered in this video lecture

In this series of coding videos, we trained our first multilayer perceptron in PyTorch.

First, we started with the XOR dataset as a warm-up exercise. Then, we moved to the MNIST handwritten digit classification dataset.

The MNIST dataset consists of 28×28 handwritten digits, which we reshaped into a vector format for our multilayer perceptron model. Since the MNIST dataset is relatively small and simple, we were able to achieve a relatively high prediction accuracy of 96%.

Additional resources if you want to learn more

If you are looking for simple but more challenging datasets, also check out EMNIST dataset, which extends the MNIST dataset with handwritten letters:

• Gregory Cohen, Saeed Afshar, Jonathan Tapson, André van Schaik (2017). EMNIST: An Extension of MNIST to Handwritten Letters, https://arxiv.org/abs/1702.05373

Another popular dataset is Fashion-MNIST, which has the same format as MNIST (28×28 grayscale images from 10 classes) but consists of fashion products:

• Han Xiao, Kashif Rasul, Roland Vollgraf (2017). Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms, https://arxiv.org/abs/1708.07747.

class PyTorchMLP(torch.nn.Module):
    def __init__(self, num_features, num_classes):
        super().__init__()

        self.all_layers = torch.nn.Sequential(
            # 1st hidden layer
            torch.nn.Linear(num_features, 50),
            torch.nn.ReLU(),
            # 2nd hidden layer
            torch.nn.Linear(50, 25),
            torch.nn.ReLU(),
            # output layer
            torch.nn.Linear(25, num_classes),
        )

    def forward(self, x):
        x = torch.flatten(x, start_dim=1)
        logits = self.all_layers(x)
        return logits

Watch Video 1 Mark complete and go to Unit 4.4 →