Lightning AI Studios: Never set up a local environment again →

Log in or create a free Lightning.ai account to track your progress and access additional course materials  

Deep Learning Fundamentals

  • Pages

    • Final certification exam

    Deep Learning Fundamentals > Unit 3 > Unit 3.5
    Course Progress:

    3.5 The PyTorch API (Parts 1-2)

    Slides

    What we covered in this video lecture

    This lecture went from a logistic regression computation graph to the PyTorch API. In Unit 2, we introduced PyTorch’s basic features: tensors. We are now stepping up our game, introducing the PyTorch Module API, which lets us define neural network models.

    You may have noticed that we define a forward method when we use PyTorch’s Module API. In the PyTorch Module context, it’s a unique method of the Module API that will implement a backward method automatically for us (we don’t see it because it happens behind the scenes.)

    Why is this useful? Using the Module class comes with certain benefits. If we use it, we can use the loss.backward() call in our training loop together with optimizer.step(). The .backward() method computes all the gradients for us (which can be pretty complicated, as we have seen in the previous lecture). Then, using the .step() method will use the loss gradients to update the model weights automatically for us.

    In this lecture, you learned the fundamental concepts of the Module API and the PyTorch training loop. Congratulations, you just learned about the most fundamental ideas behind training neural networks in PyTorch. These same concepts also apply to deep neural networks!

    Additional resources if you want to learn more

    This lecture covered the basic capabilities and usage of the torch.nn.Module class. This should suffice for implementing most neural networks. However, if you are interested in additional details, you can find the official torch.nn.Module documentation here.

    Log in or create a free Lightning.ai account to access:

    • Quizzes
    • Completion badges
    • Progress tracking
    • Additional downloadable content
    • Additional AI education resources
    • Notifications when new units are released
    • Free cloud computing credits

    Quiz: 3.5 The PyTorch API - PART 1

    What the class methods that we have to define when we instantiate a PyTorch model from torch.nn.Module (parent class)?

    Correct. Python classes typically require an __init__ if we want to define class attributes upon creating new objects from that class. In this context, we define the model parameters here.

    Correct. In the forward method we define how the model computes the outputs. In the case of logistic regression, this could be the computation of the class-membership probabilities.

    Incorrect. We do not have to implement a backward method. PyTorch does this automatically for us when we use torch.nn.Module as the parent class.

    Please answer all questions to proceed.

    Quiz: 3.5 The PyTorch API - PART 2

    Suppose you initialize a neural network layer using torch.nn.Linear(in_features=5, out_features=1). How many trainable parameters does this layer have?

    Incorrect. There are 5 input features and 1 output feature. How many weights and bias units does that require?

    Incorrect. There are 5 input features and 1 output feature. How many weights and bias units does that require?

    Correct. – Incorrect. There are 5 input features and 1 output feature. Hence, we require 5 weight parameters and 1 bias unit. You can double-check by printing list(torch.nn.Linear(in_features=5, out_features=1).parameters())

    Incorrect. There are 5 input features and 1 output feature. How many weights and bias units does that require?

    Please answer all questions to proceed.
    Watch Video 1

    Unit 3.5

    Videos
    Follow along in a Lightning Studio

    DL Fundamentals 3: Model Training in PyTorch

    Sebastian
    Launch Studio →
    Questions or Feedback?

    Join the Discussion