Lightning AI Studios: Never set up a local environment again →

Log in or create a free Lightning.ai account to track your progress and access additional course materials  

Deep Learning Fundamentals

  • Pages

    • Final certification exam

    Deep Learning Fundamentals > Unit 3 > Unit 3.4
    Course Progress:

    3.4 Automatic Differentiation in PyTorch

    Slides

    What we covered in this video lecture

    We can compute the gradients by hand when we work with small models such as logistic regression, as we have seen in the previous lecture. However, as our models become larger, this becomes tedious or even infeasible.

    Luckily, PyTorch supports automatic differentiation (also known as autograd) to calculate derivatives and gradients automatically. In this lecture, we saw the basic capabilities and usage of PyTorch’s autograd submodule. We will use it in the upcoming videos when implementing the training loop.

    Additional resources if you want to learn more

    If you are curious to learn more about PyTorch’s autograd feature, check out the official PyTorch Autograd documentation.

    Log in or create a free Lightning.ai account to access:

    • Quizzes
    • Completion badges
    • Progress tracking
    • Additional downloadable content
    • Additional AI education resources
    • Notifications when new units are released
    • Free cloud computing credits

    Quiz: 3.4 Automatic Differentiation in PyTorch

    Suppose you have the following function: f(x, y, z) = 4x^3 + 3y^2 + 2z.

    What is the partial derivative of f with respect to y when x=3, y=2, z=5?

    Incorrect. You computed the partial derivative of f with respect to x.

    Correct. The terms 4x^3 and 2z are constants and cancel according to the sum rule. The derivative of 3y^2 is then 6 * y, which is 12 for y=12. But don’t worry, you don’t have to follow the math. You can double-check this by using the following code:

    import torch
         from torch.autograd import grad
     x = torch.tensor(3.)
     y = torch.tensor(2., requires_grad=True)
     z = torch.tensor(5.)
     f = 4 * x**3 + 3 * y**2 + 2*z
     print(grad(f, y))
     
      
     
    Incorrect. You computed the partial derivative of f with respect to z.
     
      
     
    Incorrect. (Hint: try to use the grad function in PyTorch’s autograd).
     
     Please answer all questions to proceed.
     Watch Video 1 
     Unit 3.4 
    Videos
    Follow along in a Lightning Studio
     
    DL Fundamentals 3: Model Training in PyTorch
      Sebastian
    Launch Studio →
     
    Questions or Feedback?
    Join the Discussion