Lightning AI Studios: Never set up a local environment again →

Log in or create a free account to track your progress and access additional course materials  

4.4 Defining Efficient Data Loaders (Part 1-4)



What we covered in this video lecture

Now that we are beginning to work with larger neural networks and datasets, it’s time to worry about optimizing our data-loading pipeline. In this lecture, we discussed PyTorch’s Dataset and DataLoader classes, which help us fetch the training batches in the background using multiple processes. To avoid computational bottlenecks, using these background processes ensures that the next batch of data is ready when the model finishes its backward pass.

Additional resources if you want to learn more

Using the Dataset and DataLoader classes is the most convenient way to load data in PyTorch. However, there is also a new DataPipe class that the PyTorch team is developing. You can find out more about these optional DataPipes in the official TorchData repository here. I also wrote a tutorial about DataPipes which you might find helpful here.

Log in or create a free account to access:

  • Quizzes
  • Completion badges
  • Progress tracking
  • Additional downloadable content
  • Additional AI education resources
  • Notifications when new units are released
  • Free cloud computing credits

Quiz: 4.4 Defining Efficient Data Loaders

The drop_last argument in the DataLoader drops the last

Incorrect. While there are some special cases where it only drops the last training example, this is not generally true.

Correct. Sometimes when the batch size of the last minibatch is too small, it can lead to noisy gradient updates for the last batch in an epoch. Thus, setting the argument drop_last=True is usually recommended.

Incorrect. The drop_last has something to do with the dataset, not the workers …

The __getitem__ method defines how we

Incorrect. Think of it as “get item” not “get items”.

Incorrect. Typically, we need both the features and the labels during training.

Correct. In the __getitem__ method we define how we load a training example (including the features and the label).

Please answer all questions to proceed.
Watch Video 1

Unit 4.4