2.4 Improving Code Efficiency with Linear Algebra (Parts 1-4)

Code

• 2.4-linalg-part1.ipynb
• 2.4-linalg-part2.ipynb
• 2.4-linalg-part3.ipynb
• 2.4-linalg-part4.ipynb

What we covered in this video lecture

In this video, we saw how we could replace a Python for-loop computing of a weighted sum with a linear algebra concept called dot product. Given two vectors with the same number of elements, the dot product multiplies pairs of elements and sums the results. A common use case is computing the dot product between a feature vector and a weight vector.

Now, if we have n training examples, we can compute n dot products — the dot product between each training example and the weight vector. We can accomplish this via a for-loop over the n dot products. To make this more efficient, we can use matrix-vector multiplication.

What if we have n training examples and m feature vectors — we will encounter this later when we work with multilayer perceptrons. In this case, we can use matrix multiplication to multiply two matrices, a training example matrix, and a weight matrix.

Additional resources if you want to learn more

Linear algebra is a big topic, and there is a vast number of courses and textbooks out there. An exceptionally good course is Gilbert Strang’s Linear Algebra. However, good news is that we do not have to become linear algebra experts before we can get started leveraging it for deep learning. As we have seen in this lecture, we can regard linear algebra as a means of implementing computations more efficiently.

Watch Video 1 Mark complete and go to Unit 2.5 →