Lightning AI Studios: Never set up a local environment again →

Log in or create a free Lightning.ai account to track your progress and access additional course materials  

Deep Learning Fundamentals

  • Pages

    • Final certification exam

    Deep Learning Fundamentals > Unit 1 > Unit 1.4
    Course Progress:

    1.4 The First Machine Learning Classifier

    Slides

    References

    What we covered in this video lecture

    In this lecture, we introduced the perceptron algorithm, a binary classification algorithm inspired by how neurons in the human brain work.

    In the forward pass, the perceptron takes the input features, computes the net input, and finally applies a threshold to determine the predicted class labels. The perceptron then updates its weight and bias parameters based on whether the prediction is correct. Here, the weights and bias parameters are the model parameters learned from the training set.

    We use the letter m (as a subscript) to denote the number of features (dimensions) in a given dataset. And we use the letter n (as a superscript) to indicate the number of training examples. Another piece of technical jargon we introduced in this lecture is the concept of a training epoch. A training epoch refers to one full iteration over the training set.

    Additional resources if you want to learn more

    If you are interested in the early perceptron algorithms and how it all began, I recommend checking one of the early papers The Perceptron — A Perceiving and Recognizing Automaton (1957) by Frank Rosenblatt.

    While it is absolutely not necessary for this course or for understanding neural networks in general, it can be interesting to look at the perceptron from a geometrical perspective. If you are interested, I have a lecture video on the geometric intuition behind the perceptron here.

    If you can’t get enough of the perceptron, there is even a whole book devoted to the topic: Perceptrons (1969). (Note that this book was written and published to highlight the limitations of the perceptron.)

    Log in or create a free Lightning.ai account to access:

    • Quizzes
    • Completion badges
    • Progress tracking
    • Additional downloadable content
    • Additional AI education resources
    • Notifications when new units are released
    • Free cloud computing credits

    Quiz: 1.4 The First Machine Learning Classifier. Part 1

    In a binary classification problem, we work with

    This is incorrect. If a dataset has only one class, then we don’t have to use a prediction rule.

    Correct. Analogous to binary digits (0 and 1), a binary classification problem consists of only 2 class labels.

    This is incorrect. Hint: think of “binary” as binary decision or binary digits in a computer (0’s and 1’s).

    Please answer all questions to proceed.

    Quiz: 1.4 The First Machine Learning Classifier. Part 2

    The perceptron mimics exactly how neurons in the human brain work.

    Incorrect. The perceptron model and algorithm were inspired by how biological neurons work, but they still work in different ways.

    Correct. The perceptron model and algorithm were inspired by how biological neurons work, but the perceptron is a relatively simple model and does not capture all the complexity of biological neurons.

    The perceptron was first described in the 1950s. Today, scientists were able to decipher how the real learning algorithm in the human brain works.

    Incorrect. Although there are several theories, as of this writing, it is still unclear how exactly the human brain learns.

    Correct. Although there are several theories, as of this writing, it is still unclear how exactly the human brain learns.

    In the perceptron, we use a threshold to determine the class label from the so-called *net input*. The Threshold is at

    Incorrect. In a binary classification problem, we could use class labels -1 and 1 instead of 0 and 1, however, the threshold would be a different value.

    This is correct. Typically, the decision threshold is at 0.

    Incorrect. Hint: Think of the net inout value given the initial model parameters.

    Incorrect. Hint: Think of the net inout value given the initial model parameters.

    Please answer all questions to proceed.

    Quiz: 1.4 The First Machine Learning Classifier. Part 3

    Using the notation we defined in this lecture, $$x_{3}^{[5]}$$ refers to

    Correct. Subscripts refer to the feature index, superscripts to the training example index.

    Incorrect. Subscripts refer to the feature index, superscripts to the training example index.

    Based on the predictions, the perceptron updates the

    Correct. Here, the model parameters are the weights and bias unit.

    Incorrect. However, we can consider the threshold as a negative bias value as Sebastian describes in my detail in his blog post [Single-Layer Neural Networks and Gradient Descent](https://sebastianraschka.com/Articles/2015_singlelayer_neurons.html#the-unit-step-function).

    Please answer all questions to proceed.

    Quiz: 1.4 The First Machine Learning Classifier. Part 4

    Which is the correct formula for computing the net input given 2 feature variables?

    Incorrect. Hint: think of the net input as weighted sum (plus bias unit).

    Correct. The net input is a weighted sum (plus bias unit).

    Incorrect. Hint: think of the net input as weighted sum (plus bias unit).

    Incorrect. Hint: think of the net input as weighted sum (plus bias unit).

    Given the following values $$x_1 = 1.23$$, $$x_2 = -2.45$$, $$w_1=1.12$$, $$w_2=2.14$$, and $$b = 0.56$$, what is the net input?

    Correct. 1.23 * 1.12 + -2.45 * 2.14 + 0.56 = -3.31

    Incorrect. The netinput is a weighted sum (plus bias), not a product of all values.

    Incorrect. Hint: it appears that you multiplied by the bias unit instead of adding it?

    Incorrect. Hint: it appears that you added all terms instead of computing the weighted sum?

    What does “perfectly linearly separable” mean in the context of a classification problem?

    Correct. For dataset or problem to be “perfectly linearly separable”, it means a linear decision boundary should exist such that it is possible to achieve 100% training accuracy.

    Incorrect. While a linear decision boundary needs to be able to achieve 100% prediction accuracy when the problem is “perfectly linearly separable”, nonlinear decision boundaries can achieve 100% accuracy as well.

    Please answer all questions to proceed.

    Quiz: 1.4 The First Machine Learning Classifier. Part 5

    The weight $$w_j$$ corresponds to the

    Incorrect. We use superscripts to refer to the training example index.

    Correct. We use subscripts to refer to the feature index.

    If the true label is 0 and the predicted label is 0, the perceptron

    Incorrect. The prediction is correct, so we shouldn’t update the model weights.

    Correct. The prediction is correct, so we shouldn’t update the model weights.

    Incorrect. The prediction is correct, so we shouldn’t update any model parameters.

    If the true label is 1 and the predicted label is 0, the percepron changes a weight value $$w_j$$ by

    Correct. In this case, we want to make the net input larger to exceed the threshold.

    Incorrect. This would mean the weight value is updated using itself independent of the training data.

    Incorrect. We want to make the net input larger to exceed the threshold.

    Incorrect. This would mean the weight value is updated using itself independent of the training data.

    Please answer all questions to proceed.

    Quiz: 1.4 The First Machine Learning Classifier. Part 6

    Given $$x_1=1.3, w_1=2.5, b=1.2, w_2=2.4$$, what is the corresponding $$x_2$$ value?

    Correct. You rearranged the terms correctly to solve for $$x_2$$.

    Incorrect. Hint: Check if you included the $$x_1$$ term correctly.

    Incorrect. Hint: Check if you applied the parentheses correctly.

    Incorrect. Hint: Check if you included the bias unit correctly.

    Please answer all questions to proceed.
    Watch Video 1

    Unit 1.4

    Videos
    Follow along in a Lightning Studio

    DL Fundamentals 1: Intro to ML & DL

    Sebastian
    Launch Studio →
    Questions or Feedback?

    Join the Discussion