1/5/2018 Coursera | Online Courses From Top Universities.

Join for Free | Coursera

Back to Week 1 Lessons Prev Next

Gradient Descent Intuition

In this video we explored the scenario where we used one parameter θ1 and plotted its
cost function to implement a gradient descent. Our formula for a single parameter was :

Repeat until convergence:

d
θ1 := θ1 − α J (θ1 )
dθ 1

Regardless of the slope's sign for d

dθ 1
,
J (θ1 ) θ1 eventually converges to its minimum value.
The following graph shows that when the slope is negative, the value of θ1 increases and
when it is positive, the value of θ1 decreases.

On a side note, we should adjust our parameter α to ensure that the gradient descent
algorithm converges in a reasonable time. Failure to converge or too much time to obtain
the minimum value imply that our step size is wrong.

https://www.coursera.org/learn/machine-learning/supplement/QKEdR/gradient-descent-intuition 1/3
1/5/2018 Coursera | Online Courses From Top Universities. Join for Free | Coursera

How does gradient descent converge with a xed step size α?

The intuition behind the convergence is that d

dθ 1
J (θ1 ) approaches 0 as we approach the
bottom of our convex function. At the minimum, the derivative will always be 0 and thus
we get:

θ1 := θ1 − α ∗ 0

https://www.coursera.org/learn/machine-learning/supplement/QKEdR/gradient-descent-intuition 2/3
1/5/2018 Coursera | Online Courses From Top Universities. Join for Free | Coursera

Mark as completed

  

https://www.coursera.org/learn/machine-learning/supplement/QKEdR/gradient-descent-intuition 3/3