Optimizers

GRADIENT DESCENT :   It is an optimization technique which helps us to find the values of parameters(coefficients) of a function (f) that minimizes the cost function(cost). The word Gradient means slope and Descent means downward. We will decreases our slope in cost function to find the optimal value of the coefficient  It is best used when the parameters cannot be calculated analytically(means by linear algebra) and must be searched for by an optimal algorithm.  Intuition of Gradient descent: Let's consider a bowl in which you eat cereal or store fruit in. The bowl is a plot of cost function(f). A random position on the surface of bowl is the cost of the values of the coefficient(cost). The bottom of the bowl is the cost of best set of coefficients, the minimum of the function which is also known "Global Minima". Our main goal is to find the global minima point on that curve which will give us the best values of the coefficients.   Basically there are two type of minima: