In some machine learning classes I took recently, I've covered gradient descent to find the best fit line for linear regression.

In some statistics classes, I have learnt that we can compute this line using statistic analysis, using the mean and standard deviation. Why is this seemingly more simple technique not used in machine learning?

My question is, is gradient descent the preferred method for fitting linear models? If so, why? Or did the professor simply use gradient descent in a simpler setting to introduce the class to the technique? Mar 22 82 views

## 1 answer to this question.

An example you gave is one-dimensional, which is unusual in machine learning, since numerous input features are frequently present. To apply their basic approach, you must invert a matrix, which can be difficult or ill-conditioned.
The problem is usually stated as a least - square problem, which is relatively easier. Instead of using gradient descent, ordinary least square solvers could be utilized (and often are). If the number of data points is large, a typical least squares solver may be excessively expensive, but (stochastic) gradient descent could provide a solution that is as good as a more precise solution in terms of test-set error but requiring orders of magnitude less time.
If your problem is small enough to be handled quickly with an off-the-shelf least squares solver, gradient descent is probably not for you.
Because gradient descent is a general algorithm, it can be used to solve any problem involving the optimization of a cost function. The mean square error is a common cost function used in regression problems (MSE). Finding a closed form solution necessitates inverting an ill-conditioned matrix (one whose determinant is very close to zero and so does not yield a robust inverse matrix). People frequently use the gradient descent method to find a solution which does not suffer from the ill-conditionally problem to get around this problem. answered Mar 23 by
• 6,000 points

## Linear regression with gradient descent to predict house prices using area (one var) in python

Apart from instructional purposes, I'm not sure ...READ MORE

## difference between a cost function and the gradient descent equation in logistic regression?

Cost function is a way to evaluate ...READ MORE

## How do I create a linear regression model in Weka without training?

Weka is a classification algorithm. This is ...READ MORE

## Ignore the NaN and do the linear regression on remaining values

Yes, you can do this using statsmodels: import ...READ MORE

## Python code for basic linear regression

Hi @Dipti, you could try something like ...READ MORE

## Show python implementation of Lasso class - regression

Hey @Tanmay, try something like this: >>> from ...READ MORE

## What is LassoLars? - Linear regression

LassoLars is a lasso model implemented using ...READ MORE

## Can you give LassoLars python example?

Hey @Vivek, Try something like this: >>> from ...READ MORE