Just like we use the Normal equation to find out the optimum theta value in Linear Regression, can/can't we use a similar formula for Logistic Regression ? If not, why ? I'd be grateful if could someone could explain the reasoning behind it. Thank You. Mar 26 49 views

## 1 answer to this question.

Unfortunately, only one discriminative method in classification theory, linear regression, has closed form solutions (linear discriminant analysis/fisher discriminant are generative, and even they have a closed form solution due to the great simplicity of the distributions fitted).

Even with linear regression, it is considered a wonder that it "works." As far as I'm aware, proving that "you cannot solve logistic regulation in closed form" is nearly impossible; yet, the widespread consensus is that this will never be the case. You can achieve it if your features are binary only and you have a small number of them (since a solution is exponential in the number of features), as demonstrated a few years ago, but it is thought to be impossible in most cases.
So, what made it so effective for linear regression? Because, once you've computed your derivatives, you'll find that the outcome is a set of linear equations, m equations with m variables, which we know can be solved directly using matrix inversions (and other techniques). When the cost of logistic regression is differentiated, the resultant problem is no longer linear... Because it is convex (thus global optimum), but not linear, present mathematics does not give us with powerful tools to identify the optimum in closed form solution. answered Mar 30 by
• 6,000 points

## Can you give an example for isotonic regression?

Have a look at this one, It's ...READ MORE

## difference between a cost function and the gradient descent equation in logistic regression?

Cost function is a way to evaluate ...READ MORE

## How can QA team leverage BDD acceptance criteria for automation regression testing?

There is no correct response; in my ...READ MORE

## Can we use Normal Equation for Logistic Regression ?

Well not likely,  only one discriminative method ...READ MORE

## L2 regularization in Logistic regression vs NN

L2 Regularization is known as Ridge Regression. ...READ MORE

## Python code for basic linear regression

Hi @Dipti, you could try something like ...READ MORE

## Example of Logistic regression with python code

Have a look at this: import csv import ...READ MORE