Why there is the need of using regularization in machine learning problems

0 votes

It is said that regularization can help us obtain simple models over complex ones to avoid over-fitting. But for a linear classification problem:

f(x) = Wx

The complexity of the model is somewhat specified: it's linear, not quadratic or something more complex. So why do we still need regularization on the parameters? Why do we prefer smaller weights in such cases?

Feb 21, 2022 in Machine Learning by Nandini
• 5,480 points
1,386 views

1 answer to this question.

0 votes

In Machine Learning we often divide the dataset into training and test data, the algorithm while training the data can either
1) learn the data too well, even the noises which is called over fitting
2) do not learn from the data, cannot find the pattern from the data which is called under fitting.
Now, both over fitting and underfitting are problems one need to address while building models.

Regularization in Machine Learning is used to minimize the problem of overfitting, the result is that the model generalizes well on the unseen data once overfitting is minimized.
To avoid overfitting, regularization discourages learning a more sophisticated or flexible model. Regularization will try to minimize a loss function by inducing penalty.
For Example
The residual sum of squares is our optimization function or loss function in simple linear regression (RSS).

Here ,

y is the dependent variable, 
x1, x2, x3,..... xn are independent variables.
b0, b1 ,b2.........  bn, are the  coefficients estimates for different variables of x, these can also be called weights or magnitudes
Regularization will shrink these coefficients towards Zero, 
Minimizing the loss means less error and model will be a good fit.

The way regularization can be done is by
1) RIDGE also know as L-2 Regularization
2) LASSO (Least Absolute and Selection Operator) also known as L-1 Regularization

answered Feb 21, 2022 by Dev
• 6,000 points

Related Questions In Machine Learning

0 votes
1 answer

What is the process involved in machine Learning?

Discussing this on a high level, these ...READ MORE

answered May 10, 2019 in Machine Learning by Rhea
1,518 views
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 10, 2019 in Machine Learning by Shridhar
1,162 views
0 votes
1 answer
0 votes
1 answer

What is the best machine learning website?

Hi@akhtar You can find lots of website which ...READ MORE

answered May 20, 2020 in Machine Learning by MD
• 95,460 points
776 views
0 votes
1 answer

Assumptions of Naïve Bayes and Logistic Regression

There are very few difference between Naive ...READ MORE

answered Feb 7, 2022 in Machine Learning by Nandini
• 5,480 points
492 views
0 votes
1 answer
0 votes
1 answer

A simple explanation of Naïve Bayes Classification

Naive Bayes Classification uses probability to classify ...READ MORE

answered Feb 22, 2022 in Machine Learning by Nandini
• 5,480 points
490 views
0 votes
1 answer

What is the difference between classification and prediction?

Classification is about classifying categorical variables in ...READ MORE

answered Feb 25, 2022 in Machine Learning by Dev
• 6,000 points
1,273 views
0 votes
1 answer
0 votes
1 answer

What is difference between loss function and RMSE in Machine Learning?

The loss function is a function of ...READ MORE

answered Feb 23, 2022 in Machine Learning by Dev
• 6,000 points
2,879 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP