Is batch normalization useful for small networks

0 votes
We know batch normalization (BN) speeds up training of deep neural networks. But does it help with small neural networks as well? I have been experimenting with a 6-layer convolutional-MLP network and I cannot see any benefit for BN in training this network.
Mar 3, 2022 in Machine Learning by Dev
• 6,000 points
677 views

1 answer to this question.

0 votes
Batch normalization is a technique that is commonly employed with very deep neural networks. With each mini-batch, the outputs of layers after many layers fluctuate, and the layer must keep chasing a shifting objective.

However, because the fluctuations remain within a restricted range and do not confront the difficulty of a shifting target, this is not a significant issue for shallow neural networks. So, for shallow neural networks, you may train without batch normalization and everything will work well.
answered Mar 3, 2022 by Nandini
• 5,480 points

Related Questions In Machine Learning

0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
1,181 views
0 votes
2 answers
0 votes
0 answers

How is AI different from Deep learning?

How is AI different from Deep Learning? READ MORE

Apr 22, 2019 in Others by Halder
359 views
0 votes
0 answers

Use inverse transform with deep learning. Conceptual clarity needed

I have built a deep learning model ...READ MORE

Mar 21, 2020 in Python by Anan
• 180 points
1,484 views
0 votes
1 answer

Role of the bias in neural networks.

Hi@akhtar, The activation function in Neural Networks takes ...READ MORE

answered Jul 15, 2020 in Machine Learning by MD
• 95,440 points
1,294 views
0 votes
1 answer

Why is random_state required for ridge & lasso regression classifiers?

This is because the regression coefficients of ...READ MORE

answered Mar 2, 2022 in Machine Learning by Nandini
• 5,480 points
870 views
0 votes
1 answer

What is the difference between linear regression and logistic regression?

Hi Dev, to answer your question Linear Regression ...READ MORE

answered Feb 2, 2022 in Machine Learning by Nandini
• 5,480 points
955 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP