Is batch normalization useful for small networks

0 votes
We know batch normalization (BN) speeds up training of deep neural networks. But does it help with small neural networks as well? I have been experimenting with a 6-layer convolutional-MLP network and I cannot see any benefit for BN in training this network.
Mar 3 in Machine Learning by Dev
• 6,000 points
40 views

1 answer to this question.

0 votes
Batch normalization is a technique that is commonly employed with very deep neural networks. With each mini-batch, the outputs of layers after many layers fluctuate, and the layer must keep chasing a shifting objective.

However, because the fluctuations remain within a restricted range and do not confront the difficulty of a shifting target, this is not a significant issue for shallow neural networks. So, for shallow neural networks, you may train without batch normalization and everything will work well.
answered Mar 3 by Nandini
• 5,480 points

Related Questions In Machine Learning

0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
543 views
0 votes
2 answers

How do neural networks used in AI and deep learning learn?

A neural network is a hardware or software ...READ MORE

answered Apr 8, 2019 in Others by SA
• 1,050 points
363 views
0 votes
0 answers

How is AI different from Deep learning?

How is AI different from Deep Learning? READ MORE

Apr 22, 2019 in Others by Halder
150 views
0 votes
0 answers

Use inverse transform with deep learning. Conceptual clarity needed

I have built a deep learning model ...READ MORE

Mar 21, 2020 in Python by Anan
• 180 points
862 views
0 votes
1 answer

Role of the bias in neural networks.

Hi@akhtar, The activation function in Neural Networks takes ...READ MORE

answered Jul 15, 2020 in Machine Learning by MD
• 95,340 points
648 views
0 votes
1 answer
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP