Epoch vs Iteration when training neural networks

Question

What is the difference between epoch and iteration when training a multi-layer perceptron?

Answer 1

Epoch can be understood as the number of times the algorithm scans the entire data. For example if we set epoch = 10 then the algorithm will scan the entire data ten times.

Whereas, Iteration is the number of times a certain batch is passed via an algorithm. In the context of Neural Networks iteration is forward and backward pass.
For Example : a dataset having 20 samples, and batch size =4, epochs= 5

Then, in each epoch, there will be 5 batches (20/4=5). Each batch will be passed by the algorithm there will be 5 iterations per epoch. Since the epochs is set as 5.
Thus, total number of iterations will be 5*5 = 25

Supercharge Your Skills with Our Machine Learning Course!