keras model fit generator several times slower than model fit

0 votes
In Keras 1.2.2, there are referencing merge and it has multiprocessing included, but model.fit_generator() is still about 4-5x slower than model.fit() due to disk reading speed limitations. How can this be sped up, say through additional multiprocessing?

Can anyone help me with this?
May 31, 2019 in Python by ana1504.k
• 7,910 points
3,113 views

1 answer to this question.

0 votes
You may want to check out the workers and max_queue_size parameters of fit_generator() in the documentation. Essentially, more workers create more threads for loading the data into the queue that feeds data to your network. There is a chance that filling the queue might cause memory problems, though, so you might want to decrease max_queue_size to avoid this.
answered May 31, 2019 by SDeb
• 13,300 points

Related Questions In Python

+1 vote
2 answers

Replace every character of string by character whose ASCII value is K times more than it

Refer below code for your problem statement def ...READ MORE

answered May 17, 2020 in Python by Kumar Sambhawam

edited May 18, 2020 by Gitika 17,272 views
0 votes
1 answer

Why is reading lines from stdin much slower in C++ than Python?

tl;dr: Because of different default settings in ...READ MORE

answered Dec 18, 2020 in Python by Nikita
1,213 views
0 votes
1 answer

How to get all related Django model objects in Python?

This actually gives you the property names ...READ MORE

answered Nov 14, 2018 in Python by Nymeria
• 3,560 points

edited Dec 18, 2018 by Nymeria 6,812 views
0 votes
1 answer

How to fit a histogram using Python?

Here is an example working on py2.6 ...READ MORE

answered Dec 26, 2018 in Python by Nymeria
• 3,560 points
6,526 views
0 votes
1 answer

Tensorflow ImportError: cannot import name'abs' error

Hi@akhtar, You may get this error, if you ...READ MORE

answered Apr 23, 2020 in Machine Learning by MD
• 95,460 points
6,162 views
0 votes
1 answer
0 votes
1 answer

AttributeError: module 'coco' has no attribute 'CocoConfig'

Hi@akhtar, You need to import the coco module ...READ MORE

answered Jul 13, 2020 in Machine Learning by MD
• 95,460 points
1,770 views
+1 vote
1 answer

Index of predicted wrong data in Keras, how to find it?

Simply, use: model.predict() pred = model.predict(x_test) indices = [i for ...READ MORE

answered Sep 28, 2018 in Python by Priyaj
• 58,100 points
5,823 views
0 votes
1 answer

How to get rid of tensorflow verbose messages with Keras?

If you are using TensorFlow 0.12, you ...READ MORE

answered Jan 24, 2019 in Python by SDeb
• 13,300 points
4,631 views
+1 vote
1 answer

Django model iterate fields

You can try the following: getattr(foo.__class__, <field_name>)   This should ...READ MORE

answered Apr 10, 2019 in Python by SDeb
• 13,300 points
5,095 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP