In LightFM does the training set need to be the same size as the test set?

0 votes

I'm working on a pure collaborative filtering model in LightFM using a simple user-item matrix.

When I have a training matrix that has more rows than the test matrix, I encounter this error: "Incorrect number of features in user_features"

It seems this is caused by line 789 in lightfm.py:

if not user_features.shape[1] == self.user_embeddings.shape[0]:
    raise ValueError('Incorrect number of features in user_features')

Investigating this - it looks like it is being caused by line 778 in lightfm.py:

n_users, n_items = test_interactions.shape

Which suggests the size of the test dataset should match the train. And making them the same size does fix my error.

How can I persuade LightFM to let me have train and test datasets of different sizes?

Sep 21, 2018 in Python by ariaholic
• 7,320 points
17 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

I would change the wording. Often in ML, the rows are the samples and in this interpretation your single row-sample is a tuple or something similar (user, item, value). The number of rows in test and train surely has no limitation! But there is one in terms of dimensions. So only in terms of dimensions (which users and which items are observed) those datasets need a 1:1 correspondence. That's maybe a strong assumption, but you can imagine the problem predicting user 11, where there was no user 11 in training (there is no latent-vector which was build).

answered Sep 21, 2018 by charlie_brown
• 7,710 points

Related Questions In Python

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

How to get the size of a string in Python?

If you are talking about the length ...READ MORE

answered Jun 4, 2018 in Python by ariaholic
• 7,320 points
48 views
–1 vote
2 answers

How to find the size of a string in Python?

following way to find length of string  x ...READ MORE

answered Mar 29 in Python by rajesh
• 1,210 points
40 views
0 votes
1 answer
0 votes
1 answer

How to set window size in Tkinter Python?

The window size adjustment can be done ...READ MORE

answered May 8 in Python by Giri
11 views
0 votes
1 answer

How to hide a window in the constructor immediately after creation?

You can use QtCore.QTimer class Example(QWidget):     def __init__(self, app):         QWidget.__init__(self)         QTimer.singleShot(0, ...READ MORE

answered Apr 17, 2018 in Python by anonymous
16 views
0 votes
1 answer

How to correctly return an a dictionary as an output in zappier code using python?

David here, from the Zapier Platform team. ...READ MORE

answered Dec 3, 2018 in Python by charlie_brown
• 7,710 points
58 views
0 votes
1 answer

What is the preferred way to concatenate strings in python??

If the strings you are concatenating are ...READ MORE

answered Dec 20, 2018 in Python by charlie_brown
• 7,710 points
38 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.