How can I change the loss function when the shape of my data changes

0 votes

Since my data is too large, I use pd.read_csv('',chunksize=). I am using categorical_crossentropy as my loss function, however, on the last chunk, I have just one target because of chunking. So I get the error:

You are passing a target array of shape (2110, 1) while using as loss categorical_crossentropy.

What I have tried:

Now I know I can use binary_crossentropy for that last chunk. So this is what I did:

X_train, X_test, y_train, y_test = train_test_split(train_data, train_labels, shuffle=True, test_size=0.3)
if y_train.shape[1] == 1:
   loss = 'binary_crossentropy'
   loss = 'categorical_crossentropy'

When I do this, I get the error:

IndexError: indexp 1 is out of bounds for axis 0 with size 1

If i just use the binary_crossentropy function, I get the same error. My data is one-hot encoded. How can I resolve this error? Thanks

Mar 26, 2020 in Python by Anan
• 180 points

Hey, @Anan,

Could you please share your full code.

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In Python

0 votes
1 answer

How can I change the data type to string in Python?

You can use str(variablename) This is called conversion ...READ MORE

answered Dec 17, 2018 in Python by Shuvodip
0 votes
1 answer

How can I get the domain name of my site within a Django template?

Hello kartik, The variation of the context processor ...READ MORE

answered Apr 23, 2020 in Python by Niroj
• 82,580 points
+2 votes
2 answers

In a list of dictionaries, how can I find the minimum calue in a common dictionary field.

There are several options. Here is a ...READ MORE

answered Apr 10, 2018 in Python by charlie_brown
• 7,780 points
0 votes
1 answer

How can I compare the content of two files in Python?

Assuming that your file unique.txt just contains ...READ MORE

answered Apr 16, 2018 in Python by charlie_brown
• 7,780 points
+1 vote
1 answer

Index of predicted wrong data in Keras, how to find it?

Simply, use: model.predict() pred = model.predict(x_test) indices = [i for ...READ MORE

answered Sep 28, 2018 in Python by Priyaj
• 58,100 points
0 votes
1 answer

Neural Networks in Deep Learning

Neural networks as the name suggests are ...READ MORE

answered Oct 8, 2018 in Data Analytics by kurt_cobain
• 9,390 points
0 votes
1 answer

How to get rid of tensorflow verbose messages with Keras?

If you are using TensorFlow 0.12, you ...READ MORE

answered Jan 24, 2019 in Python by SDeb
• 13,250 points