What is logits softmax and softmax cross entropy with logits in Python

0 votes

I was going through the tensorflow API docs here. In the tensorflow documentation, they used a keyword called logits. What is it? In a lot of methods in the API docs it is written like

tf.nn.softmax(logits, name=None)

If what is written is those logits are only Tensors, why keeping a different name like logits?

Another thing is that there are two methods I could not differentiate. They were

tf.nn.softmax(logits, name=None)
tf.nn.softmax_cross_entropy_with_logits(logits, labels, name=None)

What are the differences between them? The docs are not clear to me. I know what tf.nn.softmaxdoes. But not the other. An example will be really helpful.

Nov 12, 2018 in Python by Anirudh
• 2,080 points

1 answer to this question.

–1 vote

Suppose you have two tensors, where y_hat contains computed scores for each class (for example, from y = W*x +b) and y_true contains one-hot encoded true labels.

y_hat  = ... # Predicted label, e.g. y = tf.matmul(X, W) + b
y_true = ... # True label, one-hot encoded

If you interpret the scores in y_hat as unnormalized log probabilities, then they are logits.

Additionally, the total cross-entropy loss computed in this manner:

y_hat_softmax = tf.nn.softmax(y_hat)
total_loss = tf.reduce_mean(-tf.reduce_sum(y_true * tf.log(y_hat_softmax), [1]))

is essentially equivalent to the total cross-entropy loss computed with the function softmax_cross_entropy_with_logits():

total_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y_hat, y_true))
answered Nov 12, 2018 by Nymeria
• 3,520 points

Related Questions In Python

+1 vote
7 answers
+1 vote
1 answer

What is the difference between range and xrange functions in Python 2.X?

xrange only stores the range params and ...READ MORE

answered Aug 22, 2018 in Python by Priyaj
• 58,120 points
0 votes
4 answers

What is a Tuple in Python and how to use it?

Tuples  are a  Unchanging sequence of values, ...READ MORE

answered Jun 21, 2020 in Python by sahil
• 540 points
+1 vote
4 answers

In Python, what is difference between Array and List?

Lists and arrays are used in Python ...READ MORE

answered Mar 15, 2019 in Python by Taj
• 1,080 points

edited Mar 18, 2019 by Omkar 122,924 views
0 votes
2 answers
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 6, 2019 in Python by Neha
• 330 points

edited Jul 8, 2019 by Kalgi 2,033 views
0 votes
0 answers
+4 votes
6 answers

Lowercase in Python

You can simply the built-in function in ...READ MORE

answered Apr 11, 2018 in Python by hemant
• 5,810 points
0 votes
1 answer

What is absolute import in Python and how is it used?

An absolute {import, path, URL} tells you exactly how ...READ MORE

answered Nov 30, 2018 in Python by Nymeria
• 3,520 points

edited Dec 10, 2018 by Nymeria 286 views
0 votes
1 answer

Is multi-threading supported in Python and can it speed up execution time as well?

The GIL does not prevent threading. All ...READ MORE

answered Nov 22, 2018 in Python by Nymeria
• 3,520 points