What is logits, softmax and softmax_cross_entropy_with_logits in Python?

0 votes

I was going through the tensorflow API docs here. In the tensorflow documentation, they used a keyword called logits. What is it? In a lot of methods in the API docs it is written like

tf.nn.softmax(logits, name=None)

If what is written is those logits are only Tensors, why keeping a different name like logits?

Another thing is that there are two methods I could not differentiate. They were

tf.nn.softmax(logits, name=None)
tf.nn.softmax_cross_entropy_with_logits(logits, labels, name=None)

What are the differences between them? The docs are not clear to me. I know what tf.nn.softmaxdoes. But not the other. An example will be really helpful.

Nov 12, 2018 in Python by Anirudh
• 2,070 points
792 views

1 answer to this question.

0 votes

Suppose you have two tensors, where y_hat contains computed scores for each class (for example, from y = W*x +b) and y_true contains one-hot encoded true labels.

y_hat  = ... # Predicted label, e.g. y = tf.matmul(X, W) + b
y_true = ... # True label, one-hot encoded

If you interpret the scores in y_hat as unnormalized log probabilities, then they are logits.

Additionally, the total cross-entropy loss computed in this manner:

y_hat_softmax = tf.nn.softmax(y_hat)
total_loss = tf.reduce_mean(-tf.reduce_sum(y_true * tf.log(y_hat_softmax), [1]))

is essentially equivalent to the total cross-entropy loss computed with the function softmax_cross_entropy_with_logits():

total_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y_hat, y_true))
answered Nov 12, 2018 by Nymeria
• 3,520 points

Related Questions In Python

+1 vote
3 answers
+1 vote
1 answer

What is the difference between range and xrange functions in Python 2.X?

xrange only stores the range params and ...READ MORE

answered Aug 22, 2018 in Python by Priyaj
• 56,940 points
282 views
0 votes
3 answers
0 votes
1 answer

In Python, what is difference between Array and List?

Lists and arrays are used in Python ...READ MORE

answered Mar 15 in Python by Taj
• 1,040 points

edited Mar 18 by Omkar 6,093 views
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 6 in Python by Neha
• 330 points

edited Jul 8 by Kalgi 348 views
+4 votes
6 answers
0 votes
1 answer

What is absolute import in Python and how is it used?

An absolute {import, path, URL} tells you exactly how ...READ MORE

answered Nov 30, 2018 in Python by Nymeria
• 3,520 points

edited Dec 10, 2018 by Nymeria 105 views
0 votes
1 answer

Is multi-threading supported in Python and can it speed up execution time as well?

The GIL does not prevent threading. All ...READ MORE

answered Nov 22, 2018 in Python by Nymeria
• 3,520 points
192 views