Not exactly. If your sigmoid output was 0.8, that doesn't mean your network produced probabilities for each class as sigmoid outputs don't add up to 1. In other words, an output of 0.8 does not imply that it is likely to belong to other classes with a probability of 0.2.

Also, with sigmoid network outputs p(y=1) in Binary Classification. Then p(y=0) = 1 - p(y=1) by definition of probability. Only for basic binary categorization, they add up to one.

Softmax activation should be used to see the probability of each class because its output will total to 1. Softmax outputs can be interpreted as probabilities.

These models, on the other hand, are deterministic rather than probabilistic. As a result, it is customary to interpret softmax results as probabilities, but there is no mathematical connection between them.

Is there a way to specify which class the probability applies to?

You can choose from a variety of thresholds, the most common of which is 0.5, but this is dependent on your data and situation. You can adjust the threshold to observe how it affects the AUC-ROC, and then interpret the results to find the ideal threshold for you.

If you wish to decide the classes, you can use the following formula, keeping in mind that the threshold is 0.5 and that you can adjust it:

predicted_classes = [1 * (x[0]>=0.5) for x in preds_sigmoid]

If the output is greater than 0.5, it is classified as second class.