How can I apply entropy and maximum entropy in terms of text mining? Can someone give me a easy, simple example

Mar 2, 2022 9,124 views

## 1 answer to this question.

Entropy is uncertainty/ randomness in the data, the more the randomness the higher will be the entropy. Information gain uses entropy to make decisions. If the entropy is less, information will be more.
Information gain is used in decision trees and random forest to decide the best split. Thus, the more the information gain the better the split and this also means lower the entropy.
The entropy of a dataset before and after a split is used to calculate information gain.
Entropy is the measure of uncertainty in the data. The effort is to reduce the entropy and maximize the information gain. The feature having the most information is considered important by the algorithm and is used for training the model.
By using Information gain you are actually using entropy.
• 6,000 points

## What is information gain? - Decision tree algo

You can use information gain to decide ...READ MORE

## What is correlation and its types?

Correlation is a statistical measure that shows ...READ MORE

## When do I use simple exponential smoothing and what is the math behind it?

Hey @Ruth, you can use this model ...READ MORE

## Naive Bayes classifier bases decision only on a-priori probabilities

You seem to have trained the model ...READ MORE

## Probability: the one true fish

What you're looking for is P(A|B), which ...READ MORE

## Use different distance formula other than euclidean distance in k means

K-means is based on variance minimization. The sum-of-variance formula ...READ MORE