What is entropy and information gain

0 votes

How can I apply entropy and maximum entropy in terms of text mining? Can someone give me a easy, simple example

Mar 2 in Machine Learning by Nandini
• 5,480 points
77 views

1 answer to this question.

0 votes
Entropy is uncertainty/ randomness in the data, the more the randomness the higher will be the entropy. Information gain uses entropy to make decisions. If the entropy is less, information will be more.
Information gain is used in decision trees and random forest to decide the best split. Thus, the more the information gain the better the split and this also means lower the entropy.
The entropy of a dataset before and after a split is used to calculate information gain.
Entropy is the measure of uncertainty in the data. The effort is to reduce the entropy and maximize the information gain. The feature having the most information is considered important by the algorithm and is used for training the model.
By using Information gain you are actually using entropy.
answered Mar 2 by Dev
• 6,000 points

Related Questions In Machine Learning

0 votes
1 answer

What is information gain? - Decision tree algo

You can use information gain to decide ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
877 views
0 votes
1 answer

What is correlation and its types?

Correlation is a statistical measure that shows ...READ MORE

answered May 10, 2019 in Machine Learning by Zulaikha
1,180 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Probability: the one true fish

What you're looking for is P(A|B), which ...READ MORE

answered Mar 25 in Machine Learning by Dev
• 6,000 points
22 views
0 votes
3 answers

How to write lines to a text file in R?

sink("outfile.txt") cat("hello") cat("\n" ...READ MORE

answered May 24, 2019 in Data Analytics by anonymous
17,310 views
0 votes
1 answer

Use different distance formula other than euclidean distance in k means

K-means is based on variance minimization. The sum-of-variance formula ...READ MORE

answered Jun 21, 2018 in Data Analytics by Sahiti
• 6,360 points
920 views
0 votes
1 answer
0 votes
1 answer

What is cross-entropy?

Cross-entropy is taken from the information theory ...READ MORE

answered Feb 23 in Machine Learning by Dev
• 6,000 points
45 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP