What is information gain - Decision tree algo

0 votes
What is Information gain in Decision tree algorithm? Can you explain how it is used? Thank you
May 14, 2019 in Machine Learning by Priya
2,447 views

1 answer to this question.

+1 vote

You can use information gain to decide which attribute goes at which level in the decision tree. By using information gain as a criterion, we try to estimate the information contained by each attribute. We are going to use some points deducted from information theory.
To measure the randomness or uncertainty of a random variable X is defined by Entropy.

For a binary classification problem with only two classes, positive and negative class.

  • If all examples are positive or all are negative then entropy will be zero i.e, low.
  • If half of the records are of positive class and half are of negative class then entropy is one i.e, high.

H(X)=\mathbb {E} _{X}[I(x)]=-\sum _{x\in \mathbb {X} }p(x)\log p(x).

By calculating the entropy measure of each attribute we can calculate their information gain. Information Gain calculates the expected reduction in entropy due to sorting on the attribute. Information gain can be calculated.

Step into the AI Future, Enroll now in the AI Tools Course today!

answered May 14, 2019 by Raj

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
2,413 views
0 votes
1 answer

What is the actual meaning implied by information gain in data mining?

Information gain works on the concept of ...READ MORE

answered Mar 2, 2022 in Machine Learning by Dev
• 6,000 points
586 views
0 votes
1 answer

What is "entropy and information gain"?

Entropy is uncertainty/ randomness in the data, ...READ MORE

answered Mar 2, 2022 in Machine Learning by Dev
• 6,000 points
21,134 views
0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
1,381 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 33,030 points
1,832 views
0 votes
1 answer

What is semi-supervised machine learning?

Hi@Ganesh, Semi-supervised machine learning is a combination of ...READ MORE

answered Jul 19, 2020 in Machine Learning by MD
• 95,460 points
1,012 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 9, 2019 in Machine Learning by Alok
1,372 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
2,560 views
0 votes
1 answer

What is greedy approach in Decision tree algorithm?

“Greedy Approach is based on the concept ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
14,888 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP