What is information gain - Decision tree algo

0 votes
What is Information gain in Decision tree algorithm? Can you explain how it is used? Thank you
May 14, 2019 in Machine Learning by Priya
1,190 views

1 answer to this question.

0 votes

You can use information gain to decide which attribute goes at which level in the decision tree. By using information gain as a criterion, we try to estimate the information contained by each attribute. We are going to use some points deducted from information theory.
To measure the randomness or uncertainty of a random variable X is defined by Entropy.

For a binary classification problem with only two classes, positive and negative class.

  • If all examples are positive or all are negative then entropy will be zero i.e, low.
  • If half of the records are of positive class and half are of negative class then entropy is one i.e, high.

H(X)=\mathbb {E} _{X}[I(x)]=-\sum _{x\in \mathbb {X} }p(x)\log p(x).

By calculating the entropy measure of each attribute we can calculate their information gain. Information Gain calculates the expected reduction in entropy due to sorting on the attribute. Information gain can be calculated.

answered May 14, 2019 by Raj

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
1,171 views
0 votes
1 answer
0 votes
1 answer

What is "entropy and information gain"?

Entropy is uncertainty/ randomness in the data, ...READ MORE

answered Mar 2 in Machine Learning by Dev
• 6,000 points
5,176 views
0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
694 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 33,050 points
649 views
0 votes
1 answer

What is semi-supervised machine learning?

Hi@Ganesh, Semi-supervised machine learning is a combination of ...READ MORE

answered Jul 19, 2020 in Machine Learning by MD
• 95,380 points
465 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 9, 2019 in Machine Learning by Alok
627 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
1,621 views
0 votes
1 answer

What is greedy approach in Decision tree algorithm?

“Greedy Approach is based on the concept ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
10,825 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP