What is information gain? - Decision tree algo

0 votes
What is Information gain in Decision tree algorithm? Can you explain how it is used? Thank you
May 14, 2019 in Machine Learning by Priya
116 views

1 answer to this question.

0 votes

You can use information gain to decide which attribute goes at which level in the decision tree. By using information gain as a criterion, we try to estimate the information contained by each attribute. We are going to use some points deducted from information theory.
To measure the randomness or uncertainty of a random variable X is defined by Entropy.

For a binary classification problem with only two classes, positive and negative class.

  • If all examples are positive or all are negative then entropy will be zero i.e, low.
  • If half of the records are of positive class and half are of negative class then entropy is one i.e, high.

H(X)=\mathbb {E} _{X}[I(x)]=-\sum _{x\in \mathbb {X} }p(x)\log p(x).

By calculating the entropy measure of each attribute we can calculate their information gain. Information Gain calculates the expected reduction in entropy due to sorting on the attribute. Information gain can be calculated.

answered May 14, 2019 by Raj

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
170 views
0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
76 views
0 votes
1 answer
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 9, 2019 in Machine Learning by Shridhar
98 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 32,440 points
60 views
0 votes
0 answers

What is semi-supervised machine learning?

What is semi-supervised machine learning? READ MORE

May 8, 2019 in Machine Learning by Ganesh
46 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
39 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
167 views
0 votes
1 answer

What is greedy approach in Decision tree algorithm?

“Greedy Approach is based on the concept ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
1,092 views