What is information gain? - Decision tree algo

0 votes
What is Information gain in Decision tree algorithm? Can you explain how it is used? Thank you
May 14, 2019 in Machine Learning by Priya
166 views

1 answer to this question.

0 votes

You can use information gain to decide which attribute goes at which level in the decision tree. By using information gain as a criterion, we try to estimate the information contained by each attribute. We are going to use some points deducted from information theory.
To measure the randomness or uncertainty of a random variable X is defined by Entropy.

For a binary classification problem with only two classes, positive and negative class.

  • If all examples are positive or all are negative then entropy will be zero i.e, low.
  • If half of the records are of positive class and half are of negative class then entropy is one i.e, high.

H(X)=\mathbb {E} _{X}[I(x)]=-\sum _{x\in \mathbb {X} }p(x)\log p(x).

By calculating the entropy measure of each attribute we can calculate their information gain. Information Gain calculates the expected reduction in entropy due to sorting on the attribute. Information gain can be calculated.

answered May 14, 2019 by Raj

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
279 views
0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
122 views
0 votes
1 answer
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 9, 2019 in Machine Learning by Shridhar
140 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 32,460 points
94 views
0 votes
0 answers

What is semi-supervised machine learning?

What is semi-supervised machine learning? READ MORE

May 8, 2019 in Machine Learning by Ganesh
79 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
85 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
279 views
0 votes
1 answer

What is greedy approach in Decision tree algorithm?

“Greedy Approach is based on the concept ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
2,356 views