What is information gain? - Decision tree algo

0 votes
What is Information gain in Decision tree algorithm? Can you explain how it is used? Thank you
May 14 in Machine Learning by Priya
62 views

1 answer to this question.

0 votes

You can use information gain to decide which attribute goes at which level in the decision tree. By using information gain as a criterion, we try to estimate the information contained by each attribute. We are going to use some points deducted from information theory.
To measure the randomness or uncertainty of a random variable X is defined by Entropy.

For a binary classification problem with only two classes, positive and negative class.

  • If all examples are positive or all are negative then entropy will be zero i.e, low.
  • If half of the records are of positive class and half are of negative class then entropy is one i.e, high.

H(X)=\mathbb {E} _{X}[I(x)]=-\sum _{x\in \mathbb {X} }p(x)\log p(x).

By calculating the entropy measure of each attribute we can calculate their information gain. Information Gain calculates the expected reduction in entropy due to sorting on the attribute. Information gain can be calculated.

answered May 14 by Raj

Related Questions In Machine Learning

0 votes
1 answer
0 votes
1 answer

What is Unsupervised Learning?

Unsupervised Learning is the training of machine ...READ MORE

answered May 8 in Machine Learning by Alok
47 views
0 votes
1 answer
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 9 in Machine Learning by Shridhar
69 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3 in Data Analytics by anonymous
• 31,840 points
42 views
0 votes
0 answers

What is semi-supervised machine learning?

What is semi-supervised machine learning? READ MORE

May 8 in Machine Learning by Ganesh
28 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer