What is greedy approach in Decision tree algorithm

0 votes

I was just reading about decision trees and I came around this term called greedy approach. Can you please explain what that is?

May 13, 2019 in Machine Learning by Kavya
14,162 views

1 answer to this question.

+2 votes

“Greedy Approach is based on the concept of Heuristic Problem Solving by making an optimal local choice at each node. By making these local optimal choices, we reach the approximate optimal solution globally.”

The algorithm can be summarized as :

1. At each stage (node), pick out the best feature as the test condition.

2. Now split the node into the possible outcomes (internal nodes).

3. Repeat the above steps until all the test conditions have been exhausted into leaf nodes.

When you start to implement the algorithm, the first question is: ‘How to pick the starting test condition?’

To make that decision, you need to have some knowledge about entropy and information gain.

Entropy: Entropy in Decision Tree stands for homogeneity. If the data is completely homogenous, the entropy is 0, else if the data is divided (50-50%) entropy is 1.

Information Gain: Information Gain is the decrease/increase in Entropy value when the node is split.

An attribute should have the highest information gain to be selected for splitting. Based on the computed values of Entropy and Information Gain, we choose the best attribute at any particular step.

answered May 13, 2019 by Upadhya

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
2,115 views
0 votes
1 answer
0 votes
1 answer

What is the process involved in machine Learning?

Discussing this on a high level, these ...READ MORE

answered May 10, 2019 in Machine Learning by Rhea
1,294 views
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 10, 2019 in Machine Learning by Shridhar
944 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 33,030 points
1,319 views
0 votes
1 answer

What is semi-supervised machine learning?

Hi@Ganesh, Semi-supervised machine learning is a combination of ...READ MORE

answered Jul 19, 2020 in Machine Learning by MD
• 95,440 points
782 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 9, 2019 in Machine Learning by Alok
1,133 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
2,259 views
0 votes
1 answer

What is information gain? - Decision tree algo

You can use information gain to decide ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
2,136 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP