What is greedy approach in Decision tree algorithm?

0 votes

I was just reading about decision trees and I came around this term called greedy approach. Can you please explain what that is?

May 13, 2019 in Machine Learning by Kavya
1,092 views

1 answer to this question.

0 votes

“Greedy Approach is based on the concept of Heuristic Problem Solving by making an optimal local choice at each node. By making these local optimal choices, we reach the approximate optimal solution globally.”

The algorithm can be summarized as :

1. At each stage (node), pick out the best feature as the test condition.

2. Now split the node into the possible outcomes (internal nodes).

3. Repeat the above steps until all the test conditions have been exhausted into leaf nodes.

When you start to implement the algorithm, the first question is: ‘How to pick the starting test condition?’

To make that decision, you need to have some knowledge about entropy and information gain.

Entropy: Entropy in Decision Tree stands for homogeneity. If the data is completely homogenous, the entropy is 0, else if the data is divided (50-50%) entropy is 1.

Information Gain: Information Gain is the decrease/increase in Entropy value when the node is split.

An attribute should have the highest information gain to be selected for splitting. Based on the computed values of Entropy and Information Gain, we choose the best attribute at any particular step.

answered May 13, 2019 by Upadhya

Related Questions In Machine Learning

0 votes
1 answer

What is Gini Index in decision tree

Gini Index is a metric to measure ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
170 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

What is clustering in Machine Learning?

Clustering is a type of unsupervised learning ...READ MORE

answered May 9, 2019 in Machine Learning by Shridhar
98 views
0 votes
2 answers

What is supervised learning?

Supervised learning is the machine learning task ...READ MORE

answered Sep 3, 2019 in Data Analytics by anonymous
• 32,440 points
60 views
0 votes
0 answers

What is semi-supervised machine learning?

What is semi-supervised machine learning? READ MORE

May 8, 2019 in Machine Learning by Ganesh
46 views
0 votes
1 answer

What is Reinforcement learning?

Alright, let me not get into the ...READ MORE

answered May 8, 2019 in Machine Learning by Alok
39 views
0 votes
1 answer
0 votes
1 answer

What is decision tree algorithm?

A decision tree is a map of ...READ MORE

answered May 13, 2019 in Machine Learning by Upadhya
167 views
0 votes
1 answer

What is information gain? - Decision tree algo

You can use information gain to decide ...READ MORE

answered May 14, 2019 in Machine Learning by Raj
116 views