Linear Discriminant Analysis vs Naive Bayes

Question

What are some advantages and disadvantages of LDA vs Naive Bayes in terms of machine learning classification?

Answer 1

There are no standards fixed as to when to use Linear Discriminant Analysis or Naive Bayes, it depends upon trials and the accuracy of the model by applying both LDA as well as Naive Bayes.
In few data sets LDA might perform well, and in other data sets chances are there that Naive Bayes will give good results.

Disadvantages of Naive Bayes:

1. Not suitable for continuous features the features have to be converted into discrete by applying any of the techniques like one-hot encoding or label encoding, but one hot encoding leads to dummy trap and also causes multicollinearity problems, thus giving poor results.
   2) Naive Bayes makes assumptions that all the features are independent of each other, but this is rarely the case with real-life data.

Advantages of Naive Bayes:

1. Performs better on small data sets, provided features are not correlated and are independent of each other.

2. Works well with categorical features

3. Naive Bayes can be used for multi-class label classification tasks.
    

Advantages of LDA

1. LDA minimizes variance in the dataset by reducing the number of features.

2. It is useful as it reduces the curse of dimensionality by effectively reducing the high-dimensional data into low-dimensional feature space.

Disadvantages of LDA

1. Makes assumption requires features to be normally distributed.

2. Does not give good results in case of unbalanced dataset.

3. Not suitable for non-linear problems.

4. Prone to overfitting