When should we use Principal Component Analysis?

Question

In machine learning, more features or dimensions can decrease a model’s accuracy since there is more data that needs to be generalized and this is known as the curse of dimensionality.

Dimensionality reduction is a way to reduce the complexity of a model and avoid overfitting. Principal Component Analysis (PCA) algorithm is used to compress a dataset onto a lower-dimensional feature to reduce the complexity of the model.

When/How should I consider that my data set has many numbers of features and I should look for PCA for dimension reduction?

Answer 1

Let me provide another view into this.

In general, you can use Principal Component Analysis for two main reasons:

1. For compression:

   • To reduce space to store your data, for example.
   • To speed up your learning algorithm (selecting the principal components with more variance). Looking at the cumulative variance of the components.

2. For visualization purposes, using 2 or 3 components.