Lesson 9.1: Feature Selection Techniques
🔹 What is Feature Selection?
Feature Selection is the process of choosing the most important features from your dataset that contribute most to the prediction.
-
Helps reduce overfitting, improve model performance, and speed up computation.
🔹 Types of Feature Selection
-
Filter Methods
-
Select features based on statistical measures.
-
Examples: Correlation, Chi-square, ANOVA.
-
Wrapper Methods
-
Use model performance to evaluate subsets of features.
-
Examples: Recursive Feature Elimination (RFE), Forward/Backward Selection.
-
Embedded Methods
-
Feature selection is part of the model training process.
-
Examples: Lasso (L1), Decision Tree Feature Importance.
🔹 Example (Using RFE)
-
n_features_to_select→ Number of top features to keep -
rfe.support_→ Boolean mask of selected features
🔹 Advantages
-
Improves model accuracy by removing irrelevant features.
-
Reduces training time.
-
Helps understand feature importance.
🔹 Disadvantages
-
Wrapper methods can be computationally expensive.
-
May remove useful features if not carefully chosen.
✅ Quick Recap:
-
Feature Selection → Pick features that maximize model performance.
-
Methods → Filter, Wrapper, Embedded.
