Lesson 7.6: K-Nearest Neighbors (KNN)
🔹 What is K-Nearest Neighbors?
K-Nearest Neighbors (KNN) is a supervised learning algorithm used for classification and regression.
-
Predicts the class or value of a new data point based on the closest points in the training data.
-
“K” = Number of nearest neighbors considered.
🔹 How KNN Works (Classification)
-
Choose K (e.g., K=5).
-
Calculate distance between the new point and all training points (e.g., Euclidean distance).
-
Identify K closest points.
-
Assign the majority class among these neighbors to the new point.
🔹 Example (Classification)
-
X_train→ Feature data -
y_train→ Class labels -
y_pred→ Predicted class
🔹 Advantages
-
Simple and easy to implement.
-
No assumption about data distribution.
-
Works for both classification and regression.
🔹 Disadvantages
-
Computationally expensive for large datasets.
-
Sensitive to irrelevant features and feature scaling.
✅ Quick Recap:
-
KNN → Predicts based on closest neighbors.
-
K choice and distance metric are crucial for performance.
