Learning AI Part 3 — Supervised Learning Algorithms

What is Supervised Learning?

Supervised Learning is one of the most common types of Machine Learning.

The word "supervised" means the algorithm learns under the guidance of someone who already knows the correct answers.

So its a pattern where a student learns from the teacher.

Machine learning works exactly the same way.

The algorithm is given:

• Input data (Features)
• Correct output (Labels)

It studies thousands or millions of examples until it learns the relationship between the input and output.

After training, it can predict answers for new data it has never seen before.

How Supervised Learning Works

It follows five simple steps.

Labeled Data
       ↓
Training
       ↓
Model Learns Patterns
       ↓
New Data
       ↓
Prediction

Labeled Data
       ↓
Training
       ↓
Model Learns Patterns
       ↓
New Data
       ↓
Prediction

Supervised learning algorithms work similarly. They are fed with a large dataset of labeled examples, and they use this data to train a model that can predict the labels for new, unseen examples.

Types of Supervised Learning

There are two types of supervised learning

1. Classification: Classification predicts categories. In classification predicts the categorical label, like an email is spam or not, an image is animal or not by seeing cat, dogs, etc.
2. Regression: Regression predicts numbers, For example, one could predict the price of a house based on its size, location, and other features or forecast the stock market. like 1000sqft room cost how much value?!.

Core Concepts in Supervised Learning

These are the building blocks of supervised learning.

Training Data

Training data is the dataset used to teach the model.

Training data is the foundation of supervised learning. It is the labeled dataset used to train the ML model. This dataset consists of input features and their corresponding output labels. The quality and quantity of training data significantly impact the model's accuracy and ability to generalize to new, unseen data.

Think of training data as a set of example problems with their correct solutions. The algorithm learns from these examples to develop a model that can solve similar problems in the future.

Features

Features are the measurable properties or characteristics of a dataset that are used as inputs to a machine learning model. They help the model identify patterns in the data and make predictions. Selecting relevant features is important for improving the model's accuracy and overall performance.

Example: When predicting house prices, the features may include:

• Size: The total area of the house.
• Number of Bedrooms: The number of bedrooms in the house.
• Location: The area or neighborhood where the house is located.
• Age of the House: The number of years since the house was built.

Labels

Labels are the known outcomes or target variables associated with each data point in the training set. They represent the "correct answers" that the model aims to predict.

In the house price prediction example, the label would be the actual price of the house.

Model

A model is a mathematical representation of the relationship between input features and output labels. It learns patterns from the training data and uses them to make predictions on new, unseen data. A model can be viewed as a function that takes input features and produces a predicted output.

Training

Training is the process of feeding the training data to the algorithm and adjusting the model's parameters to minimize prediction errors. The algorithm learns from the training data by iteratively adjusting its internal parameters to improve its prediction accuracy.

Prediction

↓

Wrong

↓

Adjust

↓

Predict Again

↓

Better

↓

Repeat

Prediction

↓

Wrong

↓

Adjust

↓

Predict Again

↓

Better

↓

Repeat

Prediction

Once the model is trained, it can be used to predict new, unseen data. This involves providing the model with the features of the new data point, and the model will output a prediction for the label. Prediction is a specific application of inference, focusing on generating actionable outputs such as classifying an email as spam or forecasting stock prices.

New House

↓

Features

↓

Model

↓

Predicted Price

New House

↓

Features

↓

Model

↓

Predicted Price

Inference

Inference is broader.

It means using the trained model to understand the data and explain what it has learned.

Instead of asking

"What is the answer?"

we ask

"Why did the model make this decision?"

Examples

Finding

• Which feature matters the most?
• How strongly does house size affect price?
• Why did the model classify this image as a dog?
• Which variables influence customer churn?

Inference includes prediction but goes beyond it by helping us interpret the model and derive insights.

Evaluation

How do we know if the model is good?

We test it.

Evaluation measures performance.

Common metrics are:

Accuracy : Percentage of correct predictions.

Precision : Out of everything the model predicted as positive, how many were actually positive?

Example (Spam Email)

Model says

20 emails are spam

Actually

18 are spam

2 are normal emails

Precision

Example (Spam Email)

Model says

20 emails are spam

Actually

18 are spam

2 are normal emails

Precision

High precision means few false positives.

Recall : Out of all the actual positive cases, how many did the model successfully find?

F1 Score: Sometimes precision is high but recall is low, or vice versa.

The F1-score combines both into one balanced metric using their harmonic mean. use it when both missing positives and falsely predicting positives matter.

Generalization

This is one of the most important ideas, a model should not just memorize training data, it should also perform well on new, unseen data.

Overfitting

Overfitting means the model memorized the training data instead of learning the underlying patterns.

Imagine a student who memorizes every question in last year's exam.

If the exam repeats,

they score 100%.

If the questions change,

they fail.

That is overfitting.

Underfitting

Underfitting is the opposite.

The model is too simple.

It never learns the patterns properly.

Example

A student studies only one page for the exam.

Result

Bad marks on

• Practice questions
• Actual exam

Similarly, an underfit model performs poorly on both training and new data.

Cross-Validation

How do we know if the model will work well on unseen data?

Instead of training and testing only once, we use cross-validation.

One common approach is k-fold cross-validation:

1. Split the dataset into k equal parts (folds).
2. Train the model on k−1 folds.
3. Test it on the remaining fold.
4. Repeat until every fold has been used as the test set once.
5. Average the results.

Fold 1 → Test
Fold 2–5 → Train

Fold 2 → Test
Fold 1,3,4,5 → Train

...

Average all scores

Fold 1 → Test
Fold 2–5 → Train

Fold 2 → Test
Fold 1,3,4,5 → Train

...

Average all scores

Regularization

Sometimes the model becomes too complex and starts overfitting.

Regularization prevents this by adding a penalty to the model during training if it becomes overly complex.

Think of it like a teacher saying:

"Don't memorize every tiny detail. Learn the important concepts."

This encourages the model to find simpler patterns that generalize better.

There are two common types:

L1 Regularization (Lasso)

Adds a penalty based on the absolute values of the model's coefficients.

• Can shrink some coefficients all the way to 0.
• Effectively removes less important features.
• Useful for feature selection.

L2 Regularization (Ridge)

Adds a penalty based on the squares of the model's coefficients.

• Shrinks coefficients toward zero but usually does not make them exactly zero.
• Keeps all features while reducing the influence of overly large coefficients.
• Often improves stability and reduces overfitting.

Complete Flow of Supervised Learning

Collect Labeled Data
          │
          ▼
Choose Features + Labels
          │
          ▼
Train the Model
          │
          ▼
Model Learns Patterns
          │
          ▼
Evaluate the Model
(Accuracy, Precision, Recall, F1, etc.)
          │
          ▼
Check for Underfitting or Overfitting
          │
          ▼
Use Cross-Validation and Regularization if Needed
          │
          ▼
Deploy the Model
          │
          ▼
Predict on New, Unseen Data (Prediction)
          │
          ▼
Explain Insights or Relationships (Inference)

Collect Labeled Data
          │
          ▼
Choose Features + Labels
          │
          ▼
Train the Model
          │
          ▼
Model Learns Patterns
          │
          ▼
Evaluate the Model
(Accuracy, Precision, Recall, F1, etc.)
          │
          ▼
Check for Underfitting or Overfitting
          │
          ▼
Use Cross-Validation and Regularization if Needed
          │
          ▼
Deploy the Model
          │
          ▼
Predict on New, Unseen Data (Prediction)
          │
          ▼
Explain Insights or Relationships (Inference)