What Is a Feature?

You want a program to price houses.

A model cannot look at brick, mortar or a garden. It only crunches numbers (from topic 25: at bottom, everything is a numeric array).

So the question becomes

You answer that question with features.

A feature is

• A sample is one thing you want to predict about — one house, one email, one patient.
• A feature is one number describing it — area, bedrooms, age.

Collect the features for every sample as columns and you get the design matrix X:

A model learns a mapping from this matrix to a target value (the answer you want, e.g., the price).

In scikit-learn the convention is fixed and worth memorizing:

• X is 2-D with shape (n_samples, n_features)
• the target y is 1-D with shape (n_samples,)

Features are numeric in the matrix even when the original data was text or categorical — turning them numeric is exactly what encoding does (a later topic).

import numpy as np

# 3 houses; features = [area_sqm, bedrooms, age_years]
X = np.array([
    [70, 2, 10],
    [95, 3, 5],
    [120, 3, 2],
])
y = np.array([250, 340, 410])   # price in $k
print(X.shape)   # (3, 3) -> (n_samples, n_features)

Take the X from the code above and read it out loud, like a table:

code
Copy
            area  bedrooms  age      ->  price (y)
house 1      70       2      10     ->   250
house 2      95       3       5     ->   340
house 3     120       3       2     ->   410
             ^ feature columns    ^ one row = one sample

• Row 1 says: "a 70 m², 2-bedroom, 10-year-old house sold for $250k."
• The model's job: learn how the three columns combine to predict the price.
• Sanity peek: as area grows (70 → 95 → 120), price grows (250 → 340 → 410). Area carries signal about price. That is what makes it a good feature.

Notice one house is described by three numbers, and ten houses would be ten rows. Same idea, any size.

Features differ by what they measure, and each type has preferred handling:

• Numerical / continuous — age, price, temperature; any real value in a range.
• Numerical / discrete (count) — number of clicks, rooms; whole numbers.
• Categorical (nominal) — color, city; unordered labels. "Red" is not greater than "blue".
• Ordinal — size S/M/L; ordered labels with meaningful rank. M really is between S and L.
• Binary — yes/no, 0/1 flags.
• Datetime — timestamps, often broken into year, month, day-of-week, hour.

Columns and features are not the same thing.

• Not every column should be a feature. An ID column is usually dropped (no signal).
• One column can spawn several features. A single timestamp can yield hour-of-day, is-weekend and days-since-signup.

Carry this analogy through the topic: your model is a chef, and the features are the ingredients.

A world-class chef cannot cook a great meal from rotten tomatoes.

• Great ingredients (informative features) → even a simple recipe (a linear model) shines.
• Bad ingredients (useless columns) → no recipe saves the dish.

Good ML is less about exotic models and more about representing information through the right features.

A model can only exploit signal that is present in its inputs.

Then no amount of tuning helps — this is why feature engineering and feature selection (later topics) have outsized impact.

The failure modes to remember:

• Irrelevant features add noise and slow training.
• Redundant features (the same information twice) do the same, and can destabilize some models.
• Leakage-prone features (information that would not exist at prediction time — a separate topic) make evaluation dishonest: the model looks brilliant in testing and fails in production.

Why AI cares: every model in this course — regression, trees, neural nets, LLM training data — consumes features. Before you ever choose an algorithm, you decide what to feed it. That decision is the first and often the biggest lever on accuracy.