Python Mean, Median, Mode and Deviation

Mean and median are the first things to compute for every numeric feature and target:

from statistics import mean, median
delivery_hours = [2, 2, 3, 3, 10]
print(mean(delivery_hours), median(delivery_hours))   # 4 3

A large gap between them signals skew: here one 10-hour delivery lifts the mean to 4 while the typical delivery takes 3. That affects ML in two ways:

• Baselines: predicting the mean minimises average squared error; predicting the median minimises average absolute error. Pick the baseline that matches the metric you'll report, because a model should be compared with the strongest simple alternative.
• Missing values: filling gaps with the mean drags imputed values towards outliers; the median is the usual safer choice for skewed features.

Compute these on the training data only, then apply the same values to validation and test data, as explained in the last section.

The mode, the most frequent value, matters most for categorical features, such as delivery region, vehicle type or day of week, where mean and median don't exist:

from statistics import multimode
regions = ["north", "south", "north", "east", "south"]
print(multimode(regions))     # ['north', 'south']: a tie

multimode() returns every value tied for most frequent, so ties are visible instead of being broken arbitrarily. In ML, the mode is the standard way to fill missing categorical values and the natural baseline for classification: always predicting the most common class (DummyClassifier(strategy="most_frequent") in scikit-learn).

That baseline is also a warning. If 92% of parcels arrive on time, a classifier that always says "on time" is 92% accurate and completely useless. Knowing the mode and its share of the data tells you what accuracy a model must beat before it's worth anything.

Standard deviation measures typical distance from the mean, in the feature's own units. In ML it's used to standardise features into z-scores:

z = (value − mean) ÷ standard deviation

After standardisation every feature has mean 0 and standard deviation 1. That matters because many algorithms, including k-nearest neighbours, k-means, regularised regression and neural networks, compare or penalise features numerically. Without scaling, a feature measured in metres would dominate one measured in kilometres purely because its numbers are bigger.

from statistics import mean, pstdev
distance_km = [1.2, 3.5, 0.8, 5.1, 2.4]
mu, sigma = mean(distance_km), pstdev(distance_km)
print([round((d - mu) / sigma, 2) for d in distance_km])

scikit-learn's StandardScaler uses the population standard deviation (dividing by n), as pstdev() does. Tree-based models such as decision trees and random forests don't need scaling, because they split on thresholds rather than distances.

Summary statistics are learned from data, so in ML they're part of the model and must follow the same rules as the model:

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X_train, X_test = train_test_split(X, test_size=0.25, random_state=0)
scaler = StandardScaler().fit(X_train)      # learn mean and sd from training rows only
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)    # reuse the training statistics

Computing the mean and standard deviation on the whole dataset before splitting lets information about the test rows leak into training, which makes test scores slightly optimistic. Putting the scaler inside a Pipeline with the model guarantees it's refitted correctly inside every cross-validation fold.

Summaries also compress: two features with the same mean and standard deviation can have very different shapes, such as one cluster versus two. Look at percentiles and a histogram too, as in data distribution and percentiles.

Task: describe delivery durations with all four summaries and explain how one long delay changes their interpretation.

from statistics import mean, median, multimode, pstdev

normal = [2, 3, 3, 4, 3]
with_delay = normal + [15]

for label, hours in [("normal week", normal), ("with one delay", with_delay)]:
    print(f"{label:<15} mean {mean(hours):.2f}  median {median(hours)}  "
          f"mode {multimode(hours)}  sd {pstdev(hours):.2f}")

normal week     mean 3.00  median 3  mode [3]  sd 0.63
with one delay  mean 5.00  median 3.0  mode [3]  sd 4.51

One 15-hour delay moves the mean from 3 to 5 and multiplies the standard deviation sevenfold, while the median and mode stay at 3. A scaler fitted on the second week would compress all normal deliveries into a narrow band of z-scores. For the worked check, multimode helps represent ties among most frequent values.

scikit-learn's preprocessing guide explains scaling, and its common pitfalls page covers leakage. Continue with data distribution and percentiles and linear regression. Practise with the Python exercises and Python quiz.