Two different ways to rescale a feature — and two different sets of assumptions that make picking the wrong one a real, common mistake.
"Just rescale the features" sounds like one step with one right answer, but it isn't. Normalization (most commonly min-max scaling) and standardization(Z-score scaling) both take a feature's raw values and put them on a comparable scale — but they do it with different formulas, produce outputs with fundamentally different properties, and fail in different ways when a dataset has outliers or a non-normal shape. Treating them as interchangeable synonyms for "scaling" is one of the most common tuning mistakes in applied data science.
Min-max normalization rescales a feature to a fixed, bounded range — almost always [0, 1] — by subtracting the feature's minimum value and dividing by its range (max minus min): x' = (x − min) / (max − min). Every rescaled value is guaranteed to land inside that bounded interval, and the formula makes no assumption whatsoever about the shape of the underlying distribution. The cost of that guarantee is that min and max are exactly the two most outlier-sensitive statistics a dataset has — a single extreme value becomes one of the two endpoints that define the entire scale.
Standardization (Z-score scaling) instead subtracts the feature's mean and divides by its standard deviation: z = (x − μ) / σ, producing a rescaled feature with mean 0 and standard deviation 1. Critically, standardization does not produce a fixed bounded range — a value can still be arbitrarily large or small, it's just expressed in units of "standard deviations from the mean" instead of raw units. Standardization implicitly works best when a feature is reasonably close to a normal (Gaussian-like) distribution, which is exactly why methods that assume normally distributed features — linear regression's coefficient interpretation, PCA — pair naturally with it. And because the mean and standard deviation are computed from every point rather than just the two extremes, a single outlier shifts them only modestly rather than redefining the entire scale.
Min-max normalization's fixed [0, 1] output is exactly what certain algorithms require — bounded activation functions, image pixel-intensity scaling, distance metrics that assume every feature contributes on a comparable, bounded scale. But that guarantee is purchased by anchoring the entire scale to the dataset's single minimum and single maximum, which is precisely what a bad or extreme data point corrupts. Standardization never promises a bounded range at all — a rescaled value can still be arbitrarily large — but because the mean and standard deviation are computed by averaging over everypoint, one extreme value only nudges them, it doesn't redefine them. That's also why standardization pairs naturally with methods that assume roughly normal-distributed features (linear regression coefficient interpretation, PCA's variance-maximizing directions): those methods are already reasoning in terms of mean and spread, which is exactly what a Z-score is built from.
False, and it's a genuinely common source of degraded model performance, not an inconsequential stylistic choice. Normalization's fixed, bounded output range makes it useful — sometimes required — for algorithms built around bounded inputs, but it is highly vulnerable to outliers: a single extreme value can compress every other value in a feature into a tiny, low-resolution sliver of the range, as the diagram above shows. Standardization has no fixed bounded range at all, which can itself be a problem for algorithms that specifically need bounded [0,1] inputs — but it is considerably more robust to outliers and pairs more naturally with methods that assume roughly normal-distributed features. Picking normalization on a dataset with real outliers, or picking standardization for an algorithm that specifically requires a bounded 0–1 input range, are both genuine, common data-science tuning mistakes— the two techniques solve related but distinct problems, and the "right" one depends on the actual outlier profile of your data and the actual input requirements of your algorithm, not on habit.
Explains the two most common feature-rescaling techniques in data science and machine learning — min-max normalization, which maps a feature to a fixed bounded range but is highly sensitive to outliers, and Z-score standardization, which has no fixed bounded range but is considerably more robust to outliers and pairs naturally with methods assuming roughly normal-distributed features — and why treating them as interchangeable is a real, common source of degraded model performance.
Both techniques get lumped together under the umbrella of "feature scaling" or "putting features on the same scale," which is technically true but hides the fact that they have opposite strengths and weaknesses. Min-max normalization guarantees a bounded [0,1] output regardless of distribution shape, at the cost of being defined entirely by the dataset's two most extreme points (min and max) — so a single outlier can compress every other value into a tiny sliver of that range. Standardization guarantees no such bound, but computes its parameters (mean, standard deviation) as averages across every point, making it far less fragile to any one extreme value, and it implicitly assumes the feature is reasonably close to normally distributed.
Min-max normalization: x' = (x − min) / (max − min), which maps the minimum value to 0, the maximum value to 1, and every other value to a proportional point in between. Because min and max are single data points, they carry zero averaging protection against an outlier — if one point in the dataset is unusually extreme, it directly becomes the new min or max, and the range (max − min) grows to match it, shrinking every other rescaled value toward the opposite end of the interval.
Z-score standardization: z = (x − μ) / σ, where μ is the feature's mean and σ its standard deviation, computed across the entire dataset. Because both μ and σ are averages over every observation, one extreme value pulls them only in proportion to its share of the total (shrinking as the sample size grows), rather than redefining the scale outright the way min/max do. The resulting z-values have mean 0 and standard deviation 1 by construction, but carry no upper or lower bound — an outlier can still produce an arbitrarily large |z|, it just does so without distorting the values around it nearly as much.
Reach for min-max normalization when an algorithm specifically requires bounded inputs — certain neural network activation functions, image pixel-intensity scaling, distance-based methods where you want every feature confined to a comparable [0,1] interval — and when the feature's outlier profile is genuinely clean, or outliers have already been handled (capped, removed, or transformed) beforehand. Reach for standardization when the feature is reasonably close to normally distributed, when the downstream method assumes or benefits from that (linear/logistic regression coefficient interpretation, PCA, many regularized linear models, k-means and other methods using Euclidean distance across differently-scaled features), or when the data has outliers you can't simply discard, since standardization degrades far more gracefully than a fixed min-max range does.
No — both are linear (affine) transformations. Subtracting a constant and dividing by a constant preserves the relative spacing and shape of the distribution; it does not make a skewed distribution normal, and it does not remove outliers. Standardization "assumes" normality only in the sense that its output is most interpretable and most useful to downstream methods when the input is roughly normal — it does not enforce or create normality.
It depends on the architecture and the input type. Image pixel values are conventionally normalized to [0,1] or [-1,1] to match how activation functions and image processing pipelines expect inputs. For tabular features feeding standard dense layers, standardization is often preferred, especially when features have differing outlier profiles, since it keeps gradient-based training more numerically stable without one outlier feature dominating the loss landscape.
No. Neither normalization nor standardization removes, caps, or corrects an outlier — they only rescale the feature. Normalization actually makes an outlier's damage more visible in the rest of the data (by compressing everything else), while standardization dilutes but does not eliminate its influence on the mean and standard deviation. If outliers are a data-quality problem, they generally need to be addressed separately (capping/winsorizing, transformation, robust scaling, or removal) before or instead of a plain min-max/Z-score rescale.
Yes — robust scaling, which subtracts the median and divides by the interquartile range (IQR) instead of the mean/std or min/max, is specifically designed to resist outlier influence, since both the median and IQR are far less sensitive to extreme values than a mean, standard deviation, minimum, or maximum. It's a common third option when a feature has real outliers you don't want to (or can't) remove.
Yes. Whichever technique you use, the min/max (for normalization) or mean/std (for standardization) must be computed only from the training set and then applied unchanged to validation and test data. Fitting the scaler on the full dataset before splitting leaks information from validation/test data into training, inflating apparent performance in a way that won't hold up on genuinely new data.
Try our AI and Data Science Studio
More calculators, simulators, and guides for this discipline.