The same model, the same confusion matrix, and two summary numbers that tell almost opposite stories — because one of them has a huge pile of true negatives quietly propping it up.
Both ROC-AUC and PR-AUC summarize a classifier across every possible decision threshold in a single number between 0 and 1. On a roughly balanced dataset they usually agree closely enough that it doesn't matter which one you report. Under severe class imbalance — fraud detection, rare-disease screening, defect detection on a production line where 99%+ of parts pass — they can disagree dramatically, and the disagreement is not a coincidence. It comes directly from what sits in each metric's denominator.
ROC-AUC is built from TPR and FPR, and both of those rates are normalized by class size — TPR divides by the total positives, FPR divides by the total negatives. That normalization is exactly what makes ROC-AUC insensitive to the positive/negative ratio, which is a real strength when you genuinely care about both classes symmetrically. But under severe imbalance it becomes the problem: with 1,000 negatives sitting in the denominator, the model can generate 50 false positives — a number an operations team would consider a serious false-alarm problem — and FPR barely moves. PR-AUC swaps FPR's huge, imbalance-inflated denominator for precision's denominator, TP + FP— the count of alerts actually raised, with no true negatives anywhere in the formula. That's why PR-AUC is the more honest metric for rare-positive-class problems: it reports performance in terms of the class you actually care about finding, not diluted by a negative class that's trivially easy to get right by doing nothing.
Not under class imbalance — and the two curves above are the same model. A ROC-AUC of 0.95 is a genuinely good number on a balanced 50/50 dataset. On a 1%-positive dataset, an AUC that high is common even for models with a serious false-alarm problem, because the metric is being computed against a sea of easy true negatives that were never the hard part of the task. The failure mode isn't that ROC-AUC is "wrong" — it's mathematically doing exactly what it's defined to do. The failure is reporting it alone as proof of quality on a rare-positive-class problem without also checking PR-AUC, precision at a realistic operating threshold, or the raw false-positive count a human or downstream system will actually have to deal with. A fraud model, a rare-disease screen, or a defect detector should always be evaluated with PR-AUC (or precision/recall at threshold) as the primary number when the positive class is rare — ROC-AUC is a useful secondary check, not the headline metric.
Explains why ROC-AUC and PR-AUC can tell opposite stories about the same classifier under severe class imbalance, using a shared 1%-fraud-rate confusion matrix to show how ROC-AUC's false-positive-rate denominator (dominated by true negatives) hides a false-alarm problem that PR-AUC's precision denominator exposes directly.
Both metrics summarize the same underlying tradeoff — how well a classifier ranks positives above negatives across every threshold — and on roughly balanced data they usually move together, so many practitioners default to reporting ROC-AUC out of habit without checking whether their dataset is actually balanced. The confusion shows up the moment someone applies that habit to a rare-event problem: a model can post an eye-catching 0.95 ROC-AUC while still being nearly useless in production, because ROC-AUC was never measuring the thing that matters operationally — how trustworthy a positive prediction actually is.
ROC-AUC plots True Positive Rate (TPR = TP / (TP+FN)) against False Positive Rate (FPR = FP / (FP+TN)) as the decision threshold sweeps from 0 to 1. Both axes are rates normalized by class size, which is why ROC-AUC is invariant to the positive/negative ratio — a real strength for balanced problems.
PR-AUC plots Precision (TP / (TP+FP)) against Recall (TP / (TP+FN)). Precision has no true-negative term at all, so it is directly sensitive to how many of the positive predictions were actually correct — exactly the quantity that determines how much false-alarm noise a human reviewer or downstream automation has to absorb. When negatives vastly outnumber positives, FP can be large in absolute terms while still being tiny relative to TN, so FPR barely rises even as precision collapses.
Use PR-AUC as the primary metric whenever the positive class is rare and false positives carry a real cost — fraud detection, rare-disease screening, anomaly and defect detection, intrusion detection. Use ROC-AUC when classes are roughly balanced, or when both classes genuinely matter symmetrically (a coin-flip-adjacent binary decision with no rare minority class). In practice it rarely hurts to report both alongside a confusion matrix at the operating threshold you actually plan to deploy — the single-number summary is convenient for model comparison, but the confusion matrix is what tells you whether the number means what you think it means.
It's rare but possible — the two curves are built from different axis pairs, so a model that ranks a few borderline cases differently can shift them slightly apart even on balanced data. In practice the large, consistent gaps that matter for decision-making almost always trace back to class imbalance.
There's no hard cutoff, but once the positive class drops below roughly 10% of the dataset, ROC-AUC starts becoming noticeably optimistic relative to real-world precision, and below 1-2% (fraud, rare disease, defect rates) the gap can be dramatic, as in the fraud example above. When in doubt, plot both curves — the divergence itself tells you which one to trust.
No — PR-AUC is bounded above by the class prevalence in a meaningful sense (a random classifier scores PR-AUC ≈ prevalence, not 0.5 like ROC-AUC), so a PR-AUC of 0.35 on a 1%-prevalence problem can still represent a huge improvement over random guessing. Always compare PR-AUC against that no-skill baseline (the prevalence), not against an absolute threshold like 0.5 or 0.9.
Average Precision is the standard way PR-AUC is actually computed in most libraries (including scikit-learn's average_precision_score) — it's a step-function approximation of the area under the precision-recall curve that avoids the interpolation artifacts a naive trapezoidal AUC calculation can introduce on a curve that isn't monotonic. For practical purposes, "PR-AUC" and "average precision" refer to the same quantity.
Not necessarily retrain differently, but definitely evaluate differently: check precision and recall at the specific threshold you plan to deploy, consider class weighting or resampling (SMOTE, undersampling the majority class) during training if precision at usable recall is still too low, and make sure your train/test split preserves the true class ratio (stratified splitting) so the PR-AUC estimate itself is trustworthy.
Try our AI and Data Science Studio
More calculators, simulators, and guides for this discipline.