Can ROC-AUC and PR-AUC ever disagree even without class imbalance?

It's rare but possible — the two curves are built from different axis pairs, so a model that ranks a few borderline cases differently can shift them slightly apart even on balanced data. In practice the large, consistent gaps that matter for decision-making almost always trace back to class imbalance.

Is there a rule of thumb for how imbalanced is "severe enough" to switch to PR-AUC?

There's no hard cutoff, but once the positive class drops below roughly 10% of the dataset, ROC-AUC starts becoming noticeably optimistic relative to real-world precision, and below 1-2% (fraud, rare disease, defect rates) the gap can be dramatic, as in the fraud example above. When in doubt, plot both curves — the divergence itself tells you which one to trust.

Does a low PR-AUC always mean the model is bad?

No — PR-AUC is bounded above by the class prevalence in a meaningful sense (a random classifier scores PR-AUC ≈ prevalence, not 0.5 like ROC-AUC), so a PR-AUC of 0.35 on a 1%-prevalence problem can still represent a huge improvement over random guessing. Always compare PR-AUC against that no-skill baseline (the prevalence), not against an absolute threshold like 0.5 or 0.9.

How does average precision (AP) relate to PR-AUC?

Average Precision is the standard way PR-AUC is actually computed in most libraries (including scikit-learn's average_precision_score) — it's a step-function approximation of the area under the precision-recall curve that avoids the interpolation artifacts a naive trapezoidal AUC calculation can introduce on a curve that isn't monotonic. For practical purposes, "PR-AUC" and "average precision" refer to the same quantity.

Should I retrain the model differently if PR-AUC is low but ROC-AUC is high?

Not necessarily retrain differently, but definitely evaluate differently: check precision and recall at the specific threshold you plan to deploy, consider class weighting or resampling (SMOTE, undersampling the majority class) during training if precision at usable recall is still too low, and make sure your train/test split preserves the true class ratio (stratified splitting) so the PR-AUC estimate itself is trustworthy.

← AI and Data Science Studio

Concept Explainer · AI & Data Science

ROC-AUC vs. PR-AUC

The same model, the same confusion matrix, and two summary numbers that tell almost opposite stories — because one of them has a huge pile of true negatives quietly propping it up.

Both ROC-AUC and PR-AUC summarize a classifier across every possible decision threshold in a single number between 0 and 1. On a roughly balanced dataset they usually agree closely enough that it doesn't matter which one you report. Under severe class imbalance — fraud detection, rare-disease screening, defect detection on a production line where 99%+ of parts pass — they can disagree dramatically, and the disagreement is not a coincidence. It comes directly from what sits in each metric's denominator.