Precision vs recall: what they measure and when to optimize each

Precision tells you, of everything your model labeled positive, what fraction was actually positive. Recall tells you, of all the truly positive items, what fraction your model successfully found. Formally, precision = TP / (TP + FP) and recall = TP / (TP + FN) where TP, FP, FN come from the confusion matrix (Google ML Crash Course: Accuracy, precision, recall).

Key definitions and the confusion matrix

• True positive (TP): an actual positive predicted as positive
• False positive (FP): an actual negative predicted as positive
• False negative (FN): an actual positive predicted as negative
• True negative (TN): an actual negative predicted as negative

These four counts compose the confusion matrix and underpin precision and recall (Google ML Crash Course: Thresholds and the confusion matrix).

Metric	What it answers	Formula	Optimize when
Precision	Of predicted positives, how many are correct?	TP / (TP + FP)	False positives are costly.
Recall	Of actual positives, how many were found?	TP / (TP + FN)	False negatives are costly.

Guidance on when to prioritize each metric is provided in the Crash Course: emphasize recall when missing positives is expensive; emphasize precision when being wrong about positives is expensive (Google ML Crash Course: Accuracy, precision, recall).

A small worked example with explicit relevance set

In this hypothetical example, define the task and universe:

• Candidate universe: 12 items your system could label as relevant (positives) or not (negatives).
• Ground truth relevance: 5 of the 12 are actually relevant (this must be known to compute recall).

Model results:

• Predicted positives: 4 items. Of these, 3 are truly relevant and 1 is not. So TP = 3, FP = 1.
• Missed relevant items: 2 of the 5 relevant items were not predicted positive. So FN = 2.

Calculations:

• Precision = TP / (TP + FP) = 3 / (3 + 1) = 3/4 = 0.75.
• Recall = TP / (TP + FN) = 3 / (3 + 2) = 3/5 = 0.60.

Why defining the universe matters: precision depends only on the items you predicted positive, but recall depends on all actual positives. If you don’t know how many relevant items exist in the universe, you can’t directly compute recall; at best, you estimate it over a judged pool.

Why precision and recall trade off

Most binary classifiers output scores that you convert to positive/negative using a threshold. Raising the threshold typically reduces both true and false positives, which tends to increase precision (fewer false positives) but decrease recall (more false negatives). Lowering the threshold does the opposite (Google ML Crash Course: Thresholds and the confusion matrix; Accuracy, precision, recall). Because precision improves as FP decreases and recall improves as FN decreases, tuning the threshold usually reveals an inverse relationship between them (Accuracy, precision, recall).

When datasets are imbalanced, precision-recall curves can be more informative than ROC curves for comparing models across thresholds (ROC and AUC).

Practical choices: when to favor each

• Favor recall when missing a true positive is high cost or risk (for example, safety-critical detection). The goal is to catch as many actual positives as possible, accepting some false alarms if necessary (Accuracy, precision, recall).
• Favor precision when a false alarm is expensive (for example, escalating a customer case incorrectly). The goal is that positive labels are trustworthy, even if some positives are missed (Accuracy, precision, recall).

Classification metrics vs AI citation visibility

Precision and recall evaluate a classifier against known ground truth labels. They do not measure whether an AI answer cites your site. AI citation visibility is a separate question about how often and where a system chooses to show sources.

• For what a citation is and how often they are correct, see How accurate are AI citations?
• Mean Reciprocal Rank (MRR) measures the average rank position of the first correct item, not precision or recall. For an overview, see What does the MRR metric represent?

Checklist: evaluate with precision and recall

• Define the candidate universe: what set of items are eligible to be labeled in this evaluation.
• Establish ground truth: which items in that universe are actually positive (relevant). Without this, recall cannot be computed.
• Choose a threshold (for score-based models) and record TP, FP, FN, TN on a held-out set.
• Compute precision = TP / (TP + FP); compute recall = TP / (TP + FN) (Accuracy, precision, recall).
• Inspect the tradeoff by varying the threshold; use precision-recall curves for imbalanced data (ROC and AUC).
• Pick the operating point that reflects your error costs: prioritize recall if false negatives are worse; prioritize precision if false positives are worse (Accuracy, precision, recall).

FAQs

Is high precision always better than high recall?

No. Which to prioritize depends on the relative cost of false positives versus false negatives in your application. The Crash Course recommends optimizing recall when false negatives are more expensive, and precision when false positives are more expensive (Accuracy, precision, recall).

Does changing the threshold change precision and recall?

Yes. Increasing the classification threshold generally decreases both true and false positives (often raising precision and lowering recall). Decreasing the threshold generally increases both true and false positives (often lowering precision and raising recall) (Thresholds and the confusion matrix).

What if my dataset is highly imbalanced?

Accuracy can be misleading. Consider precision, recall, and F1, and compare models using precision-recall curves, which are better suited to imbalanced data (Accuracy, precision, recall; ROC and AUC).