Re-ranking results

Cheap recall, then expensive precision

Retrieve fifty candidates with a fast method, then re-score them with a model that reads the query and the passage together. That two-stage shape gives you accuracy you could not afford across the whole corpus.

A cross-encoder is far more accurate than vector similarity because it can attend to the query while reading the passage — and far too slow to run over everything, which is exactly why it belongs in stage two.

Keeping fewer, better passages usually beats keeping more. Irrelevant context does not just waste tokens, it actively distracts the generation step.