O&A Consulting · Answer

How do you measure whether your retrieval pipeline is any good?

Three independent layers — and never let a metric that improves when you discard data stand on its own. Intrinsic scores like silhouette measure whether clusters are tidy, not whether they are useful, and they are flattered by throwing data away: dropping the hardest 85% of one corpus raised its agreement score from 0.62 to 0.87 with no change to the method at all. Compare at equal coverage.

RetrievalClusteringMeasurementRAGEvaluation
01 Answer

The metric that lies

Intrinsic clustering metrics reward compact, well-separated groups. Nothing in them cares whether the groups mean anything, and — more dangerously — nothing stops them improving when you simply exclude the hard cases.

Measured directly: taking a fixed partition and discarding its worst 85% raised agreement with a reference partition from 0.62 to 0.87. The method didn't change. The data did. Any metric computed only over surviving points is an average over survivors, and none of them is immune to this — so the only honest comparison between two configurations is one at equal coverage. A config that wins on silhouette while keeping three clusters and discarding most of the corpus has won nothing.

02 Answer

Three layers, deliberately independent

  • Geometric. Silhouette, Davies-Bouldin, and friends. Cheap and worth having. Necessary, nowhere near sufficient, and never reported alone.
  • Agreement with a differently-built reference. Build a partition by a genuinely different mechanism — a graph community algorithm against an embedding-based one — and measure chance-corrected agreement. Convergence between methods that fail differently is real evidence. State the limit honestly: they are independent in algorithm, not in input, since both read the same embeddings.
  • A semantic bar. Can a person name the cluster? If not, it is noise, whatever it scored. This is the layer that catches the tidy, meaningless partition the first two will happily approve.
03 Answer

Measure chunk quality by decile, not in aggregate

An average over all chunks hides the shape of the failure. Binning by length decile shows it: in one measured corpus the shortest decile sat 2.43× further from its own cluster centroid than the longest, with about half the neighbourhood purity, and the relationship held monotonically across all ten bins. A monotone trend across deciles is a real finding; a single average is a number.

It does not generalise, and that is the point — the same measurement on verse-level chunks showed only 1.17× with the relationship inverted. This is a property of how a particular chunker fails, not a law. Which is exactly why you measure your own pipeline instead of importing someone's rule of thumb.

04 Answer

Edges need two gates, and the second one matters more

For relationships built from similarity, a single threshold on the aggregate score is not enough. Use two:

  • Magnitude. Does the relationship clear a similarity threshold at all.
  • Breadth. Does some minimum fraction of the individual cross-pairs also clear it.

Breadth is the gate that earns its keep, because it catches a high average carried by a handful of outlier pairs — two clusters joined by one document that happens to mention both. And the threshold should come from calibration against your observed distribution, never from a number that sounds strong.

05 Answer

Where measurement belongs in the pipeline

Computation is distributed — each stage knows something the others don't — but enforcement should be centralised. One checkpoint that asks "is this signal or noise" beats six stages each grading their own homework, for the same reason authors don't approve their own pull requests.

In practice that means measurement earns its own stage, with output a human reads before adjusting configuration for the next run. Real changes that came out of running it that way: an edge threshold moved from 0.10 to 0.35, a character-length gate derived from the decile analysis above, and a clustering algorithm chosen on an agreement sweep rather than a preference.

06 Further

Where this is argued properly

The short answer above is ours, and we've written at length about how we got to it: