The validation record is the product.
Almost anyone can claim speed. Very few publish their concordance. This page is intended to become the most-cited thing on the site — and it is empty today, because the pipeline it would measure has not been built.
01 · The commitment
Four things we are binding ourselves to.
Every pipeline version is validated against Genome in a Bottle reference materials before it is used for customer work.
Results are published either way. A regression appears here with the same prominence as an improvement. A validation record containing no failures is evidence of selective publication, not of quality.
Methodology and raw outputs are public, so a third party can reproduce the comparison rather than trusting a number.
No figure appears before it is measured. The table below stays empty rather than being populated from a CPU reference implementation that would not represent the shipping product.
02 · The record
Pending first pipeline run.
| Pipeline | Truth set | SNV F1 | Indel F1 | Run date | Report |
|---|---|---|---|---|---|
| — | NA12878 / HG001 | — | — | — | Not yet run |
| — | NA24385 / HG002 | — | — | — | Not yet run |
Why this matters commercially. A maintained concordance record across every pipeline version is roughly eighteen months of work a competitor cannot skip, and it is the first of three compounding moats — ahead of compliance certification and clinical workflow depth. It is also the primary evidence a CAP bioinformatics inspection asks for, so the scientific work and the accreditation work are the same work.
03 · Method
What is compared, and how.
GIAB provides high-confidence variant calls for extensively characterised reference genomes. A pipeline's output is compared against that truth set to produce precision, recall and F1, reported separately for single-nucleotide variants and for insertions and deletions, because the two have materially different error profiles.
Each published row records the pipeline version, the reference set version, the genome build and the run date — a score without those four is not reproducible and therefore not evidence.
The suite runs monthly and on every pipeline change, so a regression introduced by a dependency bump is caught by the same process that catches one introduced by an algorithm change.