Accuracy you can checkrather than take on trust.
Every pipeline version is run against Genome in a Bottle reference materials, and the concordance record is published in full — including the runs that did not improve on the previous version.
01 · Method
Public truth sets, published methodology.
Genome in a Bottle is a set of extensively characterised reference genomes produced by a public consortium, with high-confidence variant calls that function as ground truth. Comparing a pipeline's output against GIAB yields precision, recall and an F1 score for single-nucleotide variants and for insertions and deletions separately.
The suite runs monthly and on every pipeline change. Results are published either way — a regression is as informative as an improvement, and a validation record with no failures in it is not a validation record.
Methodology and raw outputs are published alongside every run so a third party can reproduce the comparison independently.
02 · The record
Pending first run.
| Pipeline | Truth set | SNV F1 | Indel F1 | Run date | State |
|---|---|---|---|---|---|
| — | NA12878 / HG001 | — | — | — | Not yet run |
| — | NA24385 / HG002 | — | — | — | Not yet run |
The validation suite executes once the FPGA pipeline exists (engineering Build Phase 3). Until then this table stays empty rather than populated with figures from a CPU reference implementation, which would not represent the shipping product.
03 · Quality control
Per-sample, not just per-pipeline.
Concordance validates the pipeline. Quality control validates each sample. Coverage, duplication rate and assay-specific thresholds are computed and stored as first-class queryable data rather than buried in a report artifact.
A sample falling below its thresholds is flagged and its results are withheld from automatic release, so a poor-quality run cannot be mistaken for a clean one. The thresholds a sample was judged by travel with the analysis record — the job carries its own criteria rather than referencing a configuration that may since have changed.