<#2390 abotyper: nf-test fixtures in the current o...
# test-data
g
#2390 abotyper: nf-test fixtures in the current output layout; remove the old exon 6/7 data Pull request opened by fmobegi Test data for nf-core/abotyper, as asked in the 1.0.0 release review (nf-core/abotyper#56): the nf-test fixtures move out of the pipeline repository (
tests/fixtures/
, 575K) into this branch. Added • `abotyper/per_sample_processing/<sample>/`: the intermediate files the module and subworkflow tests use, laid out like the pipeline's output (
alignment/
,
clair3/
,
clair2metrics/
,
mpileup/
and the sample's
.ABOPhenotype.txt
). They come from the pipeline's test run on
ABO_AA_barcode01
and
ABO_BO1_barcode06
. • `abotyper/fixtures/`: sample tables for
abotyper-aggregate --samples
, an empty FASTQ used as a clean negative control, the second pipeline test's samplesheet and a README. Removed • The old
per_sample_processing/<sample>/exon6|exon7/
files and
refs/
(exon 6 and 7 references,
ABO_Database.fasta
). They belong to the earlier two-reference set-up; the pipeline now ships its own NG_006669.2 reference and no longer reads them. The FASTQ files and
samplesheet_test_abotyper.csv
are unchanged. nf-core/test-datasets
Review on #2390 abotyper: nf-test fixtures in the current output layout; remove the old exon 6/7 data Review approved by maxulysse on fmobegi's pull request nf-core/test-datasets
Review on #2390 abotyper: nf-test fixtures in the current output layout; remove the old exon 6/7 data Reviewed and changes requested by pinin4fjords on fmobegi's pull request AI-assisted review (Claude, on behalf of @pinin4fjords). Thanks for moving the fixtures here. The move itself looks clean: the per-sample files are byte-identical to the old
tests/fixtures/
, every path the nf-core/abotyper#81 tests read exists, and nothing current uses the files this removes. I ran the #81 tests against this branch and they pass. The documentation needs more before this merges, though. The nf-core test-data spec asks for each branch's README to give "the source of the data, how it was generated, and license information" (Documentation of test data), and says test data "MUST be publicly available and have licenses that allow public reuse" (License of test data). Other branches do this file by file (e.g. rnaseq, nanoseq, raredisease, hlatyping). Here the branch root
README.md
is still the generic one, and
abotyper/fixtures/README.md
only covers the fixtures. Could you replace the root README with an abotyper one covering: 1. Consent and licence for the human reads. The pipeline's
conf/test_full.config
says the donor runs "cannot be published", and its README credits Stem Cell Donors Australia registry samples. Please state who owns these reads, that the donors consented to public release, and the licence. Comparable human-data branches use public, consented sources (1000 Genomes for hlatyping, GIAB HG002 for longraredisease). 2. Where the FASTQs come from and how they were made. • The source run, flow cell, kit and basecaller model. • The exact extraction and subsampling commands with tool versions. The only record is the body of #1711 (MAPQ >= 20, primary only, seqkit subsampling,
bedtools bamtofastq
). • How each sample's genotype label was confirmed. The descriptions disagree: •
conf/test_full.config
says "whole libraries from a real run, not downsampled (67 to 198 reads each)". • A test comment calls them synthetic reads. • The files themselves have repeated read IDs (`ABO_AA_barcode01`: 92 reads, 71 unique IDs; `ABO_O1O1_barcode08`: 90 reads, 23 unique), which looks like merged per-exon extractions. Please give one account here and fix
test_full.config
to match. 3. The reference. This removes
abotyper/refs/REF.README.txt
, the only provenance note on the branch. Please say the files are aligned to NG_006669.2 (with that link), and which region BED Clair3 used. 4. How the
per_sample_processing/
files were produced.
They don't match what the pipeline produces now: • the BAM
@PG
is
minimap2 -t 12 -a
with no
-x map-ont
, which the pipeline sets (
conf/modules.config
); • the Clair3 files are Clair3 1.2.0 with an unnamed model (
model_path=clair3_model_dir
) and local paths (
/data/ABO-analysis-31-Oct-2023/...
) in their headers; • the metrics tables have 51 panel positions against 135 in abotyper-core 0.4.1. For each group, please give the step that made it, the pipeline commit, the abotyper-core and tool versions, the Clair3 model and the command. Then either say how to regenerate them, or regenerate them with the current pipeline (which I'd prefer, since they would then match what the tests exercise). 5. Names that aren't pipeline outputs. "Laid out like the pipeline's output" isn't quite right: • the pipeline doesn't publish
mpileup/
(see
docs/output.md
); •
clair2metrics/*.clair2metrics.tsv
and
*.merged.AlignmentStatistics.tsv
aren't output names, and the two files are byte-identical. Please use the real output names, or say what each file is. 6. Deliberate oddities. •
ABO_BO1_barcode06
is only a partial set. • In
samplesheet_controls_multirow.csv
,
ABO_BB
with barcode 7 points at
barcode05
. • The sample tables refer to
sample_files/
paths that aren't on the branch. • The
Haplotypes.tsv
rows are named
synthetic_*
. One line each. The consent point (1) is the one that has to be settled before merge. The rest is documentation. nf-core/test-datasets
Review on #2390 abotyper: nf-test fixtures in the current output layout; remove the old exon 6/7 data Review approved by pinin4fjords on fmobegi's pull request AI-assisted review (Claude, on behalf of @pinin4fjords). Thanks, this is a big improvement, and thanks for clearing up that the original reads were shareable cell-line data from a commercial partner, not donor samples. Please ignore my earlier comment about rewriting the branch and force pushing: since the cell-line data were shareable, there's nothing to remove from the history. The README now covers what the test-data spec asks for: source, generation and licence. I checked it against the files: • Running
simulate_abo_reads.py
with abotyper-core 0.4.1 rebuilds all ten FASTQs,
alleles.fasta
and
truth.tsv
byte for byte. • Read counts and per-allele splits match the tables, and the read IDs are unique. • The BAM and VCF headers match the table: minimap2 2.30 with
-x map-ont
, and Clair3 2.0.5 with
r1041_e82_400bps_sup_v500
. The metrics have 135 panel positions. • I ran the nf-core/abotyper#81 tests (at ca27c71) on a VM against this branch. In the full
-profile test
run, all ten samples type as their intended genotype. Every
per_sample_processing/
file matches that run's output except the timestamped
log.txt
. Three small README fixes inline, then this is good from my side. nf-core/test-datasets