Hi all, I'm developing a new module angsd/dosaf for estimating site allele frequencies (SAF likelihoods) from BAM files — see the tracking issue here: github.com/nf-core/modules/issues/12185 Before raising the module PR I need to add test data to the

modules

branch of test-datasets. I've prepared a minimal dataset derived from the 1000 Genomes Project (4 Finnish individuals, low-coverage WGS, subsetted to a small chr20 region): •

GRCh38_chr20.fa

+

.fai

— reference FASTA •

homo_sapiens_ancestor_chr20.fa

+

.fai

— EPO ancestral FASTA (for

-anc

) •

angsd_syk.errors

— precomputed error model (for

-GL 4

) •

angsd_inbreeding.indF

— per-individual inbreeding coefficients (for

-doSAF 2

) •

HG00349/350/351/358.FIN.chr20.test.bam

+

.bai

— 4 low-coverage WGS BAMs The reference and ancestral FASTA are required inputs for ANGSD's

-doSaf

mode and can't be derived from an upstream module within the test, hence the need to store them here. The

.errors

and

.indF

files are precomputed auxiliary inputs — I wasn't sure of the best placement for tool-specific files like these, so any guidance on that would be welcome. Happy to raise the test-datasets PR once I have a green light. Does the additional files sound reasonable?

How big will the files be?

Thanks for replying to my query :) The largest files include x4 bam files:

176K	HG00349.FIN.chr20.test.subset.bam
172K	HG00350.FIN.chr20.test.subset.bam
160K	HG00351.FIN.chr20.test.subset.bam
188K	HG00358.FIN.chr20.test.subset.bam

1 x subsetted chr20 reference fasta

992K	GRCh38_chr20_1Mb.fa

1 x subsetted ancestral consensus fasta

988K	homo_sapiens_ancestor_chr20_1Mb.fa

angsd_syk.errors

and

angsd_inbreeding.indF

will be tiny <10K

Ok, suitably small then. Somewhere in data/genomics/homo_sapiens/ I guess, maybe inside a new folder there. I don't know the tool at all. @James Fellows Yates, do you have any suggestion as to where to put these files? test-datasets is a mess, and has been for a long time, and we keep saying we need a better alternative but we don't manage to figure out what that would be or have anyone with the impetus to do it.

Yeah it is a mess, just need someone with time to spearhead it :\ I think the FA/FAI files are fine in

genomics/homoe_sapiens/genome/

, as we now have chr1, chr21, chr22 with dedicated folders. For the BAM file they can go in

genomics/homo_sapiens/illumina/bam

(just please follow the naming scheme where the you have a sample name, the cromosome etc. (I think you have that) The

err/.indF

can go in the dedicated ANGSD directory under

delete_me

(don't worry we won't delete it) https://github.com/nf-core/test-datasets/tree/modules/data/delete_me/angsd The most important thing is when you add the entries to the README of the files, link across them all - so make sure you say that the ANGSD files come are derived from the FA/BAM files etc.

Not delete_me

I only say that because the ANGSD folder already exists there

bah

I think it would be a bad idea to split it up

you mean like by adding all the files into three separate places 😉

Different context, tjhe `.fa`/`.bam` files are not specific to ANGSD

that is true

I know I know

Thanks both, following your advice I will place the files as follows then: • Reference and ancestral FASTAs →

data/genomics/homo_sapiens/genome/chr20/

• BAMs + indices →

data/genomics/homo_sapiens/illumina/bam/

• .errors` and `.indF →

data/delete_me/angsd/

(following the existing convention for tool-specific files) I will make sure the README entries cross-link everything so the provenance from the FASTAs and BAMs through to the ANGSD auxiliary files is clear. Will raise the PR within the next few days. On the broader point about test-datasets — I would be more than happy to help with any future restructuring effort if you're looking for people to spearhead it. I can't promise huge chunks of time but I am keen to contribute where I can 😀

grr at

delete_me

happy to help!

I still really dislike adding more files to delete_me. One option for those two files though is to generate them during the test execution itself. This is done in a number of current modules: github.com/search?q=repo%3Anf-core%2Fmod…&amp;type=code cowpy has a very simple example:

def text = "hello"
def text_file = new File('hello.txt')
text_file.text = text

input[0] = [
    [ id:'hello' ],
    file(text_file, checkIfExists: true),
]

happy to switch to that approach to avoid adding to delete_me. Will remove the two files from the branch now.