Hi! I think we are pretty close to our first relea...
# methylongj
Jin Khoo
04/08/2025, 10:14 AMHi! I think we are pretty close to our first release. In terms of the test dataset, I have 3 samples, (homo sapiens - ONT) and (arabidopsis - ONT and PacBio) as the minimal test dataset. Is this sufficient for the test?
s
Sateesh_Peri
04/08/2025, 12:35 PMthats good....you have any full size datasets in mind to run full tests on AWS prior release ?
j
Jin Khoo
04/08/2025, 12:58 PMI tried to look up in nf-core/test-data folder, can this consider as a full dataset?
Jin Khoo
04/08/2025, 12:59 PMhttps://github.com/nf-core/test-datasets/tree/methylong/test_data/test_modbam
these that I uploaded
Jin Khoo
04/08/2025, 1:04 PMeach sample is about 20mb in size, is a subsample of long reads from Arabidopsis chr1
s
Sateesh_Peri
04/08/2025, 1:04 PMno usually we put full sized datasets on S3.....do you have any full size samples that can be uploaded to S3 ?
f
Felix Lenner
04/08/2025, 1:13 PMIf you want full-sized human datasets I could suggest something from https://downloads.pacbcloud.com/public/revio/2024Q4/WGS/GIAB_trio/ (https://www.pacb.com/connect/datasets/) for PacBio or s3://ont-open-data/giab_2025.01/ (https://epi2me.nanoporetech.com/giab-2025.01/) for ONT.
j
Jin Khoo
04/08/2025, 1:33 PMI dont have any full dataset available to upload on public yet
Jin Khoo
04/08/2025, 1:33 PM@Felix Lenner thanks a bunch!
p
Phil Ewels
04/08/2025, 1:34 PMFor full size datasets we usually try to fetch directly from the original host (be it SRA, ENA, pacb.com or whatever), if possible
Phil Ewels
04/08/2025, 1:35 PMIn contrast to the small test datasets, which we always self-host on github
j
Jin Khoo
04/08/2025, 1:41 PMim not really familiar in the human genomic field, am I right to assume this is the reference genome for the dataset u mentioned? @Felix Lenner
https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/405/GCF_000001405.26_GRCh38/GCF_000001405.26_GRCh38_genomic.fna.gz
Jin Khoo
04/08/2025, 1:42 PMthanks @Phil Ewels, will ftp be acceptable too in this case?
p
Phil Ewels
04/08/2025, 2:14 PMAs long as it's stable enough 👍🏻 (sometimes we have had flaky test results from FTP being unstable for large data volumes)
j
Jin Khoo
04/08/2025, 2:17 PMokay, one way to find out 👀
Jin Khoo
04/08/2025, 2:17 PMnow I have included the full dataset for both Pacbio and ONT for the aws test for the full test
Jin Khoo
04/08/2025, 2:18 PMJin Khoo
04/08/2025, 2:19 PMwill this be sufficient for the test? @Sateesh_Peri
👍 1
f
Felix Lenner
04/08/2025, 2:25 PMThat reference should be fine! @Jin Khoo
🙌 1