Charles Plessy
02/05/2025, 7:42 AMbgzip. But databases like NCBI only offer gzip-compressed genomes for download. To ease the life of users and pipeline maintainers, I propose a new module, samtools/bgzip, that takes a FASTA file as input that can be either not compressed, or gzip-compressed, or bgzip-compressed, and outputs a file that is guaranteed to be bgzip-compressed. I welcome your comments, or your reviews if you think that it is ready to go! https://github.com/nf-core/modules/pull/7433Matthias De Smet
02/05/2025, 7:44 AMCharles Plessy
02/05/2025, 7:48 AMtabix/bgzip really going to "Do The Right Thing©" on an input that is gzipped? What I want is to be sure that anything that is fed to samtools/faidx is not going to crash the module.Sateesh_Peri
02/05/2025, 7:48 AMMatthias De Smet
02/05/2025, 7:49 AMMatthias De Smet
02/05/2025, 7:50 AMArthur
02/05/2025, 8:12 AMtabix/bgzip I believe will just end up decompressing the file if it’s already compressed (either by gzip or bgzip) and won’t recompress it
• bgzip itself is fully interoperable with gzipped files [GitHub]Arthur
02/05/2025, 8:20 AMtabix_recompress or something?
Maybe like:
zcat -f ${input} | bgzip -c > ${output}Charles Plessy
02/05/2025, 8:22 AMtabix/bgzip, I did not realise that sanitises its output by decompressing files even if they were already bgzipped. In samtools/bgzip I test if the file was compressed with bgzip or plain gzip, and if it was bgzip I just return it. Would a PR along these lines be welcome? Note that it requires to add conda-forge:file to the containers and environment.
COMPRESS_TYPE=\$(file -L -b $fasta)
case "\$COMPRESS_TYPE" in
"gzip compressed data, extra field"*)
# Assuming the "extra field" implies BGZF, do nothing.
ln -s $fasta ${prefix}.bgzip.fa.gz ${output}
;;
gzip*)
# Recompress non-BGZF gzipped files
zcat $fasta | bgzip $args --threads ${task.cpu} ${output}
;;
*)
# Compress
bgzip $args --threads ${task.cpu} --stdout $fasta > ${output}
;;
esacCharles Plessy
02/05/2025, 8:24 AMMatthias De Smet
02/05/2025, 8:26 AMArthur
02/05/2025, 8:31 AM((bgzip -cd ${input} | bgzip -c ) || bgzip -c ${input}) > $outputCharles Plessy
02/05/2025, 8:42 AMMatthias De Smet
02/05/2025, 8:44 AMMatthias De Smet
02/05/2025, 8:45 AMgzip index to bgzip the file?Arthur
02/05/2025, 8:47 AMCharles Plessy
02/05/2025, 9:18 AMtabix/bgzip module… Nevertheless, I will try to do something if the issue I opened on htslib leads somewhere, or if you ping me to open a PR based on something like the patch that I showed above.Charles Plessy
02/06/2025, 4:28 AMhtsfile tool that was exist for 10 years already. So I proposed a patch to tabix/bgzip this morning. https://github.com/nf-core/modules/issues/7444Arthur
02/06/2025, 4:31 AMtabix/bgzip module. It is intended to either decompress or compress based on the given input. Given it is a widely used module used in at least 10 nf-core pipelines alone (not to mention any non-nf-core ones) I don’t think fundamentally changing the way the module works is the right thing to do.
Hence why I proposed you create a new module something like tabix/recompressCharles Plessy
02/06/2025, 4:33 AMtabix/bgzip when I proposed samtools/bgzip which is basically doing what you call tabix/recompress ? https://github.com/nf-core/modules/pull/7433Arthur
02/06/2025, 4:33 AMCharles Plessy
02/06/2025, 4:35 AMCharles Plessy
02/06/2025, 4:36 AMArthur
02/06/2025, 4:36 AMtabix/bgzip (whether that’s a bit confusing or not).Arthur
02/06/2025, 4:39 AMhtsfile replace that.
It could potentially replace the current logic in tabix/bgzip too if it’s not an extra dependency.