https://nf-co.re logo
Join Slack
Powered by
I'm running the same test (`test_all` ), from the ...
# rubber-duck
l
I'm running the same test (
test_all
), from the same git repository 
nf-core/phaseimpute
, from the same commit 
fa9fac8cd
. I'm running it on 4 different machine (PC windows, PC linux, github action, HPC). The result from a 
SHAPEIT5_phase_common
process differ ever so slightly but throw off all my nf-test assertion... This test use the same file as input (same variants md5sum), use 
--thread 1
and 
--seed 1
. This wasn't happening before and I can't figure out what changed since then. The two PC aggree on the same variants hash 
a27ecf59af876aed33316ca37189dd9f
while GA and HPC found 
797b0fd345ce761987d4b3bddac4236a
. These md5 are stable accross multiples runs for each devices. This happen only for chromosome 21, while chr22 is stable in the 4 conditions.
When comparing the inside of the bcf produced, I just have for some position, for some heterozygous individual, a flip in the haplotype 
0|1
become 
1|0
or vice versa.
I thought that maybe singularity was getting access to different library that are specific to each device. HPC use singularity-ce version 3.10.4-jammy PC WSL singularity-ce version 3.11.0 PC linux singularity 3.8.6 Strangely when I run 
singularity run <https://depot.galaxyproject.org/singularity/shapeit5:5.1.1--hb60d31d_0>
in linux and WSL I got access to my whole repository but on HPC I only got 
public_html singularity working_directory
available.
Adding 
runOptions  = '--containall'
to the singularity options doesn't solve the issue.
m
which tool produces this bcf?
l
which tool produces this bcf?
The bcf is created by the tool itself, no bcftools available in the singularity image.
Fun fact, this diff on PC WSL doesn't happen when using Docker 🫠
I think I found the culprit... The index of the 
1000GP_chr21_fixup2.vcf.gz
is a 
.tbi
in
Copy code
SHAPEIT5_phase_common \
    --seed 1 \
    --input 1000GP_chr21_fixup2.vcf.gz \
    --region chr21:16570070-16590513 \
    --thread 1 \
    --output 1000GP_chr21_16570070-16590513_chunks.bcf
For a freaking unknown reason 
.tbi
resulted in a different result when used in singularity than in docker. Using 
.csi
stabilised the output.
But this only solved the issue for one chunks and not the second one...
m
WTF
Good investigation BTW
l
This resulted in just a few haplotype flip, but this is indeed tricky.
I just found the reason for the second chunks discrepancy. My input vcf, is not sorted 🫠meow tableflip🥲 (not that, but bcftools sort does fix the issue)
And it seems that docker and singularity for the same FUC**** image doesn't treat the data similarly...
Even stranger, singularity differ only on my personnal computer on WSL and on my portable computer on linux, but not on github action nor on my HPC 😭
So... Rewritting the input file with 
bcftools 1000GP_chr21_fixup.vcf.gz -Oz -o input.vcf.gz
gives the following: • if written in 
.bcf
the result will be stable and gives the same hash A as the good one from docker (across docker and singularity) • if written in 
.vcf.gz
the result will be stable and gives the same hash B as the wrong one from singularity (across docker and singularity) • if i give the original file singularity will give hash B in singularity (only on my PCs) and hash A in singularity (HPC and GA) and on docker...
The issue might come from upstream process generating this 
1000GP_chr21_fixup.vcf.gz
even if it's stable, it might be wrongly stable...
So adding 
| bcftools view -Ob
to vcflib/vcffixup fix the problem, I also need to change the output to bcf for the other upstream process.
I nearly resolved all the issue. But now the chr22 is giving different result across docker and singularity for 
SHAPEIT5_phase_common
but for only one chunk Adding 
bcftools
to the shapeit5 container does solve the issue.
Changing the image from the galaxy project to the seqera container solve the issue... 🫠
m
You are persistent
Let's try to investigate what's different in between the two, and we can report back to Seqera cc @Phil Ewels
l
I guess one of the c++ library might not be properly installed on the galaxy one, and as such htslib might use one available on the host machine instead.
2 Views
Open in Slack
PreviousNext
Powered by