https://nf-co.re logo
Join Slack
Powered by
Hello spatialvi team! I am trying to run the pipe...
# spatialvi
i
Hello spatialvi team! I am trying to run the pipeline starting from spaceranger outputs However I am having trouble with the paths in the samplesheet.csv When my directory does not contain the proper files, I do get the fair error:
Copy code
The specified spaceranger output directory for sample 'EX173983_R' does not contain all required files for either classic Visium: raw_feature_bc_matrix.h5, tissue_positions.csv, scalefactors_json.json, tissue_hires_image.png, tissue_lowres_image.png or Visium HD bin size 8: raw_feature_bc_matrix.h5, spatial/scalefactors_json.json, spatial/tissue_hires_image.png, spatial/tissue_lowres_image.png, spatial/tissue_positions.parquet.
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>
However if I include the necessary files, then the pipeline continues but soon stops with the error:
Copy code
ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_SPATIALVI:SPATIALVI:READ_DATA (1)'
Caused by:
  Not a valid path value: '250724-LW/EX173983_bis/'
Container:
  /beegfs/home/imaring/PROJECTS/PROMOTE_Spatial/work/singularity/community.wave.seqera.io-library-harmonypy_scanorama_gcc_gxx_pruned-95f731fde0b9ddef.img
I am not super expert in running nextflow pipelines and so I don't really know what to do to troubleshoot this. Hope you can help! Thanks and have a good day!
e
Hi, Irene! Could you provide me with the log of your second run as well as the samplesheet you used?
i
Sure.
Copy code
# Nextflow run script
module load Nextflow/25.04.6
nextflow run nf-core/spatialvi -r dev --input ./samplesheet.csv --outdir ./results  -profile singularity
samplesheet.csv
e
Okay, and what's the content and sub-directory structure of the 
250724-LW/EX173983_bis
directory?
i
Copy code
EX173983_bis/
└── binned_outputs
    └── square_008um
        ├── filtered_feature_bc_matrix
        │   ├── barcodes.tsv.gz
        │   ├── features.tsv.gz
        │   └── matrix.mtx.gz
        ├── filtered_feature_bc_matrix.h5
        ├── raw_feature_bc_matrix
        │   ├── barcodes.tsv.gz
        │   ├── features.tsv.gz
        │   └── matrix.mtx.gz
        ├── raw_feature_bc_matrix.h5
        └── spatial
            ├── aligned_fiducials.jpg
            ├── aligned_tissue_image.jpg
            ├── cytassist_image.tiff
            ├── detected_tissue_image.jpg
            ├── scalefactors_json.json
            ├── tissue_hires_image.png
            ├── tissue_lowres_image.png
            └── tissue_positions.parquet
e
Ah, okay, so this is Visium HD data. You did not supply the 
--hd_bin_size 8
that's required for the pipeline to work with HD data - as you didn't specify that parameter, the pipeline thinks you're giving it non-HD data, and your directory structure doesn't follow that. Values of 2, 8 and 16 are allowed, but I see from your directory structure that you have 8mm bins.
You can read more at the documentation: https://nf-co.re/spatialvi/dev/docs/usage/#space-ranger.
i
Actually I do not think that is the problem. I checked before the outputs of the pipeline and the params_XX.json had { "input": "./samplesheet.csv", "spaceranger_reference": "https://cf.10xgenomics.com/supp/spatial-exp/refdata-gex-GRCh38-2020-A.tar.gz", "spaceranger_probeset": null, "spaceranger_save_reference": false, "hd_bin_size": 8, "qc_min_counts": 500, "qc_min_genes": 250, "qc_min_spots": 1, "qc_mito_threshold": 20.0,... so I thought by default it was already provided and not needed. Still I have just rerun it with the param 
--hd_bin_size 8
and I have the same error...
e
@Christophe Avenel, maybe you have an idea of what's going on here? I'm looking in the code for 
INPUT_CHECK
and it should auto-detect downstream input type as far as I can tell.
All the files that should be there seems to be there, so I don't know what's happening here
@Irene Marin is the 
250724-LW/EX173983_bis
directory in the project root? What's the project's overall directory structure like?
i
Yes is the project root
Copy code
├── 250724-LW
│   └── EX173983_bis
│       └── binned_outputs
├── Nextflow.sh
├── results
├── samplesheet.csv
├── scripts
└── work
Nextflow.sh is just
Copy code
#! bin/bash
# Nextflow run script
module load Nextflow/25.04.6
nextflow run nf-core/spatialvi -r dev --input ./samplesheet.csv --outdir ./results --hd_bin_size 8 -profile singularity
e
Could you try changing the samplesheet to use the full, absolute path instead, just to check whether that works?
i
Weird. Now it took a while but finally got a different error
e
Hmm, that does indeed look different. I can see that the 
feature_slice.h5
is being checked for, and since it's not present in your data directory you're getting this error. This file is, however, not listed as required for running the HD part of the pipeline, yet it tries to move it anyway (line 59-62 in 
read_data.py
). Looks like a bug, what do you think @Christophe Avenel?
c
Hej! My feeling is that some files are missing from the main folder (not inside binned_outputs). Where did you get these from @Irene Marin? Here is what I have in a typical HD outs:
Copy code
outs
├── barcode_mappings.parquet
├── binned_outputs
│   ├── square_002um
│   ├── square_008um
│   └── square_016um
├── cloupe_008um.cloupe
├── cloupe_cell.cloupe
├── feature_slice.h5
├── metrics_summary.csv
├── molecule_info.h5
├── probe_set.csv
├── segmented_outputs
├── spatial
│   ├── aligned_fiducials.jpg
│   ├── aligned_tissue_image.jpg
│   ├── cytassist_image.tiff
│   ├── detected_tissue_image.jpg
│   ├── tissue_hires_image.png
│   └── tissue_lowres_image.png
└── web_summary.html
As you can see, I have 
feature_slice.h5
, and 
spatial
that are used by 
spatialdata_io.visium_hd(...)
I believe.
e
If 
feature_slice.h5
is required to run the HD part of the pipeline, we should include it in the input checks
c
Good point, I can double check what is needed by 
visium_hd
tomorrow!
e
Awesome!
i
you are right! I do have those. I did not run spaceranger myself, these data were given so I just copied from my data dir to make a quick test and reading the docs only copied the binned_outputs. As my first error was asking only for the 
raw_feature_bc_matrix.h5, spatial/scalefactors_json.json, spatial/tissue_hires_image.png, spatial/tissue_lowres_image.png, spatial/tissue_positions.parquet.
I assumed the other files were not used. I will copy all those files and try to run it again
e
You were entirely right not to expect any other files than those listed, it's our error in the pipeline, that should have checked for all files required. We'll fix ASAP, but if you can try again with the 
feature_slice.h5
that'd be swell
c
Yes, I will make sure we specify and test all needed files for the downstream part!
i
Ok Now is running! Thanks a lot for your help! Just a final note, not critical but so you know, I do need to specify the full path in the samplesheet.csv EX173983_R, /home/imaring/PROJECTS/PROMOTE_Spatial/250724-LW/EX173983_bis if I do 250724-LW/EX173983_bis ./250724-LW/EX173983_bis /250724-LW/EX173983_bis I get error: Not a valid path value: '250724-LW/EX173983_bis' even though the initial message of the pipeline says the launchDir is :/home/imaring/PROJECTS/PROMOTE_Spatial/ Core Nextflow options revision : dev runName : sick_leavitt containerEngine : singularity container.MULTIQC : docker.io/multiqc/multiqc:v1.29 container.READ_DATA|QUALITY_CONTROLS|CLUSTERING|SPATIALLY_VARIABLE_GENES: community.wave.seqera.io/library/harmonypy_scanorama_gcc_gxx_pruned:95f731fde0b9ddef container.INTEGRATE_SDATA : community.wave.seqera.io/library/harmonypy_scanorama_gcc_gxx_pruned:95f731fde0b9ddef launchDir : /beegfs/home/imaring/PROJECTS/PROMOTE_Spatial workDir : /beegfs/home/imaring/PROJECTS/PROMOTE_Spatial/work projectDir : /home/imaring/.nextflow/assets/nf-core/spatialvi userName : imaring profile : singularity configFiles : /home/imaring/.nextflow/assets/nf-core/spatialvi/nextflow.config
2 Views
Open in Slack
PreviousNext
Powered by