We tried to run this pipeline on a publicly available Xenium dataset using Seqera Tower and AWS batch. It seems to run into the following file check error when running on AWS batch. Is this expected behaviour?

Missing required file(s) in xenium bundle '<s3://xxxx/data/spatial_transcriptomics/xenium/public/mouse/GSE272303-SNCA-tg/Line61nTg_1/outs>': [cell_boundaries.csv.gz, cell_boundaries.parquet, cell_feature_matrix.h5, cell_feature_matrix.zarr.zip, cells.csv.gz, cells.parquet, cells.zarr.zip, experiment.xenium, gene_panel.json, metrics_summary.csv, morphology.ome.tif, morphology_focus/, nucleus_boundaries.csv.gz, nucleus_boundaries.parquet, transcripts.parquet, transcripts.zarr.zip]

That is exprected. It seems to me your public avaialalbe dataset has not eveything a fully Xenium dataset bundle should have. This is quite common tbh. I saw many public datasets with missing data.

Thanks @Florian Heyl. I launched the test with

-profile test

and it also errored out with an error after the untar step. Is the pipeline typically tested in AWS or other cloud environments during testing?

The pipeline was tested on an lsf HPC. What was the error after the UNTAR step?

As far as I could tell, the following line cannot resolve an s3 URI correctly and it results in checking for an unresolved path when used with AWS batch or a local run on a VM.

def missing_required = bundle_required_files.findAll { check -> !file("${bundle_path}/${check}").exists() }