Hi, I have been trying the nf-core/hadge pipeline on my data but I'm running into the following error in the last step:

ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_HADGE:HADGE:GENETIC_DEMULTIPLEXING:GENE_SUMMARY
(...)
Command executed [/home/lb2336/.nextflow/assets/nf-core/hadge/modules/local/gene_summary/templates/gene_summary.py]:
(...)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

@Leon B, could you please send me the whole error message?

is this enough?

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Traceback (most recent call last):
    File ".command.sh", line 231, in <module>
      barcodes_df = pd.read_csv(args.barcodes, header=None, sep="       ", names=["Barcode"])
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
      return _read(filepath_or_buffer, kwds)
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 620, in _read
      parser = TextFileReader(filepath_or_buffer, **kwds)
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1620, in __init__
      self._engine = self._make_engine(f, self.engine)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1898, in _make_engine
      return mapping[engine](f, **self.options)
             ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 93, in __init__
      self._reader = parsers.TextReader(src, **kwds)
                     ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
    File "pandas/_libs/parsers.pyx", line 574, in pandas._libs.parsers.TextReader.__cinit__
    File "pandas/_libs/parsers.pyx", line 766, in pandas._libs.parsers.TextReader._get_header
    File "pandas/_libs/parsers.pyx", line 874, in pandas._libs.parsers.TextReader._tokenize_rows
    File "pandas/_libs/parsers.pyx", line 891, in pandas._libs.parsers.TextReader._check_tokenize_status
    File "pandas/_libs/parsers.pyx", line 2053, in pandas._libs.parsers.raise_parser_error
    File "<frozen codecs>", line 325, in decode
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Is your

barcodes.tsv

zipped (e.g.

.gz

)?

Ok, good catch that seemed to have indeed solved the gene_summary problem - now with the gunzipped barcodes.tsv however

NFCORE_HADGE:HADGE:CREATE_ANNDATA_MUDATA

is failing missing the barcodes.tsv.gz:

Caused by:
 Process `NFCORE_HADGE:HADGE:CREATE_ANNDATA_MUDATA (AJ_GroundTruth_1)` terminated with an error exit status (1)

Command executed [/home/lb2336/.nextflow/assets/nf-core/hadge/modules/local/create_anndata_mudata/templates/create_anndata_mudata.py]:
(...)

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Traceback (most recent call last):
    File ".command.sh", line 122, in <module>
      rna_data = sc.read_10x_mtx(args.rna_matrix)
    File "/opt/conda/lib/python3.14/site-packages/legacy_api_wrap/__init__.py", line 88, in fn_compatible
      return fn(*args_all, **kw)
    File "/opt/conda/lib/python3.14/site-packages/scanpy/readwrite.py", line 570, in read_10x_mtx
      adata = _read_10x_mtx(
          path,
      ...<5 lines>...
          is_legacy=is_legacy,
      )
    File "/opt/conda/lib/python3.14/site-packages/scanpy/readwrite.py", line 621, in _read_10x_mtx
      barcodes = pd.read_csv(path / f"{prefix}barcodes.tsv{suffix}", header=None)
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
      return _read(filepath_or_buffer, kwds)
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 620, in _read
      parser = TextFileReader(filepath_or_buffer, **kwds)
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1620, in __init__
      self._engine = self._make_engine(f, self.engine)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/parsers/readers.py", line 1880, in _make_engine
      self.handles = get_handle(
                     ~~~~~~~~~~^
          f,
          ^^
      ...<6 lines>...
          storage_options=self.options.get("storage_options", None),
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      )   
      ^
    File "/opt/conda/lib/python3.14/site-packages/pandas/io/common.py", line 765, in get_handle
      handle = gzip.GzipFile(  # type: ignore[assignment]
          filename=handle,
          mode=ioargs.mode,
          **compression_args,
      )
    File "/opt/conda/lib/python3.14/gzip.py", line 208, in __init__
      fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
                                 ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  FileNotFoundError: [Errno 2] No such file or directory: 'sample_filtered_feature_bc_matrix/barcodes.tsv.gz'

Work dir:
  /vast/palmer/pi/csei/users/lb2336/test_hadge/nfcore/work/19/628ed59c9371ba8e89e4f1a4c68724

Barcodes have to be .gz inside the folder for gene expression. The barcodes.tsv for the samplesheet (used for genetic demultiplexing) should not be .gz. So in your case you should probably copy the barcodes.tsv.gz from the gene expression matrix, move it to a different directory and unzip it. Use this copied and unzipped barcodes.tsv for the barcodes column in the samplesheet.csv and leave the barcodes.tsv.gz for the gene expression matrix as it is. I hope that helps.

ah ok I see thanks now I got it, might also be usefull to add that to the README