Daniel Lundin
08/09/2025, 8:41 AMmodules patch subcommand. I've struggled before but didn't report, but now I'm getting errors again, see attached file. The final lines are:
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/patch.py:11 │
│ 7 in patch │
│ │
│ 114 │ │ # Write the patch to a temporary location (otherwise it is printed to the screen │
│ 115 │ │ patch_temp_path = tempfile.mktemp() │
│ 116 │ │ try: │
│ ❱ 117 │ │ │ ComponentsDiffer.write_diff_file( │
│ 118 │ │ │ │ patch_temp_path, │
│ 119 │ │ │ │ component, │
│ 120 │ │ │ │ self.modules_repo.repo_path, │
│ │
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/components_ │
│ differ.py:167 in write_diff_file │
│ │
│ 164 │ │ if dsp_to_dir is None: │
│ 165 │ │ │ dsp_to_dir = to_dir │
│ 166 │ │ │
│ ❱ 167 │ │ diffs = ComponentsDiffer.get_component_diffs(from_dir, to_dir, for_git, dsp_from │
│ 168 │ │ if all(diff_status == ComponentsDiffer.DiffEnum.UNCHANGED for _, (diff_status, _ │
│ 169 │ │ │ raise UserWarning("Component is unchanged") │
│ 170 │ │ log.debug(f"Writing diff of '{component}' to '{diff_path}'") │
│ │
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/components_ │
│ differ.py:100 in get_component_diffs │
│ │
│ 97 │ │ │ │
│ 98 │ │ │ elif temp_path.exists(): │
│ 99 │ │ │ │ with open(temp_path) as fh: │
│ ❱ 100 │ │ │ │ │ new_lines = fh.readlines() │
│ 101 │ │ │ │ # The file was created │
│ 102 │ │ │ │ # Show file against /dev/null │
│ 103 │ │ │ │ diff = difflib.unified_diff( │
│ in decode:325 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x97 in position 18: invalid start byte
Googling this suggests it might indicate that a binary file is read in text mode.
(I really like the feature, so it would be great if I could get it to work.)Erik Danielsson
08/11/2025, 6:48 AMcat/fastq module have you made changes to?Daniel Lundin
08/11/2025, 6:53 AMmain.nf, so definitely a text file. Here's a diff:
$ git diff 097406c0a764cbd7dae0dd757fb6818ba95a5e22 modules/nf-core/cat/fastq/main.nf
diff --git a/modules/nf-core/cat/fastq/main.nf b/modules/nf-core/cat/fastq/main.nf
index acfb6d0..ed6acb8 100644
--- a/modules/nf-core/cat/fastq/main.nf
+++ b/modules/nf-core/cat/fastq/main.nf
@@ -23,7 +23,13 @@ process CAT_FASTQ {
if (meta.single_end) {
if (readList.size >= 1) {
"""
- cat ${readList.join(' ')} > ${prefix}.merged.fastq.gz
+ for f in ${readList.join(' ')}; do
+ if [ \$(echo \$f | grep '.gz\$') ]; then
+ cat \$f >> ${prefix}.merged.fastq.gz
+ else
+ gzip -c \$f >> ${prefix}.merged.fastq.gz
+ fi
+ done
cat <<-END_VERSIONS > versions.yml
"${task.process}":
@@ -40,8 +46,21 @@ process CAT_FASTQ {
def read2 = []
readList.eachWithIndex { v, ix -> (ix & 1 ? read2 : read1) << v }
"""
- cat ${read1.join(' ')} > ${prefix}_1.merged.fastq.gz
- cat ${read2.join(' ')} > ${prefix}_2.merged.fastq.gz
+ for f in ${read1.join(' ')}; do
+ if [ \$(echo \$f | grep '.gz\$') ]; then
+ cat \$f >> ${prefix}_1.merged.fastq.gz
+ else
+ gzip -c \$f >> ${prefix}_1.merged.fastq.gz
+ fi
+ done
+
+ for f in ${read2.join(' ')}; do
+ if [ \$(echo \$f | grep '.gz\$') ]; then
+ cat \$f >> ${prefix}_2.merged.fastq.gz
+ else
+ gzip -c \$f >> ${prefix}_2.merged.fastq.gz
+ fi
+ done
cat <<-END_VERSIONS > versions.yml
"${task.process}":Erik Danielsson
08/11/2025, 7:07 AMpatch code now, and the error is coming from the code that checks whether you created any new files inside the module's directory. Could this be the case?Daniel Lundin
08/11/2025, 7:15 AM.main.nf.sw? present, the command works. Thanks!Daniel Lundin
08/11/2025, 7:17 AMfile reports vim files like below. If there's a common pattern between different temporary files, I don't know.
$ file ./.nextflow.config.swp
./.nextflow.config.swp: Vim swap file, version 9.1, pid 1774520, user daluab, host LB-00029, file ~daluab/dev/nf-core-metatdenovo/nextflow.configErik Danielsson
08/11/2025, 7:23 AM