I'm having problems with the `modules patch` subco...
# tools
d
I'm having problems with the
modules patch
subcommand. I've struggled before but didn't report, but now I'm getting errors again, see attached file. The final lines are:
Copy code
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/patch.py:11 │
│ 7 in patch                                                                                       │
│                                                                                                  │
│   114 │   │   # Write the patch to a temporary location (otherwise it is printed to the screen   │
│   115 │   │   patch_temp_path = tempfile.mktemp()                                                │
│   116 │   │   try:                                                                               │
│ ❱ 117 │   │   │   ComponentsDiffer.write_diff_file(                                              │
│   118 │   │   │   │   patch_temp_path,                                                           │
│   119 │   │   │   │   component,                                                                 │
│   120 │   │   │   │   self.modules_repo.repo_path,                                               │
│                                                                                                  │
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/components_ │
│ differ.py:167 in write_diff_file                                                                 │
│                                                                                                  │
│   164 │   │   if dsp_to_dir is None:                                                             │
│   165 │   │   │   dsp_to_dir = to_dir                                                            │
│   166 │   │                                                                                      │
│ ❱ 167 │   │   diffs = ComponentsDiffer.get_component_diffs(from_dir, to_dir, for_git, dsp_from   │
│   168 │   │   if all(diff_status == ComponentsDiffer.DiffEnum.UNCHANGED for _, (diff_status, _   │
│   169 │   │   │   raise UserWarning("Component is unchanged")                                    │
│   170 │   │   log.debug(f"Writing diff of '{component}' to '{diff_path}'")                       │
│                                                                                                  │
│ /home/daluab/miniconda3/envs/nf-core/lib/python3.13/site-packages/nf_core/components/components_ │
│ differ.py:100 in get_component_diffs                                                             │
│                                                                                                  │
│    97 │   │   │                                                                                  │
│    98 │   │   │   elif temp_path.exists():                                                       │
│    99 │   │   │   │   with open(temp_path) as fh:                                                │
│ ❱ 100 │   │   │   │   │   new_lines = fh.readlines()                                             │
│   101 │   │   │   │   # The file was created                                                     │
│   102 │   │   │   │   # Show file against /dev/null                                              │
│   103 │   │   │   │   diff = difflib.unified_diff(                                               │
│ in decode:325                                                                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x97 in position 18: invalid start byte
Googling this suggests it might indicate that a binary file is read in text mode. (I really like the feature, so it would be great if I could get it to work.)
e
This could indeed indicate that the command is touching a binary. Could you detail what files in the
cat/fastq
module have you made changes to?
d
main.nf
, so definitely a text file. Here's a diff:
Copy code
$ git diff 097406c0a764cbd7dae0dd757fb6818ba95a5e22 modules/nf-core/cat/fastq/main.nf
diff --git a/modules/nf-core/cat/fastq/main.nf b/modules/nf-core/cat/fastq/main.nf
index acfb6d0..ed6acb8 100644
--- a/modules/nf-core/cat/fastq/main.nf
+++ b/modules/nf-core/cat/fastq/main.nf
@@ -23,7 +23,13 @@ process CAT_FASTQ {
     if (meta.single_end) {
         if (readList.size >= 1) {
             """
-            cat ${readList.join(' ')} > ${prefix}.merged.fastq.gz
+            for f in ${readList.join(' ')}; do
+                if [ \$(echo \$f | grep '.gz\$') ]; then
+                    cat \$f >> ${prefix}.merged.fastq.gz
+                else
+                    gzip -c \$f >> ${prefix}.merged.fastq.gz
+                fi
+            done
 
             cat <<-END_VERSIONS > versions.yml
             "${task.process}":
@@ -40,8 +46,21 @@ process CAT_FASTQ {
             def read2 = []
             readList.eachWithIndex { v, ix -> (ix & 1 ? read2 : read1) << v }
             """
-            cat ${read1.join(' ')} > ${prefix}_1.merged.fastq.gz
-            cat ${read2.join(' ')} > ${prefix}_2.merged.fastq.gz
+            for f in ${read1.join(' ')}; do
+                if [ \$(echo \$f | grep '.gz\$') ]; then
+                    cat \$f >> ${prefix}_1.merged.fastq.gz
+                else
+                    gzip -c \$f >> ${prefix}_1.merged.fastq.gz
+                fi
+            done
+
+            for f in ${read2.join(' ')}; do
+                if [ \$(echo \$f | grep '.gz\$') ]; then
+                    cat \$f >> ${prefix}_2.merged.fastq.gz
+                else
+                    gzip -c \$f >> ${prefix}_2.merged.fastq.gz
+                fi
+            done
 
             cat <<-END_VERSIONS > versions.yml
             "${task.process}":
e
Thanks! I've checked the
patch
code now, and the error is coming from the code that checks whether you created any new files inside the module's directory. Could this be the case?
d
You found it: Now that I've closed the source file, and there's hence no vim
.main.nf.sw?
present, the command works. Thanks!
catjam 1
Would probably be good if the tool knew about these kinds of files. There are many to ignore though.
file
reports vim files like below. If there's a common pattern between different temporary files, I don't know.
Copy code
$ file ./.nextflow.config.swp
./.nextflow.config.swp: Vim swap file, version 9.1, pid 1774520, user daluab, host LB-00029, file ~daluab/dev/nf-core-metatdenovo/nextflow.config
e
Yes that might be a good idea. Could you put this in an issue on nf-core/tools?
👍 1