André Ferreira Cunha
05/24/2025, 7:17 AMERROR ~ Error executing process > 'NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489)'
Caused by:
Process `NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489)` terminated with an error exit status (1)
Command executed:
samtools \
stats \
--threads 1 \
--reference human_plus_phix.fna \
SAMN05826489_SAMN05826489.bam \
> SAMN05826489_SAMN05826489.stats
cat <<-END_VERSIONS > versions.yml
"NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS":
samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
END_VERSIONS
Command exit status:
1
Command output:
(empty)
Work dir:
<az://rawdata/work/6d/d9aa8bd4032fd6ab43e3e53801b19a>
Container:
quay.io/biocontainers/samtools:1.20--h50ea8bc_0
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
-- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>
-- Check '.nextflow.log' file for details
The .command.log in the work folder is like this:
mkdir: can't create directory 'SAMN05826489_SAMN05826489.bam': File exists
mkdir: can't create directory 'human_plus_phix.fna': File exists
I've tried to delete the work folder and run the pipeline again but the same error occurs. At the end, I get only one sample out of five being profiled by Metaphlan in my tests. I really don't know how to solve this without turning off host reads removal.James Fellows Yates
05/24/2025, 7:42 AMJames Fellows Yates
05/24/2025, 7:42 AMnextflow.config?André Ferreira Cunha
05/24/2025, 7:54 AMprocess {
executor = 'azurebatch'
container = 'docker'
}
azure {
managedIdentity {
clientId = 'myid'
}
storage {
accountName = 'mystoragename'
}
batch {
accountName = 'mybatchaccountname'
location = 'westeurope'
autoPoolMode = true
allowPoolCreation = true
pools {
auto {
autoScale = true
vmType = 'Standard_D32d_v4'
virtualNetwork = 'myvnet'
identity {
type = 'UserAssigned'
userAssignedIdentities = [
'myid'
]
}
}
}
}
}André Ferreira Cunha
05/24/2025, 7:56 AMJames Fellows Yates
05/24/2025, 8:00 AM.nextflow.log?André Ferreira Cunha
05/24/2025, 8:04 AMJames Fellows Yates
05/24/2025, 3:18 PMMay-24 07:04:20.845 [TaskFinalizer-3] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489); work-dir=<az://rawdata/work/6d/d9aa8bd4032fd6ab43e3e53801b19a>
error [nextflow.exception.ProcessFailedException]: Process `NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489)` terminated with an error exit status (1)
May-24 07:04:20.873 [TaskFinalizer-3] DEBUG nextflow.processor.TaskRun - Unable to dump output of process 'NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489)' -- Cause: java.nio.file.NoSuchFileException: <az://rawdata/work/6d/d9aa8bd4032fd6ab43e3e53801b19a/.command.out>
May-24 07:04:20.881 [TaskFinalizer-3] DEBUG nextflow.processor.TaskRun - Unable to dump error of process 'NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826489)' -- Cause: java.nio.file.NoSuchFileException: <az://rawdata/work/6d/d9aa8bd4032fd6ab43e3e53801b19a/.command.err>
So it can't even make the `.command.out`/`.command.err` messages. This happens in the start up step before the actual command is executed too, so is happening to what is being recorded in .command.log.
You say you 'keep having this error', did you try running from scratch the pipeline multiple times, and/or did you try with -resume?
Instinctively this to me feels like that particular cloud node has a problem (but I don't know if I can get any more info on that from this log file).James Fellows Yates
05/24/2025, 3:19 PMAdam Talbot
05/24/2025, 4:53 PMAdam Talbot
05/24/2025, 4:54 PMpools.auto.userAssignedIdentities isn’t a config item so won’t do anythingAdam Talbot
05/24/2025, 5:05 PMAt the end, I get only one sample out of five being profiled by Metaphlan in my tests.
Adam Talbot
05/24/2025, 5:05 PMJames Fellows Yates
05/24/2025, 5:10 PMAndré Ferreira Cunha
05/24/2025, 6:08 PMJames Fellows Yates
05/24/2025, 6:18 PMJames Fellows Yates
05/25/2025, 5:12 AMJames Fellows Yates
05/25/2025, 5:13 AMJames Fellows Yates
05/25/2025, 5:13 AMAdam Talbot
05/25/2025, 8:34 AM-profile test?André Ferreira Cunha
05/26/2025, 8:53 AM-profile testJames Fellows Yates
05/26/2025, 9:56 AMAndré Ferreira Cunha
05/26/2025, 10:13 AMNFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS output actually used for host removal? Can I bypass it?James Fellows Yates
05/26/2025, 10:19 AMwithName: *.SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS {
when: false
}
To a config file and give it to the pipeline with -c. But I've not tried it myself before.
But it is VERY strange that the test profile works with the same process and not in your full run...James Fellows Yates
05/26/2025, 10:19 AMAndré Ferreira Cunha
05/26/2025, 2:00 PMwithName: 'SAMTOOLS_STATS' {
when = false
}André Ferreira Cunha
05/26/2025, 2:02 PMAdam Talbot
05/26/2025, 3:17 PMJames Fellows Yates
05/26/2025, 7:18 PMAdam Talbot
05/26/2025, 10:15 PMAdam Talbot
05/26/2025, 10:16 PMJames Fellows Yates
05/27/2025, 5:42 AMAdam Talbot
05/27/2025, 8:53 AMAdam Talbot
05/27/2025, 8:56 AMAdam Talbot
05/27/2025, 9:02 AMwithName: 'SAMTOOLS_STATS' {
disk = 128GB
}
To reserve a chunk of disk on the node at runtime.Adam Talbot
05/27/2025, 9:05 AMProcess `NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826487)` terminated with an error exit status (1)
2. After completion, it did not upload the .command.out and .command.log to remote storage
May-24 17:43:59.262 [TaskFinalizer-2] DEBUG nextflow.processor.TaskRun - Unable to dump error of process 'NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826487)' -- Cause: java.nio.file.NoSuchFileException: <az://rawdata/work/f8/3f88df384251b69bd065cb0b96d34b/.command.err>
May-24 17:43:59.271 [TaskFinalizer-2] DEBUG nextflow.processor.TaskRun - Unable to dump output of process 'NFCORE_TAXPROFILER:TAXPROFILER:SHORTREAD_HOSTREMOVAL:SAMTOOLS_STATS (SAMN05826487)' -- Cause: java.nio.file.NoSuchFileException: <az://rawdata/work/f8/3f88df384251b69bd065cb0b96d34b/.command.out>
3. Nextflow was unable to see what happened because it can't see the log files and exitcode
I have a PR open to catch these situations better: https://github.com/nextflow-io/nextflow/pull/5803André Ferreira Cunha
05/27/2025, 1:37 PMSAMN05826487
SAMN05826489
SAMN05827028
SAMN05826517
SAMN05826516
All associated runs had been already merged into sample FASTQs since I downloaded everything using fastq-dl with --group-by-sample .
I was using the config file I sent earlier in this thread and I think you can see the pipeline parameters in the nextflow.log files.André Ferreira Cunha
05/27/2025, 1:40 PM