Ok no it's failing agian, it keeps being aborted b...
# taxprofiler
j
Ok no it's failing agian, it keeps being aborted because it can't publish files to s3...
@Maxime Garcia @Rike Hanssen does this look like a fusion issue?
Copy code
Aug-28 08:00:32.887 [PublishDir-4] DEBUG nextflow.processor.PublishDir - Failed to publish file: <s3://nf-core-awsmegatests/scratch/5gThtWHC12qu4j/06/a195b4f75bbcdffc971a0c9594c0df/MOCK_001_Illumina_Hiseq_3000_1.bam.bai>; to: <s3://nf-core-awsmegatests/taxprofiler/results-ec6a749e2a4d528ada5c13b6f8be6b5d808f7383/MOCK_001_Illumina_Hiseq_3000_1.bam.bai> [copy] -- attempt: 4; reason: class Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 cannot be cast to class java.lang.String (Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 is in unnamed module of loader ch.artecat.grengine.load.BytecodeClassLoader @26101efc; java.lang.String is in module java.base of loader 'bootstrap')
r
not immediatly to me. Looks more like something is funky with a closure here:
Copy code
reason: class Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 cannot be cast to class java.lang.String (Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 is in unnamed module of loader ch.artecat.grengine.load.BytecodeClassLoader @26101efc; java.lang.String is in module java.base of loader 'bootstrap')
j
It's running fine on my cluster though
and all the local tests pass...
The closure thing is failing on
Failed to publish file: <s3://nf-core-awsmegatests/scratch/5gThtWHC12qu4j/06/a195b4f75bbcdffc971a0c9594c0df/MOCK_001_Illumina_Hiseq_3000_1.bam.bai>;
which seems like a cloud saving error
r
can you point me to the code where this file is being published
I think?
We don't explicitly pick up any
.bai
file in modules.config, so I assume it's the template
modules.config
code
Everything gets aborted which I find odd
r
@Edmund the log has lines about s3 tags:
Copy code
java.lang.ClassCastException: class Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 cannot be cast to class java.lang.String (Script9C18B823B9A324380AE1870F9572036F$_run_closure3$_closure6 is in unnamed module of loader ch.artecat.grengine.load.BytecodeClassLoader @26101efc; java.lang.String is in module java.base of loader 'bootstrap')
	at nextflow.cloud.aws.nio.S3Path.getTagsList(S3Path.java:556)
	at nextflow.cloud.aws.nio.S3FileSystemProvider.copy(S3FileSystemProvider.java:555)
do you think this may be related:
Copy code
tags: [
            'compute_env': 'aws_ireland_fusionv2_nvme_cpu_snapshots',
            'pipeline': { workflow.manifest.name },
            'session': { workflow.sessionId },
            'timestamp': { new Date().format('yyyy-MM-dd_HH-mm-ss') },
            'architecture': 'x86_64',
            'fusion': 'enabled',
            'fusionSnapshots': 'enabled'
        ]
@James Fellows Yates to test you could set the
tags
in the
publishDir
to empty
sad cowboy 1
j
Eek OK, let me make a new branch
Wait, do you mean delete them entirely?
Or set to
null
?
r
no need, you should be able to overwrite it in the nextflow.config
j
That's a lot of tags to replace ๐Ÿ˜… but OK
Ah you mean here?
image.png
r
๐Ÿ˜„ whatever works for you
yep
j
Oooh, not my per-module tag in
modules.config
?
image.png
This sort o fempty/
๐Ÿ‘ 1
r
now I am atually wondering if we are messing with peoples publishing ๐Ÿคจ
j
๐Ÿš€ launching
image.png
Should I try removing it entirely?
r
sure
j
That seems to be working
It worked
image.png
s
It failed again. https://github.com/nf-core/taxprofiler/pull/636 I missed checking that the AWS tests had passed before doing the releas, I only saw the green tick from the GitHub actions.
j
Yeah don't worry I will trigger manually Tomorrow
s
ok thank you!!