https://nf-co.re logo
Join Slack
Powered by
hello! Are the AWS test GA working? I get failure ...
# pipeline-maintainers
t
hello! Are the AWS test GA working? I get failure within 10 sec without any useful errors being logged (during launch workflow via seqera platform) github.com/nf-core/eager/…/96718657381 Could this have to do with me not being part of the nf-core org on the seqera platform? 😅
r
hit and miss. Reruning it usually fixes it, but also Phil is working on updating the action at the moment which should make it better. no it has nothing to do with your membership in platform, the tokens that are used are from a service account.
t
so I just keep retrying until it works? 🤔
r
yeah, but i also added you to platform now, so you can also just submit there directly
🙏 1
t
thanks Rike!
I don’t think I can add a pipeline to the launchpad, and eager is not on there atm. Meanwhile, from the nf-core website I can only launch v 2.5.3, while I’m trying to test a specific dev branch. Do I need further permissions, or am I misunderstanding how to use the platform?
r
no you are right launch users cannot. Let me bump your perm temporarily
done
t
thanks! I added eager to the runlist.
👍 1
So, I’ve now tried to run this multiple times on the sequera platform and all jobs keep failing with error
Copy code
Missing AWS Batch job queue -- provide it by using the process `queue` directive
I assume this is something that gets configured at the template level? I’ve tried runs providing additional nf-core profiles 
awsbatch
or 
aws_tower
, to no avail. Is there some configuration I should be including that I am somehow missing? I have looked for some documentation of how all this should be set up on the nf-core website as well, but I could only find this bytesize from 2021, which I assume is outdated, since it mostly talks about the GA setup.
r
hm let me take a look, this looks like it either gets submitted with an invalid CE or the queue got deleted on the AWS side.
🙏 1
Copy code
WARN: The following invalid input values have been detected:
* --bam_mapping_quality_threshold: 25
* --awscli: /home/ec2-user/miniconda/bin/aws
* --bwaalnl: 1024
* --bwaalnn: 0.04
* --awsregion: eu-west-1
* --run_bam_filtering: true
* --gatk_ploidy: 2
* --bam_unmapped_type: discard
🚫
t
well, it’s running at least 🙆 What did you do differently? 😅
Those options are my bad, but should not affect the run. homer disappear
r
we got an on-demand fallback to catch spot reclamations in case fusion snapshotting isn't able to handle them. For some reason I had to set the spot queue explicitely:
Copy code
process {
    queue = { task.attempt > 1 ? 'TowerForge-6nSba1hDuNFP08JP7SjigJ' : 'TowerForge-29o9qiv0F5P6DSWkbWgj7w-work' }
}
for other pipelines it's fine not to. 🤷 tbh this is a ll a bit of a hack. i am planning a hackathon group to make time to clean this up
another needed element: docs homer disappear
t
I see! that makes sense. Is that something that’s okay for me to specify as well? I want to fix the warnings above, which would require another snapshot update it seems >_<
I would also tweak the profile to not have failing jobs and retries due to memory limits
r
yes that's fine
you should have the permissions, otherwise we can also add the pipeline with the dev branch + config to the launchpad
t
perfect. Thanks Rike! You’ve been massively helpful, as always! 🎉
4 Views
Open in Slack
PreviousNext
Powered by