Hi, I noticed that the aws workflow results that a...
# aws-megatests
n
Hi, I noticed that the aws workflow results that are published on nf-co.re are missing the report. The process ran during the test and produced a file, but it does not show in the AWS results. Is there some configuration that I missed?
j
Which report exactly?
n
genomeassembler is creating a QC report (should be in
report/
) which does not show up here nf-co.re/genomeassembler/…/results-a72d… but all other outputs do?
j
image.png
confused dog 1
yeah definitely present in the bucket...
Does the release run logs on Seqera Platform say anything?
n
I am confused about several things: • I am not sure what the screenshot you pasted is from / what I should take from it. It does not list the folder
report
? • There were two runs on platform: this one was successful, however, I think the merge triggered another test, which failed. I guess the results come from the successful one (disctracted_bernard). The logs seem fine to me, however, also on platform the
outputs
tab lists nothing? • I do not know how to inspect the output bucket via platform; when I look in the task workdir, the report is okay, except that it seems like fonts cannot be fetched from googlefonts?
j
Sorry - the screenshot is from the AWS S3 Interface. I was confirming the nf-core website was just mirroring the bucket file list. This implies either there was a problem publishing the report directory from the work dir to the final results dir, or IT just didn't execute. When you check on platform: there is nothing in the logs about
failed to publish
? We've there is nothing in date explorer? You should be able to see everything in there I believe
There might be a 'inspect workdir' button or something like that ok the specific task window you access via the task table (sorry for slightly vague. Descriptions - on train with crap data)
n
These are the last 3 lines from the log:
Copy code
[ee/508a90] Submitted process > NFCORE_GENOMEASSEMBLER:GENOMEASSEMBLER:REPORT (REPORT)
-[nf-core/genomeassembler] Pipeline completed successfully-
WARN: Slack plugin: Failed to add reaction 'white_check_mark': missing_scope (add reactions:write scope to your Slack app)
🤔 1
j
Ok that does look ok
Question is then is the file in the work dir I guess
If it's still available
n
image.png
This is confusing me a bit
In the workdir of
REPORT
, the report is generated
j
🥲
@Rike Hanssen @Edmund @Phil Ewels Platform experts, any suggestions what else Niklas could do to investigate without us having to get into the AWS Dashboard
n
I also know nothing about AWS or platform. If this needs another full_test run, that is fine for me, but I would adapt the test to avoid some AWS specific failures..
r
do you have it specified in the
tower.yml
?
n
Nope
r
ok then platform doesn't know that it should be a report. Can you add it there? Here are the docs on how to docs.seqera.io/platform-cloud/reports/overview#…
👀 1
n
Thanks
r
you can also check any of the bigger pipelines. rnaseq or sarek have a relatively detailed list if you want something to c&p
n
Just to clarify, this affects the reporting tab in tower right? But does not explain why the
report
folder does not show up in outputs?
r
yes the reporting tab in tower
ah the outputs in platform. This is sth one of us needs to configure in platform for all of nf-core. We'll need to enable lineage and it will also require pipelines to use
outputDir
. I will add it to my things to do at the hackathon. Thanks for flagging, I had forgotten about this
👍 1
n
Ok, great, then I will update
tower.yml
for now
👍 1
p
@Rike Hanssen I think this is a pipeline folder called
report
which is not being published to s3, not Platform features at all..?
Expectation is that the directory would show on the nf-core website
I think we're talking cross-purposes here 👆🏻
r
ah sorry, saw a screenshot with a missing report tab and didn't read the rest carefully: • Platform Reports: needs tower.yml • Platform OUtputs: needs lineage (can enable at the hackathon) •
results/reports
not being published to AWS: no idea. • wbsite mirroring: nothing broken it sounds like
which process is publishing to
reports
?
there are some warnings in the log of the REPORT process
Copy code
[WARNING] Could not fetch resource <https://fonts.googleapis.com/css2?family=Maven+Pro:ital,wght@0,400;0,700;1,400;1,700&display=swap>: HttpExceptionRequest Request {
Is this the file that should have been moved to the reports subfolder?
n
the
REPORT
process publishes to
reports/
. These warnings are there, but they do not stop the report from rendering, it is created in the workdir (
report.html
)
results/reports
not being published to AWS: no idea.
That seems to be the issue, but it is reassuring that it at least there is no (obvious) gross misunderstanding on my side...
👍 1
I also dont do anything weird for the report publishing: github.com/nf-core/genomeassembler/blob/…/report.config, and the reporting module itself is overall similar to the quarto_report nf-core module
j
Thinking about this... maybe you can somehow run the test again, and when getting to the end monitor the logs again and somehow monitor the s3 bucket/work directory?
n
I will probably run the test again when I do the next patch release
👍 1