https://nf-co.re logo
Join Slack
Powered by
<@U044S8RL7K7> do you know what's going on? :point...
# pixelator
p
@Adrien Coulier do you know what's going on? 👆
j
@Adrien Coulier is on vacation, so I am the one responsible for the ones triggered today at least.
p
ok cool - so they're being triggered manually / intentionally then?
j
Nope.
I didn't know they existed...
I made a patch release today, so I am guessing that is what triggered this.
p
Yeah, I think that they get triggered on PR approval to 
main
and maybe again on push to 
main
(?)
j
I'll add looking into this to out backlog. I didn't mean to spam. Sorry for that.
p
To be honest, maybe no need - sounds like it's working as intended
Idea is to run full-size tests at appropriate times prior to pipeline release, and notify devs here after long-running full size test if there was a failure
j
Ok. Then things are as expected. 🙂 Thanks for checking in.
We did a big new product release yesterday, and unfortunately some bugs snuck through the validation, so we needed to do a few quick patches to get everything up to scratch, hence the noise here.
👍🏻 1
p
No probs, congrats!
j
Thanks!
m
Since your megatests are not really megatests since they only last 10min or so. Shouldn't we do that mainly just with regular CI on GitHub?
r
Since your megatests are not really megatests since they only last 10min or so. Shouldn't we do that mainly just with regular CI on GitHub?
then there wouldn't be any results in the AWS results page, right?
m
oh right
but I do think they differ from 'real life' expected results, no?
a
They are based on simulated data, so the results lack the same diversity a real run would have unfortunately 😕 We had a filtered dataset that took too long to run for regular test that we could use though 🤔 I'll add that to our backlog!
p
the full size test is meant to be a realistic dataset, eg. from a publication
it's set up so that it can run for hours if it needs to, that's kind of the idea. These tests sometimes pick up stuff that the smaller test datasets miss. It also gives useful reference outputs for folks to see on the website.
a
I see, the runs can get kind of expensive though, how is the cost handled? Also where is such data typically hosted?
p
We have credits from AWS for this which cover run costs and hosting on s3
❤️ 1
As long as it's not crazy expensive, and as long as you're not triggering loads of tests (eg. 1-2 per release) it should be fine.
Like, 10s of $s per run is fine.
If it's 100s then maybe we need to think about it
a
Alright, we'll discuss this internally and see what we can find 👍
p
Looking for docs on the topic but we don't have much. Only thing points to this bytesize but it's from 2021 (!) and is hugely outdated 🥲
7 Views
Open in Slack
PreviousNext
Powered by