I guess we need to come to a consensus on how to p...
# configs
j
I guess we need to come to a consensus on how to pass user specific information to a cluster... I've seen this come up a few times now particular for info such as manually specifying a partition, or an account name, and there seem to be two camps: 1. Use a custom parameter, e.g. `--parameter a`nd use call it with the config with ${params.parameter} 2. Specify as a bash env variable e.g. `PARAMETER`and use
sys.getenv("PARAMETER")
to call it within the config We should decide on one way to make sure the advice is consistent, because people might get confused when getting conflicting info during reviews... While I recognise 1 is more convenient, I prefer 2 as it reduces chances of getting the 'unrecognised parameter' warning as the parameter is not in the pipeline defaults. I know you can add to parameter to ignore parameters validation, but I think it makes it slightly less portable because if a user move to a new cluster, they will think the parameter will work there. Whereas the bash variable is more likely to be considered specific to that HPC etc. On the flip side, the potential change to configs may (?) Make the
getenv
system to not work, so might not work in the long run... Can have a discussion in a thread on this post , but if you have a strong opinion can vote out right with emojis for option 1️⃣ and option 2️⃣
🔢 1
m
how about a logic/function in the template, which does both in the background?
j
What do you mean?
m
have a function which syncs paramters with env variables
j
Not sure I follow... ?
As in you can set either or?
m
exactly
j
Hmmm maybe, although I can't see in my head exactly how that would work though...
(given variability of the names of parameters that could be defined)(
p
I don't see any way forcing either of those solutions would improve things. While allowing for both sounds nice it feels like there would be same problem as for
check_max
in user configs all over. But I don't think this is a big problem now, nor if someone needs to check the documentation when moving to a new cluster. If it has been an issue for review (?), maybe it needs to be clarified that either is okay (along with an recommendation to follow what appears most natural for the resource)?