This message was deleted.

hi, yeah this has been in our backlog for a while.

unfortunately all i can contribute are user requests at this point🤣

no worries, that's great! We'll get back to running the weekly polls to know what to prioritize - those have been pretty useful

This is probably a stretch but it would be cool to support an expression language in the placeholders (whether in env.yaml or pipeline.yaml). For example my pipeline generates date-based output folders. I can inject {{now}} to ensure consistency across the notebooks (avoids problems in the edge case where you run the pipeline at 11:59pm), but I have to keep applying the same .strftime format across the notebooks. Would be better if I could apply the xform in the pipeline.yaml using {{now.strftime(‘%Y%m%d’)}}.

expressions are actually supported but undocumented. we use jinja so any jinja valid expresion will work. we haven't documented it since encouraging logic in the env.yaml might cause beginners to put too much stuff there, but it's possible. the challenge with

{{now}}

, is that we pass it as a string, so you'd need to parse it as an datetime then back to a string, which isn't ideal. what about supporting dotted paths in

env.yaml

?

# env.yaml
now: functions.now # executes the function and assigns the returned value to now

# functions.py

def now():
    # your logic goes here
    return something

I ended up just making a util function that I use to keep things reasonably DRY

hey, what Python version are you using? we'll release this in the next major update (0.20), but this will also drop support for python 3.6

i'm on 3.10

ok, all good then

Coming back to this. First, my util function was no good because it prevented me from using Ploomber's "product" feature to pass info across stages. So I tried the Jinja thing and it worked great. Happy to use that for now but I could definitely see it being useful to be able to format {{now}} for pipeline outputs. We timestamp different pipeline runs.

we had another question today that would be solved with the dynamic params thing (pass a function as parameter, the task receives the output of the function). I'll add it to the backlog, it's easy to implement re env.yaml params: great! turned out to be a lot simpler than I thought, it was like 20 lines of code