https://ploomber.io/ logo
Join Slack
Powered by
This message was deleted.
# ask-anything
s
This message was deleted.
e
create an 
env.yaml
file and then reference the parameter in the tasks:
Copy code
#env.yaml
some_param: some_value

# pipeline.yaml
- source: some-task.py
  params:
    some_param: '{{some_param}}'

- source: another-task.py
  params:
    some_param: '{{some_param}}'
does this work?
both tasks will get 
some_param='some-value'
m
when i use ploomber scaffold in cli, it creates a environment.yml file. I should use this?
so for example let's say I want to use the same csv file in several tasks but the csv file isn't create by anything upstream
e
environment.yml is to declare your conda/pip dependencies, so create an env.yaml check this out
yeah, a path to a csv is a common use case for a parameter. the recipe I shared should work. one tip, you can define the path like this:
Copy code
# env.yaml
path_to_csv: '{{here}}/path/to/data.csv'
{{here}}
will be replaced for the directory where the env.yaml is located, this is more robust since you'll get an absolute path that you can load anywhere regardless of the current working directory
👍 1
m
ah nice, that is a good tip. Just to be clear once the param is defined in the env.yaml file it needs to be re specified in the pipeline.yaml to be used in a task?
kinda preferable to go from env.yaml to task to skip retyping
e
yeah, you have to re-specified it. we've gotten this feedback before. any thoughts? how would you like the API to look like to be more succinct
m
env.yaml seems redundant, why not just specify params in pipeline.yaml?
does {{here}} work in pipeline.yaml?
e
yes {{here}} works in pipeline.yaml. sure you can define them without an env.yaml
Copy code
# pipeline.yaml
- source: some-task.py
  params:
    some_param: '{{here}}/some/path.csv'

- source: another-task.py
  params:
    some_param: '{{here}}/some/path.csv'
m
yea i guess this is preferable to the env.yaml file solution
its just too easy to retype it incorrectly
once source of truth is a good design philosophy I guess
thank you a ton as always!
e
sure!
m
the doc u linked to is also very helpful !
🙌 1
5 Views
Open in Slack
PreviousNext
Powered by