Hi team, Is there any way to pass the profile dynamically? That's my bash command currently, bash_command=f"$DBT_VENV_PATH_1_5/bin/edr send-report --s3-bucket-name {s3_bucket}", and I need to add the profiles

Hi @Paulo Henrique Orind, the typical solution to this is to populate the

profiles.yml

using environment variables. Here's an example.

I meant not having the profiles.yml file

No, there's currently no way of running

edr

without a

profiles.yml

that defines how to connect to the Elementar schema. Care to elaborate why can't you have the

profiles.yml

file?

I can set it up, but since we're triggering the EDR from a DAG, we prefer not to store a profiles file in S3 solely for this purpose

You don't need to. What you'd want to do in this case is to have a

profiles.yml

file on your Airflow worker that is purely a template, like this:

profile:
  target: prod
  outputs:
    prod:
      type: postgres
      host: 127.0.0.1
      # IMPORTANT: Make sure to quote the entire Jinja string here
      user: "{{ env_var('DBT_USER') }}"
      password: "{{ env_var('DBT_PASSWORD') }}"
      ....

Then, you run

edr

with those environment variables filled from your secret manager, not S3. What do you think?

Good one, it works, thanks!