https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# ask-elementary-ai
s
This message was deleted.
k
A message from kapa.ai
The error message 
The profile 'elementary' does not have a target named 'production'.
indicates that your 
profiles.yml
file under the 
elementary
profile does not define a target called 
production
. This is a common issue when upgrading or changing configurations. How to resolve: 1. Check your `profiles.yml`: Make sure that under the 
elementary
profile, you have a target named 
production
. For example:
Copy code
yaml    elementary:      target: production      outputs:        production:          type: snowflake          # ... other connection details ...        dev:          type: snowflake          # ... other connection details ...
2. If you only have 
dev
and 
prod
targets: If your 
profiles.yml
only has 
dev
and 
prod
(not 
production
), but you are running a command with 
--target production
or similar, you will get this error. You should either: - Change your command to use 
--target prod
, or - Add a 
production
target to your profile. Example from a similar case: > The profile 'elementary' does not have a target named 'default'. The valid target names for this profile are: - dev - prod The solution was to either add 
target: dev
to the profile or run the command with 
-t dev/prod
[see here](https://slack.com/archives/C02CTC89LAX/p1699363455840319). 3. The 
--env
flag is cosmetic: If you are using 
--env prod
or similar, note that this only affects the report/alerts cosmetically. The actual connection is determined by the 
--target
parameter [see here](https://slack.com/archives/C02CTC89LAX/p1699363455840319). Summary: Update your 
profiles.yml
to include a 
production
target, or change your command to use a target that exists in your profile (e.g., 
prod
or 
dev
). This should resolve the error after your upgrade. If you need more specific help, please share your 
profiles.yml
(with sensitive info redacted) and the exact command you are running.
A message from kapa.ai
Based on the knowledge sources, Elementary CLI requires access to a profiles.yml file with an elementary profile to connect to your data warehouse. There is currently no supported way to run Elementary without a profiles.yml file, even in production environments. This is confirmed by trusted support staff: > No, there’s currently no way of running edr without a profiles.yml that defines how to connect to the Elementary schema. > — [Trusted Support Staff 369](https://slack.com/archives/C02CTC89LAX/p1705397377889689) If your production environment does not use a profiles.yml, this will cause the error you are seeing. While you can use environment variables within profiles.yml to avoid hardcoding secrets, the file itself must exist and be accessible to Elementary in all environments, including production. Summary: - Elementary requires a profiles.yml file in all environments, including production. - If it works locally and in CI (where profiles.yml is present), but fails in production (where it is missing), this is expected behavior. - You should ensure that a valid profiles.yml is available in your production environment, possibly templated with environment variables for secrets. If you need to avoid storing the file in source control, you can generate it dynamically or inject it as a secret, but it must be present at runtime for Elementary to function properly. Let me know if you need guidance on securely managing profiles.yml in production!
A message from kapa.ai
To securely manage 
profiles.yml
in production for Elementary, the recommended approach is to avoid hardcoding credentials and instead use environment variables and secret managers. Here are the best practices based on the knowledge sources: 1. Use Environment Variables in profiles.yml Define your 
profiles.yml
with Jinja 
env_var
functions to reference environment variables for all sensitive fields (user, password, keyfile, etc.). For example:
Copy code
yaml    elementary:      target: prod      outputs:        prod:          type: snowflake          account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}"          user: "{{ env_var('SNOWFLAKE_USER') }}"          password: "{{ env_var('SNOWFLAKE_PASSWORD') }}"          role: "{{ env_var('SNOWFLAKE_ROLE') }}"          warehouse: "{{ env_var('SNOWFLAKE_WAREHOUSE') }}"          database: "{{ env_var('SNOWFLAKE_DATABASE') }}"          schema: "{{ env_var('SNOWFLAKE_SCHEMA') }}"
This way, no secrets are stored in the file itself, and you can inject the actual values at runtime via your orchestration tool or CI/CD system [example](https://slack.com/archives/C02CTC89LAX/p1705397377889689). 2. Inject Secrets at Runtime - In Airflow or similar orchestrators, use your secret manager (e.g., AWS Secrets Manager) to inject environment variables into the worker/container that runs Elementary. The 
profiles.yml
can be a template with all sensitive values coming from environment variables [example](https://slack.com/archives/C02CTC89LAX/p1665489623025739). - In GitHub Actions, you can store the entire 
profiles.yml
as a secret and pass it to the workflow, or generate it dynamically in a job step using secrets as environment variables [example](https://slack.com/archives/C02CTC89LAX/p1682351403423809). 3. Do Not Store profiles.yml in Source Control The file does not need to be in your repo. You can generate it on the fly in your CI/CD pipeline or orchestration job, using secrets and environment variables, and then clean it up after the run [example](https://slack.com/archives/C02CTC89LAX/p1682351403423809). 4. Minimal Example for Airflow/Kubernetes - Use a minimal 
profiles.yml
with all credentials as environment variables. - Inject secrets using your orchestrator’s secret management (e.g., Airflow’s KubernetesPodOperator with secrets) [example](https://slack.com/archives/C02CTC89LAX/p1665489623025739). Summary: - Always use environment variables for secrets in 
profiles.yml
. - Inject those variables at runtime using your orchestrator or CI/CD’s secret management. - Do not commit 
profiles.yml
with secrets to source control. - The file must exist at runtime, but it can be generated or injected securely. If you need a specific example for your orchestration tool, let me know!
A message from kapa.ai
To fix the error:
Copy code
The profile 'elementary' does not have a target named 'production'. The valid target names for this profile are:    - default
you need to ensure that your 
profiles.yml
(provided to Elementary in your GitHub Action) contains a target named 
production
under the 
elementary
profile, or that your GitHub Action is referencing the correct existing target (in your case, 
default
). How to fix: ### 1. Update your 
profiles.yml
to include a 
production
target If you want to use 
production
as your target, your 
profiles.yml
should look like this:
Copy code
yaml elementary:   target: production   outputs:     production:       type: <your_warehouse_type>       # ... other connection details ...     default:       type: <your_warehouse_type>       # ... other connection details ...
Then, in your GitHub Action, you can run:
Copy code
yaml edr report --target production
### 2. Or, use the existing 
default
target If your 
profiles.yml
only has a 
default
target, make sure your GitHub Action uses:
Copy code
yaml edr report --target default
or omits the 
--target
flag (since 
default
is the default target). ### 3. How to provide 
profiles.yml
in GitHub Actions You can store your 
profiles.yml
as a GitHub secret (e.g., 
ELEMENTARY_PROFILES_YML
) and pass it to the Elementary GitHub Action using the 
profiles-yml
input:
Copy code
yaml - name: Run Elementary   uses: elementary-data/run-elementary-action@v1.8   with:     warehouse-type: <your_warehouse_type>     profiles-yml: ${{ secrets.ELEMENTARY_PROFILES_YML }}     edr-command: edr report --target production
[See example](https://slack.com/archives/C02CTC89LAX/p1694694572757179) Summary: - Make sure the target you specify in your command exists in your 
profiles.yml
under the 
elementary
profile. - Either add a 
production
target to your 
profiles.yml
, or use the 
default
target in your command. - Pass the correct 
profiles.yml
to the GitHub Action as a secret. If you need to dynamically generate the 
profiles.yml
in your workflow, you can do so in a step before running Elementary, as shown in [this example](https://slack.com/archives/C02CTC89LAX/p1682351403423809). Let me know if you need a sample workflow!
A message from kapa.ai
New message text here
A message from kapa.ai
The error message:
Copy code
Error: Source freshness for invocation id effb4596-477e-45cd-8d93-2405e95b4783 were already uploaded.
means that Elementary has already uploaded the source freshness results for this specific dbt invocation. This is a built-in safeguard to prevent duplicate uploads of the same results. What does this mean? - Each time you run 
dbt source freshness
, it generates a new 
sources.json
file with a unique invocation ID. - When you run 
edr run-operation upload-source-freshness
, Elementary checks if the results for that invocation ID have already been uploaded. - If you try to upload the same results again (without re-running 
dbt source freshness
), you will get this error. How to resolve: - Re-run 
dbt source freshness
before running 
edr run-operation upload-source-freshness
to generate a new invocation ID and fresh results. - Make sure your CI workflow does not attempt to upload the same 
sources.json
multiple times without regenerating it. This behavior is by design to avoid duplicate data in the Elementary tables. If you need to upload new results, always ensure you have a fresh run of 
dbt source freshness
first. If you need to debug further, check that your workflow is not reusing the same 
sources.json
file or re-running the upload step without a new dbt run. If you have more questions or need to adjust your workflow, let me know!
35 Views
Open in Slack
PreviousNext
Powered by