K
05/30/2024, 8:13 AMK
05/30/2024, 8:41 AMK
05/31/2024, 7:38 AMDenis Kokorin
05/31/2024, 10:06 AMpip install ...
dbt deps
edr report ...
#or
edr monitor ...K
05/31/2024, 10:21 AMK
06/05/2024, 1:12 PMname: Run Elementary
on:
# Run the action when a push to the main branch of the repository is made.
push:
branches: [ "main", "master" ]
# Run the action when a pull request to the main branch is opened.
pull_request:
branches: [ "main", "master" ]
# Run the action in a scheduled manner every hour.
schedule:
- cron: '0 9,13,19 * * 1-5'
# Allows you to run this workflow manually from the Actions tab.
workflow_dispatch:
env:
SNOWFLAKE_ACCOUNT: ${{ secrets.GITHUB_TOKEN }}
SNOWFLAKE_USER: ${{ secrets.SNOWFLAKE_USER }}
SNOWFLAKE_PASSWORD: ${{ secrets.SNOWFLAKE_PASSWORD }}
SNOWFLAKE_DB: ${{ secrets.SNOWFLAKE_DB }}
SNOWFLAKE_WAREHOUSE: ${{ secrets.SNOWFLAKE_WAREHOUSE }}
jobs:
elementary:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout dbt project
uses: actions/checkout@v3
- name: Run Elementary
uses: elementary-data/run-elementary-action@v1.11
with:
warehouse-type: snowflake # Type of warehouse to use (bigquery, snowflake, redshift, etc.)
adapter-version: 1.5.0 # The dbt-adapter version that should be used (If not provided the latest version will be used).
profiles-yml: ${{ secrets.elementary }} # Content of ~/.dbt/profiles.yml, should have an `elementary` profile.
edr-command:
edr monitor
--slack-token ${{ secrets.elementary_slack_token }}
--slack-channel-name "tech-alerts-data-issues"
&&
edr report
--file-path "report.html"
&&
edr send-report
--slack-token ${{ secrets.elementary_slack_token }}
--slack-channel-name "tech-alerts-data-issues"
- name: Upload report
uses: actions/upload-artifact@v3
with:
name: report.html
path: report.html
- name: Upload log
if: always()
uses: actions/upload-artifact@v3
with:
name: edr.log
path: edr.logK
06/05/2024, 1:12 PM2024-06-05 13:07:54,200 Failed to get Elementary dbt package version: {'returncode': 1, 'cmd': ['dbt', '--log-format', 'json', 'run-operation', 'get_elementary_dbt_pkg_version', '--project-dir', '/edr_stager_dbt_project'], 'output': b'{"data": {"log_version": 3, "version": "=1.5.11"}, "info": {"category": "", "code": "A001", "extra": {}, "invocation_id": "0e01a258-16fb-43ea-9611-1657f3a6676a", "level": "info", "msg": "Running with dbt=1.5.11", "name": "MainReportVersion", "pid": 53, "thread": "MainThread", "ts": "2024-06-05T13:07:52.302819Z"}}\n{"data": {"profile_name": "elementary", "target_name": "default"}, "info": {"category": "", "code": "A005", "extra": {}, "invocation_id": "0e01a258-16fb-43ea-9611-1657f3a6676a", "level": "info", "msg": "target not specified in profile \'elementary\', using \'default\'", "name": "MissingProfileTarget", "pid": 53, "thread": "MainThread", "ts": "2024-06-05T13:07:52.307216Z"}}\n{"data": {"adapter_name": "snowflake", "adapter_version": "=1.5.0"}, "info": {"category": "", "c
Traceback (most recent call last):
File "/entrypoint.py", line 148, in <module>
main()
File "/entrypoint.py", line 139, in main
install_edr(args.adapter, args.project_dir, args.profile_target)
File "/entrypoint.py", line 62, in install_edr
command_results = subprocess.run(
File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['dbt', '--log-format', 'json', 'run-operation', 'get_elementary_dbt_pkg_version', '--project-dir', '/edr_stager_dbt_project']' returned non-zero exit status 1.Denis Kokorin
06/05/2024, 1:15 PM"target not specified in profile \'elementary\', using \'default\'", "name": "MissingProfileTarget"
K
06/05/2024, 1:19 PMK
06/05/2024, 1:20 PMDenis Kokorin
06/05/2024, 1:20 PMK
06/05/2024, 1:21 PM2024-06-05 13:17:44,393 Failed to get Elementary dbt package version: {'returncode': 1, 'cmd': ['dbt', '--log-format', 'json', 'run-operation', 'get_elementary_dbt_pkg_version', '--project-dir', '/edr_stager_dbt_project'], 'output': b'{"data": {"log_version": 3, "version": "=1.5.11"}, "info": {"category": "", "code": "A001", "extra": {}, "invocation_id": "0f64e51d-98d8-4b57-a8f1-c2df71f4e659", "level": "info", "msg": "Running with dbt=1.5.11", "name": "MainReportVersion", "pid": 54, "thread": "MainThread", "ts": "2024-06-05T13:17:42.500196Z"}}\n{"data": {"adapter_name": "snowflake", "adapter_version": "=1.5.0"}, "info": {"category": "", "code": "E034", "extra": {}, "invocation_id": "0f64e51d-98d8-4b57-a8f1-c2df71f4e659", "level": "info", "msg": "Registered adapter: snowflake=1.5.0", "name": "AdapterRegistered", "pid": 54, "thread": "MainThread", "ts": "2024-06-05T13:17:42.805355Z"}}\n{"data": {"reason": "saved manifest not found. Starting full parse."}, "info": {"category": "", "code": "I024", "extra": {}, "i
Traceback (most recent call last):
File "/entrypoint.py", line 148, in <module>
main()
File "/entrypoint.py", line 139, in main
install_edr(args.adapter, args.project_dir, args.profile_target)
File "/entrypoint.py", line 62, in install_edr
command_results = subprocess.run(
File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['dbt', '--log-format', 'json', 'run-operation', 'get_elementary_dbt_pkg_version', '--project-dir', '/edr_stager_dbt_project']' returned non-zero exit status 1.K
06/05/2024, 1:22 PMElementary needs to install some packages for any EDR commandI would need to give permissions to write packages in this case
K
06/05/2024, 2:06 PM