https://linen.dev logo
Join Slack
Powered by
Hello, i'm having trouble here with `--days-back` ...
# community-support
l
Hello, i'm having trouble here with 
--days-back
command. I execute this command: 
["edr", "monitor", "--slack-token", SLACK_TOKEN, "--slack-channel-name", "tech-g4-infra"]
Until yesterday it was returning this on log: 
2024-05-02 14:36:32 — INFO — Running dbt run -m elementary_cli.alerts.alerts_v2 --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project
Today it started to return the command with the 
days_back
option, even though i don't set it 
2024-05-02 15:34:43 — INFO — Running dbt run -m elementary_cli.alerts.alerts_v2 --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project --vars {"days_back": 1}
And now i'm getting this error on the log execution (complete error on the thread):
Copy code
`File "/usr/local/lib/python3.9/site-packages/elementary/monitor/data_monitoring/alerts/data_monitoring_alerts.py", line 131, in _get_suppressed_alerts
    (current_time_utc - last_sent_time).total_seconds() / 3600
TypeError: can't subtract offset-naive and offset-aware datetimes
Copy code
airflow-worker-gv2nw
*** Reading remote log from <gs://us-east1-data-engineering-p-4f99987a-bucket/logs/dag_id=dbt_analytics_hourly_job/run_id=scheduled__2024-05-02T22:10:00+00:00/task_id=task_trigger_elementary_slack_notification/attempt=1.log>.
[2024-05-02, 20:32:01 -03] {taskinstance.py:1104} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: dbt_analytics_hourly_job.task_trigger_elementary_slack_notification scheduled__2024-05-02T22:10:00+00:00 [queued]>
[2024-05-02, 20:32:01 -03] {taskinstance.py:1104} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: dbt_analytics_hourly_job.task_trigger_elementary_slack_notification scheduled__2024-05-02T22:10:00+00:00 [queued]>
[2024-05-02, 20:32:01 -03] {taskinstance.py:1309} INFO - Starting attempt 1 of 1
[2024-05-02, 20:32:01 -03] {taskinstance.py:1328} INFO - Executing <Task(PythonOperator): task_trigger_elementary_slack_notification> on 2024-05-02 22:10:00+00:00
[2024-05-02, 20:32:01 -03] {standard_task_runner.py:57} INFO - Started process 1161751 to run task
[2024-05-02, 20:32:01 -03] {standard_task_runner.py:84} INFO - Running: ['airflow', 'tasks', 'run', 'dbt_analytics_hourly_job', 'task_trigger_elementary_slack_notification', 'scheduled__2024-05-02T22:10:00+00:00', '--job-id', '591080', '--raw', '--subdir', 'DAGS_FOLDER/dados/dbt_analytics_hourly_job.py', '--cfg-path', '/tmp/tmp8_1jcb5g']
[2024-05-02, 20:32:01 -03] {standard_task_runner.py:85} INFO - Job 591080: Subtask task_trigger_elementary_slack_notification
[2024-05-02, 20:32:02 -03] {task_command.py:414} INFO - Running <TaskInstance: dbt_analytics_hourly_job.task_trigger_elementary_slack_notification scheduled__2024-05-02T22:10:00+00:00 [running]> on host airflow-worker-gv2nw
[2024-05-02, 20:32:02 -03] {taskinstance.py:1547} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='airflow' AIRFLOW_CTX_DAG_ID='dbt_analytics_hourly_job' AIRFLOW_CTX_TASK_ID='task_trigger_elementary_slack_notification' AIRFLOW_CTX_EXECUTION_DATE='2024-05-02T22:10:00+00:00' AIRFLOW_CTX_TRY_NUMBER='1' AIRFLOW_CTX_DAG_RUN_ID='scheduled__2024-05-02T22:10:00+00:00'
[2024-05-02, 20:33:47 -03] {dbt_jobs.py:30} INFO - CONTENT:     ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

You are using Elementary 0.14.0, however version 0.15.0 is available.
Consider upgrading by running: "/usr/local/bin/python -m pip install --upgrade elementary-data"

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2024-05-02 23:32:09 — INFO — Running with edr=0.14.0
2024-05-02 23:32:09 — INFO — Installing packages for edr internal dbt package...
2024-05-02 23:32:09 — INFO — Running dbt --log-format json deps --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project
2024-05-02 23:32:19 — INFO — Running with dbt=1.7.13
2024-05-02 23:32:19 — INFO — Updating lock file in file path: /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project/package-lock.yml
2024-05-02 23:32:19 — INFO — Installing dbt-labs/dbt_utils
2024-05-02 23:32:19 — INFO — Installed from version 0.8.6
2024-05-02 23:32:19 — INFO — Updated version available: 1.1.1
2024-05-02 23:32:19 — INFO — Installing elementary-data/elementary
2024-05-02 23:32:19 — INFO — Installed from version 0.14.0
2024-05-02 23:32:19 — INFO — Updated version available: 0.15.0
2024-05-02 23:32:19 — INFO — 
2024-05-02 23:32:19 — INFO — Updates available for packages: ['dbt-labs/dbt_utils', 'elementary-data/elementary']                 
Update your versions in packages.yml, then run dbt deps
[33;20m2024-05-02 23:32:53 — WARNING — You are using incompatible versions between edr (0.14.0) and Elementary's dbt package (0.15.0).
 To fix please run:
pip install --upgrade elementary-data
[0m
2024-05-02 23:33:03 — INFO — Elementary's database and schema: '"g4-data.elementary"'
2024-05-02 23:33:03 — INFO — Running internal dbt run to populate alerts
2024-05-02 23:33:03 — INFO — Running dbt run -m elementary_cli.alerts.alerts_v2 --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project
[0m23:33:09  Running with dbt=1.7.13
[0m23:33:09  target not specified in profile 'elementary', using 'default'
[0m23:33:13  Registered adapter: bigquery=1.7.7
[0m23:33:14  Found 41 models, 2 operations, 6 tests, 6 sources, 0 exposures, 0 metrics, 1376 macros, 0 groups, 0 semantic models
[0m23:33:14  
[0m23:33:15  
[0m23:33:15  Running 1 on-run-start hook
[0m23:33:15  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
[0m23:33:15  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [[32mOK[0m in 0.00s]
[0m23:33:15  
[0m23:33:15  Concurrency: 8 threads (target='default')
[0m23:33:15  
[0m23:33:15  1 of 1 START sql incremental model elementary.alerts_v2 ........................ [RUN]
[0m23:33:24  1 of 1 OK created sql incremental model elementary.alerts_v2 ................... [[32mMERGE (0.0 rows, 4.6 MiB processed)[0m in 9.42s]
[0m23:33:24  
[0m23:33:24  Running 1 on-run-end hook
[0m23:33:24  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
[0m23:33:24  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [[32mOK[0m in 0.00s]
[0m23:33:24  
[0m23:33:24  
[0m23:33:24  Finished running 1 incremental model, 2 hooks in 0 hours 0 minutes and 10.60 seconds (10.60s).
[0m23:33:24  
[0m23:33:24  [32mCompleted successfully[0m
[0m23:33:24  
[0m23:33:24  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
2024-05-02 23:33:25 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_pending_alerts", "macro_args": {"days_back": 1, "type": null}} --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project
2024-05-02 23:33:36 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_last_alert_sent_times", "macro_args": {"days_back": 1, "type": null}} --project-dir /usr/local/lib/python3.9/site-packages/elementary/monitor/dbt_project

Traceback (most recent call last):
  File "/usr/local/bin/edr", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/elementary/cli/cli.py", line 67, in invoke
    return super().invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/elementary/monitor/cli.py", line 364, in monitor
    success = data_monitoring.run_alerts(
  File "/usr/local/lib/python3.9/site-packages/elementary/monitor/data_monitoring/alerts/data_monitoring_alerts.py", line 295, in run_alerts
    sorted_alerts = self._sort_alerts(
  File "/usr/local/lib/python3.9/site-packages/elementary/monitor/data_monitoring/alerts/data_monitoring_alerts.py", line 101, in _sort_alerts
    suppressed_alerts = self._get_suppressed_alerts(alerts, alerts_last_sent_times)
  File "/usr/local/lib/python3.9/site-packages/elementary/monitor/data_monitoring/alerts/data_monitoring_alerts.py", line 131, in _get_suppressed_alerts
    (current_time_utc - last_sent_time).total_seconds() / 3600
TypeError: can't subtract offset-naive and offset-aware datetimes

[2024-05-02, 20:33:47 -03] {taskinstance.py:1826} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/gcs/dags/utils/dbt_jobs.py", line 32, in execute_job
    response.raise_for_status()
  File "/opt/python3.8/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: <https://dbt-analytics-jobs-l6mn3tpara-ue.a.run.app/elementary-slack-notification>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/gcs/dags/dados/dbt_analytics_hourly_job.py", line 35, in trigger_elementary_slack_notification
    DbtAnalytics().execute_job("elementary-slack-notification")
  File "/home/airflow/gcs/dags/utils/dbt_jobs.py", line 44, in execute_job
    raise Exception(
Exception: Task executed successfully. Job has 0 errors.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/python3.8/lib/python3.8/site-packages/airflow/operators/python.py", line 181, in execute
    return_value = self.execute_callable()
  File "/opt/python3.8/lib/python3.8/site-packages/airflow/operators/python.py", line 198, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/home/airflow/gcs/dags/dados/dbt_analytics_hourly_job.py", line 37, in trigger_elementary_slack_notification
    raise AirflowException(e)
airflow.exceptions.AirflowException: Task executed successfully. Job has 0 errors.
[2024-05-02, 20:33:47 -03] {taskinstance.py:1346} INFO - Marking task as FAILED. dag_id=dbt_analytics_hourly_job, task_id=task_trigger_elementary_slack_notification, execution_date=20240502T221000, start_date=20240502T233201, end_date=20240502T233347
[2024-05-02, 20:33:47 -03] {standard_task_runner.py:104} ERROR - Failed to execute job 591080 for task task_trigger_elementary_slack_notification (Task executed successfully. Job has 0 errors.; 1161751)
[2024-05-02, 20:33:48 -03] {local_task_job_runner.py:225} INFO - Task exited with return code 1
[2024-05-02, 20:33:48 -03] {taskinstance.py:2656} INFO - 2 downstream tasks scheduled from follow-on schedule check
We've changed to version 
0.14.1
and the problem got solved
34 Views
Open in Slack
PreviousNext
Powered by