https://linen.dev logo
Join Slack
Powered by
Hello <@U088WCP3B4P> ! We have been running elemen...
# ask-elementary-ai
a
Hello @kapa.ai ! We have been running elementary for some time with great success. However recently after a deployment, the issue below appeared when running the edr send-report cmd. Can you take a look and help me resolve this ? This happens right at the end of the cmd when it has already sent the report to GCS but hasn't sent the report yet to Slack. Logs :
Copy code
[2025-07-24, 09:45:23 CEST] {pod_manager.py:472} INFO - [base] 2025-07-24 07:45:22 — INFO — Uploading to GCS bucket "data-static-data-tech-eu"
[2025-07-24, 09:45:23 CEST] {pod_manager.py:472} INFO - [base] 2025-07-24 07:45:23 — INFO — Uploaded report to GCS.
[2025-07-24, 09:45:23 CEST] {pod_manager.py:472} INFO - [base] 2025-07-24 07:45:23 — INFO — Updated GCS bucket's website.
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base] 2025-07-24 07:45:23 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 30, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /usr/local/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /usr/local/bigquery-dbt --target elementary-prod
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base] Traceback (most recent call last):
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/bin/edr", line 8, in <module>
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     sys.exit(cli())
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]              ^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1442, in __call__
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return self.main(*args, **kwargs)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1363, in main
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     rv = self.invoke(ctx)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]          ^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/cli/cli.py", line 66, in invoke
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return super().invoke(ctx)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1830, in invoke
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return _process_result(sub_ctx.command.invoke(sub_ctx))
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1226, in invoke
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return ctx.invoke(self.callback, **ctx.params)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 794, in invoke
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return callback(*args, **kwargs)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 34, in new_func
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return f(get_current_context(), *args, **kwargs)
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/monitor/cli.py", line 741, in send_report
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     sent_report_successfully = data_monitoring.send_report(
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 220, in send_report
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     self.send_test_results_summary(
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 295, in send_test_results_summary
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     summary_test_results = tests_api.get_test_results_summary(
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 111, in get_test_results_summary
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     return [
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]            ^
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/elementary/monitor/api/tests/tests.py", line 112, in <listcomp>
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     TestResultSummarySchema(
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]   File "/usr/local/lib/python3.11/site-packages/pydantic/v1/main.py", line 347, in __init__
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base]     raise validation_error
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base] pydantic.v1.error_wrappers.ValidationError: 1 validation error for TestResultSummarySchema
[2025-07-24, 09:45:33 CEST] {pod_manager.py:472} INFO - [base] subscribers -> 0
[2025-07-24, 09:45:35 CEST] {pod_manager.py:490} INFO - [base]   none is not an allowed value (type=type_error.none.not_allowed)
k
A message from kapa.ai
Thank you for providing detailed logs and context. The error you're encountering:
Copy code
pydantic.v1.error_wrappers.ValidationError: 1 validation error for TestResultSummarySchema subscribers -> 0 none is not an allowed value (type=type_error.none.not_allowed)
indicates that the 
subscribers
field in the data being passed to the 
TestResultSummarySchema
is 
None
, but the schema expects a non-null value (likely a list or similar). This type of error is a recurring theme in Elementary when a field in the underlying data warehouse table is 
NULL
(or 
None
in Python), but the corresponding Pydantic schema does not allow nulls. This has been seen with other fields in different schemas, such as 
test_results_query
, 
test_results_description
, and 
compiled_sql
in previous issues and Slack threads. The general workaround has been to update the problematic records in the relevant table to set the field to an empty value (e.g., an empty list or string) instead of 
NULL
or to delete those records if they are not needed anymore. What you can do: 1. Identify the Table and Field: The error is with the 
subscribers
field. You need to find the table (likely 
elementary_test_results
or a similar summary/results table) where this field is present. 2. Update NULL Values: Update all records where 
subscribers
is 
NULL
to an empty list (
[]
) or an empty string (
''
), depending on the expected type. For example, if it's a list:
Copy code
sql    UPDATE <your_database>.<elementary_schema>.<relevant_table>    SET subscribers = '[]'    WHERE subscribers IS NULL;
If it's a string, use 
''
instead of 
'[]'
. 3. Rerun the Command: After updating the records, rerun your 
edr send-report
command. This approach is consistent with previous solutions for similar issues in Elementary, where schema validation errors were caused by 
NULL
values in the database. For example, see the advice for 
compiled_sql
and 
test_results_query
fields in other threads and issues, where updating or deleting problematic records resolved the error [Slack thread example](https://slack.com/archives/C02CTC89LAX/p1664550366911369). If the problem persists: - Double-check the type expected for 
subscribers
in your version of Elementary. - If you are running an older version, consider upgrading, as many of these schema validation issues have been addressed in newer releases. - If you are unable to resolve the issue, please provide the schema of the relevant table and a sample of the problematic record (with sensitive data redacted). Let me know if you need help with the exact SQL command or further troubleshooting!
a
@kapa.ai Hmm i looked at every elementary table and there was no column named subscribers. If it's supposed to be a column value, i don't really know where to search since there are a lot of columns.
k
A message from kapa.ai
Thank you for checking the tables. Based on the knowledge sources, the 
subscribers
field is not a physical column in an Elementary-managed table by default. Instead, it is a property that is configured in the 
meta
block of your dbt model or test YAML files, and then Elementary reads this metadata during report generation and alerting. The value is extracted from the dbt artifacts, not from a dedicated database column. If you are seeing a Pydantic validation error about 
subscribers
, it means that somewhere in the process, Elementary is expecting the 
subscribers
field to be present (and not null) in the metadata for a model or test, but it is missing or set to 
None
. This can happen if: - The 
subscribers
property is not set in your dbt YAML files for some models/tests. - The property is set to 
null
or is omitted, and the code expects a non-null value (like an empty list or string). What to check: - Review your dbt model and test YAML files and ensure that, if you use the 
subscribers
property, it is always set to a valid value (e.g., an empty list 
[]
if there are no subscribers, or a list of user tags). - Example configuration:
Copy code
yaml   models:     - name: my_model_name       meta:         subscribers: []
or
Copy code
yaml   models:     - name: my_model_name       meta:         subscribers: ["@jessica.jones"]
[See documentation for reference](https://docs.elementary-data.com/features/alerts-and-incidents/owners-and-subscribers). - If you have not set 
subscribers
anywhere, but Elementary is still expecting it, it may be due to a bug or a change in how your dbt artifacts are being parsed after the recent deployment. Summary: You will not find a 
subscribers
column in the database tables. Instead, check your dbt YAML files for the 
subscribers
property in the 
meta
block. Make sure it is never 
null
or missing if your configuration or alert fields expect it. If you are still having trouble, please share a sample of your dbt YAML configuration for further troubleshooting. If you need more specific guidance, let me know which version of Elementary you are using and whether you have recently changed your dbt model/test configurations or upgraded dbt/Elementary.
a
@kapa.ai Ah i just saw in my sources.yml file that i have multiple subscribers field that are empty. Example :
Copy code
sources:
  - database: data-dwh
    name: oasis
    schema: oasis
    tables:
      - identifier: banner
        name: banner
        loaded_at_field: meta_processing_time
        freshness:
          warn_after: { count: 12, period: hour }
          filter: DATE(meta_processing_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)
        meta:
          owner: "@captain-data"
          subscribers:
          description: Test to check the freshness of the homepage banner ingestion
          channel: data-alerts
          alert_suppression_interval: 24
Should i just remove the subscribers field ?
k
A message from kapa.ai
New message text here
a
@kapa.ai Thank you, this was indeed the issue and it is now fixed 😉
k
A message from kapa.ai
You're welcome! I'm glad to hear that resolving the empty 
subscribers
field in your YAML fixed the issue. This approach is consistent with Elementary's requirements for alert configuration, where fields like 
subscribers
should either be omitted or set to a valid value (such as an empty list or a list of user tags) to avoid schema validation errors during report generation. If you have any more questions or run into other issues, feel free to ask!
32 Views
Open in Slack
PreviousNext
Powered by