Hi all! I enable profiling but got an error called...
# troubleshoot
m
Hi all! I enable profiling but got an error called ‘partial_unexpected_list’. (from snowflake) [2022-06-07 163451,395] ERROR {datahub.ingestion.source.ge_data_profiler:852} - Encountered exception while profiling <http://tm_internal_db.chuxuan.cg|<>table> Traceback (most recent call last): File “/opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/datahub/ingestion/source/ge_data_profiler.py”, line 840, in _generate_single_profile ).generate_dataset_profile() File “/opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/datahub/ingestion/source/ge_data_profiler.py”, line 601, in generate_dataset_profile self.query_combiner.flush() File “/opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/datahub/utilities/sqlalchemy_query_combiner.py”, line 396, in flush let.switch() File “/opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/datahub/ingestion/source/ge_data_profiler.py”, line 235, in <lambda> return self.query_combiner.run(lambda: method(self, *args, **kwargs)) File “/opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/datahub/ingestion/source/ge_data_profiler.py”, line 440, in _get_dataset_column_sample_values str(v) for v in res[“partial_unexpected_list”] KeyError: ‘partial_unexpected_list’
m
try disabling query combiner and see if that fixes it.
Copy code
config:
  profiling:
    query_combiner_enabled: False
m
profiling: enabled: True query_combiner_enabled: False include_field_sample_values: True ‘failures’: {<table>: [“Profiling exception ‘partial_unexpected_list’“]}, My table is simple(4 rows). Two columns (v1, v2) the value is just number.
@mammoth-bear-12532 If I enable turn_off_expensive_profiling_metrics, it can ingest successfully. But there is no sample values.
b
@dazzling-judge-80093 can you please take a look - what's going wrong?