Hello! I get this error when trying to get the profiling for my snowflake's tables:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 924, in _generate_single_profile
    profile = _SingleDatasetProfiler(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 511, in generate_dataset_profile
    self.query_combiner.flush()
  File "/usr/local/lib/python3.9/site-packages/datahub/utilities/sqlalchemy_query_combiner.py", line 395, in flush
    let.switch()
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 236, in <lambda>
    return self.query_combiner.run(lambda: method(self, *args, **kwargs))
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/ge_data_profiler.py", line 490, in _get_dataset_column_sample_values
    str(v) for v in res["partial_unexpected_list"]
KeyError: 'partial_unexpected_list'

Can you help me? Thanks!

Hi @worried-zebra-47870 which source are you using

snowflake

or

snowflake-beta

? Can you please try disabling query combiner as mentioned here - https://datahubspace.slack.com/archives/C029A3M079U/p1654634984140369?thread_ts=1654634462.516299&cid=C029A3M079U

I'm using snowflake, this field doesn't seem to be available

Can you share your profiling config ? The field should be set at same level as profiling.enabled

profiling:
    enabled: true
    query_combiner_enabled: False

profiling:
          enabled: true
          query_combiner_enabled: false

I still get the error

oh! Is this causing entire ingestion to fail , or only some profiles ? Would it be possible for you to share the logs to understand this better ?

No only some profiles are failing. I'll try to reproduce the problem, I just removed the profiling config to get started!

okay. That would work. Or You can also disable sample values ingestion and continue with other profiling by setting

profiling
    enabled: True
    include_field_sample_values: False

Thanks for all the replies 🙂 I'm having trouble going through (and understand) all the parameters, but right now profiling is not important.

got it 🙂