Hi Team, I am trying datahub integration with great expectations. I have added following action mentioned below. However, after I run the checkpoint I see my assertion results in local but the same is not exported to datahub. Any clue on what might be missing ?

- name: datahub_action
    action:
      module_name: datahub.integration.great_expectations.action
      class_name: DatahubValidationAction
      server_url: http://{datahub-url}:8080

I installed required module

pip install "acryl-datahub[great-expectations]"

too. See results of

great_expectations checkpoint run sample_checkpoint

below (PFA).

@famous-waitress-64616 might be able to help with this!

Hi Ishwar, how are you deploying datahub? Are you sure your

datahub-url

is correct and that you don't need a token to make requests to gms? If that all looks good, then perhaps @big-carpet-38439 may have more ideas

Hi @famous-waitress-64616, thanks for the response. The datahub instance is deployed on a GCP cluster. I have verified its gms server url in GX code and it is correct. Also, it doesn't require token for communication. I have created various datasets, domains, tags etc. from local via open api and it didn't ask for any token at that time.

Hi @nutritious-lighter-88459 what kind of datasource are you validating ? Is it sqlalchemy based or any other ? Currently, DataHubValidationAction only supports emitting assertions for sqlalchemy based datasets. If you are already using sqlalchemy based datasource, I'd suggest enabling debug logs as mentioned here to share the logs with us to help through troubleshooting.

This is a MySQL dataset (a table within MySQL)

Hi Mayuri, as Siddharth mentioned it is MySql datasource. So yes, it is Sqlalchemy based. I was able to export one (out of 5) assertion from my suite. I am not sure why others were not visible in datahub. And even that was gone after I tried to delete and add ingestion again. I will try to share logs in a while.

MySQL is supported. We'll know better about the behavior after logs are shared.

Hi Mayuri, Attaching the logs as requested. Also, sharing checkpoint and expectations suite files for reference.

Looks like a bug in DataHubValidationAction. should be fixed by https://github.com/datahub-project/datahub/pull/8243

Thank you for your help Mayuri. We will monitor the PR for when it gets merged and following which we will upgrade and retest our flow.