Hello, I'm trying to ingest stats from AWS Athena ...
# ingestion
m
Hello, I'm trying to ingest stats from AWS Athena to Datahub. I enabled profiling and get the following warning:
Copy code
[2021-11-30 11:32:03,180] INFO     {datahub.ingestion.source.sql.sql_common:512} - Profiling example_db.table_example (this may take a while)
[2021-11-30 11:32:06,581] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:32:09,996] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:32:13,366] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:32:16,821] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:32:20,337] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:32:23,738] WARNING  {great_expectations.dataset.sqlalchemy_dataset:1715} - No recognized sqlalchemy types in type_list for current dialect.
[2021-11-30 11:33:26,261] INFO     {datahub.ingestion.run.pipeline:61} - sink wrote workunit profile-example_db.table_example
The stats & queries tabs are still inactive. Here's my ingestion yml file
Copy code
source:
  type: athena
  config:
    aws_region: us-east-2
    work_group: primary

    username: ...
    password: ...
    database: example_db

    s3_staging_dir: "..."
    
    profiling:
      enabled: true

sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"
Am I doing smth wrong?
b
AWS Athena stats... Okay @helpful-optician-78938 @mammoth-bear-12532 do we know of anyone else using stats with Athena right now? Seems to be some weirdness in the compatibility between GE and Athena types (sigh)
a
Following… :)
b
ping @helpful-optician-78938 - any word on this part?