https://datahubproject.io logo
#ingestion
Title
# ingestion
a

adamant-sugar-28445

06/15/2022, 9:07 AM
When I ingested the Hive metadata of a table, the schema shown in datahub looked OK but there're only scant information ("is_view" and "view_definition") in the Properties tab. I expect to see Hive details similar to those in https://demo.datahubproject.io/dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,long_tail[…]ctive_customer_ltv,PROD)/Properties?is_lineage_mode=false. How can I achieve that? The config file I used: `source`:
type: sqlalchemy
config:
platform: hive
connect_uri: "XXXX"
include_views: False
table_pattern:
allow:
- "<MdatabaseName>.<table>"
schema_pattern:
allow:
- "<databaseName>"
deny:
- "<otherSchemas>"
options:
connect_args:
auth: '<AuthStrategy>'
sink:
type: "datahub-rest"
config:
server: "<serverURI>"
h

hundreds-photographer-13496

06/15/2022, 10:53 AM
Which sqlalchemy dialect pypi package have you installed for hive ?
a

adamant-sugar-28445

06/16/2022, 2:45 AM
Oh, I upgraded the SQLAlchemy version to 1.3.24 and it worked. I didn't think the package played the role in providing Properties. Thanks a lot, @hundreds-photographer-13496
h

hundreds-photographer-13496

06/16/2022, 4:39 AM
Glad it worked. I was actually checking for pyhive dialect pypi package. Curious - which sqlalchemy version were you using earlier ?
4 Views