A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

hello, I used the command “datahub ingest -c xxx.yaml” to ingest the hive table, but the COMMENT in table did not show in the datahub. Is it a bug? Or datahub don’t support the hive comment imported？

Hi <@U022R8MK5UJ>! Are you able to help with this one?

Screenshot 2022-07-07 at 9.33.13 AM.png

Screenshot 2022-07-07 at 9.28.49 AM.png

Hey <@U03N06PBY5R>, are you using the <https://datahubproject.io/docs/generated/ingestion/sources/hive/|Hive plugin>? All table parameters (including comment) are imported into DataHub during ingestion, as shown below.

Can you help run a `describe format` in Hive and see if the comment field is there?

same issue here with my coworkers, in my case looking for column comments/descriptions

Screenshot 2022-07-08 at 12.14.06 PM.png

it works for me. can you share the describe formatted output?

image.png

Using
acryl-datahub[hive]==0.8.40
or
acryl-PyHive==0.6.13

with same results

There is any scpecific parameter to enable comments?

using datahub 0.8.40 at this moment during the tests, I’m asking to my coworker the crawler config

hi <@U022R8MK5UJ> I was building my own lab, and it doesnt work, we are using data bricks, not sure if this is the issue:

```source:
  type: hive
  config:
    host_port: <http://adb-4911762757593392.12.azuredatabricks.net:443|adb-4911762757593392.12.azuredatabricks.net:443>
    username: token
    password: xxxxx
    scheme: 'databricks+pyhive'
    database: silver
    platform: delta-lake
    profiling:
      enabled: true
    table_pattern:
      allow:
        - silver.calendar_calendars

    options:
      connect_args:
        http_path: 'sql/protocolv1/o/4911762757593392/0426-111319-t6tf09gf'

sink:
  type: datahub-rest
  config:
    server: <http://xxxxx:8080>```

the global table comment appears, but not per field comment/description

If I run delta-lake crawler got this, the description is in json format: