A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hello, I am new to datahub and currently able to ingest metadata from hive. Can someone guide me  or provide me an example/documentation on how to ingest the metadata description for the tables/columns.

It should've pulled in the descriptions by default. Can you check your ingestion logs and see if there are any errors?

Also, can you confirm that your tables in hive already have the documentation at the table/column level?

<@U01C3DMG2GL> I am using some sample test tables and I just have the comments added in the create table for every column and table.

Hi <@U02GJ5PMLRE>, when you run the ingestion command for a hive recipe,  on the console logs do you see any errors or exceptions?

No I don't see any errors, it says pipeline completed successfully

In the recipe, when you define the sink, is that a file sink? If so, could you please take a peek at the metadata change event corresponding to the table you are referring to?

Also, in the sink report at the end of the ingestions process, are there any warnings?

This is the recipe i am using .
```source:
  type: hive
  config:
    # Coordinates
    username: hive
    password: hive
    host_port: localhost:10000
    database: datahub_test # optional, if not specified, ingests from all databases
    options:
      connect_args:
        auth: CUSTOM
sink:
  type: "datahub-rest"
  config:
    server: "<http://localhost:8080>"```

No, i dont see any warnings When doing the ingestion process.