Do we support Hive 1.2 ingestion? Looks like current hive ingestion used pyhive which is only compatible with Hive 2

I don't think anyone has built a Hive 1.x connector yet 😕

Maybe we can build it. All we need is to connect to hive metastore and get table schemas

Would be great to get that contribution @mysterious-lamp-91034! Let us know how we can help.

@loud-island-88694 Thanks! I definitely will have some questions in the future. Then I will ask you

awesome - also, please let us know if this is blocking your POC of DataHub. We can look into prioritizing this

Thanks @loud-island-88694 This is my first question, I followed https://datahubproject.io/docs/metadata-ingestion/adding-source and started from step 4) I updated pip dependencies of plugins and add my hive1x in entry_points https://gist.github.com/wizardxz/8ec1bf9aa94e416a24c856192f45351f#file-gistfile1-txt-L284 But when I run

datahub check plugins

I don't see my hive1x source. Is the build required? Is there a way to debug new source without build? Thanks

Could you try rerunning

pip install -e '.'

in the metadata-ingestion directory’s venv?

trying

@mysterious-lamp-91034 the "datahub ingest" command will pull from hive server. We just implemented a spark-lineage emitter that pushes metadata out from a spark job. A hive emitter would need to be built that does something similar. We don't have it currently, but might be pretty easy to build by following the spark-lineage template.