Hey all, anyone encountered DataBricks Hive ingestion error

ValueError: ('# Detailed Table Information', None, None) is not in list

? I encounter this for all tables, but all database names are ingested fine.

@careful-pilot-86309 ^

Hey @billowy-book-26360 can you try using the recipe below:

source:
    type: hive
    config:
        host_port: '<http://dbc-xxxx.cloud.databricks.com:443|dbc-xxxx.cloud.databricks.com:443>'
        scheme: databricks+connector
        options:
            connect_args:
                http_path: <http-path>
                access_token: <access-token>

Make sure you have sqlalchemy-databricks and databricks-sql-connector installed. It worked for me. Let me know if you run into issues.

Thanks @hundreds-photographer-13496 that seems to be working! I don't see detailed table info like Type, Provider, Location etc. in the UI. How would I see if that was being ingested with this connector?

Thats great! Unfortunately, detailed table info is not ingested when using

scheme : databricks+connector

The fix for the ValueError you mentioned earlier will be available soon in acryl-pyhive python package . Once that's available, you can go back to using recipe

scheme: databricks+pyhive

as mentioned in hive docs where tables along with detailed table info will be ingested. I will update here once it's available. cc: @dazzling-judge-80093

Thanks @hundreds-photographer-13496, that's great. I'll look out for the update. Strangely I no longer get the ValueError using

databricks+pyhive

after a few reboots and some additional unrelated pip installs.

Interesting! Do you see detailed table info like Type, Provider, Location etc ?

No, I re-ran

databricks+connector

ingestion but Properties tab under Hive objects (tables) is blank. When I use

databricks+connector

I see is_view=True under Properties tab and view_definition is blank.