Hi DataHub team, I’m seeing this weird error when...
# troubleshoot
p
Hi DataHub team, I’m seeing this weird error when trying to ingest Snowflake datasets:
Copy code
{snowflake.connector.CArrowIterator:372} - [Snowflake Exception] unknown arrow internal data type(0) for TIMESTAMP_LTZ data
Weirdly it was working before this and then started failing on a second run a few mins later. Can anyone help me understand this issue? More detailed log:
Copy code
> datahub ingest -c ../../zwa-snowflake.recipe.yaml
.
.
.
[2023-02-17 01:35:46,224] INFO     {datahub.cli.ingest_cli:120} - Starting metadata ingestion
\[2023-02-17 01:35:47,083] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:1389} - Checking current version
[2023-02-17 01:35:47,247] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:1395} - Checking current role
-[2023-02-17 01:35:47,328] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:1401} - Checking current warehouse
[2023-02-17 01:35:47,424] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:1408} - Checking current edition
|[2023-02-17 01:35:49,021] ERROR    {snowflake.connector.CArrowIterator:372} - [Snowflake Exception] unknown arrow internal data type(0) for TIMESTAMP_LTZ data
\[2023-02-17 01:35:49,032] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:627} - The role REVEFI_ROLE has `MANAGE GRANTS` privilege. This is not advisable and also not required.
/[2023-02-17 01:35:49,565] ERROR    {snowflake.connector.CArrowIterator:372} - [Snowflake Exception] unknown arrow internal data type(0) for TIMESTAMP_LTZ data
[2023-02-17 01:35:49,566] INFO     {datahub.ingestion.source.snowflake.snowflake_v2:627} - The role REVEFI_ROLE has `MANAGE GRANTS` privilege. This is not advisable and also not required.
[2023-02-17 01:35:49,566] ERROR    {datahub.ingestion.source.snowflake.snowflake_v2:224} - permission-error => No databases found. Please check permissions.
[2023-02-17 01:35:49,763] INFO     {datahub.cli.ingest_cli:133} - Finished metadata ingestion
✅ 1
Closing this out: uninstalling pyarrow fixed this for me.
Copy code
pip uninstall pyarrow
d
ahh, cool, thanks for the heads up