https://datahubproject.io logo
Join Slack
Powered by
We are getting below two messages in UI while sele...
# troubleshoot
c
We are getting below two messages in UI while selecting any snowflake table/view. Validation error of type FieldUndefined: Field 'latestVersion' in type 'GetSchemaBlameResult' is undefined @ 'getSchemaBlame/latestVersion' (code undefined) The variables input contains a field name 'categories' that is not defined for input object type 'GetSchemaBlameInput' (code undefined) What needs to be done to resolve it?
b
hey Hiral! this usually occurs when GMS and your frontend are out of sync as their respective graphql schemas don't align. I would try upgrading your instance and see that resolves your issue
c
Hi Chris,
Thank you for looking into this and providing a possible solution. We will look into this.
One more thing, lineage option is grayed out for all snowflake tables/views in datahub, I am new to datahub, can you let me know what needs to be done to get lineage information?
@bulky-soccer-26729 Can you let me know the solution about lineage question I have above?
b
hey Hiral sorry i'm just seeing this
hmm do you have your snowflake recipe on hand? can you send that this way with secret values obfuscated?
c
@bulky-soccer-26729 Sorry for delayed response. Here is recipe. source: type: snowflake config: include_table_lineage: true role: xxxxxxxxx warehouse: xxxxxxxx check_role_grants: true ignore_start_time_lineage: true password: xxxxxxxx account_id: xxxxxxxxx profiling: enabled: true include_view_lineage: true stateful_ingestion: enabled: true schema_pattern: allow: - ACCESS_HISTORY database_pattern: allow: - SNOWFLAKE username: xxxxxxxxx pipeline_name: 'urnlidataHubIngestionSource:b94b5e80-3501-43bf-8a34-4b0649a68eb6'
b
okay cool everything here in your recipe looks reasonable to me.. I guess a few things - first are you using the Enterprise edition of snowflake? can you also try adding 
upstream_lineage_in_report: true
to your recipe under 
config
?
c
@bulky-soccer-26729 I added upstrea_lineage_in_report: true option in recipe and ran ingestoin. I do not have "Test" option in UI, but checked randomly few tables, some of them has lineage enabled and some of them don't.
So I am guessing tables which do not have upstream or downstream relationship, those tables will not have lineage tan enabled right?
b
that's correct! if there's no lineage for a table then the lineage tab will be grayed out
so the fact that you're getting lineage for some things makes me think that things are actually working properly
c
Thank you for confirmation
@bulky-soccer-26729
When we try to ingest snowflake database, ingestion is failing.
Copy code
'ProgrammingError: (snowflake.connector.errors.ProgrammingError) 090105 (22000): Cannot perform SELECT. This session does not have a '
           "current database. Call 'USE DATABASE', or use a qualified name.\n"
           '[SQL: \n'
           'select table_catalog, table_schema, table_name\n'
           'from information_schema.tables\n'
           "where last_altered >= to_timestamp_ltz(1663086530849, 3) and table_type= 'BASE TABLE'\n"
           '            ]\n'
           '(Background on this error at: <http://sqlalche.me/e/13/f405>)\n'
           '[2022-09-14 16:28:52,024] INFO     {datahub.entrypoints:187} - DataHub CLI version: 0.8.41 at '
           '/tmp/datahub/ingest/venv-snowflake-0.8.41/lib/python3.9/site-packages/datahub/__init__.py\n'
           '[2022-09-14 16:28:52,024] INFO     {datahub.entrypoints:190} - Python version: 3.9.9 (main, Dec 21 2021, 10:03:34) \n'
           '[GCC 10.2.1 20210110] at /tmp/datahub/ingest/venv-snowflake-0.8.41/bin/python3 on '
           'Linux-5.4.196-108.356.amzn2.x86_64-x86_64-with-glibc2.31\n'
           "[2022-09-14 16:28:52,024] INFO     {datahub.entrypoints:193} - GMS config {'models': {}, 'versions': {'linkedin/datahub': {'version': "
           "'v0.8.42', 'commit': '4f35a6c43dcd058e4e85b1ed7e4818100ab224e0'}}, 'managedIngestion': {'defaultCliVersion': '0.8.41', 'enabled': True}, "
           "'statefulIngestionCapable': True, 'supportsImpactAnalysis': True, 'telemetry': {'enabledCli': True, 'enabledIngestion': False}, "
           "'datasetUrnNameCasing': False, 'retention': 'true', 'datahub': {'serverType': 'prod'}, 'noCode': 'true'}\n",
           "2022-09-14 16:28:53.137401 [exec_id=2dc5382a-f673-489f-b9bf-4cf1328b7bf7] INFO: Failed to execute 'datahub ingest'",
           '2022-09-14 16:28:53.137719 [exec_id=2dc5382a-f673-489f-b9bf-4cf1328b7bf7] INFO: Caught exception EXECUTING '
           'task_id=2dc5382a-f673-489f-b9bf-4cf1328b7bf7, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
           '    self.event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 89, in run_until_complete\n'
           '    return f.result()\n'
           '  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
           '    raise self._exception\n'
           '  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
           '    result = coro.send(None)\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 112, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}
Execution finished with errors.
Here is recipe.
source: type: snowflake config: username: DATAHUB_USER password: xxxxx role: xxxx warehouse: xxxxx check_role_grants: true account_id: xxxxx include_table_lineage: true include_view_lineage: true ignore_start_time_lineage: true upstream_lineage_in_report: true profiling: enabled: true stateful_ingestion: enabled: true database_pattern: allow: - SNOWFLAKE schema_pattern: allow: - ACCOUNT_USAGE pipeline_name: 'urnlidataHubIngestionSource:b94b5e80-3501-43bf-8a34-4b0649a68eb6'
@bulky-soccer-26729
Any update on this?
b
hey Hiral can you post this in #ingestion? this isn't my area of expertise and at a glance I'm not sure what the problem is..
c
Sure. Thx
@bulky-soccer-26729 For following error: Validation error of type FieldUndefined: Field 'latestVersion' in type 'GetSchemaBlameResult' is undefined @ 'getSchemaBlame/latestVersion' (code undefined) The variables input contains a field name 'categories' that is not defined for input object type 'GetSchemaBlameInput' (code undefined) You suggested below: hey Hiral! this usually occurs when GMS and your frontend are out of sync as their respective graphql schemas don't align. I would try upgrading your instance and see that resolves your issue We have following versions. Let me know anything needs to be updated. DataHub CLI version: 0.8.41 Python version: 3.9.9 GMS : v0.8.42
b
so yeah this issue is because the graphql schemas from datahub-frontend and gms are out of sync. I would suggest updating both of those just to get them synced
are you still seeing the issue?
c
yes
What should be the versions?
b
i don't think it matters as long as your frontend and gms are in sync - i would suggest upgrading to latest which i believe is v0.8.44
2 Views
Open in Slack
PreviousNext
Powered by