Hi, Getting these errors while trying to ingest s...
# troubleshoot
b
Hi, Getting these errors while trying to ingest snowflake on datahub. Have read access to tables and queries. Please advise on a process. DataHub CLI version: 0.10.1.1 Python version: 3.9.6 (default, Mar 10 2023, 201638) [Clang 14.0.3 (clang-1403.0.22.14.1)] Deployment method - Docker on local machine
Copy code
~~~~ Execution Summary - RUN_INGEST ~~~~
Execution finished with errors.
{'exec_id': '426ca545-5c66-4adb-98b0-7f2fffbfcd0f',
 'infos': ['2023-04-14 02:35:39.958717 INFO: Starting execution for task with name=RUN_INGEST',
           "2023-04-14 02:55:31.996839 INFO: Failed to execute 'datahub ingest'",
           '2023-04-14 02:55:32.010133 INFO: Caught exception EXECUTING task_id=426ca545-5c66-4adb-98b0-7f2fffbfcd0f, name=RUN_INGEST, '
           'stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
           '    task_event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
           '    return future.result()\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 231, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
 'errors': []}
🔍 1
📖 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!
a
this could mean your recipe is misconfigured- have you been able to successfully ingest in the past?
@hundreds-photographer-13496 could you take a look here?
h
Hey @brave-room-48783 since when did this stop working ? Is there anything more to logs (error stack trace, etc) that can help us identify the issue ?
b
Hey, yes at the end it showed some segmentation fault
This is the docker resource allocation I am using - CPUs - 4 Memory - 7 GB Swap - 2 GB Virtual Disk Limit - 72 GB
a
Can you please share the additional error stack trace ?
b
Will share it with you within a day
here you go @hundreds-photographer-13496
Tried reinstalling everything too
h
can you share the recipe ? Have you enabled classification ?
b
This is the yml I used
And idk about classification, whatever the default settings are I went with them
h
By recipe, I mean the snowflake recipe in yaml form
b
Here's that
a
1. This error appears to have encountered during profiling. Can you confirm if you get this error even when profiling is disabled. 2. What memory is allocated to datahub-actions container. Please make sure to allocate enough memory, the ingestion process is currently consuming 610MB already.
b
1. Will update you on this 2. datahub-actions image is 5.18 GBs , how do I see image specific memory allocations?
This is the docker resource allocation I am using - CPUs - 4 Memory - 7 GB Swap - 2 GB Virtual Disk Limit - 72 GB