Hello, everyone! I'm new to Datahub Project. I mad...
# troubleshoot
g
Hello, everyone! I'm new to Datahub Project. I made a setup for a POC a few days ago and today I'm trying to ingest metadata from Metabase. Unfortunately, I'm getting some errors. Everything is ok about permissions and access to Metabase API (I checked by me with a python script). But, before metadata ingestion ends, I get this error below:
Copy code
---- (full traceback above) ----
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/entrypoints.py", line 138, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/home/pbraz/.local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
File "/home/pbraz/.local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/telemetry/telemetry.py", line 202, in wrapper
    raise e
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/telemetry/telemetry.py", line 194, in wrapper
    res = func(*args, **kwargs)
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/utilities/memory_leak_detector.py", line 102, in wrapper
    res = func(*args, **kwargs)
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/cli/ingest_cli.py", line 92, in run
    pipeline.run()
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/ingestion/run/pipeline.py", line 181, in run
    for wu in itertools.islice(
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/ingestion/source/metabase.py", line 541, in get_workunits
    yield from self.emit_card_mces()
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/ingestion/source/metabase.py", line 240, in emit_card_mces
    chart_snapshot = self.construct_card_from_api_data(card_info)
File "/home/pbraz/.local/lib/python3.8/site-packages/datahub/ingestion/source/metabase.py", line 258, in construct_card_from_api_data
    card_response = self.session.get(card_url)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)

ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')"))
[2022-03-24 17:02:52,053] INFO     {datahub.entrypoints:161} - DataHub CLI version: 0.8.31.1 at /home/pbraz/.local/lib/python3.8/site-packages/datahub/__init__.py
[2022-03-24 17:02:52,053] INFO     {datahub.entrypoints:164} - Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) 
[GCC 9.3.0] at /usr/bin/python3 on Linux-5.13.0-1019-gcp-x86_64-with-glibc2.29
[2022-03-24 17:02:52,053] INFO     {datahub.entrypoints:167} - GMS config {}
I made some searches trying to discover the reason for these errors. My first guess was the API request rate limit, but I found in the documentation that just login requests have rate limits (see here). My second try was to search for the error on the internet and I found not so similar situation but with the same error (see here). It could be possible that Metabase has a security control for User-Agent headers? The user created for this POC is receiving this email every time I try to ingest the metadata from Metabase: Does someone have some idea what I could probably be doing wrong? Thanks for your attention!
l
Hi @gentle-camera-33498! Gentle reminder to please post large blocks of code/stacktrace in threads - it’s a MASSIVE help for the Core Team to keep track of all open threads across our support channels! teamwork I’ll escalate this one with the team today so we can get you some direction 🙂
g
Ops! Ok, Maggie! I think I found the problem. I will make local tests to prove the idea. If works, I can create a Pull Request or pass the information to someone to fix.
l
That would be amazing!!