Has anyone used Preset to do a superset recipe <https datahu DataHub #troubleshoot

Has anyone used Preset to do a superset recipe? <...

enough-zoo-71516

11/10/2021, 5:41 PM

Has anyone used Preset to do a superset recipe? https://datahubproject.io/docs/metadata-ingestion/source_docs/superset/ I just created a Preset (https://preset.io/ - managed superset) account today and was wondering how to figure out the

connect_uri

. Just wondering if anyone else has already got Datahub + Preset running?

enough-zoo-71516

11/10/2021, 5:45 PM

These are the errors i get: Looks like maybe it cant make a connection to my preset url (which looks like https://b2a******.us2a.app.preset.io/).

Copy code

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task
    self._execute_task_with_callbacks(context)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
    result = self._execute_task(context, self.task)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
    result = execute_callable(context=context)
  File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 151, in execute
    return_value = self.execute_callable()
  File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 162, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/usr/local/airflow/dags/datahub.preset.pipeline/datahub.preset.pipeline.py", line 43, in ingest_from_preset
    pipeline = Pipeline.create(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 136, in create
    return cls(config)
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 104, in __init__
    self.source: Source = source_class.create(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/superset.py", line 133, in create
    return cls(ctx, config)
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/superset.py", line 113, in __init__
    self.access_token = login_response.json()["access_token"]
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 901, in json
    return complexjson.loads(
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2214, in email_alert
    send_email(self.task.email, subject, html_content)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 55, in send_email
    return backend(
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 102, in send_email_smtp
    send_mime_email(e_from=smtp_mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 210, in send_mime_email
    conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 244, in _get_smtp_connection
    else smtplib.SMTP(host=host, port=port, timeout=timeout)
  File "/usr/local/lib/python3.9/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.9/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.9/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/local/lib/python3.9/socket.py", line 844, in create_connection
    raise err
  File "/usr/local/lib/python3.9/socket.py", line 832, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1752, in handle_failure
    self.email_alert(error)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2216, in email_alert
    send_email(self.task.email, subject, html_content_err)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 55, in send_email
    return backend(
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 102, in send_email_smtp
    send_mime_email(e_from=smtp_mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 210, in send_mime_email
    conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/email.py", line 244, in _get_smtp_connection
    else smtplib.SMTP(host=host, port=port, timeout=timeout)
  File "/usr/local/lib/python3.9/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.9/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.9/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/local/lib/python3.9/socket.py", line 844, in create_connection
    raise err
  File "/usr/local/lib/python3.9/socket.py", line 832, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address
[2021-11-10, 17:29:11 UTC] {standard_task_runner.py:88} ERROR - Failed to execute job 7802 for task ingest_from_preset
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
    args.func(args, dag=self.dag)
  File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
    _run_task_by_selected_method(args, dag, ti)
  File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
    _run_raw_task(args, ti)
  File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
    ti._run_raw_task(
  File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task
    self._execute_task_with_callbacks(context)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
    result = self._execute_task(context, self.task)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
    result = execute_callable(context=context)
  File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 151, in execute
    return_value = self.execute_callable()
  File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 162, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/usr/local/airflow/dags/datahub.preset.pipeline/datahub.preset.pipeline.py", line 43, in ingest_from_preset
    pipeline = Pipeline.create(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 136, in create
    return cls(config)
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 104, in __init__
    self.source: Source = source_class.create(
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/superset.py", line 133, in create
    return cls(ctx, config)
  File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/superset.py", line 113, in __init__
    self.access_token = login_response.json()["access_token"]
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 901, in json
    return complexjson.loads(
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

green-football-43791

11/15/2021, 7:53 PM

Hey @enough-zoo-71516! We'll connect with folks from the preset team and see if they can help us make the connector preset-compatible.

2 Views

Open in Slack

Previous Next