Hello I am currently trying to connect a Microsoft...
# troubleshoot
g
Hello I am currently trying to connect a Microsoft SQL Server to Datahub but I am getting “No module named ‘Pyodbc’”. I have ODBC Drive 17 installed. I have reinstalled Pyodbc but it does nothing. I am using Windows. Thanks!
👀 1
b
just checking, did you try to pip install in administrator mode?
a
@gray-ghost-82678, hi could you post the full stack trace here?
g
Copy code
~~~~ Execution Summary - RUN_INGEST ~~~~
Execution finished with errors.
{'exec_id': '3609e1c1-0205-4400-bf96-bccb49f67bf2',
 'infos': ['2023-02-10 01:19:22.404509 INFO: Starting execution for task with name=RUN_INGEST',
           "2023-02-10 01:19:26.463622 INFO: Failed to execute 'datahub ingest'",
           '2023-02-10 01:19:26.465369 INFO: Caught exception EXECUTING task_id=3609e1c1-0205-4400-bf96-bccb49f67bf2, name=RUN_INGEST, '
           'stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
           '    task_event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
           '    return future.result()\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 231, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
 'errors': []}

~~~~ Ingestion Logs ~~~~
Obtaining venv creation lock...
Acquired venv creation lock
venv setup time = 0
This version of datahub supports report-to functionality
datahub  ingest run -c /tmp/datahub/ingest/3609e1c1-0205-4400-bf96-bccb49f67bf2/recipe.yml --report-to /tmp/datahub/ingest/3609e1c1-0205-4400-bf96-bccb49f67bf2/ingestion_report.json
[2023-02-10 01:19:24,463] INFO     {datahub.cli.ingest_cli:165} - DataHub CLI version: 0.9.6
[2023-02-10 01:19:24,484] INFO     {datahub.ingestion.run.pipeline:179} - Sink configured successfully. DataHubRestEmitter: configured to talk to <http://datahub-gms:8080>
[2023-02-10 01:19:24,998] ERROR    {datahub.entrypoints:213} - Command failed: Failed to configure the source (mssql): No module named 'pyodbc'
Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 114, in _add_init_error_context
    yield
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 192, in __init__
    self.source = source_class.create(
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/source/sql/mssql.py", line 193, in create
    return cls(config, ctx)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/source/sql/mssql.py", line 123, in __init__
    for inspector in self.get_inspectors():
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/source/sql/mssql.py", line 229, in get_inspectors
    url = self.config.get_sql_alchemy_url()
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/source/sql/mssql.py", line 75, in get_sql_alchemy_url
    import pyodbc  # noqa: F401
ModuleNotFoundError: No module named 'pyodbc'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/entrypoints.py", line 171, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 350, in wrapper
    raise e
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 302, in wrapper
    res = func(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/utilities/memory_leak_detector.py", line 95, in wrapper
    return func(ctx, *args, **kwargs)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 179, in run
    pipeline = Pipeline.create(
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 303, in create
    return cls(
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 191, in __init__
    with _add_init_error_context(f"configure the source ({source_type})"):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/tmp/datahub/ingest/venv-mssql-0.9.6/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 116, in _add_init_error_context
    raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to configure the source (mssql): No module named 'pyodbc'
a
Cc: @gentle-hamburger-31302
g
@astonishing-answer-96712 @gentle-hamburger-31302 any updates?
g
Hi @gray-ghost-82678 Let me reproduce the issue on windows machine, tomorrow I will give you the update
g
Thanks! @gentle-hamburger-31302
g
Hi @gray-ghost-82678 Could you please paste the command you are executing for ingestion
?
g
Copy code
source:
  type: mssql
  config:
    # Coordinates
    host_port: localhost:1433
    database: DemoDatabase

    # Credentials
    username: admin
    password: password

    # Options
    use_odbc: "True"
    uri_args:
      driver: "ODBC Driver 17 for SQL Server"
      Encrypt: "yes"
      TrustServerCertificate: "Yes"
      ssl: "True"
Im pretty much using this format for ingestion. I got this from your documentation.
@gentle-hamburger-31302
g
Command
datahub ingest run ….
or python3 -m datahub ingest run ….
^ @gray-ghost-82678
g
ok I just tried python3 -m datahub ingest run and im getting a different error.
@gentle-hamburger-31302 I was able to get it to work with python -m datahub ingest run thanks!