Hi,
I am trying to ingest data from my Hive, but I...
# troubleshootm
microscopic-mechanic-13766
04/21/2022, 10:47 AMHi,
I am trying to ingest data from my Hive, but I can't get it right as it prints the following error:
Copy code
'[2022-04-21 10:41:04,600] INFO {datahub.cli.ingest_cli:86} - Starting metadata ingestion\n'
'[2022-04-21 10:41:05,303] ERROR {datahub.entrypoints:119} - File '
'"/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/datahub/entrypoints.py", line 105, in main\n'
' 102 def main(**kwargs):\n'
' 103 # This wrapper prevents click from suppressing errors.\n'
' 104 try:\n'
'--> 105 sys.exit(datahub(standalone_mode=False, **kwargs))\n'
' 106 except click.exceptions.Abort:\n'
' ..................................................\n'
' kwargs = {}\n'
' datahub = <Group datahub>\n'
" click.exceptions.Abort = <class 'click.exceptions.Abort'>\n"
' ..................................................\n'
'\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 1130, in __call__\n'
' 1128 def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:\n'
' (...)\n'
'--> 1130 return self.main(*args, **kwargs)\n'
' ..................................................\n'
' self = <Group datahub>\n'
' args = ()\n'
' t.Any = typing.Any\n'
" kwargs = {'standalone_mode': False,\n"
" 'prog_name': 'python3 -m datahub'}\n"
' ..................................................\n'
'\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 1055, in main\n'
' rv = self.invoke(ctx)\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 1657, in invoke\n'
' return _process_result(sub_ctx.command.invoke(sub_ctx))\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 1657, in invoke\n'
' return _process_result(sub_ctx.command.invoke(sub_ctx))\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 1404, in invoke\n'
' return ctx.invoke(self.callback, **ctx.params)\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/click/core.py", line 760, in invoke\n'
' return __callback(*args, **kwargs)\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/datahub/telemetry/telemetry.py", line '
'194, in wrapper\n'
' 181 def wrapper(*args: Any, **kwargs: Any) -> Any:\n'
' (...)\n'
' 190 return res\n'
' 191 # Catch general exceptions\n'
' 192 except Exception as e:\n'
' 193 telemetry_instance.ping(category, action, f"error:{get_full_class_name(e)}")\n'
'--> 194 raise e\n'
' 195 # System exits (used in ingestion and Docker commands) are not caught by the exception handler,\n'
' ..................................................\n'
' args = ()\n'
' Any = typing.Any\n'
" kwargs = {'config': '/tmp/datahub/ingest/36f9165c-d27e-44aa-b49a-b08a77157764.yml',\n"
" 'dry_run': False,\n"
" 'preview': False,\n"
" 'strict_warnings': False}\n"
" telemetry_instance.ping = <method 'Telemetry.ping' of <datahub.telemetry.telemetry.Telemetry object at 0x7faf304431c0> "
'telemetry.py:110>\n'
" category = 'datahub.cli.ingest_cli'\n"
" action = 'run'\n"
' ..................................................\n'
'\n'
'File "/tmp/datahub/ingest/venv-36f9165c-d27e-44aa-b49a-b08a77157764/lib/python3.9/site-packages/datahub/telemetry/telemetry.py", line '
'188, in wrapper\n'
' 181 def wrapper(*args: Any, **kwargs: Any) -> Any:\n'
' (...)\n'
' 184 action = func.__name__\n'
' 185 \n'
' 186 telemetry_instance.ping(category, action, "started")\n'
' 187 try:\n'
'--> 188 res = func(*args, **kwargs)\n'
' 189 telemetry_instance.ping(category, action, "completed")\n'
' ..................................................\n'
' args = ()\n'
' Any = typing.Any\n'
" kwargs = {'config': '/tmp/datahub/ingest/36f9165c-d27e-44aa-b49a-b08a77157764.yml',\n"
" 'dry_run': False,\n"
" 'preview': False,\n"
" 'strict_warnings': False}\n"
" action = 'run'\n"
" func.__name__ = 'run'\n"
" telemetry_instance.ping = <method 'Telemetry.ping' of <datahub.telemetry.telemetry.Telemetry object at 0x7faf304431c0> "s
square-activity-64562
04/21/2022, 10:50 AMGentle reminder to please post large blocks of code/stack trace in Slack message threads - it’s a HUGE help for the Core Team to keep track of which questions are still unaddressed across our various support channels!
square-activity-64562
04/21/2022, 10:51 AMPlease share the following
• What version of CLI are you using?
• What version of server are you using?
• full logs in text format (instead of screenshots) from the ingestion that fails. Please do not remove any parts of the log (mask the secret if any secret is being shown)
• the recipe in text format (instead of screenshots) masking the secrets
m
microscopic-mechanic-13766
04/21/2022, 11:16 AMI am currrently using the 0.8.32.1 CLI. I am using Hive 3.1.2 version. I am using the ES 7.16.1 version
microscopic-mechanic-13766
04/21/2022, 11:16 AMmicroscopic-mechanic-13766
04/21/2022, 11:18 AM Copy code
source:
type: hive
config:
host_port: 'hive-server:10000'
database: null
username: '${HIVE_USERNAME}'
password: '${HIVE_PASSWORD}'
sink:
type: datahub-rest
config:
server: '<http://datahub-gms:8080>'s
square-activity-64562
04/21/2022, 11:26 AM``` | ' raise ValueError("Password should be set if and only if in LDAP or CUSTOM mode; "\n'
| '\n'
| 'ValueError: Password should be set if and only if in LDAP or CUSTOM mode; Remove password or use one of those modes\n',```This seems to be the problem. Does this help? If not, I can ask someone else to look at this as I am not familiar with hive.
m
microscopic-mechanic-13766
04/22/2022, 7:22 AMAs you highlighted, the password was being the problem. By simply removing it the ingestion process succeded. Thanks for the help Aseem!
👍 1
2 Views