Hi good day. I just finished a test ingestion and ...
# troubleshoots
stale-petabyte-93467
03/16/2022, 8:16 AMHi good day. I just finished a test ingestion and I would like to know where I can see the errors or failures. Based from the report failures is null but I am getting an error message.
Copy code
Sink (datahub-rest) report:
{'downstream_end_time': datetime.datetime(2022, 3, 16, 16, 7, 18, 188112),
'downstream_start_time': datetime.datetime(2022, 3, 16, 16, 0, 51, 276126),
'downstream_total_latency_in_seconds': 386.911986,
'failures': [],
'gms_version': 'v0.8.29',
'records_written': 2495,
'warnings': []}
Pipeline finished with failures
[2022-03-16 16:07:19,211] INFO {datahub.telemetry.telemetry:159} - Sending Telemetry
[2022-03-16 16:07:29,226] INFO {datahub.telemetry.telemetry:159} - Sending Telemetryd
dazzling-judge-80093
03/16/2022, 8:27 AMWhat error message did you get?
s
stale-petabyte-93467
03/16/2022, 8:36 AM Copy code
Pipeline finished with failuresd
dazzling-judge-80093
03/16/2022, 8:42 AMIs this all the logs? It is weird it is saying failure even though in the summary there isn’t anything
s
stale-petabyte-93467
03/16/2022, 10:08 AMYeah which is why I am interested to see if there are any other log locations
d
dazzling-judge-80093
03/16/2022, 10:16 AMIt should have written some info logs/warning on the screen.
You can enable debug logging as well with ->
datahub --debug …s
square-activity-64562
03/16/2022, 10:21 AM@stale-petabyte-93467 There should be some logs above "Sink (datahub-rest) report". Can you please share those too?
square-activity-64562
03/16/2022, 10:22 AMCan you please share the following with error reports? This would help team in helping you out
• Are you using UI based ingestion, or Python CLI through command line, or programatically via python SDK or programmatically via Java emitter?
• full logs in text format (instead of screenshots) from the ingestion that fails. Please do not remove any parts of the log (mask the secret if any secret is being shown)
• the recipe in text format (instead of screenshots)
s
stale-petabyte-93467
03/16/2022, 10:30 AMThe logs above are too long. This is why I am asking if it is stored somewhere?
stale-petabyte-93467
03/16/2022, 10:30 AMI used Python CLI using datahub ingest -c
stale-petabyte-93467
03/16/2022, 10:30 AMRunning a VM with CENTOS 7.14
s
square-activity-64562
03/16/2022, 10:32 AMThey are not stored anywhere at the moment
square-activity-64562
03/16/2022, 10:33 AMWhat recipe are you using?
square-activity-64562
03/16/2022, 10:33 AMIf you are not comfortable sharing the logs you can search them yourself for any exceptions or errors
d
dazzling-judge-80093
03/16/2022, 10:34 AMyou can write to file the logs if you redirect the output to file:
Copy code
datahub ingest ... > datahub.log 2>&1s
stale-petabyte-93467
03/16/2022, 12:10 PMLet me pipe the output.
I am using a recipe for snowflake. Let me rerun it tomorrow and update with logs
d
dazzling-judge-80093
03/16/2022, 1:02 PMthanks