Hey folks, i'm trying to ingest some data from my ...
# troubleshooti
icy-piano-35127
03/31/2022, 6:55 PMHey folks, i'm trying to ingest some data from my redshift datasource but it's running for like 5 hours (and we have about 24 tables). Seems like that the ingestion finished but the status is wrong in the ingestion section. What can i do to help you to debug this?
icy-piano-35127
03/31/2022, 6:57 PMCan it be the kafka configuration? Or there's no relation? Because i was looking in the kafka pods log and theres a warning:
i
incalculable-ocean-74010
03/31/2022, 7:39 PMDid you enable profiling on this ingestion?
i
icy-piano-35127
03/31/2022, 7:50 PMYes
icy-piano-35127
03/31/2022, 7:50 PMI'll try without profiling and see if it's work, probably yes because the amount of data in our tables
icy-piano-35127
03/31/2022, 7:51 PMOne of them have like 64gb
b
big-carpet-38439
04/01/2022, 9:15 PMYeah its most likely that the container that is running ingestion is having trouble keeping up if you have profiling enabled - one way to combat this is to increase the resources assigned to the
datahub-actionsthank you 1
i
icy-piano-35127
04/04/2022, 11:52 AMJohn, i've tryied to remove the profiling but it's still taking a long time (more than 24h because i forgot to turn it of 🤦♂️ )
icy-piano-35127
04/04/2022, 1:01 PMI think that i found the motive of that. The point is the redshift lineage. Basically it is taking too long to check every archive that generates the table using the copy statement because it's more than 100+ archives
b
big-carpet-38439
04/04/2022, 9:54 PMI see - cc @dazzling-judge-80093 due to the lineage scaling issue. If you disable lineage extraction that should also help increase the latency
big-carpet-38439
04/04/2022, 9:54 PMWe would recommend generally having different sources with different schedules for these things due to the difference in execution cost
i
icy-piano-35127
04/05/2022, 12:35 PMCool @big-carpet-38439 ! I've disabled it for now. Thanks for helping
2 Views