Channels
Hey there’s a bug with `looker` ingestion source -...
# ingestionb
bumpy-activity-74405
08/30/2021, 6:40 AMHey there’s a bug with
lookerm
mammoth-bear-12532
08/31/2021, 3:42 AM
@bumpy-activity-74405: thanks for the PR. It seems simple enough, but I'm curious. What is the use-case for ingesting deleted dashboards? Do you expect to search for them in DataHub? Would it be confusing to find a deleted dashboard?
b
bumpy-activity-74405
08/31/2021, 5:33 AMI don’t want to see or be able to find deleted dashboards. Running ingestion daily so dashboards deleted after last run would get hidden by the next one. The way the code was written (adding a status aspect) I thought that was the intention, the problem is that deleted dashboards never got ingested.
m
mammoth-bear-12532
08/31/2021, 6:46 AM
that makes sense! Thanks for the explanation 🙂
mammoth-bear-12532
08/31/2021, 7:02 AM
@high-hospital-85984 makes some good points in his review... this seems to fall into the general category of -> detecting deleted entries cheaply and soft-deleting them on the next ingest. This is a common problem across our sources and needs a common solution IMO.
b
bumpy-activity-74405
09/02/2021, 11:17 AMHey @high-hospital-85984, @mammoth-bear-12532 I’ve made the ingestion of deleted dashbaords optional - https://github.com/linkedin/datahub/pull/3158/files. I hope that works for you.
h
high-hospital-85984
09/02/2021, 11:26 AMWorks for me. We anyway basically hardcode the dashboards we're interested (as regex and numbers aren't the best combination 😅 ).
b
bumpy-activity-74405
09/02/2021, 11:30 AMOut of curiosity - what do you do if a dashboard gets deleted?
h
high-hospital-85984
09/02/2021, 11:44 AMuse your flag.
high-hospital-85984
09/02/2021, 11:44 AMor soft-delete manually