Hey all - I'm using GraphQL and am having trouble ...
# troubleshootr
rhythmic-stone-77840
07/19/2022, 12:42 AMHey all - I'm using GraphQL and am having trouble setting up a filter for downstream/upstream lineage. I'd like to pull out all datasets that have an upstream lineage of 0, but I don't understand how to get the filter to work for this. Current query in 🧵
rhythmic-stone-77840
07/19/2022, 12:44 AMCurrently this will pull out all BQ datasets and give me their immediate upstream & down lineage counts
Copy code
{
search(input: { type: DATASET, query: "*", start: 0, count: 10, filters: {field: "platform", value:"urn:li:dataPlatform:bigquery"}}) {
start
count
total
searchResults {
entity {
urn
type
...on Dataset {
name
upstream: lineage(input: {direction: UPSTREAM}){
total
}
downstream: lineage(input: {direction: DOWNSTREAM}) {
total
}
}
}
}
}rhythmic-stone-77840
07/19/2022, 12:44 AMIs it possible to add a filter on the upstream lineage to only pull out totals that are 0?
b
bulky-soccer-26729
07/21/2022, 4:22 PMhey Laura! I'm not actually sure we have that sort of filter supported at the moment. What are you planning on doing with the results of this query? Would it be possible to apply a post-query filter on the results to get what you want?
r
rhythmic-stone-77840
07/21/2022, 5:19 PMI was hoping to use it to get a list of tables that don't have any upstreams since it either means they're landing data, or they're an adhoc table that needs to be cleaned up. I can apply a post-query filter, I was just hoping there was a way to push that processing into GraphQL instead.
2 Views