https://datahubproject.io logo
Join Slack
Powered by
How can I search for datasets with a filter on a f...
# troubleshoot
a
How can I search for datasets with a filter on a field that matches the field against one of a list of values? The query below only matches tags 
urn:li:tag: user identifier
, but I want to match either 
urn:li:tag:user identifier
or 
urn:li:tag:email address
.
Copy code
query getSearchResultsForMultiple {
  searchAcrossEntities(input: {
    types: [DATASET], 
    query: "",
    start: 0,
    count:1000,
    orFilters: [
      {
        and: [
          {
            field: "fieldTags",
            values: ["urn:li:tag:user identifier", "urn:li:tag:email address"],
            condition: EQUAL
          }
        ]
      }
    ]
  }) {
    start
    count
    total
    searchResults {
      entity {
        urn
        type
      }
    }
  }
}
✅ 1
From the logs I saw the following, it seems that both the 
value
and the 
values
fields in the filter is set.
Copy code
01:02:06 [Thread-16104] DEBUG c.l.d.g.r.s.SearchAcrossEntitiesResolver - Executing search for multiple entities: entity types [DATASET], query , filters: null, start: 0, count: 1000
01:02:06 [Thread-16104] DEBUG c.l.metadata.search.SearchService - Searching Search documents entities: [dataset], input: , postFilters: {or=[{and=[{condition=EQUAL, field=fieldTags.keyword, value=urn:li:tag:user identifier, values=[urn:li:tag:user identifier, urn:li:tag:email address]}]}]}, sortCriterion: null, from: 0, size: 1000
01:02:06 [Thread-16105] DEBUG c.l.m.s.e.ElasticSearchService - Searching Search documents entityName: dataset, input: , postFilters: {or=[{and=[{condition=EQUAL, field=fieldTags.keyword, value=urn:li:tag:user identifier, values=[urn:li:tag:user identifier, urn:li:tag:email address]}]}]}, sortCriterion: null, from: 0, size: 100
Then the query sent to elastic search only filters on 
value
instead of 
values
.
Copy code
"should": [
                          {
                            "match": {
                              "fieldTags.keyword": {
                                "query": "urn:li:tag:user identifier",
                                "operator": "OR",
                                "prefix_length": 0,
                                "max_expansions": 50,
                                "fuzzy_transpositions": true,
                                "lenient": false,
                                "zero_terms_query": "NONE",
                                "auto_generate_synonyms_phrase_query": true,
                                "boost": 1.0
                              }
                            }
                          },
                          {
                            "match": {
                              "editedFieldTags.keyword": {
                                "query": "urn:li:tag:user identifier",
                                "operator": "OR",
                                "prefix_length": 0,
                                "max_expansions": 50,
                                "fuzzy_transpositions": true,
                                "lenient": false,
                                "zero_terms_query": "NONE",
                                "auto_generate_synonyms_phrase_query": true,
                                "boost": 1.0
                              }
                            }
                          }
                        ]
@big-carpet-38439 @brainy-tent-14503 ^^^
It works fine for fields that don't require pair field handling like "platform". Happy to send a PR once you guys confirm.
b
@orange-night-91387 ☝️ - This is likely a candidate to add to the misc search fixes bucket.
o
Yeah this looks like a miss, if you want to put up a PR would be happy to accept. If you don't mind adding in a case to the unit test on the SearchRequestHandlerTest with a case that tests this would be even more happy 😄 Looks like our current unit test doesn't test the use of values in a single filter, just having multiple single value filters. If not I can write one up and add it to the PR you put up
a
Sure thing, I will send a PR shortly
o
Thank you!!
a
Thank you for your quick review @orange-night-91387! Can you merge it so that we can start testing it?
Open in Slack
PreviousNext
Powered by