Hi, I am facing issue while running restore indexes job, I am trying to host new datahub from existing database it is unable to fetch policy urn, can somebody help what is the issue, why Elastic search query is failing ?

2022-11-04 11:27:15.085 ERROR 1 --- [pool-7-thread-1] c.d.authorization.DataHubAuthorizer : Failed to retrieve policy urns! Skipping updating policy cache until next refresh. start: 0, count: 30
com.datahub.util.exception.ESQueryException: Search query failed:
at com.linkedin.metadata.search.elasticsearch.query.ESSearchDAO.executeAndExtract(ESSearchDAO.java:73) ~[metadata-io.jar!/:na]
at com.linkedin.metadata.search.elasticsearch.query.ESSearchDAO.search(ESSearchDAO.java:100) ~[metadata-io.jar!/:na]
at com.linkedin.metadata.search.elasticsearch.ElasticSearchService.search(ElasticSearchService.java:67) ~[metadata-io.jar!/:na]
at com.linkedin.entity.client.JavaEntityClient.search(JavaEntityClient.java:280) ~[restli-client.jar!/:na]
at com.datahub.authorization.PolicyFetcher.fetchPolicies(PolicyFetcher.java:50) ~[auth-impl.jar!/:na]
at com.datahub.authorization.PolicyFetcher.fetchPolicies(PolicyFetcher.java:42) ~[auth-impl.jar!/:na]
at com.datahub.authorization.DataHubAuthorizer$PolicyRefreshRunnable.run(DataHubAuthorizer.java:229) ~[auth-impl.jar!/:na]

Hi Nirmit, are you restoring only the index or restoring from a backup? It seems there may be policies referenced that were erased as well?

yes, I have erased entire elastic search data using below api

curl -X DELETE '<http://localhost:9200/_all>'

now when I use frontend it throws 500 error, which I figured out to be related to no data found in elastic search. so I was trying to run, restore indices, but it is unable to process, can you suggest some how to achieve this ?

@big-carpet-38439 do you know if we’re able to restore indices without a backup?

I think some sort of job can be created that can create the indices data from db