Hello all, I am trying to fetch entities using the search API

<http://localhost:8080/entities?action=search>

with this body as payload

{
  "input": "",
  "entity": "dataset",
  "start": 9999,
  "count": 10000
}

I could able to fetch only upto 10,000 records. Is there any way to get entire records based on entity type and environment. If I give start as '0' and end as '10000', getting the response. Not getting the remaining records.

Hello, 10k records is a limitation of ElasticSearch which is the engine we use for search within DataHub. The best way to fetch all data from DataHub is to iteratively run sequential searches that return less than 10k records.

okay. Thanks @incalculable-ocean-74010