Hi guys, I am trying to fetch data from pinot using golang application. After fetching the data from table I have to iterate(for loop) it in order to print the data. Since the efficiency of the application is considerably going down in the above method. Is there any alternate method to get the data without iterating? I am following this link for reference - https://docs.pinot.apache.org/users/clients/golang cc: @Arun Prasath

Is it the iteration that is slow, or could there be other issues (eg huge response size, deserialization, etc)?

We have a huge amount of data so the iteration makes it slow.

it depends on the limit size in query how many rows you fetch .. I guess you are concerned aboutr latency added because of pinot query response json processing?

Yes. The same issue. Because the limit value is huge

what is the use case and the query

UseCase - Select all the values from the table, store it in a structure so that we can pass it as a json to the user. Query - General select query but the limit is all the data in the table (If there is 1 lakh record, we have to parse it all)

you will have to limit query since pinot only returns 10 rows by default .. so you have to put your upper limit of count of rows you want to fetch

Is there any other way we can handle the latency

this is the processing latency ..you can calculate how many milliseconds its taking to process one json record and then get an estimate what wi;ll be the processing time for n no of records .. also check which line no is conusming max amount of processsing time