Hi All, Pinot server stopped or crashed for 1 hour. Kafka ingestion from clients are in progress and they are not stopped. There are more data inflows say 10Million records as offset. I am using one Pinot Server. After an hour, server got restarted. Problem: Pinot Server to catch up those 10M kafka offset records for a topic takes more time - 2-4 hours as well. How to avoid this and i am not planning to loose the KAFKA data as well, Retention period for KAFKA is 6hrs. Also 10M records continue to increase, since the KAFKA client ingestion is not all stopped..... What kind of design or solution that pino supports to overcome or waiting for 2-5 hours for the catch-up mechanism.....?

Have you right sized your cluster?

I don't have cluster, i have only single instance of pinot server. To overcome this issue at this point in time. can i just increase the pino server instance alone.... to start with... Will this avoid the catch-up issue on the restart of pino-servers?

You need to find where the bottleneck is. For example, how mnay partitions you have in Kafka, how is the CPU/mem on the server, how often are you flushing etc. 2-4 hours to catch up usually means your setup is far from optimal

If yes, then the current pino eco system, ./pinot-admin.sh StartServer \ script does not consider or take the inputs from -config=../conf/pinot-server.conf file. It takes default values always for certain ports 8090, 8098, etc....

I don't think that should be the case.