Hi Team , I am trying to executing backfill job using pinot-ingestion job.Basically , I am trying create offline segment using pinot-inestion job can we fix offline segments size without using minion while executing backfill job if too much data is there? Can anyone please help with the same?

if you meant such ingestion job, looks like it has to ingest one file into one segment but you might spread the input data into multiple files as a workaround, if that’s feasible

I am using regex under inculdeFilePattern , so that it takes all the files and generate offline segment . If the record is too much , it will create multiple offline segment. Basically , i want to generate offline segment based on number of record/size using pinot-ingestion job

hmm.. for pinot-ingestion job, I didn’t see configs to tune output segment size and it’s one segment perf file from the implementation. but I might have looked at the wrong code, so could you share the job spec you used so I could double check. thank you