Apache Pinot

hey, yes but when i run my query the trace has the following output

`"numServersQueried": 1,`
    `"numServersResponded": 1,`
    `"numSegmentsQueried": 1,`
    `"numSegmentsProcessed": 1,`
    `"numSegmentsMatched": 1,`

so i think all my records have been saved inside one segment only

every file you ingest creates one segment

ah ok, so 1 segment per file... ok so i simple can split my big csv into smaller chunks

in that way pinot will do a parallel search on each segment

however i set REFRESH, so if i put another file i think it will override the previous one no?

typically you partition the data on some key and name them segment_0, segment_1 ... segment_N

```executionFrameworkSpec:
  name: 'standalone'
  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
  segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '/tmp/pinot-quick-start/rawdata/'
includeFileNamePattern: 'glob:**/*.csv'
outputDirURI: '/tmp/pinot-quick-start/segments/'
overwriteOutput: true
pinotFSSpecs:
  - scheme: file
    className: org.apache.pinot.spi.filesystem.LocalPinotFS
recordReaderSpec:
  dataFormat: 'csv'
  className: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReader'
  configClassName: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReaderConfig'
tableSpec:
  tableName: 'test'
  schemaURI: '<http://pinot-quickstart:9000/tables/test/schema>'
  tableConfigURI: '<http://pinot-quickstart:9000/tables/test>'
pinotClusterSpecs:
  - controllerURI: '<http://pinot-quickstart:9000>'```


i mean.... i f i put more .csv inside rawdata

each csv will have a different segment (it will be automatically generated)

it will have a segment name as table_{idx_id}

ok so if i have rawdata/1.csv     /rawdata/2.csv it will create the segment automatically, right? i mean, i should not change that yaml file right ?

the layout will be /segments/test_0 /segments/test_1 …

ok i do the same test but splitting my giant csv file into a smaller chunks

<@UGRJA9TEH> 3x times faster ! now the segments are 38, before only 1...for ~37M documents

```    "numDocsScanned": 37991563,
    "numEntriesScannedInFilter": 0,
    "numEntriesScannedPostFilter": 75983126,```
