baarath
04/22/2025, 6:57 PMexecutionFrameworkSpec:
name: 'standalone'
segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '/input_data/'
includeFileNamePattern: '*.csv'
outputDirURI: '/output_segments/'
overwriteOutput: true
createMetadataTarGz: true
segmentCreationJobParallelism: 4
pinotFSSpecs:
-
scheme: file
className: org.apache.pinot.spi.filesystem.LocalPinotFS
recordReaderSpec:
dataFormat: 'csv'
className: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReader'
tableSpec:
tableName: 'test_table'
schemaURI: 'http://localhost:9000/tables/test_table/schema'
tableConfigURI: 'http://localhost:9000/tables/test_table'
pinotClusterSpecs:
-
controllerURI: 'http://localhost:9000'
And I’m getting following error in ss. Can anyone help me on this.baarath
04/23/2025, 5:54 AMbaarath
04/24/2025, 6:53 AMincludeFileNamePattern: '*.csv' instead of
includeFileNamePattern: 'glob:**/*.csv' which causing issue.
https://dev.startree.ai/docs/pinot/recipes/csv-files-spaces-column-names