Hi all I’m trying to stand up a batch process to i...
# troubleshooting
b
Hi all I’m trying to stand up a batch process to ingest data into pinot offline table. below is my job spec yaml file.
Copy code
executionFrameworkSpec:
  name: 'standalone'
  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
  segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '/input_data/'
includeFileNamePattern: '*.csv'
outputDirURI: '/output_segments/'
overwriteOutput: true
createMetadataTarGz: true
segmentCreationJobParallelism: 4
pinotFSSpecs:
  - 
    scheme: file
    className: org.apache.pinot.spi.filesystem.LocalPinotFS
recordReaderSpec:
  dataFormat: 'csv'
  className: 'org.apache.pinot.plugin.inputformat.csv.CSVRecordReader'
tableSpec:
  tableName: 'test_table'
  schemaURI: 'http://localhost:9000/tables/test_table/schema'
  tableConfigURI: 'http://localhost:9000/tables/test_table'
pinotClusterSpecs:
  - 
    controllerURI: 'http://localhost:9000'
And I’m getting following error in ss. Can anyone help me on this.
Do I need to have minon container separately launched FYI I use Docker composer to deploy pinot cluster (version 1.2.0). As of now i have separate container for broker, server & controller. Should i have seperate container for Minon. Is that what the issue ?
Had it resolved. Below link helped me to resolve the issue. Seems like i used
includeFileNamePattern: '*.csv'
instead of
includeFileNamePattern: 'glob:**/*.csv'
which causing issue. https://dev.startree.ai/docs/pinot/recipes/csv-files-spaces-column-names