[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023/11/24 22:59:02.064 ERROR [LaunchDataIngestionJobCommand] [main] Got exception to generate IngestionJobSpec for data ingestion job - 
org.yaml.snakeyaml.constructor.ConstructorException: Cannot create property=recordReaderSpec for JavaBean=org.apache.pinot.spi.ingestion.batch.spec.SegmentGenerationJobSpec@9a6fbc0
 in 'string', line 1, column 1:
    {
    ^
Cannot create property=type for JavaBean=org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec@4d96250
 in 'string', line 2, column 23:
      "recordReaderSpec": {
                          ^
Unable to find property 'type' on class: org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec
 in 'string', line 3, column 13:
        "type": "SegmentCreationAndTar",
                ^

 in 'string', line 2, column 23:
      "recordReaderSpec": {
                          ^

	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:283) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:169) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:320) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:264) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:247) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:201) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:185) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:493) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.Yaml.loadAs(Yaml.java:473) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.getSegmentGenerationJobSpec(IngestionJobLauncher.java:100) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:111) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
Caused by: org.yaml.snakeyaml.constructor.ConstructorException: Cannot create property=type for JavaBean=org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec@4d96250
 in 'string', line 2, column 23:
      "recordReaderSpec": {
                          ^
Unable to find property 'type' on class: org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec
 in 'string', line 3, column 13:
        "type": "SegmentCreationAndTar",
                ^

	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:283) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:169) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:264) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:247) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:261) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	... 21 more
Caused by: org.yaml.snakeyaml.error.YAMLException: Unable to find property 'type' on class: org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec
	at org.yaml.snakeyaml.introspector.PropertyUtils.getProperty(PropertyUtils.java:155) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.introspector.PropertyUtils.getProperty(PropertyUtils.java:145) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.getProperty(Constructor.java:299) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:223) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:169) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:264) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:247) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:261) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	... 21 more
Cannot create property=recordReaderSpec for JavaBean=org.apache.pinot.spi.ingestion.batch.spec.SegmentGenerationJobSpec@9a6fbc0
 in 'string', line 1, column 1:
    {
    ^
Cannot create property=type for JavaBean=org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec@4d96250
 in 'string', line 2, column 23:
      "recordReaderSpec": {
                          ^
Unable to find property 'type' on class: org.apache.pinot.spi.ingestion.batch.spec.RecordReaderSpec
 in 'string', line 3, column 13:
        "type": "SegmentCreationAndTar",
                ^

 in 'string', line 2, column 23:
      "recordReaderSpec": {
                          ^

	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:283)
	at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:169)
	at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:320)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:264)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:247)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:201)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:185)
	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:493)
	at org.yaml.snakeyaml.Yaml.loadAs(Yaml.java:473)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.getSegmentGenerationJobSpec(IngestionJobLauncher.java:100)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:111)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)

long

'2023-11-28T03:55:20.886+0000'

"dateTimeFieldSpecs": [
    {
      "name": "LOG_TS",
      "dataType": "STRING",
      "format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",
      "granularity": "1:MILLISECONDS"
    }

[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023/11/28 10:46:04.436 ERROR [LaunchDataIngestionJobCommand] [main] Got exception to kick off standalone data ingestion job - 
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=00/000095_0>
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.IllegalArgumentException: Cannot convert value: '2023-11-28T10:44:30.601+0000' to type: LONG
	at org.apache.pinot.spi.data.FieldSpec$DataType.convert(FieldSpec.java:532) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.NullValueTransformer.<init>(NullValueTransformer.java:76) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformer(CompositeTransformer.java:73) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.<init>(TransformPipeline.java:60) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171)
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)
Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=00/000095_0>
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Cannot convert value: '2023-11-28T10:44:30.601+0000' to type: LONG
	at org.apache.pinot.spi.data.FieldSpec$DataType.convert(FieldSpec.java:532)
	at org.apache.pinot.segment.local.recordtransformer.NullValueTransformer.<init>(NullValueTransformer.java:76)
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformer(CompositeTransformer.java:73)
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.<init>(TransformPipeline.java:60)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101)
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112)
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265)
	... 5 more

{
  "schemaName": "uapitest",
  "dimensionFieldSpecs": [
    {
      "name": "log_id",
      "dataType": "STRING"
    },
    {
      "name": "date_id",
      "dataType": "STRING"
    },
    {
      "name": "time_id",
      "dataType": "INT"
    },
    {
      "name": "request_type_desc",
      "dataType": "STRING"
    },
    {
      "name": "success_ind",
      "dataType": "STRING"
    },
    {
      "name": "schema_version",
      "dataType": "STRING"
    },
    {
      "name": "agency_name",
      "dataType": "STRING"
    },
    {
      "name": "branch",
      "dataType": "STRING"
    },
    {
      "name": "target_branch",
      "dataType": "STRING"
    },
    {
      "name": "agent",
      "dataType": "STRING"
    },
    {
      "name": "pseudo_city_code",
      "dataType": "STRING"
    },
    {
      "name": "ur_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "origin_application_code",
      "dataType": "STRING"
    },
    {
      "name": "provider_code",
      "dataType": "STRING"
    },
    {
      "name": "provider_code_list",
      "dataType": "STRING"
    },
    {
      "name": "supplier_code_list",
      "dataType": "STRING"
    },
    {
      "name": "total_number_of_travellers",
      "dataType": "INT"
    },
    {
      "name": "total_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "total_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "travel_origin",
      "dataType": "STRING"
    },
    {
      "name": "travel_destination",
      "dataType": "STRING"
    },
    {
      "name": "travel_startdate",
      "dataType": "STRING"
    },
    {
      "name": "travel_enddate",
      "dataType": "STRING"
    },
    {
      "name": "total_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "galileo_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "apollo_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "worldspan_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "axess_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "aircontent_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "railcontent_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "merchandising_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "services_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "trm_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "app_server_host",
      "dataType": "STRING"
    },
    {
      "name": "one_way_shop_ind",
      "dataType": "STRING"
    },
    {
      "name": "request_bytes",
      "dataType": "INT"
    },
    {
      "name": "response_bytes",
      "dataType": "INT"
    },
    {
      "name": "host_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "optional_service_qty",
      "dataType": "INT"
    },
    {
      "name": "travel_segments",
      "dataType": "INT"
    },
    {
      "name": "travel_dates",
      "dataType": "STRING"
    },
    {
      "name": "supplier_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "flight_number",
      "dataType": "STRING"
    },
    {
      "name": "class_of_service",
      "dataType": "STRING"
    },
    {
      "name": "availability_source",
      "dataType": "STRING"
    },
    {
      "name": "number_in_party",
      "dataType": "INT"
    },
    {
      "name": "air_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "air_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "hotel_code",
      "dataType": "STRING"
    },
    {
      "name": "hotel_search_by_addr_ind",
      "dataType": "STRING"
    },
    {
      "name": "hotel_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "hotel_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "car_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "car_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "request_base_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "response_base_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "base_difference_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "request_total_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "response_total_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "total_difference_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "ssl_term_region",
      "dataType": "STRING"
    },
    {
      "name": "error_level_code",
      "dataType": "STRING"
    },
    {
      "name": "error_code",
      "dataType": "STRING"
    },
    {
      "name": "error_type_code",
      "dataType": "STRING"
    },
    {
      "name": "error_text",
      "dataType": "STRING"
    },
    {
      "name": "soap_fault_text",
      "dataType": "STRING"
    },
    {
      "name": "etl_id",
      "dataType": "STRING"
    },
    {
      "name": "load_date",
      "dataType": "STRING"
    },
    {
      "name": "created_by",
      "dataType": "STRING"
    },
    {
      "name": "created_dt",
      "dataType": "STRING"
    },
    {
      "name": "updated_by",
      "dataType": "STRING"
    },
    {
      "name": "updated_dt",
      "dataType": "STRING"
    },
    {
      "name": "traceid",
      "dataType": "STRING"
    },
    {
      "name": "fare_rules_filter_category_codes",
      "dataType": "STRING"
    },
    {
      "name": "request_rate_plan_type",
      "dataType": "STRING"
    },
    {
      "name": "number_of_adults",
      "dataType": "INT"
    },
    {
      "name": "number_of_rooms",
      "dataType": "INT"
    },
    {
      "name": "guarantee_type",
      "dataType": "STRING"
    },
    {
      "name": "other_guarantee_info",
      "dataType": "STRING"
    },
    {
      "name": "response_rate_plan_type",
      "dataType": "STRING"
    },
    {
      "name": "response_message_code",
      "dataType": "STRING"
    },
    {
      "name": "booking_confirmation",
      "dataType": "STRING"
    },
    {
      "name": "status",
      "dataType": "STRING"
    },
    {
      "name": "language_code",
      "dataType": "STRING"
    },
    {
      "name": "pickup_location_type",
      "dataType": "STRING"
    },
    {
      "name": "return_location_type",
      "dataType": "STRING"
    },
    {
      "name": "pickup_location_number",
      "dataType": "INT"
    },
    {
      "name": "return_location_number",
      "dataType": "INT"
    },
    {
      "name": "rate_code",
      "dataType": "STRING"
    },
    {
      "name": "rate_category",
      "dataType": "STRING"
    },
    {
      "name": "rate_period",
      "dataType": "STRING"
    },
    {
      "name": "discount_number",
      "dataType": "INT"
    },
    {
      "name": "acriss_vehicle_code",
      "dataType": "STRING"
    },
    {
      "name": "rate_for_period",
      "dataType": "DOUBLE"
    },
    {
      "name": "base_rate",
      "dataType": "DOUBLE"
    },
    {
      "name": "cds_transaction_id",
      "dataType": "STRING"
    },
    {
      "name": "cds_execution_time",
      "dataType": "LONG"
    },
    {
      "name": "cds_success_failure_flag",
      "dataType": "STRING"
    },
    {
      "name": "cds_error_message",
      "dataType": "STRING"
    },
    {
      "name": "estimated_total_amount",
      "dataType": "DOUBLE"
    },
    {
      "name": "mandatory_charge_total",
      "dataType": "DOUBLE"
    },
    {
      "name": "air_conditioning",
      "dataType": "STRING"
    },
    {
      "name": "transmission_type",
      "dataType": "STRING"
    },
    {
      "name": "vehicle_class",
      "dataType": "STRING"
    },
    {
      "name": "category",
      "dataType": "STRING"
    },
    {
      "name": "vehicle_type",
      "dataType": "STRING"
    },
    {
      "name": "door_count",
      "dataType": "INT"
    },
    {
      "name": "aux1",
      "dataType": "STRING"
    },
    {
      "name": "aux2",
      "dataType": "STRING"
    },
    {
      "name": "aux3",
      "dataType": "STRING"
    },
    {
      "name": "cancellation_text",
      "dataType": "STRING"
    },
    {
      "name": "requested_negotiated_ratecodes_count",
      "dataType": "INT"
    },
    {
      "name": "rate_category_count",
      "dataType": "INT"
    },
    {
      "name": "number_of_rate_plan_types",
      "dataType": "INT"
    },
    {
      "name": "negotiated_rate_codes_in_response",
      "dataType": "STRING"
    },
    {
      "name": "response_negotiated_ratecode_count",
      "dataType": "INT"
    },
    {
      "name": "requested_vendor_location_count",
      "dataType": "INT"
    },
    {
      "name": "max_wait_time",
      "dataType": "INT"
    },
    {
      "name": "response_properties_count",
      "dataType": "INT"
    },
    {
      "name": "tax",
      "dataType": "DOUBLE"
    },
    {
      "name": "surcharge",
      "dataType": "DOUBLE"
    },
    {
      "name": "commision_indicator",
      "dataType": "STRING"
    },
    {
      "name": "percentage",
      "dataType": "DOUBLE"
    },
    {
      "name": "galileo_guarantee",
      "dataType": "STRING"
    },
    {
      "name": "galileo_deposit",
      "dataType": "STRING"
    },
    {
      "name": "worldspan_deposit_guarantee_prepayment",
      "dataType": "STRING"
    },
    {
      "name": "uapi_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "session_key",
      "dataType": "STRING"
    },
    {
      "name": "agency_country",
      "dataType": "STRING"
    },
    {
      "name": "source_ipaddress",
      "dataType": "STRING"
    },
    {
      "name": "ach_transactionid",
      "dataType": "STRING"
    },
    {
      "name": "db2_locktime",
      "dataType": "LONG"
    },
    {
      "name": "db2_number_of_locks",
      "dataType": "INT"
    },
    {
      "name": "override_pcc",
      "dataType": "STRING"
    },
    {
      "name": "meta_search",
      "dataType": "STRING"
    },
    {
      "name": "solution_result",
      "dataType": "STRING"
    },
    {
      "name": "ptc_list",
      "dataType": "STRING"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "LOG_TS",
      "dataType": "STRING",
      "format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",
      "granularity": "1:MILLISECONDS"
    }
  ],
  "primaryKeyColumns": [
    "log_id"
  ]
}

bash-4.2# cat pinot_table_config.json 

{
  "OFFLINE": {
    "tableName": "uapitest_OFFLINE",
    "tableType": "OFFLINE",
    
    "segmentsConfig": {
      "schemaName": "uapitest",
      "replication": "2",
      "replicasPerPartition": "2",
      "timeColumnName": "LOG_TS",
      "segmentPushFrequency": "HOURLY",
      "segmentPushType": "APPEND",
      "minimizeDataMovement": false
    },

    "tenants": {
      "broker": "DefaultTenant",
      "server": "DefaultTenant"
    },

    "tableIndexConfig": {
      "invertedIndexColumns": [],
      "noDictionaryColumns": [],
      "rangeIndexColumns": [],
      "rangeIndexVersion": 2,
      "autoGeneratedInvertedIndex": false,
      "createInvertedIndexDuringSegmentGeneration": false,
      "sortedColumn": [],
      "bloomFilterColumns": [],
      "loadMode": "MMAP",
      "onHeapDictionaryColumns": [],
      "varLengthDictionaryColumns": [],
      "enableDefaultStarTree": false,
      "enableDynamicStarTreeCreation": false,
      "aggregateMetrics": false,
      "nullHandlingEnabled": false,
      "optimizeDictionary": false,
      "optimizeDictionaryForMetrics": false,
      "noDictionarySizeRatioThreshold": 0.85
    },

    "metadata": {},
    "quota": {},
    "routing": {},
    "query": {},

    "ingestionConfig": {
      "continueOnError": false,
      "rowTimeValueCheck": false,
      "segmentTimeValueCheck": true
    },

    "isDimTable": false,

    "tableDataConfig": {
      "s3Config": {
        "s3BucketName": "xxxxxxxx",
        "awsAccessKey": "xxxxxxx",
        "awsSecretKey": "xxxxxxxxxxx"
      },

      "proxyConfig": {
        "httpProxy": "xxxxxxx:8080",
        "httpsProxy": "xxxxxx:8080",
        "ftpProxy": "xxxxxx:8080"
      }
    }
  }
}

"dateTimeFieldSpecs": [
    {
      "name": "LOG_TS",
      "dataType": "LONG",
      "format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",
      "granularity": "1:MILLISECONDS"
    }

[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023/11/28 10:57:29.972 ERROR [LaunchDataIngestionJobCommand] [main] Got exception to kick off standalone data ingestion job - 
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=00/000095_0>
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.IllegalArgumentException: Cannot convert value: '2023-11-28T10:55:52.079+0000' to type: LONG
	at org.apache.pinot.spi.data.FieldSpec$DataType.convert(FieldSpec.java:532) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.NullValueTransformer.<init>(NullValueTransformer.java:76) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformer(CompositeTransformer.java:73) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.<init>(TransformPipeline.java:60) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171)
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)
Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=00/000095_0>
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Cannot convert value: '2023-11-28T10:55:52.079+0000' to type: LONG
	at org.apache.pinot.spi.data.FieldSpec$DataType.convert(FieldSpec.java:532)
	at org.apache.pinot.segment.local.recordtransformer.NullValueTransformer.<init>(NullValueTransformer.java:76)
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformer(CompositeTransformer.java:73)
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.<init>(TransformPipeline.java:60)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101)
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112)
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265)
	... 5 more

bash-4.2# cat pinot_table_config.json 

{
  "OFFLINE": {
    "tableName": "uapi_OFFLINE",
    "tableType": "OFFLINE",
    
    "segmentsConfig": {
      "schemaName": "uapi",
      "replication": "2",
      "replicasPerPartition": "2",
      "timeColumnName": "LOG_TS",
      "segmentPushFrequency": "HOURLY",
      "segmentPushType": "APPEND",
      "minimizeDataMovement": false
    },

    "tenants": {
      "broker": "DefaultTenant",
      "server": "DefaultTenant"
    },

    "tableIndexConfig": {
      "invertedIndexColumns": [],
      "noDictionaryColumns": [],
      "rangeIndexColumns": [],
      "rangeIndexVersion": 2,
      "autoGeneratedInvertedIndex": false,
      "createInvertedIndexDuringSegmentGeneration": false,
      "sortedColumn": [],
      "bloomFilterColumns": [],
      "loadMode": "MMAP",
      "onHeapDictionaryColumns": [],
      "varLengthDictionaryColumns": [],
      "enableDefaultStarTree": false,
      "enableDynamicStarTreeCreation": false,
      "aggregateMetrics": false,
      "nullHandlingEnabled": false,
      "optimizeDictionary": false,
      "optimizeDictionaryForMetrics": false,
      "noDictionarySizeRatioThreshold": 0.85
    },

    "metadata": {},
    "quota": {},
    "routing": {},
    "query": {},

    "ingestionConfig": {
      "continueOnError": false,
      "rowTimeValueCheck": false,
      "segmentTimeValueCheck": true
    },

    "isDimTable": false,

    "tableDataConfig": {
      "s3Config": {
        "s3BucketName": "xxx",
        "awsAccessKey": "xxxx",
        "awsSecretKey": "xxxx"
      },

      "proxyConfig": {
        "httpProxy": "xxxx:8080",
        "httpsProxy": "xxxx:8080",
        "ftpProxy": "xxxxx:8080"
      }
    }
  }
}

bash-4.2# cat pinot_schema_config.json 
{
  "schemaName": "uapi",
  "dimensionFieldSpecs": [
    {
      "name": "log_id",
      "dataType": "STRING"
    },
    {
      "name": "date_id",
      "dataType": "STRING"
    },
    {
      "name": "time_id",
      "dataType": "INT"
    },
    {
      "name": "request_type_desc",
      "dataType": "STRING"
    },
    {
      "name": "success_ind",
      "dataType": "STRING"
    },
    {
      "name": "schema_version",
      "dataType": "STRING"
    },
    {
      "name": "agency_name",
      "dataType": "STRING"
    },
    {
      "name": "branch",
      "dataType": "STRING"
    },
    {
      "name": "target_branch",
      "dataType": "STRING"
    },
    {
      "name": "agent",
      "dataType": "STRING"
    },
    {
      "name": "pseudo_city_code",
      "dataType": "STRING"
    },
    {
      "name": "ur_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "origin_application_code",
      "dataType": "STRING"
    },
    {
      "name": "provider_code",
      "dataType": "STRING"
    },
    {
      "name": "provider_code_list",
      "dataType": "STRING"
    },
    {
      "name": "supplier_code_list",
      "dataType": "STRING"
    },
    {
      "name": "total_number_of_travellers",
      "dataType": "INT"
    },
    {
      "name": "total_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "total_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "travel_origin",
      "dataType": "STRING"
    },
    {
      "name": "travel_destination",
      "dataType": "STRING"
    },
    {
      "name": "travel_startdate",
      "dataType": "STRING"
    },
    {
      "name": "travel_enddate",
      "dataType": "STRING"
    },
    {
      "name": "total_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "galileo_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "apollo_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "worldspan_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "axess_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "aircontent_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "railcontent_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "merchandising_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "services_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "trm_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "app_server_host",
      "dataType": "STRING"
    },
    {
      "name": "one_way_shop_ind",
      "dataType": "STRING"
    },
    {
      "name": "request_bytes",
      "dataType": "INT"
    },
    {
      "name": "response_bytes",
      "dataType": "INT"
    },
    {
      "name": "host_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "optional_service_qty",
      "dataType": "INT"
    },
    {
      "name": "travel_segments",
      "dataType": "INT"
    },
    {
      "name": "travel_dates",
      "dataType": "STRING"
    },
    {
      "name": "supplier_locator_code",
      "dataType": "STRING"
    },
    {
      "name": "flight_number",
      "dataType": "STRING"
    },
    {
      "name": "class_of_service",
      "dataType": "STRING"
    },
    {
      "name": "availability_source",
      "dataType": "STRING"
    },
    {
      "name": "number_in_party",
      "dataType": "INT"
    },
    {
      "name": "air_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "air_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "hotel_code",
      "dataType": "STRING"
    },
    {
      "name": "hotel_search_by_addr_ind",
      "dataType": "STRING"
    },
    {
      "name": "hotel_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "hotel_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "car_segments_booked",
      "dataType": "INT"
    },
    {
      "name": "car_segments_cancelled",
      "dataType": "INT"
    },
    {
      "name": "request_base_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "response_base_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "base_difference_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "request_total_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "response_total_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "total_difference_price",
      "dataType": "DOUBLE"
    },
    {
      "name": "ssl_term_region",
      "dataType": "STRING"
    },
    {
      "name": "error_level_code",
      "dataType": "STRING"
    },
    {
      "name": "error_code",
      "dataType": "STRING"
    },
    {
      "name": "error_type_code",
      "dataType": "STRING"
    },
    {
      "name": "error_text",
      "dataType": "STRING"
    },
    {
      "name": "soap_fault_text",
      "dataType": "STRING"
    },
    {
      "name": "etl_id",
      "dataType": "STRING"
    },
    {
      "name": "load_date",
      "dataType": "STRING"
    },
    {
      "name": "created_by",
      "dataType": "STRING"
    },
    {
      "name": "created_dt",
      "dataType": "STRING"
    },
    {
      "name": "updated_by",
      "dataType": "STRING"
    },
    {
      "name": "updated_dt",
      "dataType": "STRING"
    },
    {
      "name": "traceid",
      "dataType": "STRING"
    },
    {
      "name": "fare_rules_filter_category_codes",
      "dataType": "STRING"
    },
    {
      "name": "request_rate_plan_type",
      "dataType": "STRING"
    },
    {
      "name": "number_of_adults",
      "dataType": "INT"
    },
    {
      "name": "number_of_rooms",
      "dataType": "INT"
    },
    {
      "name": "guarantee_type",
      "dataType": "STRING"
    },
    {
      "name": "other_guarantee_info",
      "dataType": "STRING"
    },
    {
      "name": "response_rate_plan_type",
      "dataType": "STRING"
    },
    {
      "name": "response_message_code",
      "dataType": "STRING"
    },
    {
      "name": "booking_confirmation",
      "dataType": "STRING"
    },
    {
      "name": "status",
      "dataType": "STRING"
    },
    {
      "name": "language_code",
      "dataType": "STRING"
    },
    {
      "name": "pickup_location_type",
      "dataType": "STRING"
    },
    {
      "name": "return_location_type",
      "dataType": "STRING"
    },
    {
      "name": "pickup_location_number",
      "dataType": "INT"
    },
    {
      "name": "return_location_number",
      "dataType": "INT"
    },
    {
      "name": "rate_code",
      "dataType": "STRING"
    },
    {
      "name": "rate_category",
      "dataType": "STRING"
    },
    {
      "name": "rate_period",
      "dataType": "STRING"
    },
    {
      "name": "discount_number",
      "dataType": "INT"
    },
    {
      "name": "acriss_vehicle_code",
      "dataType": "STRING"
    },
    {
      "name": "rate_for_period",
      "dataType": "DOUBLE"
    },
    {
      "name": "base_rate",
      "dataType": "DOUBLE"
    },
    {
      "name": "cds_transaction_id",
      "dataType": "STRING"
    },
    {
      "name": "cds_execution_time",
      "dataType": "LONG"
    },
    {
      "name": "cds_success_failure_flag",
      "dataType": "STRING"
    },
    {
      "name": "cds_error_message",
      "dataType": "STRING"
    },
    {
      "name": "estimated_total_amount",
      "dataType": "DOUBLE"
    },
    {
      "name": "mandatory_charge_total",
      "dataType": "DOUBLE"
    },
    {
      "name": "air_conditioning",
      "dataType": "STRING"
    },
    {
      "name": "transmission_type",
      "dataType": "STRING"
    },
    {
      "name": "vehicle_class",
      "dataType": "STRING"
    },
    {
      "name": "category",
      "dataType": "STRING"
    },
    {
      "name": "vehicle_type",
      "dataType": "STRING"
    },
    {
      "name": "door_count",
      "dataType": "INT"
    },
    {
      "name": "aux1",
      "dataType": "STRING"
    },
    {
      "name": "aux2",
      "dataType": "STRING"
    },
    {
      "name": "aux3",
      "dataType": "STRING"
    },
    {
      "name": "cancellation_text",
      "dataType": "STRING"
    },
    {
      "name": "requested_negotiated_ratecodes_count",
      "dataType": "INT"
    },
    {
      "name": "rate_category_count",
      "dataType": "INT"
    },
    {
      "name": "number_of_rate_plan_types",
      "dataType": "INT"
    },
    {
      "name": "negotiated_rate_codes_in_response",
      "dataType": "STRING"
    },
    {
      "name": "response_negotiated_ratecode_count",
      "dataType": "INT"
    },
    {
      "name": "requested_vendor_location_count",
      "dataType": "INT"
    },
    {
      "name": "max_wait_time",
      "dataType": "INT"
    },
    {
      "name": "response_properties_count",
      "dataType": "INT"
    },
    {
      "name": "tax",
      "dataType": "DOUBLE"
    },
    {
      "name": "surcharge",
      "dataType": "DOUBLE"
    },
    {
      "name": "commision_indicator",
      "dataType": "STRING"
    },
    {
      "name": "percentage",
      "dataType": "DOUBLE"
    },
    {
      "name": "galileo_guarantee",
      "dataType": "STRING"
    },
    {
      "name": "galileo_deposit",
      "dataType": "STRING"
    },
    {
      "name": "worldspan_deposit_guarantee_prepayment",
      "dataType": "STRING"
    },
    {
      "name": "uapi_responsetime_ms",
      "dataType": "LONG"
    },
    {
      "name": "session_key",
      "dataType": "STRING"
    },
    {
      "name": "agency_country",
      "dataType": "STRING"
    },
    {
      "name": "source_ipaddress",
      "dataType": "STRING"
    },
    {
      "name": "ach_transactionid",
      "dataType": "STRING"
    },
    {
      "name": "db2_locktime",
      "dataType": "LONG"
    },
    {
      "name": "db2_number_of_locks",
      "dataType": "INT"
    },
    {
      "name": "override_pcc",
      "dataType": "STRING"
    },
    {
      "name": "meta_search",
      "dataType": "STRING"
    },
    {
      "name": "solution_result",
      "dataType": "STRING"
    },
    {
      "name": "ptc_list",
      "dataType": "STRING"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "LOG_TS",
      "dataType": "LONG",
      "format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",
      "granularity": "1:MILLISECONDS"
    }
  ],
  "primaryKeyColumns": [
    "log_id"
  ]
}

[root@shldvatlb109 ~]# python3 uapi-pinot5.py 
[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023/11/28 18:18:52.671 ERROR [LaunchDataIngestionJobCommand] [main] Got exception to kick off standalone data ingestion job - 
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:122) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
Caused by: java.lang.NullPointerException
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.getSegmentUriToTarPathMap(SegmentPushUtils.java:366) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.BaseSegmentPushJobRunner.run(BaseSegmentPushJobRunner.java:143) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:150) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	... 13 more
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:122)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171)
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)
Caused by: java.lang.NullPointerException
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.getSegmentUriToTarPathMap(SegmentPushUtils.java:366)
	at org.apache.pinot.plugin.ingestion.batch.common.BaseSegmentPushJobRunner.run(BaseSegmentPushJobRunner.java:143)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:150)
	... 13 more

{
  "OFFLINE": {
    "tableName": "uapitest_OFFLINE",
    "tableType": "OFFLINE",
    "segmentsConfig": {
      "schemaName": "uapitest",
      "replication": "2",
      "replicasPerPartition": "2",
      "timeColumnName": "LOG_TS",
      "segmentPushFrequency": "HOURLY",
      "segmentPushType": "APPEND",
      "minimizeDataMovement": false
    },
    "tenants": {
      "broker": "DefaultTenant",
      "server": "DefaultTenant"
    },
    "tableIndexConfig": {
      "invertedIndexColumns": [],
      "noDictionaryColumns": [],
      "rangeIndexColumns": [],
      "rangeIndexVersion": 2,
      "autoGeneratedInvertedIndex": false,
      "createInvertedIndexDuringSegmentGeneration": false,
      "sortedColumn": [],
      "bloomFilterColumns": [],
      "loadMode": "MMAP",
      "onHeapDictionaryColumns": [],
      "varLengthDictionaryColumns": [],
      "enableDefaultStarTree": false,
      "enableDynamicStarTreeCreation": false,
      "aggregateMetrics": false,
      "nullHandlingEnabled": false,
      "optimizeDictionary": false,
      "optimizeDictionaryForMetrics": false,
      "noDictionarySizeRatioThreshold": 0.85
    },
    "metadata": {},
    "quota": {},
    "routing": {},
    "query": {},
    "ingestionConfig": {
      "continueOnError": false,
      "rowTimeValueCheck": false,
      "segmentTimeValueCheck": true
    },
    "isDimTable": false,
    "tableDataConfig": {
      "recordReaderSpec": {
        "dataFormat": "tar",
        "className": "org.apache.pinot.plugin.inputformat.TarRecordReader",
        "configClassName": "org.apache.pinot.plugin.inputformat.TarRecordReaderConfig"
      },
      "pinotFSSpecs": [
        {
          "scheme": "s3",
          "className": "org.apache.pinot.plugin.filesystem.S3PinotFS",
          "configs": {
            "s3BucketName": "xxxxxx",
            "awsAccessKey": "xxxxx",
            "awsSecretKey": "/xxxxxx",
            "region": "us-east-1"
          }
        }
      ],
      "proxyConfig": {
        "httpProxy": "<http://xxxxx:8080>",
        "httpsProxy": "<http://xxxxx:8080>",
        "ftpProxy": "<http://xxxx:8080>"
      },
      "inputDirURI": "<s3://tvlp-ds-pinot-dev/pinot_segments/>",
      "includeFileNamePattern": "glob:**/*",
      "outputDirURI": "<s3://tvlp-ds-pinot-dev/pinot_segments/>",
      "overwriteOutput": true
    }
  }
}

tar.gz

pushJobSpec:
  pushAttempts: 2
  pushRetryIntervalMillis: 1000

outputDirURI

outputDirURI: '<s3://tvlp-ds-pinot-dev/pinot_segments/2023/02/27/00/>'

outputDirURI

/tmp/uapitest_OFFLINE_2023-11-28-20_2023-11-28-20_9/v3
# ll
total 124
-rw-r--r-- 1 root root  3748 Nov 28 20:30 columns.psf
-rw-r--r-- 1 root root    16 Nov 28 20:30 creation.meta
-rw-r--r-- 1 root root 23039 Nov 28 20:30 index_map
-rw-r--r-- 1 root root 91329 Nov 28 20:30 metadata.properties

outputDirURI: '<s3://tvlp-ds-pinot-dev/pinot_segments/>'

segmentMetadataPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentMetadataPushJobRunner'

executionFrameworkSpec:
  name: 'standalone'
  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
  segmentMetadataPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentMetadataPushJobRunner'
  #  name: 'standalone'
  #  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftSegmentGenerationJobRunner'
  #  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '<s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=01/>'
includeFileNamePattern: 'glob:**/*'
outputDirURI: '<s3://tvlp-ds-pinot-dev/pinot_segments/>'
overwriteOutput: true
pinotFSSpecs:
  - scheme: s3
    className: org.apache.pinot.plugin.filesystem.S3PinotFS
    configs:
      region: us-east-1
recordReaderSpec:
  dataFormat: 'parquet'
  className: 'org.apache.pinot.plugin.inputformat.parquet.ParquetRecordReader'
  configClassName: 'org.apache.pinot.plugin.inputformat.parquet.ParquetRecordReaderConfig'
    #recordTransformerSpec:
    #  transformerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftRecordTransformer'
    #  dataSchema:
    #    dataSource: 'uapi'
    #    granularitySpec:
    #      dataType: 'LONG'
    #      segmentGranularity: 'HOUR'
    #      queryGranularity: 'NONE'
pinotClusterSpecs:
  - controllerURI: '<http://shldvatlb109.tvlport.net:9000>'
tableSpec:
  tableName: 'uapitest_OFFLINE'
    #s3Config:
    #  awsAccessKey: 'xxxx'
    #  awsSecretKey: 'xxxxx'
    #proxyConfig:
    #  httpProxy: 'xxxxx:8080'
    #  httpsProxy: 'xxxxx:8080'
    #  ftpProxy: 'xxxxx:8080'
pushJobSpec:
  pushAttempts: 2
  pushRetryIntervalMillis: 1000

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2023/11/29 15:42:57.864 ERROR [LaunchDataIngestionJobCommand] [main] Got exception to kick off standalone data ingestion job - 
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:122) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:33) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.Command.call(Command.java:29) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.access$1300(CommandLine.java:145) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at picocli.CommandLine.execute(CommandLine.java:2078) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
Caused by: org.apache.pinot.spi.utils.retry.AttemptsExceededException: Operation failed after 2 attempts
	at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:65) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:173) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:106) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner.uploadSegments(SegmentTarPushJobRunner.java:64) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.BaseSegmentPushJobRunner.run(BaseSegmentPushJobRunner.java:148) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:150) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	... 13 more
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:122)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171)
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)
Caused by: org.apache.pinot.spi.utils.retry.AttemptsExceededException: Operation failed after 2 attempts
	at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:65)
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:173)
	at org.apache.pinot.segment.local.utils.SegmentPushUtils.pushSegments(SegmentPushUtils.java:106)
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner.uploadSegments(SegmentTarPushJobRunner.java:64)
	at org.apache.pinot.plugin.ingestion.batch.common.BaseSegmentPushJobRunner.run(BaseSegmentPushJobRunner.java:148)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:150)
	... 13 more

[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.NumberFormatException: empty String
	at jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1842) ~[?:?]
	at jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110) ~[?:?]
	at java.lang.Double.parseDouble(Double.java:543) ~[?:?]
	at org.apache.pinot.common.utils.PinotDataType$11.toDouble(PinotDataType.java:622) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.common.utils.PinotDataType$8.convert(PinotDataType.java:478) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.common.utils.PinotDataType$8.convert(PinotDataType.java:426) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.DataTypeTransformer.transform(DataTypeTransformer.java:106) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.transform(CompositeTransformer.java:95) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.processPlainRow(TransformPipeline.java:97) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.processRow(TransformPipeline.java:92) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.RecordReaderSegmentCreationDataSource.gatherStats(RecordReaderSegmentCreationDataSource.java:67) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.RecordReaderSegmentCreationDataSource.gatherStats(RecordReaderSegmentCreationDataSource.java:37) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:173) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112) ~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265) ~[pinot-batch-ingestion-standalone-1.0.0-shaded.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152)
	at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121)
	at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:130)
	at org.apache.pinot.tools.Command.call(Command.java:33)
	at org.apache.pinot.tools.Command.call(Command.java:29)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171)
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)
Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=01/000053_0>
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Caught exception while transforming data type for column: surcharge
	at org.apache.pinot.segment.local.recordtransformer.DataTypeTransformer.transform(DataTypeTransformer.java:112)
	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.transform(CompositeTransformer.java:95)
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.processPlainRow(TransformPipeline.java:97)
	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.processRow(TransformPipeline.java:92)
	at org.apache.pinot.segment.local.segment.creator.RecordReaderSegmentCreationDataSource.gatherStats(RecordReaderSegmentCreationDataSource.java:67)
	at org.apache.pinot.segment.local.segment.creator.RecordReaderSegmentCreationDataSource.gatherStats(RecordReaderSegmentCreationDataSource.java:37)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:173)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:145)
	at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.init(SegmentIndexCreationDriverImpl.java:101)
	at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:112)
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:265)
	... 5 more
Caused by: java.lang.NumberFormatException: empty String
	at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1842)
	at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
	at java.base/java.lang.Double.parseDouble(Double.java:543)
	at org.apache.pinot.common.utils.PinotDataType$11.toDouble(PinotDataType.java:622)
	at org.apache.pinot.common.utils.PinotDataType$8.convert(PinotDataType.java:478)
	at org.apache.pinot.common.utils.PinotDataType$8.convert(PinotDataType.java:426)
	at org.apache.pinot.segment.local.recordtransformer.DataTypeTransformer.transform(DataTypeTransformer.java:106)
	... 15 more

-11-thread-5] Starting TaskMetricsEmitter with running frequency of 300 seconds.
2023/11/29 23:36:32.927 INFO [BasePeriodicTask] [pool-11-thread-5] [TaskRequestId: auto] Start running task: TaskMetricsEmitter
2023/11/29 23:36:32.931 INFO [BasePeriodicTask] [pool-11-thread-5] [TaskRequestId: auto] Finish running task: TaskMetricsEmitter in 4ms
2023/11/29 23:37:23.149 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:37:49.147 INFO [ControllerResponseFilter] [grizzly-http-server-14] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapitest/schema>, content-type null status code 200 OK
2023/11/29 23:37:49.214 INFO [PinotQueryResource] [grizzly-http-server-12] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:37:49.223 INFO [PinotQueryResource] [grizzly-http-server-12] Query: select * from uapitest limit 10 Time: 9
2023/11/29 23:37:49.223 INFO [ControllerResponseFilter] [grizzly-http-server-12] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:37:50.324 INFO [PinotQueryResource] [grizzly-http-server-17] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:37:50.331 INFO [PinotQueryResource] [grizzly-http-server-17] Query: select * from uapitest limit 10 Time: 7
2023/11/29 23:37:50.331 INFO [ControllerResponseFilter] [grizzly-http-server-17] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:37:51.491 INFO [ControllerResponseFilter] [grizzly-http-server-20] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapi1/schema>, content-type null status code 200 OK
2023/11/29 23:37:51.568 INFO [PinotQueryResource] [grizzly-http-server-9] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:37:51.572 INFO [PinotQueryResource] [grizzly-http-server-9] Query: select * from uapi1 limit 10 Time: 4
2023/11/29 23:37:51.572 INFO [ControllerResponseFilter] [grizzly-http-server-9] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:37:53.176 INFO [ControllerResponseFilter] [grizzly-http-server-13] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapi/schema>, content-type null status code 200 OK
2023/11/29 23:37:53.289 INFO [PinotQueryResource] [grizzly-http-server-5] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:37:53.293 INFO [PinotQueryResource] [grizzly-http-server-5] Query: select * from uapi limit 10 Time: 4
2023/11/29 23:37:53.293 INFO [ControllerResponseFilter] [grizzly-http-server-5] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:38:23.151 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:39:23.154 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:40:22.628 INFO [PeriodicTaskScheduler] [pool-11-thread-6] Starting SegmentStatusChecker with running frequency of 300 seconds.
2023/11/29 23:40:22.628 INFO [BasePeriodicTask] [pool-11-thread-6] [TaskRequestId: auto] Start running task: SegmentStatusChecker
2023/11/29 23:40:22.629 INFO [ControllerPeriodicTask] [pool-11-thread-6] Processing 3 tables in task: SegmentStatusChecker
2023/11/29 23:40:22.634 INFO [ServerTableSizeReader] [pool-11-thread-6] Reading segment sizes from 0 servers for table: uapi_OFFLINE with timeout: 30000ms
2023/11/29 23:40:22.635 INFO [CompletionServiceHelper] [pool-11-thread-6] Finished reading information for table: uapi_OFFLINE
2023/11/29 23:40:22.639 INFO [ServerTableSizeReader] [pool-11-thread-6] Reading segment sizes from 0 servers for table: uapitest_OFFLINE with timeout: 30000ms
2023/11/29 23:40:22.639 INFO [CompletionServiceHelper] [pool-11-thread-6] Finished reading information for table: uapitest_OFFLINE
2023/11/29 23:40:22.644 INFO [ServerTableSizeReader] [pool-11-thread-6] Reading segment sizes from 0 servers for table: uapi1_OFFLINE with timeout: 30000ms
2023/11/29 23:40:22.644 INFO [CompletionServiceHelper] [pool-11-thread-6] Finished reading information for table: uapi1_OFFLINE
2023/11/29 23:40:22.644 INFO [ControllerPeriodicTask] [pool-11-thread-6] Finish processing 3/3 tables in task: SegmentStatusChecker
2023/11/29 23:40:22.644 INFO [BasePeriodicTask] [pool-11-thread-6] [TaskRequestId: auto] Finish running task: SegmentStatusChecker in 16ms
2023/11/29 23:40:23.156 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:41:23.157 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:41:24.503 INFO [ControllerResponseFilter] [grizzly-http-server-19] Handled request from 172.19.0.1 GET <http://shldvatlb109.tvlport.net:9000/tables/uapitest_OFFLINE/schema>, content-type null status code 200 OK
2023/11/29 23:41:24.579 INFO [ControllerResponseFilter] [grizzly-http-server-15] Handled request from 172.19.0.1 GET <http://shldvatlb109.tvlport.net:9000/tables/uapitest_OFFLINE>, content-type null status code 200 OK
2023/11/29 23:41:32.931 INFO [PeriodicTaskScheduler] [pool-11-thread-7] Starting TaskMetricsEmitter with running frequency of 300 seconds.
2023/11/29 23:41:32.931 INFO [BasePeriodicTask] [pool-11-thread-7] [TaskRequestId: auto] Start running task: TaskMetricsEmitter
2023/11/29 23:41:32.934 INFO [BasePeriodicTask] [pool-11-thread-7] [TaskRequestId: auto] Finish running task: TaskMetricsEmitter in 3ms
2023/11/29 23:42:23.158 INFO [LeadControllerUtils] [ControllerLeadershipFetchingThread] Getting Helix leader: 172.19.0.3_9000, Helix version: 1.0.4, mtime: 1699641101367
2023/11/29 23:42:59.685 INFO [ControllerResponseFilter] [grizzly-http-server-22] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapitest/schema>, content-type null status code 200 OK
2023/11/29 23:42:59.773 INFO [PinotQueryResource] [grizzly-http-server-18] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:42:59.779 INFO [PinotQueryResource] [grizzly-http-server-18] Query: select * from uapitest limit 10 Time: 6
2023/11/29 23:42:59.779 INFO [ControllerResponseFilter] [grizzly-http-server-18] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:43:08.169 INFO [ControllerResponseFilter] [grizzly-http-server-16] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapitest/schema>, content-type null status code 200 OK
2023/11/29 23:43:08.234 INFO [PinotQueryResource] [grizzly-http-server-10] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:43:08.240 INFO [PinotQueryResource] [grizzly-http-server-10] Query: select * from uapitest limit 10 Time: 6
2023/11/29 23:43:08.240 INFO [ControllerResponseFilter] [grizzly-http-server-10] Handled request from 172.31.172.141 POST <http://shldvatlb109.tvlport.net:9000/sql>, content-type application/json; charset=UTF-8 status code 200 OK
2023/11/29 23:43:09.659 INFO [ControllerResponseFilter] [grizzly-http-server-21] Handled request from 172.31.172.141 GET <http://shldvatlb109.tvlport.net:9000/tables/uapi1/schema>, content-type null status code 200 OK
2023/11/29 23:43:09.760 INFO [PinotQueryResource] [grizzly-http-server-24] url string passed is : <http://172.19.0.4:8099/query/sql>
2023/11/29 23:43:09.767 INFO [PinotQueryResource] [grizzly-http-server-24] Query: select * from uapi1 limit 10 Time: 8
2023/11/29 23:43:09.767 INFO [Control

Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - <s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=01/000053_0>

executionFrameworkSpec:
  name: 'standalone'
  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
  segmentMetadataPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentMetadataPushJobRunner'
  #  name: 'standalone'
  #  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftSegmentGenerationJobRunner'
  #  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftPushJobRunner'
jobType: SegmentTarPush
inputDirURI: '<s3://tvlp-ds-pinot-dev/uapi-pinot/log_date=2023-02-27/log_hour=01/>'
includeFileNamePattern: 'glob:**/*'
outputDirURI: '<s3://tvlp-ds-pinot-dev/pinot_segments/>'
overwriteOutput: true
pinotFSSpecs:
  - scheme: s3
    className: org.apache.pinot.plugin.filesystem.S3PinotFS
    configs:
      region: us-east-1
recordReaderSpec:
  dataFormat: 'parquet'
  className: 'org.apache.pinot.plugin.inputformat.parquet.ParquetRecordReader'
  configClassName: 'org.apache.pinot.plugin.inputformat.parquet.ParquetRecordReaderConfig'
    #recordTransformerSpec:
    #  transformerClassName: 'org.apache.pinot.plugin.inputformat.thrift.ThriftRecordTransformer'
    #  dataSchema:
    #    dataSource: 'uapi'
    #    granularitySpec:
    #      dataType: 'LONG'
    #      segmentGranularity: 'HOUR'
    #      queryGranularity: 'NONE'
pinotClusterSpecs:
  - controllerURI: '<http://shldvatlb109.tvlport.net:9000>'
tableSpec:
  tableName: 'uapitest_OFFLINE'
    #s3Config:
    #  awsAccessKey: 'xxxx'
    #  awsSecretKey: 'xxxxx'
    #proxyConfig:
    #  httpProxy: 'xxxxx:8080'
    #  httpsProxy: 'xxxxx:8080'
    #  ftpProxy: 'xxxxx:8080'
pushJobSpec:
  pushAttempts: 2
  pushRetryIntervalMillis: 1000

jobType: SegmentTarPush

bash-4.2# cd /opt/pinot/bin && /opt/pinot/bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile /opt/pinot/configs/batch_ingestion_job_spec.yaml
[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

<s3://tvlp-ds-pinot-dev/pinot_segments/>

[root@shldvatlb109 ~]# docker exec -it pinot-server /bin/bash
bash-4.2# vi /opt/pinot/configs/batch_ingestion_job_spec.yaml
bash-4.2# cd /opt/pinot/bin && /opt/pinot/bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile /opt/pinot/configs/batch_ingestion_job_spec.yaml
[0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:gc-pinot-server.log instead.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-orc/pinot-orc-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-clp-log/pinot-clp-log-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-1.0.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-1.0.0-jar-with-dependencies.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
bash-4.2#

jobType: SegmentTarPush

SegmentCreationAndTarPush

jobType: SegmentTarPush

jobType: SegmentTarPush