Jonathan Meyer
06/23/2021, 10:24 AMID1.parquet -> prefix_ID1.segment
Anyway to make this work using segmentNameGeneratorSpec.type
?
Maybe using a particular file structure like data/ID/file.parquet
?
Thanks !sp
06/23/2021, 4:26 PMUnrecognized VM option 'PrintGCDateStamps'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Sheetal
06/24/2021, 1:15 AMRK
06/24/2021, 11:42 AMRK
06/24/2021, 11:44 AMSyed Akram
06/24/2021, 1:34 PMRK
06/28/2021, 1:13 AMRK
06/29/2021, 5:16 PMJonathan Meyer
06/30/2021, 1:58 PMElon
06/30/2021, 3:47 PMUnexpected response status: 500 for request {"sql" : "select AAA, BBB from CCC limit 2147483647" } to url <http://XXXX>:XXXX/query/sql, with headers {Accept=[application/json], Content-Type=[application/json]}, full response null
Mohamed Sultan
07/01/2021, 7:09 AMJonathan Meyer
07/01/2021, 3:22 PM.Values.{controller|broker}.external.enabled
set to true
by default ?
Maybe having it to false
would be a safer alternative for first time users not knowing the chart well enough yetJonathan Meyer
07/01/2021, 6:03 PMjainendra tarun
07/01/2021, 7:48 PMJonathan Meyer
07/02/2021, 3:44 PM/segments/{tableName}/metadata
controller API endpoint only returns metadata of one segment (a random one, it seems) and not of all of them ?
The description states Get the server metadata for all table segments
Fritz Budiyanto
07/03/2021, 6:59 PMCluster structure is not set up for cluster: pinot-quickstart
What went wrong here ?
cd incubator-pinot/kubernetes/helm/pinot
helm install -n pinot-quickstart2 pinot .
Session establishment complete on server pinot-zookeeper/172.20.47.166:2181, sessionid = 0x1012fddef2b0003, negotiated timeout = 30000
zookeeper state changed (SyncConnected)
MBean HelixZkClient:Key=pinot-quickstart.Broker_pinot-broker-0.pinot-broker-headless.pinot-quickstart2.svc.cluster.local_8099,Type=SPECTATOR has been registered.
MBean HelixZkClient:Key=pinot-quickstart.Broker_pinot-broker-0.pinot-broker-headless.pinot-quickstart2.svc.cluster.local_8099,PATH=Root,Type=SPECTATOR has been registered.
KeeperState: SyncConnected, instance: Broker_pinot-broker-0.pinot-broker-headless.pinot-quickstart2.svc.cluster.local_8099, type: SPECTATOR
Handle new session, instance: Broker_pinot-broker-0.pinot-broker-headless.pinot-quickstart2.svc.cluster.local_8099, type: SPECTATOR
Handling new session, session id: 1012fddef2b0003, instance: Broker_pinot-broker-0.pinot-broker-headless.pinot-quickstart2.svc.cluster.local_8099, instanceTye: SPECTATOR, cluster: pinot-quickstart
fail to createClient.
org.apache.helix.HelixException: Cluster structure is not set up for cluster: pinot-quickstart
at org.apache.helix.manager.zk.ZKHelixManager.handleNewSession(ZKHelixManager.java:1124) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-47a75e5093129cc280de4c118434ccb337cd3da1]
Azri Jamil
07/04/2021, 5:31 AMexecutionFrameworkSpec:
name: 'standalone'
segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndUriPush
inputDirURI: '<gs://mdm-datalake/ais/sentences/>'
outputDirURI: '/tmp/ais-pinot/sentences/'
includeFileNamePattern: 'glob:**/**.parquet'
overwriteOutput: true
pinotFSSpecs:
- scheme: file
className: org.apache.pinot.spi.filesystem.LocalPinotFS
- scheme: gs
className: org.apache.pinot.plugin.filesystem.GcsPinotFS
configs:
projectId: 'aton-analytics'
gcpKey: '/var/pinot/controller/config/gcs-datalake-key.json'
recordReaderSpec:
dataFormat: 'parquet'
className: 'org.apache.pinot.plugin.inputformat.parquet.ParquetRecordReader'
tableSpec:
tableName: 'sentence'
pinotClusterSpecs:
- controllerURI: '<http://localhost:9000>'
Jackie
07/05/2021, 4:40 PMPrashant Pandey
07/06/2021, 8:39 AMjainendra tarun
07/07/2021, 2:45 PMCharles
07/08/2021, 3:11 AMRuslan Rodriquez
07/08/2021, 10:41 AMDeepak Mishra
07/08/2021, 11:53 AMCarlos Domínguez
07/08/2021, 9:41 PMKamal Chavda
07/10/2021, 6:15 PMExecuting command: StartController -configFileName /tmp/controller.conf
Caught exception while starting controller, exiting.
java.lang.RuntimeException: Error: Unable to find controller config file /tmp/controller.conf
I've created the controller.conf file and placed it in the /tmp/pinot-s3-docker folder on my EC2 instance. Any help would be greatly appreciated.Zsolt Takacs
07/12/2021, 11:18 AM"completionMode": "DOWNLOAD"
. The download failed three times on the replica server, and the segment is stuck in error state. If I try to reload it the log says it's reloading a consuming segment, but nothing changes. How can I recover from this state?Kamal Chavda
07/12/2021, 6:04 PM"name": "created_date",
"dataType": "STRING",
"format" : "1:MILLISECONDS:SIMPLE_DATE_FORMAT:YYYY-MM-dd HH24:MI:<http://SS.MS|SS.MS>",
"granularity": "1:MILLISECONDS"
example from csv file: 2020-03-01 073108.792457. I keep on getting failed to generate pinot segment and java illegal argument exception error
java.lang.IllegalArgumentException: Illegal pattern component: I
at org.joda.time.format.DateTimeFormat.parsePatternTo(DateTimeFormat.java:566) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.joda.time.format.DateTimeFormat.createFormatterForPattern(DateTimeFormat.java:687) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.joda.time.format.DateTimeFormat.forPattern(DateTimeFormat.java:177) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.spi.data.DateTimeFormatPatternSpec.<init>(DateTimeFormatPatternSpec.java:57) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.spi.data.DateTimeFormatSpec.<init>(DateTimeFormatSpec.java:59) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.core.indexsegment.generator.SegmentGeneratorConfig.setTime(SegmentGeneratorConfig.java:212) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.core.indexsegment.generator.SegmentGeneratorConfig.<init>(SegmentGeneratorConfig.java:138) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:95) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$run$0(SegmentGenerationJobRunner.java:199) ~[pinot-batch-ingestion-standalone-0.7.1-shaded.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
Has anyone run into this issue? Do I need to convert the dates to EPOCH when generating CSV?Bruce Ritchie
07/12/2021, 6:13 PMPrashant Pandey
07/12/2021, 7:31 PM