Jonathan Meyer
04/09/2021, 2:55 PMingestionConfig.transformConfigs
occur before ingestionConfig.filterConfig
? [realtime table]
I'm seeing errors on some messages due to invalid transformations (-> missing field - since this table isn't for this type of event)Brian Olsen
04/10/2021, 2:00 AMBrian Olsen
04/10/2021, 2:00 AMMatt
04/10/2021, 2:55 AMOguzhan Mangir
04/10/2021, 9:37 AMserver:
name: server
ports:
netty: 8098
admin: 8097
replicaCount: 1
dataDir: /var/pinot/server/data/index
segmentTarDir: /var/pinot/server/data/segment
persistence:
enabled: true
accessMode: ReadWriteOnce
size: "30Gi"
mountPath: /var/pinot/server/data
storageClass: "alicloud-disk-available"
#storageClass: "ssd"
jvmOpts: "-Xms512M -Xmx1G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:/opt/pinot/gc-pinot-server.log -Dplugins.include=pinot-hdfs"
log4j2ConfFile: /opt/pinot/conf/pinot-server-log4j2.xml
pluginsDir: /opt/pinot/plugins
service:
annotations: {}
clusterIP: ""
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
type: ClusterIP
port: 8098
nodePort: ""
resources: {}
nodeSelector: {}
affinity: {}
tolerations: []
podAnnotations: {}
updateStrategy:
type: RollingUpdate
# Extra configs will be appended to pinot-server.conf file
extra:
configs: |-
pinot.set.instance.id.to.hostname=true
pinot.server.instance.realtime.alloc.offheap=true
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.server.storage.factory.hdfs.hadoop.conf.path=/opt/hadoop/conf/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
here is my server config. I want to use hdfs as deep storage. I put related hdfs jars under the pinot, but server pod not starting when i add the -Dplugins.include=pinot-hdfs
Akash
04/13/2021, 3:01 PMbin/start-server.sh -zkAddress pinot1.plan:2181,pinot2.plan:2181,pinot3.plan:2181 -configFileName conf/server.conf
Server configs are
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.instance.dataDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/index
pinot.server.instance.segmentTarDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/segmentTar
I am getting following error
2021/04/13 15:00:49.231 INFO [MBeanRegistrar] [Start a Pinot [SERVER]] MBean HelixCallback:Change=MESSAGES_CONTROLLER,Key=PinotCluster.Server_10.10.211.27_8098,Type=PARTICIPANT has been registered.
2021/04/13 15:00:49.232 INFO [MBeanRegistrar] [Start a Pinot [SERVER]] MBean HelixCallback:Change=HEALTH,Key=PinotCluster.Server_10.10.211.27_8098,Type=PARTICIPANT has been registered.
2021/04/13 15:00:49.598 INFO [Reflections] [Start a Pinot [SERVER]] Reflections took 313 ms to scan 1 urls, producing 5 keys and 151 values
2021/04/13 15:00:49.645 ERROR [PinotFSFactory] [Start a Pinot [SERVER]] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs
java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_275]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_275]
at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:80) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:268) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:239) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:220) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:53) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
2021/04/13 15:00:49.650 ERROR [StartServiceManagerCommand] [Start a Pinot [SERVER]] Failed to start a Pinot [SERVER] at 0.513 since launch
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
Caused by: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_275]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_275]
at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:80) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:268) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:239) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:220) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:53) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
... 9 more
Aaron Wishnick
04/13/2021, 3:52 PMselect * from foo limit 10
I get: "message": "MergeResponseError: responses for table: foo from servers: [10.20.67.239_O] got dropped due to data schema inconsistency.",Kevin Vu
04/13/2021, 4:17 PMPhúc Huỳnh
04/14/2021, 3:14 AMRealtimetoOfflines
task in QC env has been stop. but i cannot find any errors log.
• Another the question is the realtime segments will be remove after convert to Offline Segments Table ?Hakob Avjyan
04/14/2021, 2:25 PMAaron Wishnick
04/14/2021, 4:44 PMLaxman Ch
04/16/2021, 7:56 AMwget -qO temp.zip <https://jitpack.io/com/github/apache/incubator-pinot/pinot-distribution/e22be7c3a39e840321d3658e7505f21768b228d6/pinot-distribution-e22be7c3a39e840321d3658e7505f21768b228d6-shaded.jar>
However, I can see the shaded artifact is available under maven central here.
https://repo1.maven.org/maven2/org/apache/pinot/pinot-distribution/0.7.1/
Any suggestions on how to proceed here?
@Kishore G @Xiang FuJonathan Meyer
04/16/2021, 9:11 AM[
{
"errorCode": 500,
"message": "MergeResponseError:\nData schema mismatch between merged block: [eventTimeString(STRING),communityId(STRING),eventType(STRING),ibcustomer(STRING),newsId(STRING),timeString(STRING),userId(STRING)] and block to merge: [eventTimeString(STRING),communityId(STRING),ibcustomer(STRING),newsId(STRING),timeString(STRING),type(STRING),userId(STRING)], drop block to merge"
}
]
I'm aware of pinot.server.instance.reload.consumingSegment
setting - can you confirm setting it to true
will solve this problem ?
If so, maybe the error message should contain this bit of information ? or a reference to the doc ?Phúc Huỳnh
04/16/2021, 11:00 AMcopy
function ?Jonathan Meyer
04/16/2021, 3:57 PMQuery Console
Tab
Problem :
• I click on a table, the schema appears below it - the schema is valid (the one corresponding to this table)
• I then click on the other table, the schema of the previous table remains
• If I reload the page and click on the other table first, its schema remains after I click on the other table
Info:
• The schemas really are different
• Actually the API calls when inspecting the page return the correct schemas
• So it seems it only is a UI-related issue
• I don't think I've seen this issue before
• Force reloading Firefox page (ignore cache) doesn't solve the issue
Ultimately it's only a UI inconsistency, so not a big deal, but got me thinking about my schemas for a bit 😄Hakob Avjyan
04/16/2021, 5:40 PMApache Pino Error
{'errorCode': 410, 'message': 'BrokerResourceMissingError'}
This may be triggered by:
Issue 1002 - The database returned an unexpected error
This is what I got from superset documentation
Your query failed because of an error that occurred on the database. This may be due to a syntax error, a bug in your query, or some other internal failure within the database. This is usually not an issue within Superset, but instead a problem with the underlying database that serves your query.
What's the best fix for this?Gabriel Lucano
04/16/2021, 8:51 PMSurendra
04/16/2021, 11:38 PM2021/04/16 16:59:54.827 WARN [TimeRetentionStrategy] [pool-10-thread-5] REALTIME segment: <>__0__982__20210312T0906Z of table: <>_REALTIME has invalid end time: -9223372036854775808 MILLISECONDS
2021/04/16 16:59:54.827 WARN [TimeRetentionStrategy] [pool-10-thread-5] REALTIME segment: <>__0__983__20210312T0936Z of table: <>_REALTIME has invalid end time: -9223372036854775808 MILLISECONDS
2021/04/16 16:59:54.827 WARN [TimeRetentionStrategy] [pool-10-thread-5] REALTIME segment: <>__0__984__20210312T1006Z of table: <>_REALTIME has invalid end time: -9223372036854775808 MILLISECONDS
Oguzhan Mangir
04/18/2021, 7:07 PMRavikumar Maddi
04/19/2021, 6:52 AMRavikumar Maddi
04/19/2021, 2:32 PMbin/pinot-ingestion-job.sh -jobSpec examples/batch/airlineStats/ingestionJobSpec.yaml
The error:
ERROR [LaunchDataIngestionJobCommand] [main] Error: "-jobSpec" is not a valid option
Need your help...Nikhil Sulegaon
04/19/2021, 6:28 PMOFFLINE
table in our Pinot cluster has a lot of BAD
segments. Is there a list of common reasons that might make a segment go BAD
?Phúc Huỳnh
04/20/2021, 4:40 AM21/04/20 03:03:42 ERROR org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand: Got exception to kick off standalone data ingestion job -
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.spark.SparkSegmentGenerationJobRunner
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:144)
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:117)
at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:132)
at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.main(LaunchDataIngestionJobCommand.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at <http://org.apache.spark.deploy.SparkSubmit.org|org.apache.spark.deploy.SparkSubmit.org>$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.nio.file.FileSystemNotFoundException: Provider "gs" not installed
at java.nio.file.Paths.get(Paths.java:147)
at org.apache.pinot.plugin.filesystem.GcsPinotFS.copy(GcsPinotFS.java:262)
at org.apache.pinot.plugin.ingestion.batch.spark.SparkSegmentGenerationJobRunner.run(SparkSegmentGenerationJobRunner.java:344)
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:142)
... 15 more
Akash
04/20/2021, 5:23 PMNeil Teng
04/20/2021, 5:23 PMElon
04/21/2021, 2:43 AMMohamed Sultan
04/21/2021, 3:12 PMAshish
04/21/2021, 9:29 PMRavikumar Maddi
04/22/2021, 9:35 AMElon
04/22/2021, 6:39 PM