Oguzhan Mangir
04/10/2021, 9:37 AMserver:
name: server
ports:
netty: 8098
admin: 8097
replicaCount: 1
dataDir: /var/pinot/server/data/index
segmentTarDir: /var/pinot/server/data/segment
persistence:
enabled: true
accessMode: ReadWriteOnce
size: "30Gi"
mountPath: /var/pinot/server/data
storageClass: "alicloud-disk-available"
#storageClass: "ssd"
jvmOpts: "-Xms512M -Xmx1G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:/opt/pinot/gc-pinot-server.log -Dplugins.include=pinot-hdfs"
log4j2ConfFile: /opt/pinot/conf/pinot-server-log4j2.xml
pluginsDir: /opt/pinot/plugins
service:
annotations: {}
clusterIP: ""
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
type: ClusterIP
port: 8098
nodePort: ""
resources: {}
nodeSelector: {}
affinity: {}
tolerations: []
podAnnotations: {}
updateStrategy:
type: RollingUpdate
# Extra configs will be appended to pinot-server.conf file
extra:
configs: |-
pinot.set.instance.id.to.hostname=true
pinot.server.instance.realtime.alloc.offheap=true
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.server.storage.factory.hdfs.hadoop.conf.path=/opt/hadoop/conf/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
here is my server config. I want to use hdfs as deep storage. I put related hdfs jars under the pinot, but server pod not starting when i add the -Dplugins.include=pinot-hdfs
Xiang Fu
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
should have same indentation as previous configs-Dplugins.include=pinot-hdfs
then you may also need to specify all other plugins e.g,
-Dplugins.include=pinot-hdfs,pinot-avro,pinot-parquet
Oguzhan Mangir
04/10/2021, 9:47 AMXiang Fu
-Dplugins.include
Oguzhan Mangir
04/10/2021, 9:53 AMXiang Fu
Oguzhan Mangir
04/10/2021, 10:00 AMXiang Fu
Daniel Lavoie
04/10/2021, 1:35 PM