https://pinot.apache.org/ logo
Channels
Powered by
a

Akash

04/20/2021, 5:23 PM
With respect to this, how do we specify the jar location and what jar we need to supply ?
m

Mayank

04/20/2021, 5:23 PM
@Xiang Fu ^^
a

Akash

04/20/2021, 5:25 PM
Copy code
export HADOOP_HOME=/local/hadoop/
export HADOOP_CONF_DIR=/local/hadoop/etc/hadoop/
export JAVA_OPTS="-Xms12G -Xmx28G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/plugins/ -Dlog4j2.configurationFile=file:/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/conf/pinot-server-log4j2.xml"
/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/bin/start-server.sh  -zkAddress hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181 -configFileName /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/conf/server.conf
Server Conf
Copy code
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.server.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.instance.dataDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/data/PinotServer/index
pinot.server.instance.segmentTarDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/data/PinotServer/segmentTar
Error which I am getting
Copy code
2021/04/20 17:19:31.724 ERROR [StartServiceManagerCommand] [Start a Pinot [SERVER]] Failed to start a Pinot [SERVER] at 3.081 since launch
java.lang.RuntimeException: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:336) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
Caused by: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:71) ~[pinot-hdfs-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	... 9 more
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) ~[pinot-parquet-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67) ~[pinot-hdfs-0.7.0-SNAPSHOT-shaded.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e]
	... 9 more
x

Xiang Fu

04/20/2021, 5:31 PM
can you try to put to put hadoop jar into your java classpath ?
or put it into pinot plugin directory
a

Akash

04/20/2021, 5:32 PM
which specific jar ?
hadoop-common ?
x

Xiang Fu

04/20/2021, 5:32 PM
hadoop common
a

Akash

04/20/2021, 5:32 PM
There are lots of jars
Cool. Thanks.
m

Mayank

04/20/2021, 5:32 PM
Thanks @Xiang Fu.
@Akash Could you please confirm if this solves the issue, we can add that into Faq.
a

Akash

04/20/2021, 6:05 PM
The server script is quitting now.
Copy code
2021/04/20 18:04:01.026 INFO [HelixInstanceDataManagerConfig] [Start a Pinot [SERVER]] InstanceDataManagerConfig, key: reload.consumingsegment , value: false
2021/04/20 18:04:01.026 INFO [HelixInstanceDataManagerConfig] [Start a Pinot [SERVER]] InstanceDataManagerConfig, key: id , value: Server_10.10.211.27_8098
2021/04/20 18:04:01.026 INFO [HelixInstanceDataManagerConfig] [Start a Pinot [SERVER]] InstanceDataManagerConfig, key: readmode , value: mmap
2021/04/20 18:04:01.027 INFO [HelixInstanceDataManager] [Start a Pinot [SERVER]] HelixInstanceDataManagerConfig: Instance Id: Server_10.10.211.27_8098
	Instance Data Dir: /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/data/PinotServer/index
	Instance Segment Tar Dir: /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/data/PinotServer/segmentTar
	Bootstrap Segment Dir: null
	Read Mode: mmap
	Segment format version: null
2021/04/20 18:04:01.029 INFO [SegmentBuildTimeLeaseExtender] [Start a Pinot [SERVER]] Initialized segment build time lease extender executor
2021/04/20 18:04:01.030 INFO [HelixInstanceDataManager] [Start a Pinot [SERVER]] Initialized Helix instance data manager
2021/04/20 18:04:02.382 INFO [FunctionRegistry] [Start a Pinot [SERVER]] Initialized FunctionRegistry with 119 functions: [fromepochminutesbucket, arrayunionint, codepoint, mod, sha256, year, yearofweek, upper, arraycontainsstring, arraydistinctstring, bytestohex, tojsonmapstr, trim, timezoneminute, sqrt, togeometry, normalize, fromepochdays, arraydistinctint, exp, jsonpathlong, yow, toepochhoursrounded, lower, toutf8, concat, ceil, todatetime, jsonpathstring, substr, dayofyear, contains, jsonpatharray, arrayindexofint, fromepochhoursbucket, arrayindexofstring, minus, arrayunionstring, toepochhours, toepochdaysrounded, millisecond, fromepochhours, arrayreversestring, dow, doy, min, toepochsecondsrounded, strpos, jsonpath, tosphericalgeography, fromepochsecondsbucket, max, reverse, hammingdistance, stpoint, abs, timezonehour, toepochseconds, arrayconcatint, quarter, md5, ln, toepochminutes, arraysortstring, replace, strrpos, jsonpathdouble, stastext, second, arraysortint, split, fromepochdaysbucket, lpad, day, toepochminutesrounded, fromdatetime, fromepochseconds, arrayconcatstring, base64encode, ltrim, arraysliceint, chr, sha, plus, base64decode, month, arraycontainsint, toepochminutesbucket, startswith, week, jsonformat, sha512, arrayslicestring, fromepochminutes, remove, dayofmonth, times, hour, rpad, arrayremovestring, now, divide, bigdecimaltobytes, floor, toepochsecondsbucket, toepochdaysbucket, hextobytes, rtrim, length, toepochhoursbucket, bytestobigdecimal, toepochdays, arrayreverseint, datetrunc, minute, round, dayofweek, arrayremoveint, weekofyear] in 1352ms
2021/04/20 18:04:02.383 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing query executor of class: org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl
2021/04/20 18:04:02.387 INFO [ServerQueryExecutorV1Impl] [Start a Pinot [SERVER]] Default timeout for query executor : 15000
2021/04/20 18:04:02.387 INFO [ServerQueryExecutorV1Impl] [Start a Pinot [SERVER]] Trying to build SegmentPrunerService
2021/04/20 18:04:02.389 INFO [SegmentPrunerService] [Start a Pinot [SERVER]] Adding segment pruner: DataSchemaSegmentPruner
2021/04/20 18:04:02.389 INFO [SegmentPrunerService] [Start a Pinot [SERVER]] Adding segment pruner: ColumnValueSegmentPruner
2021/04/20 18:04:02.389 INFO [SegmentPrunerService] [Start a Pinot [SERVER]] Adding segment pruner: SelectionQuerySegmentPruner
2021/04/20 18:04:02.389 INFO [ServerQueryExecutorV1Impl] [Start a Pinot [SERVER]] Trying to build QueryPlanMaker
2021/04/20 18:04:02.390 INFO [InstancePlanMakerImplV2] [Start a Pinot [SERVER]] Initializing plan maker with maxInitialResultHolderCapacity: 10000, numGroupsLimit: 100000
2021/04/20 18:04:02.390 INFO [ServerQueryExecutorV1Impl] [Start a Pinot [SERVER]] Trying to build QueryExecutorTimer
2021/04/20 18:04:02.390 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing query scheduler
2021/04/20 18:04:02.393 INFO [QuerySchedulerFactory] [Start a Pinot [SERVER]] Using FCFS query scheduler
2021/04/20 18:04:02.394 INFO [ResourceManager] [Start a Pinot [SERVER]] Initializing with 48 query runner threads and 96 worker threads
2021/04/20 18:04:02.399 INFO [QueryScheduler] [Start a Pinot [SERVER]] Query log max rate: 10000.0
2021/04/20 18:04:02.399 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing Netty query server on port: 8098
2021/04/20 18:04:02.400 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing transform functions
2021/04/20 18:04:02.410 INFO [ServerInstance] [Start a Pinot [SERVER]] Finish initializing server instance
2021/04/20 18:04:02.413 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Did not find any fs classes in the configuration
2021/04/20 18:04:02.414 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Got scheme hdfs, initializing class org.apache.pinot.plugin.filesystem.HadoopPinotFS
2021/04/20 18:04:02.414 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Initializing PinotFS for scheme hdfs, classname org.apache.pinot.plugin.filesystem.HadoopPinotFS
Error is gone
@Xiang Fu
Copy code
/usr/bin/java -Xms12G -Xmx28G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/plugins/ -Dlog4j2.configurationFile=file:/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/conf/pinot-server-log4j2.xml -cp /local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/plugins -classpath /local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar:/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/lib/pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar -Dapp.name=start-server -Dapp.pid=51665 -Dapp.repo=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/lib -Dapp.home=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin -Dbasedir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin org.apache.pinot.tools.admin.PinotServer -zkAddress hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181 -configFileName /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/conf/server.conf
I am directly running this command but the App is existing right away
Quitting*
Copy code
2021/04/20 18:42:41.346 INFO [InstancePlanMakerImplV2] [Start a Pinot [SERVER]] Initializing plan maker with maxInitialResultHolderCapacity: 10000, numGroupsLimit: 100000
2021/04/20 18:42:41.346 INFO [ServerQueryExecutorV1Impl] [Start a Pinot [SERVER]] Trying to build QueryExecutorTimer
2021/04/20 18:42:41.346 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing query scheduler
2021/04/20 18:42:41.348 INFO [QuerySchedulerFactory] [Start a Pinot [SERVER]] Using FCFS query scheduler
2021/04/20 18:42:41.349 INFO [ResourceManager] [Start a Pinot [SERVER]] Initializing with 48 query runner threads and 96 worker threads
2021/04/20 18:42:41.353 INFO [QueryScheduler] [Start a Pinot [SERVER]] Query log max rate: 10000.0
2021/04/20 18:42:41.354 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing Netty query server on port: 8098
2021/04/20 18:42:41.355 INFO [ServerInstance] [Start a Pinot [SERVER]] Initializing transform functions
2021/04/20 18:42:41.364 INFO [ServerInstance] [Start a Pinot [SERVER]] Finish initializing server instance
2021/04/20 18:42:41.367 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Did not find any fs classes in the configuration
2021/04/20 18:42:41.367 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Got scheme hdfs, initializing class org.apache.pinot.plugin.filesystem.HadoopPinotFS
2021/04/20 18:42:41.367 INFO [PinotFSFactory] [Start a Pinot [SERVER]] Initializing PinotFS for scheme hdfs, classname org.apache.pinot.plugin.filesystem.HadoopPinotFS
After this
x

Xiang Fu

04/20/2021, 6:47 PM
you mean the server exits after this?
a

Akash

04/20/2021, 6:47 PM
Yes
x

Xiang Fu

04/20/2021, 6:47 PM
hmm
a

Akash

04/20/2021, 6:47 PM
No error
It just stops.
x

Xiang Fu

04/20/2021, 6:47 PM
is there console output or pinot-server-logs for the reason?
a

Akash

04/20/2021, 6:50 PM
The above is console output.
I will add some more logs at the place its quiting.
x

Xiang Fu

04/20/2021, 6:50 PM
sure
is there any log files on the working directory?
a

Akash

04/20/2021, 6:52 PM
Copy code
2021/04/20 18:43:58.393 INFO [MBeanRegistrar] [Start a Pinot [SERVER]] MBean HelixCallback:Change=HEALTH,Key=PinotCluster.Server_10.10.211.27_8098,Type=PARTICIPANT has been registered.
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-protobuf 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-kafka-2.0 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-avro 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-distribution 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-csv 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-s3 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-segment-uploader-default 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-yammer 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-batch-ingestion-standalone 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.402 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-confluent-avro 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-thrift 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-orc 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-batch-ingestion-spark 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-gcs 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-hdfs 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-batch-ingestion-hadoop 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-adls 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-json 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-minion-builtin-tasks 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-parquet 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.403 INFO [Utils] [Start a Pinot [SERVER]] Using pinot-segment-writer-file-based 0.7.0-SNAPSHOT-b9b31e5d2ecdc954934cdc1143d6f729b9f22c0e
2021/04/20 18:43:58.408 INFO [DefaultHelixStarterServerConfig] [Start a Pinot [SERVER]] External config key: pinot.server.segment.fetcher.hdfs.class, value: org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
2021/04/20 18:43:58.408 INFO [DefaultHelixStarterServerConfig] [Start a Pinot [SERVER]] External config key: pinot.server.segment.fetcher.protocols, value: file,http,hdfs
2021/04/20 18:43:58.408 INFO [DefaultHelixStarterServerConfig] [Start a Pinot [SERVER]] External config key: pinot.server.instance.enable.split.commit, value: true
2021/04/20 18:43:58.408 INFO [DefaultHelixStarterServerConfig] [Start a Pinot [SERVER]] E
x

Xiang Fu

04/20/2021, 6:52 PM
from the starter cmd, there seems to be some redundent:
Copy code
-cp /local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.0-SNAPSHOT-bin/plugins
a

Akash

04/21/2021, 4:42 PM
This is still happening again and agin.
Server is quitting.
Copy code
org.apache.hadoop.conf.Configuration hadoopConf = new org.apache.hadoop.conf.Configuration();
On this specific line.
x

Xiang Fu

04/21/2021, 6:28 PM
hmmm, do you mean this line failed the server?
a

Akash

04/21/2021, 6:28 PM
Yes. Its very strange.
x

Xiang Fu

04/21/2021, 6:28 PM
@Ting Chen can you help check what’s your setup
a

Akash

04/21/2021, 6:30 PM
This happens only when i add hadoop-common jar in classpath.
Installation works fine without specifying HDFS as deep storage.
x

Xiang Fu

04/21/2021, 6:31 PM
yeah, i don’t know what could be an issue, we’ve used this in Uber for multiple years
a

Akash

04/21/2021, 6:31 PM
How are you passing hadoop-common jar dependency in your installation ?
x

Xiang Fu

04/21/2021, 6:32 PM
either in classpath or put it into pinot-hdfs plugin
a

Akash

04/21/2021, 6:33 PM
only hadoop-common right or any other hadoop dependency is also required ?
x

Xiang Fu

04/21/2021, 6:37 PM
hmm, can you try to add more jars
hadoop-auth-2.7.1.jar hadoop-mapreduce-client-core-2.7.1.jar hadoop-common-2.7.1.jar
a

Akash

04/21/2021, 6:38 PM
Sure. Will do.
x

Xiang Fu

04/21/2021, 6:39 PM
Another way is to add your hadoop classpath to your java classpath
so it will load all the hadoop related jars
a

Akash

04/21/2021, 7:28 PM
I have added lots of jar on plugin dir but Pinot is failing to load it.
Copy code
2021/04/21 19:26:47.763 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-mapreduce-client-core-2.7.0.jar]
2021/04/21 19:26:47.764 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-common-2.7.0.jar]
2021/04/21 19:26:47.764 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-auth-2.7.0.jar]
2021/04/21 19:26:47.764 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/pinot-hdfs-0.7.1-shaded.jar]
2021/04/21 19:26:47.764 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-hdfs-2.7.0.jar]
2021/04/21 19:26:47.764 ERROR [PluginManager] [main] Failed to load plugin [pinot-hdfs] from dir [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs]
java.lang.ClassCastException: java.net.URL cannot be cast to java.lang.Comparable
	at java.util.ComparableTimSort.countRunAndMakeAscending(ComparableTimSort.java:320) ~[?:1.8.0_275]
	at java.util.ComparableTimSort.sort(ComparableTimSort.java:188) ~[?:1.8.0_275]
	at java.util.Arrays.sort(Arrays.java:1246) ~[?:1.8.0_275]
	at org.apache.pinot.spi.plugin.PluginManager.createClassLoader(PluginManager.java:195) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.load(PluginManager.java:187) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:157) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:123) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.<init>(PluginManager.java:104) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.<clinit>(PluginManager.java:46) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:182) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.tools.admin.PinotBroker.main(PinotBroker.java:35) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
2021/04/21 19:26:47.769 INFO [PluginManager] [main] Trying to load plugin [pinot-hdfs] from location [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs]
2021/04/21 19:26:47.769 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-mapreduce-client-core-2.7.0.jar]
2021/04/21 19:26:47.769 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-common-2.7.0.jar]
2021/04/21 19:26:47.769 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-auth-2.7.0.jar]
2021/04/21 19:26:47.769 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/pinot-hdfs-0.7.1-shaded.jar]
2021/04/21 19:26:47.770 INFO [PluginManager] [main] Successfully loaded plugin [pinot-hdfs] from jar file [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/hadoop-hdfs-2.7.0.jar]
2021/04/21 19:26:47.770 ERROR [PluginManager] [main] Failed to load plugin [pinot-hdfs] from dir [/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs]
java.lang.ClassCastException: java.net.URL cannot be cast to java.lang.Comparable
	at java.util.ComparableTimSort.countRunAndMakeAscending(ComparableTimSort.java:320) ~[?:1.8.0_275]
	at java.util.ComparableTimSort.sort(ComparableTimSort.java:188) ~[?:1.8.0_275]
	at java.util.Arrays.sort(Arrays.java:1246) ~[?:1.8.0_275]
	at org.apache.pinot.spi.plugin.PluginManager.createClassLoader(PluginManager.java:195) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.load(PluginManager.java:187) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:157) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:123) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.<init>(PluginManager.java:104) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.spi.plugin.PluginManager.<clinit>(PluginManager.java:46) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:182) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
	at org.apache.pinot.tools.admin.PinotBroker.main(PinotBroker.java:35) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
Copy code
[akashmishra@hotpanel1.mlan ~]$ ls -al hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/pinot-file-system/pinot-hdfs/
total 12980
drwxrwxr-x 2 akashmishra wwwaccess    4096 Apr 21 18:46 .
drwxrwxr-x 6 akashmishra wwwaccess    4096 Apr 21 15:43 ..
-rw-r--r-- 1 akashmishra wwwaccess   70600 Apr 21 18:46 hadoop-auth-2.7.0.jar
-rw-r--r-- 1 akashmishra wwwaccess 3425940 Apr 21 18:36 hadoop-common-2.7.0.jar
-rw-r--r-- 1 akashmishra wwwaccess 8235347 Apr 21 18:43 hadoop-hdfs-2.7.0.jar
-rw-r--r-- 1 akashmishra wwwaccess 1528797 Apr 21 18:46 hadoop-mapreduce-client-core-2.7.0.jar
-rw-r--r-- 1 akashmishra wwwaccess    9016 Apr 21 14:06 pinot-hdfs-0.7.1-shaded.jar
This is the content of plugin dir
x

Xiang Fu

04/21/2021, 7:31 PM
can you try to put them into java classpath
a

Akash

04/21/2021, 7:44 PM
Copy code
/usr/bin/java -Xms12G -Xmx28G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/ -Dlog4j2.configurationFile=file:/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/pinot-server-log4j2.xml -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins -classpath /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/lib/*:/home/akashmishra/hadooplib/hadoop-auth-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-common-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-hdfs-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-mapreduce-client-core-2.7.0.jar -Dapp.name=start-server -Dapp.pid=3319822 -Dapp.repo=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/lib -Dapp.home=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin -Dbasedir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin org.apache.pinot.tools.admin.PinotServer -zkAddress hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181 -configFileName /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/server.conf
Running this command right now. Its started quitting again at the same place
x

Xiang Fu

04/21/2021, 7:45 PM
hmm, ok, I will take a look
is it jdk 8?
a

Akash

04/21/2021, 7:46 PM
yes
Copy code
[akashmishra@hotpanel1.mlan ~]$ java -version
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-b01)
OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)
x

Xiang Fu

04/21/2021, 7:46 PM
what’s your controller conf
it’s also failing right?
a

Akash

04/21/2021, 7:50 PM
Copy code
controller.data.dir=<hdfs://bihadoop/tmp/users/akashmishra/pinot/data/>
controller.local.temp.dir=<hdfs://bihadooop/tmp/users/akashmishra/pinot/tmp/>
controller.zk.str=hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181
controller.enable.split.commit=true
controller.access.protocols.http.port=9000
controller.helix.cluster.name=PinotController
pinot.controller.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.controller.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.controller.segment.fetcher.protocols=file,http,hdfs
pinot.controller.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
Server conf
Copy code
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.server.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.instance.dataDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/index
pinot.server.instance.segmentTarDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/segmentTar
Copy code
/usr/bin/java -Xms8G -Xmx12G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-controller.log -Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/plugins/ -Dlog4j2.configurationFile=file:/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/pinot-controller-log4j2.xml -classpath /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/lib/*:/home/akashmishra/hadooplib/hadoop-auth-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-common-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-hdfs-2.7.0.jar:/home/akashmishra/hadooplib/hadoop-mapreduce-client-core-2.7.0.jar -Dapp.name=start-controller -Dapp.pid=3339741 -Dapp.repo=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/lib -Dapp.home=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin -Dbasedir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin org.apache.pinot.tools.admin.PinotController -configFileName /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/controller.conf
This controller command
x

Xiang Fu

04/21/2021, 9:07 PM
i tired it on my mac and it works fine
what’s your os?
a

Akash

04/21/2021, 9:15 PM
This is crazy, I have added following code in HadoopPinotFS
Copy code
try {
      <http://LOGGER.info|LOGGER.info>("Getting Class org.apache.hadoop.conf.Configuration");
      Class<?> clazz = Class.forName("org.apache.hadoop.conf.Configuration");
      <http://LOGGER.info|LOGGER.info>("Building object");
      Object conf = clazz.newInstance();
    } catch (Exception e) {
      LOGGER.error("Failed instantiating org.apache.hadoop.conf.Configuration " );
      LOGGER.error("Exception is ", e);
    }
    Configuration hadoopConf = new Configuration();
    <http://LOGGER.info|LOGGER.info>("Configuration loaded");
and it stops here
Copy code
2021/04/21 21:13:49.444 INFO [HadoopPinotFS] [Start a Pinot [SERVER]] Starting to initialized HadoopPinotFS
2021/04/21 21:13:49.444 INFO [HadoopPinotFS] [Start a Pinot [SERVER]] New Config path is  /local/hadoop/etc/hadoop/
2021/04/21 21:13:49.444 INFO [HadoopPinotFS] [Start a Pinot [SERVER]] Getting Class org.apache.hadoop.conf.Configuration
[akashmishra@hotpanel1.mlan ~]$
Somehow org.apache.hadoop.conf.Configuration is creating some issues when running code.
Copy code
NAME="CentOS Linux"
VERSION="8"
x

Xiang Fu

04/21/2021, 9:17 PM
what’s inside: /local/hadoop/etc/hadoop/
a

Akash

04/21/2021, 9:18 PM
Copy code
[akashmishra@hotpanel1.mlan ~]$ ls -al /local/hadoop/etc/hadoop/
total 188
drwxr-xr-x 2 root root  4096 Apr 12 22:09 .
drwxr-xr-x 3 root root  4096 Apr 12 22:07 ..
-rw-r--r-- 1 root root  4437 Apr 12 22:09 capacity-scheduler.xml
-rw-r--r-- 1 root root  1335 Jan 27 18:28 configuration.xsl
-rw-r--r-- 1 root root   318 Jan 27 18:28 container-executor.cfg
-rw-r--r-- 1 root root   873 Apr 12 22:09 core-site.xml
-rw-r--r-- 1 root root    29 Apr 12 22:09 dfs.exclude
-rw-r--r-- 1 root root  4482 Apr 12 22:09 fair-scheduler.xml
-rw-r--r-- 1 root root  3670 Jan 27 18:28 hadoop-env.cmd
-rw-r--r-- 1 root root  4197 Apr 12 22:09 hadoop-env.sh
-rw-r--r-- 1 root root  2598 Jan 27 18:28 hadoop-metrics2.properties
-rw-r--r-- 1 root root  2490 Jan 27 18:28 hadoop-metrics.properties
-rw-r--r-- 1 root root  9683 Jan 27 18:28 hadoop-policy.xml
-rw-r--r-- 1 root root  6451 Apr 12 22:09 hdfs-site.xml
-rw-r--r-- 1 root root  1449 Jan 27 18:28 httpfs-env.sh
-rw-r--r-- 1 root root  1657 Jan 27 18:28 httpfs-log4j.properties
-rw-r--r-- 1 root root    21 Jan 27 18:28 httpfs-signature.secret
-rw-r--r-- 1 root root   620 Jan 27 18:28 httpfs-site.xml
-rw-r--r-- 1 root root  3518 Jan 27 18:28 kms-acls.xml
-rw-r--r-- 1 root root  1527 Jan 27 18:28 kms-env.sh
-rw-r--r-- 1 root root  1631 Jan 27 18:28 kms-log4j.properties
-rw-r--r-- 1 root root  5511 Jan 27 18:28 kms-site.xml
-rw-r--r-- 1 root root 12259 Apr 12 22:09 log4j.properties
-rw-r--r-- 1 root root   951 Jan 27 18:28 mapred-env.cmd
-rw-r--r-- 1 root root  1418 Apr 12 22:09 mapred-env.sh
-rw-r--r-- 1 root root  4113 Jan 27 18:28 mapred-queues.xml.template
-rw-r--r-- 1 root root  2162 Apr 12 22:09 mapred-site.xml
-rw-r--r-- 1 root root   758 Jan 27 18:28 mapred-site.xml.template
-rw-r--r-- 1 root root    10 Jan 27 18:28 slaves
-rw-r--r-- 1 root root  2316 Jan 27 18:28 ssl-client.xml.example
-rw-r--r-- 1 root root  2268 Jan 27 18:28 ssl-server.xml.example
-rw-r--r-- 1 root root  2250 Jan 27 18:28 yarn-env.cmd
-rw-r--r-- 1 root root  4957 Apr 12 22:09 yarn-env.sh
-rw-r--r-- 1 root root   198 Apr 12 22:09 yarn.exclude
-rw-r--r-- 1 root root  6923 Apr 12 22:09 yarn-site.xml
All of our Hadoop configuration.
x

Xiang Fu

04/21/2021, 9:18 PM
is there any logs in your working directory
or dump
this sudden stopping seems very weird
a

Akash

04/21/2021, 9:20 PM
Is there any complex shading going on with respect to “org.apache.hadoop.conf.Configuration” ?
Copy code
org.apache.hadoop.conf.Configurable
This one is loading fine.
m

Mayank

04/21/2021, 9:21 PM
Side note @Akash @Xiang Fu if you are using latest master, there is now an api to return all relevant configs of pinot configs: https://github.com/apache/incubator-pinot/pull/6817
x

Xiang Fu

04/21/2021, 9:21 PM
I think Akash is on 0.7.1
m

Mayank

04/21/2021, 9:22 PM
He did mention he also debugged using master I think
x

Xiang Fu

04/21/2021, 9:22 PM
i see
a

Akash

04/21/2021, 9:22 PM
I can deploy current master. I have tried all, master, 0.6.0 and 0.7.1
x

Xiang Fu

04/21/2021, 9:22 PM
i think the issue here is pinot doesnt start
so no api is accessible
m

Mayank

04/21/2021, 9:23 PM
I see
a

Akash

04/21/2021, 9:23 PM
Yes. All i can really drill down is that when java is trying to instantiate “org.apache.hadoop.conf.Configuration” then it just stops
I have even downloaded all the 2.7.0 jars from maven to just be sure and added to classpath.
x

Xiang Fu

04/21/2021, 10:05 PM
hmmm, @Oguzhan Mangir have you seen similar things when you are trying out the hdfs?
a

Akash

04/21/2021, 10:30 PM
Copy code
docker run -ti \
    --net=host        \
    --name pinot-server \
    -v /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/:/opt/pinot/configs/ \
    -v /local/hadoop/etc/hadoop:/opt/pinot/hadoopConf \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log" \
    -d apachepinot/pinot:0.7.1 StartServer \
    -zkAddress hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181 \
    -configFileName /opt/pinot/configs/server.conf
How can i pass hadoop jar via Docker ?
x

Xiang Fu

04/21/2021, 10:31 PM
use -v to mount
a

Akash

04/21/2021, 10:34 PM
Can i specify
Copy code
-d apachepinot/pinot:0.7.1 StartServer -classpath /opt/pinot/lib/*:/opt/pinot/hadoopjar/*
and it will pick up ?
In pinot-admin.sh we are already setting classpath
Copy code
exec "$JAVACMD" $ALL_JAVA_OPTS \
  -classpath "$CLASSPATH" \
  -Dapp.name="pinot-admin" \
  -Dapp.pid="$$" \
  -Dapp.repo="$REPO" \
  -Dapp.home="$BASEDIR" \
  -Dbasedir="$BASEDIR" \
  org.apache.pinot.tools.admin.PinotAdministrator \
  "$@"
Let me ask very dumb question here, how can i set java classpath while running ./pinot-admin.sh ?
btw even this is crashing
Copy code
docker run -ti \
    --net=host        \
    --name pinot-server \
    -v /home/akashmishra/hpgraph/apache-pinot-incubating-0.7.1-bin/conf/:/opt/pinot/configs/ \
    -v /local/hadoop/etc/hadoop:/opt/pinot/hadoopConf \
    -v /home/akashmishra/hadooplib/hadoop-common-2.7.0.jar:/opt/pinot/hadooplib/hadoop-common-2.7.0.jar \
    -v /home/akashmishra/hadooplib/hadoop-hdfs-2.7.0.jar:/opt/pinot/hadooplib/hadoop-hdfs-2.7.0.jar \
    -e CLASSPATH_PREFIX="/opt/pinot/hadooplib/hadoop-common-2.7.0.jar:/opt/pinot/hadooplib/hadoop-hdfs-2.7.0.jar" \
    -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc-pinot-server.log" \
    -d <http://dockerio.badoo.com/bi/pinot:0.1|dockerio.badoo.com/bi/pinot:0.1> StartServer \
    -zkAddress hotpanel1.mlan:2181,hotpanel3.mlan:2181,bi24.mlan:2181 \
    -configFileName /opt/pinot/configs/server.conf
x

Xiang Fu

04/21/2021, 11:15 PM
I think the way you specify classpath is correct
I was using something like:
Copy code
CLASSPATH_PREFIX="/Users/xiangfu/.m2/repository/org/apache/hadoop/hadoop-common/2.7.0/hadoop-common-2.7.0.jar:/Users/xiangfu/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/xiangfu/.m2/repository/org/apache/hadoop/hadoop-auth/2.7.0/hadoop-auth-2.7.0.jar" bin/pinot-admin.sh LaunchDataIngestionJob  -jobSpecFile examples/batch/airlineStats/ingestionJobSpec.yaml
a

Akash

04/21/2021, 11:16 PM
Do we need guava-19.0.jar & hadoop-auth-2.7.0.jar ?
It’s looks like Pinot needs almost all of the hadoop-common and all dependent jars to run.
x

Xiang Fu

04/21/2021, 11:52 PM
when we build pinot-hdfs, we make hadoop-common to be provided runtime
but still, the issue is why it quits
and there is no exception
can you check if log4j use sync logger?
a

Akash

04/22/2021, 12:01 AM
Found the problem.
Copy code
CLASSPATH_PREFIX="/local/hadoop/share/hadoop/common/*;/local/hadoop/share/hadoop/common/lib/*;"
It needs all the commons jar and it’s dependencies.
Will validate again tomorrow and update the docs.
m

Mayank

04/22/2021, 12:03 AM
🙏🙏
Powered by