https://pinot.apache.org/ logo
a

Akash

04/13/2021, 3:01 PM
I am trying to add HDFS as deep storage and running following command
Copy code
bin/start-server.sh -zkAddress pinot1.plan:2181,pinot2.plan:2181,pinot3.plan:2181 -configFileName conf/server.conf
Server configs are
Copy code
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.segment.fetcher.protocols=file,http,hdfs
pinot.server.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.instance.dataDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/index
pinot.server.instance.segmentTarDir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/data/PinotServer/segmentTar
I am getting following error
Copy code
2021/04/13 15:00:49.231 INFO [MBeanRegistrar] [Start a Pinot [SERVER]] MBean HelixCallback:Change=MESSAGES_CONTROLLER,Key=PinotCluster.Server_10.10.211.27_8098,Type=PARTICIPANT has been registered.
2021/04/13 15:00:49.232 INFO [MBeanRegistrar] [Start a Pinot [SERVER]] MBean HelixCallback:Change=HEALTH,Key=PinotCluster.Server_10.10.211.27_8098,Type=PARTICIPANT has been registered.
2021/04/13 15:00:49.598 INFO [Reflections] [Start a Pinot [SERVER]] Reflections took 313 ms to scan 1 urls, producing 5 keys and 151 values
2021/04/13 15:00:49.645 ERROR [PinotFSFactory] [Start a Pinot [SERVER]] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs
java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_275]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_275]
	at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:80) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:268) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:239) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:220) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:53) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
2021/04/13 15:00:49.650 ERROR [StartServiceManagerCommand] [Start a Pinot [SERVER]] Failed to start a Pinot [SERVER] at 0.513 since launch
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
Caused by: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.HadoopPinotFS
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_275]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_275]
	at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:80) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:268) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:239) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:220) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:53) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 9 more
d

Daniel Lavoie

04/13/2021, 3:04 PM
You can pass the plugin dir config through the
JAVA_OPTS
env variable
a

Akash

04/13/2021, 3:06 PM
Thank, when i tried to add -Dplugin.dir in start-server.sh script then i got the following error.
Copy code
2021/04/13 15:05:48.160 ERROR [StartServiceManagerCommand] [Start a Pinot [SERVER]] Failed to start a Pinot [SERVER] at 1.305 since launch
java.lang.RuntimeException: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
Caused by: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:71) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 9 more
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 9 more
Looks like pinot is not able to load hdfs configs.
Copy code
hadoop.conf.path=/local/hadoop/etc/hadoop/
pinot.server.storage.factory.hdfs.hadoop.conf.path=/local/hadoop/etc/hadoop/
is there any issue with these configurations ?
o

Oguzhan Mangir

04/13/2021, 3:09 PM
-Dplugins.dir
not -Dplugin.dir
also you have not to pass -Dplugins.include
a

Akash

04/13/2021, 3:10 PM
Copy code
export JAVA_OPTS="-Dplugins.dir=/home/akashmishra/hpgraph/apache-pinot-incubating-0.6.0-bin/plugins/"
I am not using this, but still the same error.
Copy code
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
o

Oguzhan Mangir

04/13/2021, 3:10 PM
can you try without -Dplugins.include ?
a

Akash

04/13/2021, 3:11 PM
I am not using -Dplugins.include anywhere. AFAIU, this is not set anywhere.
o

Oguzhan Mangir

04/13/2021, 3:13 PM
remove this config
Copy code
hadoop.conf.path=/local/hadoop/etc/hadoop/
you already set pinot.server.storage.factory.hdfs.hadoop.conf.path this is not about your problem
1
a

Akash

04/13/2021, 3:13 PM
Which one ?
The problem still persists.
d

Daniel Lavoie

04/13/2021, 3:19 PM
Do you want to try the
pinot-admin.sh StartServer
instead?
a

Akash

04/13/2021, 3:20 PM
Same results.
Let me debug log and come up with more better information for you guys to help me 🙂
Copy code
2021/04/13 17:01:28.443 ERROR [PinotFSFactory] [Start a Pinot [SERVER]] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs
java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:71) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 10 more
2021/04/13 17:01:28.447 ERROR [StartServiceManagerCommand] [Start a Pinot [SERVER]] Failed to start a Pinot [SERVER] at 1.134 since launch
java.lang.RuntimeException: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:74) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.SegmentFetcherAndLoader.<init>(SegmentFetcherAndLoader.java:71) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.server.starter.helix.HelixServerStarter.start(HelixServerStarter.java:316) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startServer(PinotServiceManager.java:150) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:95) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:260) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:57) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:260) [pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
Caused by: java.lang.RuntimeException: Could not initialize HadoopPinotFS
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:71) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 9 more
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) ~[pinot-parquet-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67) ~[pinot-hdfs-0.6.0-shaded.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:54) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-bb646baceafcd9b849a1ecdec7a11203c7027e21]
	... 9 more
Still problem
I have also added Hadoop and hadoop_conf env
Copy code
export HADOOP_HOME=/local/hadoop/
export HADOOP_CONF_DIR=/local/hadoop/etc/hadoop/