RK
06/02/2021, 11:26 AMElon
06/02/2021, 5:10 PMXiang Fu
RK
06/02/2021, 5:15 PMKen Krugler
06/02/2021, 5:43 PM<hdfs://xxx>
for your staging directory.
2. You need to use <hdfs://xxx>
for your outputDirURI
.configs:
section inside of the pinot FS specs section, which has hadoop.conf.path
. E.g. something like:
``````pinotFSSpecs:
- # scheme: used to identify a PinotFS.
# E.g. local, hdfs, dbfs, etc
scheme: hdfs
className: org.apache.pinot.plugin.filesystem.HadoopPinotFS
configs:
hadoop.conf.path: '/root/hadoop-ops/config/master/'
Xiang Fu
RK
06/02/2021, 6:22 PMXiang Fu
/home/rah/hybrid/staging/pinot-plugins.tar.gz
RK
06/02/2021, 6:30 PMXiang Fu
RK
06/02/2021, 6:37 PMKen Krugler
06/02/2021, 6:57 PMhadoop.conf.path
is in the wrong section. You have it as part of the file
specification, but it needs to be part of the hdfs
specification.file
scheme section from the pinotFSSpecs
configurationRK
06/02/2021, 7:02 PMXiang Fu
RK
06/03/2021, 6:17 AMXiang Fu
hadoop.conf.path
RK
06/03/2021, 6:48 AMXiang Fu
RK
06/03/2021, 9:26 AMXiang Fu
RK
06/03/2021, 9:50 AMXiang Fu
RK
06/03/2021, 9:54 AMXiang Fu
RK
06/03/2021, 9:56 AMXiang Fu
RK
06/03/2021, 9:57 AMXiang Fu
RK
06/03/2021, 9:58 AMKen Krugler
06/03/2021, 2:33 PMRK
06/03/2021, 2:43 PM