Hello everyone I have recently started Using Apache Pinot and I am able to integrate my Kafka with Pinot. I have done the setup in local and once I started  Pinot using bin/quick-start-batch.sh. I am able to see all the Pinot details on localhost:9000. I want to add one user Authentication feature here so someone use this localhost:9000. It should ask for credential and then it should go to pinot home page. I checked multiple documents and youtube videos but could not find any reference for the same. Kindly suggest /guide me how Can i implement the same.

Team Someone Please help.

I know that @User has added support for authentication. Let's give him a minute. In the meanwhile I can try to find the PRs he added.

Okay thanks @User

You can refer to this PR for now: https://github.com/apache/incubator-pinot/pull/6613 @User

Hi RK - the PR above actually achieves the scenario you describe. It’ll officially become available in Pinot 0.8.0+, but you can already use the snapshot to try out the feature. There’s already a configuration reference here: https://docs.pinot.apache.org/operators/tutorials/authentication-and-authorization

@User inside bin folder I have multiple .Sh files kindly guide me in which file I need to add all these properties. . And after adding all these properties simply I need to start bin/quick-start-batch.sh or need to do some additional setups. controller.admin.access.control.principals=admin controller.admin.access.control.principals.admin.password=verysecret controller.admin.access.control.factory.class=org.apache.pinot.controller.api.access.BasicAuthAccessControlFactory controller.admin.access.control.principals=admin,user controller.admin.access.control.principals.admin.password=verysecret controller.admin.access.control.principals.user.password=secret controller.admin.access.control.principals.user.tables=myusertable,baseballStats,stuff controller.admin.access.control.principals.user.permissions=READ # the factory class property is different for the broker pinot.broker.access.control.class=org.apache.pinot.broker.broker.BasicAuthAccessControlFactory pinot.broker.access.control.principals=admin,user pinot.broker.access.control.principals.admin.password=verysecret pinot.broker.access.control.principals.user.password=secret pinot.broker.access.control.principals.user.tables=baseballStats,otherstuff

Pinot is typically configured using *.properties files. You can create e.g.

controller.properties

and

broker.properties

and store the respective properties there. Then, you can use the various starter scripts like

bin/start-controller.sh -configFileName controller.properties

to bring up the individual pinot components with you new configs. (The commands also support a

--help

flag, and there’s a detailed reference here: https://docs.pinot.apache.org/basics/getting-started/advanced-pinot-setup)

1. Started Zookeer Manually using below command: bin/pinot-admin.sh StartZookeeper \ -zkPort 2191 -----------------------_------ 2. Created One controller.properties file in bin folder and added following properties. controller.admin.access.control.principals=admin controller.admin.access.control.principals.admin.password=verysecret
 controller.segment.fetcher.auth.token=Basic YWRtaW46dmVyeXNlY3JldA controller.admin.access.control.factory.class=org.apache.pinot.controller.api.access.BasicAuthAccessControlFactory controller.admin.access.control.principals=admin,user 
controller.admin.access.control.principals.admin.password=verysecret
 controller.admin.access.control.principals.user.password=secret
 controller.admin.access.control.principals.user.tables=myusertable,baseballStats
,stuff controller.admin.access.control.principals.user.permissions=READ ------------------------------------------------ 3. Added these 3 extra properties also since it was throwing error to add these parameter in config file. controller.access.protocols.http.port=2191 controller.zk.str=localhost:2181 controller.helix.cluster.name=PinotCluster i.e. : 2021/05/08 092655.054 ERROR [StartControllerCommand] [main] Caught exception while starting controller, exiting. org.apache.commons.configuration.ConfigurationException: Pinot Controller Config Validation Error: missing controller http port, please fix 'controller.access.protocols.http.port' property in config file. ------------------------------------------------ 4. After then when I am trying to start controller with configFileName property its giving below error. apache-pinot-incubating-0.7.1-bin % bin/start-controller.sh \ -configFileName bin/controller.properties 2021/05/08 092943.096 ERROR [StartServiceManagerCommand] [main] Failed to start a Pinot [CONTROLLER] at 0.051 since launch java.lang.IndexOutOfBoundsException: Index: 0, Size: 0     at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[?:1.8.0_261]     at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_261]     at org.apache.pinot.controller.ControllerStarter.<init>(ControllerStarter.java:156) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.service.PinotServiceManager.startController(PinotServiceManager.java:115) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:91) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.lambda$startBootstrapServices$0(StartServiceManagerCommand.java:234) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:286) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startBootstrapServices(StartServiceManagerCommand.java:233) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.execute(StartServiceManagerCommand.java:183) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:130) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:164) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:184) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     at org.apache.pinot.tools.admin.PinotController.main(PinotController.java:35) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]

Hi RK - in a few hours the

quick-start-auth.sh

should become available if you don’t want to wait try this - looks like the ports got swapped: (1) instead of

controller.access.protocols.http.port=2191

simply use

controller.port=9000

and

controller.host=localhost

(2) instead of

controller.zk.str=localhost:2181

go with

controller.zk.str=localhost:2191

(or whichever port you start zookeeper on)

When I am executing bin/quick-start-batch.sh. In log I found Executing command: StartZookeeper -zkPort 2123 -dataDir /var/folders/5f/c00hxf596s19gdw84svdm7j40000gp/T/1620485847863/baseballStats/rawdata/PinotZkDir Start zookeeper at localhost:2123 in thread main Executing command: StartController -clusterName QuickStartCluster -controllerHost null -controllerPort 9000 -dataDir /var/folders/5f/c00hxf596s19gdw84svdm7j40000gp/T/1620485847863/baseballStats/rawdata/PinotControllerDir0 -zkAddress localhost:2123 Executing command: StartServiceManager -clusterName QuickStartCluster -zkAddress. so In my controller.properties I have mentioned same details for controller.zk.str=localhost:2123 controller.data.dir=/var/folders/5f/c00hxf596s19gdw84svdm7j40000gp/T/1620485357791/baseballStats/rawdata/PinotZkDir controller.helix.cluster.name=QuickStartCluster PFB the controller.properties file. controller.admin.access.control.principals=admin controller.admin.access.control.principals.admin.password=verysecret controller.segment.fetcher.auth.token=Basic YWRtaW46dmVyeXNlY3JldA controller.admin.access.control.factory.class=org.apache.pinot.controller.api.access.BasicAuthAccessControlFactory controller.admin.access.control.principals=admin,user controller.admin.access.control.principals.admin.password=verysecret controller.admin.access.control.principals.user.password=secret controller.admin.access.control.principals.user.tables=myusertable,baseballStats
,stuff controller.admin.access.control.principals.user.permissions=READ controller.port=9000 ontroller.host=localhost controller.zk.str=localhost:2123 controller.data.dir=/var/folders/5f/c00hxf596s19gdw84svdm7j40000gp/T/1620485357791/baseballStats/rawdata/PinotZkDir controller.helix.cluster.name=QuickStartCluster After That when I am starting controller with controller.properties File. I am getting java.lang.NullPointerException. Kindly Help. PFA the Log File for Controller Error.

Kindly pull and build the most recent

master

of incubator-pinot. You’ll now find a

quick-start-auth.sh

script which launches a pre-configured pinot cluster with auth

hey @Alexander Pucher I am not able to find that script in the master branch could you please share the link to that file