https://pinot.apache.org/ logo
#general
Title
# general
r

RK

06/10/2021, 6:07 PM
Is there any way to query Pinot table directly from superset without using prestro as a middleware. I.e. To access pinot table through superset I am using pinot prestro connector than in superset I am using this catalog to connect from table so basically whenever I m firing some queries from superset it's going to pinot with the help of prestro. Since I am not using any joins in query so I believe I can also connect directly superset and pinot without using prestro as a middleware. So I think this way queries will be fast. @User kindly suggest.
m

Mayank

06/10/2021, 6:15 PM
In general docs.pinot.apache.org has a lot of good information and is also searchable.
s

Srini Kadamati

06/10/2021, 6:21 PM
hi RK, I work on the Superset project and you can absolutely query Pinot directly!
Ill also link to our docs page for Pinot- https://superset.apache.org/docs/databases/pinot
r

RK

06/10/2021, 6:49 PM
@User @User yes I checked this and in SqlAIchemy URI I have mention pinot://my_broker_host:8099/query/sql?controller=http://contrller_host:9000/
But for me it was showing this issue
s

Srini Kadamati

06/10/2021, 6:57 PM
ah, have you added the necessary drivers? I think this is a missing driver issue
the
pinotdb
library (https://pypi.org/project/pinotdb/) needs to be in the same python context as Superset
if you’re using docker, we have some instructions here: https://superset.apache.org/docs/databases/dockeradddrivers
did you end up getting this working @User? 🤔
r

RK

06/11/2021, 2:42 PM
I will try in local .I was not allowed to install any driver on server so couldn't try. @User
s

Srini Kadamati

06/11/2021, 3:03 PM
got it
k

Kamal Chavda

07/19/2021, 3:50 PM
@User, were you able to resolve this? I've installed the pinotdb driver on my superset instance but no luck. Also, the documentation on Superset's site and Pinot doesn't match for the URI, https://docs.pinot.apache.org/integrations/superset
pinot://<pinot-broker-host>:<pinot-broker-port><pinot-broker-path>?controller=<pinot-controller-host>:<pinot-controller-port>
on superset site it's:
Copy code
pinot+<http://controller:5436/query?server=http://controller:5983/>
https://superset.apache.org/docs/databases/pinot
s

Srini Kadamati

07/19/2021, 3:51 PM
I bet the Pinot docs are right and the Superset docs aren’t! I’m from the superset community and can update the Superset docs if someone confirms the Pinot string is correct 🙏
k

Kamal Chavda

07/19/2021, 4:17 PM
Nothing seems to be working for me. I've got a standalone instance of Superset (1.1.0) installed from master. I've got Pinot running single cluster mode (following tutorial). Both on separate EC2 instances and I'm able to hit the host IP on port 9000.
I do see under advanced setup to configure the broker https://docs.pinot.apache.org/developers/advanced/advanced-pinot-setup#pinot-broker. The tutorial I followed doesn't mention this. On this page the property
pinot.broker.client.queryPort
is marked (legacy).
x

Xiang Fu

07/19/2021, 5:22 PM
can you check which port of pinot is exposed for controller can server?
k

Kamal Chavda

07/19/2021, 5:25 PM
Hi @User, in the controller.conf it's set to 9000 and docker run it's 9000:9000
Instance config =
Copy code
{
  "id": "Controller_null_9000",
  "simpleFields": {
    "HELIX_ENABLED": "true",
    "HELIX_ENABLED_TIMESTAMP": "1625971997542",
    "HELIX_HOST": "Controller_null",
    "HELIX_PORT": "9000"
  },
  "mapFields": {},
  "listFields": {
    "TAG_LIST": [
      "controller"
    ]
  }
}
Liveinstance config =
Copy code
{
  "id": "Controller_null_9000",
  "simpleFields": {
    "HELIX_VERSION": "0.9.8",
    "LIVE_INSTANCE": "1@d078b25c568f",
    "SESSION_ID": "1000c3ef65e0079"
  },
  "mapFields": {},
  "listFields": {}
}
x

Xiang Fu

07/19/2021, 8:21 PM
ok, do you know the broker port?
hmm, why the hostname is null
k

Kamal Chavda

07/19/2021, 9:56 PM
Hi @User, the broker port is 8099. The defaults that are part of the tutorial.
x

Xiang Fu

07/19/2021, 10:00 PM
ok
then what’s your superset connection strirng?
did you open port 8099 from your docker?
can you access the 8099 port to your docker containerr?
k

Kamal Chavda

07/20/2021, 6:59 PM
That was it! I'm able to connect! I was trying the broker container IP as suggested in documentation but that wasn't working. So ended up being the below:
Copy code
<pinot://controller:8099/query/sql?server=http://controller:9000/>
x

Xiang Fu

07/20/2021, 8:58 PM
oh, so you start controller/broker in same docker container?
k

Kamal Chavda

07/20/2021, 9:00 PM
The controller and broker containers are on the same EC2 instance for my POC.
x

Xiang Fu

07/20/2021, 9:01 PM
ic, then it’s same host different port
2 Views