https://pinot.apache.org/ logo
Join Slack
Powered by
Is there any way to query Pinot table directly fro...
# general
r
Is there any way to query Pinot table directly from superset without using prestro as a middleware. I.e. To access pinot table through superset I am using pinot prestro connector than in superset I am using this catalog to connect from table so basically whenever I m firing some queries from superset it's going to pinot with the help of prestro. Since I am not using any joins in query so I believe I can also connect directly superset and pinot without using prestro as a middleware. So I think this way queries will be fast. @User kindly suggest.
m
In general docs.pinot.apache.org has a lot of good information and is also searchable.
s
hi RK, I work on the Superset project and you can absolutely query Pinot directly!
Ill also link to our docs page for Pinot- https://superset.apache.org/docs/databases/pinot
r
@User @User yes I checked this and in SqlAIchemy URI I have mention pinot://my_broker_host:8099/query/sql?controller=http://contrller_host:9000/
But for me it was showing this issue
s
ah, have you added the necessary drivers? I think this is a missing driver issue
the 
pinotdb
library (https://pypi.org/project/pinotdb/) needs to be in the same python context as Superset
if you’re using docker, we have some instructions here: https://superset.apache.org/docs/databases/dockeradddrivers
did you end up getting this working @User? 🤔
r
I will try in local .I was not allowed to install any driver on server so couldn't try. @User
s
got it
k
@User, were you able to resolve this? I've installed the pinotdb driver on my superset instance but no luck. Also, the documentation on Superset's site and Pinot doesn't match for the URI, https://docs.pinot.apache.org/integrations/superset 
pinot://<pinot-broker-host>:<pinot-broker-port><pinot-broker-path>?controller=<pinot-controller-host>:<pinot-controller-port>
on superset site it's:
Copy code
pinot+<http://controller:5436/query?server=http://controller:5983/>
https://superset.apache.org/docs/databases/pinot
s
I bet the Pinot docs are right and the Superset docs aren’t! I’m from the superset community and can update the Superset docs if someone confirms the Pinot string is correct 🙏
k
Nothing seems to be working for me. I've got a standalone instance of Superset (1.1.0) installed from master. I've got Pinot running single cluster mode (following tutorial). Both on separate EC2 instances and I'm able to hit the host IP on port 9000.
I do see under advanced setup to configure the broker https://docs.pinot.apache.org/developers/advanced/advanced-pinot-setup#pinot-broker. The tutorial I followed doesn't mention this. On this page the property 
pinot.broker.client.queryPort
is marked (legacy).
x
can you check which port of pinot is exposed for controller can server?
k
Hi @User, in the controller.conf it's set to 9000 and docker run it's 9000:9000
Instance config =
Copy code
{
  "id": "Controller_null_9000",
  "simpleFields": {
    "HELIX_ENABLED": "true",
    "HELIX_ENABLED_TIMESTAMP": "1625971997542",
    "HELIX_HOST": "Controller_null",
    "HELIX_PORT": "9000"
  },
  "mapFields": {},
  "listFields": {
    "TAG_LIST": [
      "controller"
    ]
  }
}
Liveinstance config =
Copy code
{
  "id": "Controller_null_9000",
  "simpleFields": {
    "HELIX_VERSION": "0.9.8",
    "LIVE_INSTANCE": "1@d078b25c568f",
    "SESSION_ID": "1000c3ef65e0079"
  },
  "mapFields": {},
  "listFields": {}
}
x
ok, do you know the broker port?
hmm, why the hostname is null
k
Hi @User, the broker port is 8099. The defaults that are part of the tutorial.
x
ok
then what’s your superset connection strirng?
did you open port 8099 from your docker?
can you access the 8099 port to your docker containerr?
k
That was it! I'm able to connect! I was trying the broker container IP as suggested in documentation but that wasn't working. So ended up being the below:
Copy code
<pinot://controller:8099/query/sql?server=http://controller:9000/>
x
oh, so you start controller/broker in same docker container?
k
The controller and broker containers are on the same EC2 instance for my POC.
x
ic, then it’s same host different port
2 Views
Open in Slack
PreviousNext
Powered by