Srini Kadamati
04/19/2022, 5:13 PMHugh Miles
04/19/2022, 5:14 PM<pinot://localhost:8000/query/sql?controller=http://localhost:9000/>
pinotdb
i’ve been getting this error:
~/src/superset/venv/lib/python3.8/site-packages/pinotdb/db.py in check_sufficient_responded(self, query, queried, responded)
261 needed = fraction
262 if responded < 0 or responded < needed:
--> 263 raise exceptions.DatabaseError(
264 f"Query\n\n{query} timed out: Out of {queried}, only"
265 f" {responded} responded, while needed was {needed}"
DatabaseError: Query
select * from baseballStats
timed out: Out of -1, only -1 responded, while needed was -1
Kenny Bastani
04/19/2022, 5:17 PMpinot+<http://pinot:8000/query?server=http%3A%2F%2Fpinot%3A9000%2F>
for the connection stringHugh Miles
04/19/2022, 5:20 PM2-04-19 10:21:16,806:INFO:pinotdb.sqlalchemy:Updated pinot dialect args from {'dbapi': <module 'pinotdb' from '/Users/hugh/src/superset/venv/lib/python3.8/site-packages/pinotdb/__init__.py'>, 'debug': False, 'verify_ssl': True}: None and False
2022-04-19 10:21:16,807:INFO:pinotdb.sqlalchemy:Updated pinot dialect args from {'host': 'pinot', 'port': 8000, 'path': 'query', 'scheme': 'http', 'verify_ssl': True, 'username': None, 'password': None, 'debug': False}: <http://pinot:9000/> and False
2022-04-19 10:21:16,807:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_attempt
2022-04-19 10:21:16,837:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): pinot:8000
2022-04-19 10:21:16,844:DEBUG:superset.stats_logger:[stats_logger] (incr) test_connection_error.DBAPIError
2022-04-19 10:21:16,846:DEBUG:superset.stats_logger:[stats_logger] (incr) DatabaseRestApi.test_connection.error
2022-04-19 10:21:16,847:WARNING:superset.views.base:[SupersetError(message='(builtins.NoneType) None\n(Background on this error at: <http://sqlalche.me/e/13/dbapi)>', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Apache Pinot', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
Kenny Bastani
04/19/2022, 5:22 PMpinot+<http://localhost:8000/query?server=http%3A%2F%2Flocalhost%3A9000%2F>
pinot+<http://localhost:8000/query?server=http://localhost:9000/>
Hugh Miles
04/19/2022, 5:28 PMfrom sqlalchemy import *
...: from sqlalchemy.engine import create_engine
...: from sqlalchemy.schema import *
...:
...: engine = create_engine('pinot+<http://localhost:8000/query?server=http://localhost:9000/>')
...:
...: places = Table('baseballStats', MetaData(bind=engine), autoload=True)
...: print(select([func.count('*')], from_obj=places).scalar())
Chinmay Soman
04/19/2022, 5:28 PMKenny Bastani
04/19/2022, 5:29 PMHugh Miles
04/19/2022, 5:29 PMConnectionError: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /query/sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10730a790>: Failed to establish a new connection: [Errno 61] Connection refused'))
In [26]: from pinotdb import connect
...:
...: conn = connect(host='localhost', port=9000, path='/query/sql', scheme='http')
...: curs = conn.cursor()
...: curs.execute("""
...: select * from baseballStats
...: """)
...: for row in curs:
...: print(row)
...:
~/src/superset/venv/lib/python3.8/site-packages/pinotdb/db.py in check_sufficient_responded(self, query, queried, responded)
261 needed = fraction
262 if responded < 0 or responded < needed:
--> 263 raise exceptions.DatabaseError(
264 f"Query\n\n{query} timed out: Out of {queried}, only"
265 f" {responded} responded, while needed was {needed}"
DatabaseError: Query
select * from baseballStats
timed out: Out of -1, only -1 responded, while needed was -1
Kenny Bastani
04/19/2022, 5:30 PMcurl <http://localhost:8000/query?server=http://localhost:9000/>
Hugh Miles
04/19/2022, 5:31 PMKenny Bastani
04/19/2022, 5:31 PMlocalhost:8000
Hugh Miles
04/19/2022, 5:31 PMKenny Bastani
04/19/2022, 5:32 PMHugh Miles
04/19/2022, 5:32 PMKenny Bastani
04/19/2022, 5:32 PMnetstat -vanp tcp | grep '*.2123\|9000\|8000\|7000'
Hugh Miles
04/19/2022, 5:33 PM✦ ➜ netstat -vanp tcp | grep '*.2123\|9000\|8000\|7000'
tcp6 0 0 ::1.9000 ::1.57711 FIN_WAIT_2 407508 146808 478 0 0x0112 0x0000000c
tcp6 0 0 ::1.57711 ::1.9000 CLOSE_WAIT 407577 146808 57242 0 0x0022 0x00000000
tcp46 0 0 *.9000 *.* LISTEN 131072 131072 478 0 0x0100 0x00000006
tcp6 0 0 ::1.9000 ::1.63536 TIME_WAIT 403702 146808 478 0 0x2131 0x0000000c
tcp6 0 0 ::1.9000 ::1.63537 TIME_WAIT 400527 146808 478 0 0x2131 0x0000000c
tcp6 0 0 ::1.9000 ::1.63538 TIME_WAIT 404686 146808 478 0 0x2131 0x0000000c
Kenny Bastani
04/19/2022, 5:33 PMHugh Miles
04/19/2022, 5:34 PMdocker run \
-p 9000:9000 \
apachepinot/pinot:0.9.3 QuickStart \
-type batch
Kenny Bastani
04/19/2022, 5:34 PMHugh Miles
04/19/2022, 5:34 PMKenny Bastani
04/19/2022, 5:34 PMHugh Miles
04/19/2022, 5:34 PMKenny Bastani
04/19/2022, 5:35 PMHugh Miles
04/19/2022, 5:37 PMdocker run \
-p 9000:9000 \
-p 8000:8000 \
-p 7000:7000 \
apachepinot/pinot:0.9.3 QuickStart \
-type batch
Kenny Bastani
04/19/2022, 5:37 PMHugh Miles
04/19/2022, 5:41 PMKenny Bastani
04/19/2022, 5:41 PMHugh Miles
04/19/2022, 5:56 PM✦ ➜ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fe8b5fdf9dcb apachepinot/pinot:latest "./bin/pinot-admin.s…" 14 minutes ago Up 2 minutes 0.0.0.0:7000->7000/tcp, 0.0.0.0:8000->8000/tcp, 0.0.0.0:9000->9000/tcp, 8096-8099/tcp confident_curie
✦ ➜ netstat -vanp tcp | grep '*.2123\|9000\|8000\|7000'
tcp46 0 0 *.8000 *.* LISTEN 131072 131072 478 0 0x0100 0x00000006
tcp46 0 0 *.7000 *.* LISTEN 131072 131072 478 0 0x0100 0x00000006
tcp46 0 0 *.9000 *.* LISTEN 131072 131072 478 0 0x0100 0x00000006
tcp6 0 0 ::1.9000 ::1.57711 FIN_WAIT_2 407508 146808 478 0 0x0112 0x0000000c
tcp6 0 0 ::1.57711 ::1.9000 CLOSE_WAIT 407577 146808 57242 0 0x0022 0x00000000
Kenny Bastani
04/19/2022, 5:57 PMHugh Miles
04/19/2022, 5:57 PMpinot+<http://localhost:8000/query?server=http://localhost:9000>
Kenny Bastani
04/19/2022, 5:58 PMHugh Miles
04/19/2022, 6:00 PMKenny Bastani
04/19/2022, 6:00 PMHugh Miles
04/19/2022, 6:00 PMdocker run \
-p 9000:9000 \
-p 8000:8000 \
-p 7000:7000 \
apachepinot/pinot:0.9.3 QuickStart \
-type batch
Working Connection String:
pinot+<http://localhost:8000/query?server=http://localhost:9000>
Python Test Example:
In [2]: from pinotdb import connect
...:
...: conn = connect(host='localhost', port=8000, path='/query/sql', scheme='http')
...: curs = conn.cursor()
...: curs.execute("""
...: select * from baseballStats
...: """)
...: for row in curs:
...: print(row)
...:
Kenny Bastani
04/19/2022, 6:00 PMHugh Miles
04/19/2022, 6:01 PMahsen m
04/19/2022, 9:26 PMpinot+<http://apache-pinot-broker-0.apache-pinot-broker-headless.pinot-cluster.svc.cluster.local:8099/query?controller=http%3A%2F%2Fapache-pinot-controller-0.apache-pinot-controller-headless.pinot-cluster.svc.cluster.local%3A9000%2F>