sure! thanks for creating this!

talking of new features, I have a PR out for multivalued type support, please take a look when you get time https://github.com/prestodb/presto/pull/15093 (I somehow don't have access to add reviewers)

thanks Xiang for creating this

https://eng.uber.com/engineering-sql-support-on-apache-pinot/ Reading through this, got curious - - does the presto pinot connector utilize pruning when asking the broker for a segment list?

No

Ah. Might end up adding that down the line. Could be really useful with that time based segment pruner.

yes

at some point, we should just merge pinot broker and presto worker

Agreed. There are Pinot segment page and Pinot broker page. Pinot broker page will directly query Pinot broker, so Pinot broker will handle that logic. For Pinot segment page, presto will query every Pinot segment. It’s great to have the pruning feature in presto as well.

Typical segment query has predicate, so time based pruning will be very useful

Where would I create an issue for this? In incubator-pinot or on the presto github?

Hi team ,facing issues while accessing pinto table in presto getting Query 20211214_044329_00009_tcxbr failed: java.net.SocketTimeoutException: Connect Timeout

have you tried with newest docker image ?

we saw some groupby queries with issue

if you can share more logs on your presto coordinator/worker, that will be very useful for debugging

Hi Xiang ,Now I can connect to presto and able to do simple queries ,but while trying joins and sub quires query is failing with error:

java.lang.NullPointerException: null value in entry: Server_172.19.0.5_7000=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:32)
	at com.google.common.collect.SingletonImmutableBiMap.<init>(SingletonImmutableBiMap.java:42)
	at com.google.common.collect.ImmutableBiMap.of(ImmutableBiMap.java:72)
	at com.google.common.collect.ImmutableMap.of(ImmutableMap.java:124)
	at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:458)
	at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:437)
	at com.facebook.presto.pinot.PinotSegmentPageSource.queryPinot(PinotSegmentPageSource.java:242)
	at com.facebook.presto.pinot.PinotSegmentPageSource.fetchPinotData(PinotSegmentPageSource.java:214)
	at com.facebook.presto.pinot.PinotSegmentPageSource.getNextPage(PinotSegmentPageSource.java:161)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:280)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:245)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:424)
	at com.facebook.presto.operator.Driver.lambda$processFor$9(Driver.java:307)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:728)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:300)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1079)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:599)
	at com.facebook.presto.$gen.Presto_0_267_SNAPSHOT_ac0dc73____20211214_100300_1.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

can you explain the query ?

is presto and pinot deployed in same cluster?

for pinot side, can you try to add below into your pinot server configs:

pinot.server.instance.currentDataTableVersion=2
pinot.server.grpc.enable=true
pinot.server.grpc.port=8090

Hi Xiang Fu, we are using pinot quick start to test some use case where we have to integrate with presto to query data with pinot .i'm facing issue when i'm fetching entire data from presto throwing .com.facebook.presto.pinot.PinotException: Error when hitting host Server_172.19.0.5_7000 with pinot query "SELECT homeRuns, playerStint, groundedIntoDoublePlays, numberOfGames, AtBatting, stolenBases, tripples, hitsByPitch, teamID, numberOfGamesAsBatter, strikeouts, sacrificeFlies, caughtStealing, baseOnBalls, playerName, league, doules, yearID, hits, runsBattedIn, G_old, sacrificeHits, intentionalWalks, runs, playerID FROM baseballStats_OFFLINE LIMIT 2147483647"

Since pql endpoint is deprecated in pinot 0.11. Try to deprecate PQL support in PrestoDB. https://github.com/prestodb/presto/pull/18392

Deprecate server netty query support in Presto: https://github.com/prestodb/presto/pull/18404

Just for the awareness, I’ve deprecated PQL support in PrestoDB. https://github.com/prestodb/presto/pull/18392 and server netty query support in Presto: https://github.com/prestodb/presto/pull/18404. You might see error when upgrading to new image. The triage is to delete those configs. *And make sure your pinot server GRPC endpoint is enabled (*`pinot.server.grpc.enable=true`, default turned on from 0.11.0). Here are the deprecated catalog config:

pinot.use-pinot-sql-for-broker-queries
pinot.allow-multiple-aggregations
pinot.thread-pool-size
pinot.min-connections-per-server
pinot.max-connections-per-server
pinot.max-backlog-per-server
pinot.idle-timeout
pinot.use-streaming-for-segment-queries

Here are the deprecated session properties:

pinot.use_pinot_sql_for_broker_queries
pinot.ignore_empty_responses
pinot.connection_timeout

what should the presto configs look like for targeting standalone instance of pinot started using bin/pinot-admin.sh QuickStart -type batch following the instructions here: https://docs.pinot.apache.org/integrations/presto i am running into the following error while issuing queries from presto cli: presto:default> show tables; Query 20221006_215751_00005_m39ub failed: java.io.UncheckedIOException: java.net.UnknownHostException: pinot-controller

i was looking for the exact properties to set the connection with. was able to get things working after following the steps in the link below https://medium.com/@deepakcse2k5/connect-apache-pino-woth-presto-144a743a8626

Hi folks, I am seeing the following exception when booting presto-server with pinot connector. Is this a known issue? Using instructions: https://docs.pinot.apache.org/basics/getting-started/kubernetes-quickstart#7.-access-pinot-using-presto -- n/a -- 2022-10-20T204819.386Z ERROR main com.facebook.presto.server.PrestoServer com.google.inject.CreationException: Unable to create injector, see the following errors: 1) Error: Invalid configuration property pinot.query-options: may not be null (for class com.facebook.presto.pinot.PinotConfig.queryOptions) 1 error java.lang.RuntimeException: com.google.inject.CreationException: Unable to create injector, see the following errors: 1) Error: Invalid configuration property pinot.query-options: may not be null (for class com.facebook.presto.pinot.PinotConfig.queryOptions) 1 error at com.facebook.presto.pinot.PinotConnectorFactory.create(PinotConnectorFactory.java:94) at com.facebook.presto.connector.ConnectorManager.createConnector(ConnectorManager.java:388) at com.facebook.presto.connector.ConnectorManager.addCatalogConnector(ConnectorManager.java:235) at com.facebook.presto.connector.ConnectorManager.createConnection(ConnectorManager.java:227) at com.facebook.presto.connector.ConnectorManager.createConnection(ConnectorManager.java:213) at com.facebook.presto.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:123) at com.facebook.presto.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:98) at com.facebook.presto.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:80) at com.facebook.presto.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68) at com.facebook.presto.server.PrestoServer.run(PrestoServer.java:150) at com.facebook.presto.server.PrestoServer.main(PrestoServer.java:85) Caused by: com.google.inject.CreationException: Unable to create injector, see the following errors: 1) Error: Invalid configuration property pinot.query-options: may not be null (for class com.facebook.presto.pinot.PinotConfig.queryOptions) 1 error at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543) at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:159) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106) at com.google.inject.Guice.createInjector(Guice.java:87) at com.facebook.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:251) at com.facebook.presto.pinot.PinotConnectorFactory.create(PinotConnectorFactory.java:89) ... 10 more

Hi folks, We were looking into importing some Pinot packages into Presto codebase, and found out some

…-jdk8

packages are not being published since Pinot 0.12. Example: https://mvnrepository.com/artifact/org.apache.pinot/pinot-segment-local-jdk8 It’s interesting that it exists for other packages like pinot-common. I also noticed this PR by @Xiang Fu which gets rid of jdk8 related dependencies for Presto. Is there a new way to get these artifacts into Presto? Thanks cc @Jane Hang @Ting Chen

I had one PR to upgrade to 0.12 https://github.com/prestodb/presto/pull/19992/files

Then the final plan is to rewrite a jdk8 client in Prestodb

Hi all — silly question. I’m new to Pinot and wondering when it makes sense to use the Presto connector and move to using Pinot and Presto together?