Hi to start minion using apache pinot (e.g. start-minion.sh) to manage realtime to offline flows

Hi All I got an Exception when integrate with presto, could anyone help check ? thx

ProcessingException(errorCode:150, messagePQLParsingError org.apache.pinot.sql.parsers.SqlCompilationException: Caught exception while parsing query: *select * from 59398999_Status_logagent_performance_review_tbl1416000000692493 limit 10* at org.apache.pinot.sql.parsers.CalciteSqlParser.compileCalciteSqlToPinotQuery(CalciteSqlParser.java:322) at org.apache.pinot.sql.parsers.CalciteSqlParser.compileToPinotQuery(CalciteSqlParser.java:107) at org.apache.pinot.sql.parsers.CalciteSqlCompiler.compileToBrokerRequest(CalciteSqlCompiler.java:35) at org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:166) at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:137) at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469))

facing exception when tablename contains underscore(_)

can someone help

I think the issue is that table name cannot start with numbers

can you try:

select * from "59398999_Status_logagent_performance_review_tbl1416000000692493" limit 10

so it won’t confuse the sql compiler

yes it worked fine

cc: @Mayank @Xiang Fu @Jackie @Kishore G @Neha Pawar @Ken Krugler @Chinmay Soman

reference : https://docs.pinot.apache.org/users/tutorials/schema-evolution#update-the-schema

Thanks in advance.

Can anyone please help on this issue?

Hello everyone, I’m having some issues querying Pinot using the JDBC driver. The Pinot has been configured using EKS in AWS. The error is the following one:

org.apache.pinot.client.PinotClientException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.net.UnknownHostException: pinot-broker-0.pinot-broker-headless.pinot-quickstart.svc.cluster.local: nodename nor servname provided, or not known

I’m able to use the controller via browser

And everything works fine

Yes because you have an external url

There is an external url for the broker

You need a load balancer for you broker

You should find a load balancer attached to your broker service

Hello everyone - controller.task.frequencyInSeconds is deprecated to auto schedule offline flows

I don’t find anywhere how to setup task schedule config

if i have queries that routinely have >1k and <100k items in my IN clause, what optimizations are available to me? e.g.

# using pinot client with a super long SQL statement
userIds = $SUBQUERY
select cell, count(userId) from events where userId in userIds group by cell;

# using presto
select cell, count(userId) from events where userId in $SUBQUERY group by cell;

There is a sub query support in Pinot and if the ids are ints, it will automatically use bitmap to represent the ids

@Jackie do you have an example for this?

This feature is included in the latest Pinot release

Looks like it might be related to the low level kafka 2 consumer defaulting the kafka message key type to string instead of reading properties.