hi, I am just wondering if anyone seen pinot broker has latency spike more than pre-defined timeout period when the request rate/latency is high

for example, if broker got hundreds of requests per second with each request take seconds to finish, will pinot broker web server runs out of request handler and cause all incoming request to stuck in the some kind of pending state?

after timeout the connection handles must be released and available to use for new queries

yes, but I saw that in broker we have connection pool with default something like 30?

if the broker -> server requests take long time to finish and we have high qps to the broker, will the broker runs out all request handler and cause all incoming request to this broker to get into the “waiting state”?

yes, @User added the support for multiplexing over the same connection, not sure if that takes care of this scenario

got it. do you know if we have existing metrics exposed for monitoring such scenario? for example java jetty server has some metrics like available connection count metrics to let us get to know if we still have enough connection handler on the java web server

With the new connection pool, we have the reverse effect where broker bombards servers in a non blocking way

IIRC, the current code will drop queries that have already passed the timeout

The default value of timeout may be too big (30 sec??), and tuning that should mitigate that issue

There is a metric to monitor the wait time in broker queue

got it, thanks

will look into that for better monitoring

I think we had this metric to track the dropped request:

REQUEST_DROPPED_DUE_TO_CONNECTION_ERROR

Hi, we were wondering if anyone has integrated Grafana and whether Grafana can be used as a visualization tool for data/metrics that we have plans to store in Pinot.

we haven't tried that, there is a superset adapter

I’m talking to @User on DM, We actually integrated Pinot metrics, not the grafana-pinot integration. It would be a nice integration, especially now Pinot support Lucene integration

I'm a bit confused how about how to query a table and filter to specific time ranges. I have a timeFieldSpec that specifies a timestamp column, named

timestamp

and a segmentsConfig that specifies a timestamp column. When I try to query the timestamp column in PQL, I get an error

Encountered "timestamp >"

...

Was expecting one of

. Do I need to refer to a different name? Do I need to add dimension fields for the parts of date I want to query? Do I have to use a function to convert timestamp to a timestamp I can filter between?

timestamp is a reserved keyword

I think timestamp is a reserved keyword in SQL, so you will need to quote the column name in the query

Ah, that's it. Thanks!

If I filter for timestamps outside of the data range, my request gives a

ProcessingException(errorCode: 450,

. Is that intended? I figured it'd just return empty rows.

No, it should return your empty rows if no rows are selected

The error implies something else went wrong

Hmm, all I changed was a timestamp filter to an earlier date.

Do you have queries?

Hmm, weird, I can't reproduce it now. If/when I hit it again, I'll record them.

Unrelated, has anyone hooked up a Pinot Connector to to AWS Athena (managed Presto)? No worries if not.

no, is that even possible?