Shireen, you won’t see increased latency as long as you have cpu cycles to spare

What kind of requests do you have?

You can also expect latency to increase if your dataset does not fit into memory and you start paging in and out (for that your queries should touch most of the data)

There are different types of queries

is there a workaround to do conditional count on a field in PQL? Wanted to do something like

select sum(case when foo = '' then 0 else 1 end) from ...

no, Pinot does not support case statements as of now

Hi Team, I am working with Shireen on the Pinot project

We were getting the error: java.net.NoRouteToHostException: Cannot assign requested address (Address not available) on controller I believe it is due to port unavailability. I am running the cluster at 1500 qps... for stress testing

So to resolve the problem there are 3 steps 1) the recommended solution is to increase the port range to 1024 65535. 2) The close wait timeout is around 60 secs and recycle and reuse are turned off. So we can change it to 30, 1, 1. As recommended by "http://www.linuxbrigade.com/reduce-time_wait-socket-connections/"

# Decrease TIME_WAIT seconds net.ipv4.tcp_fin_timeout = 30 # Recycle and Reuse TIME_WAIT sockets faster net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1

Any suggestions/comments?

Also there is 1 broker and 3 servers... each using 2GB out of 8GB total RAM space

If you max out the connections on a broker, you can add more brokers.

Also, you can play with Pinot broker timeout.

is the non blocking connection pushed?

we should not be having this issue with non blocking rt

Yes, and it's enabled by default

But why the error is on controller?

they are running is old version of Pinot, asked @User to try with latest pinot code.

I guess you folks have noticed the spam ? https://cwiki.apache.org/confluence/display/PINOT/Consuming+and+Indexing+rows+in+Realtime (comments)

Yes

Trying to find somehow who as permissions to delete those

I had given permission to Subbu and few others, cant remember

deleting is one, we should probably disallow commenting?

Interesting place to do some SEO optimization 🙂. Reminds me of my work in the Bing Index team

I will be merging this change to cleanup our dependency tree: https://github.com/apache/incubator-pinot/pull/4155. If anyone has references to pinot-filesystem module in any extensions that have been built around it, please be aware that compilation might fail; you just need to remove the explicity pinot-filesystem dependency. Odds are that pinot-common is being depended on anyway, so nothing else should break.

<!here> I would like to know how people are configuring log4j outside of LinkedIn @User @User @User We try to migrate to log4j2 but its configuration is not backward compatible. https://github.com/apache/incubator-pinot/pull/4139

yes we do (Uber)

1.7.6 version