yes. you can use ThirdEye on MySQL

can you please join Apache ThirdEye channel?

We will be working on supported SQL compliant

Order By

behavior for

aggregation-groupby

queries in PQL. Please follow the issue for details: https://github.com/apache/incubator-pinot/issues/4450

It will be good to partner with LinkedIn learning and see if we can do something similar for Apache Pinot

Agree

Great idea.

Druid university sounds weird :)

Only one course can be taught there 🤷‍♂️

according to druid, that's all you need to know 🙂

What is the max heap size (i.e., -Xmx) for Pinot controller for prod usage in LinkedIn? In Uber, we used to stay with 4g. Recently we ran into multiple OOM issue. Upon heapdump analysis, it shows the main suspect is usage of com.codahale.metrics.SlidingTimeWindowArrayReservoir in Helix ZkClientPathMonitor.java. It uses a hardcoded 1hr slide window which seems too long for histogram for a busy prod server: https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/monitoring/mbeans/ZkClientPathMonitor.java#L130-L137

Controller memory totally depends on your cluster characteristics: the number of tables, the number of segments you have, etc.. We use 40g in our large clusters (1000s of tables), and 16g for smaller number of tables. Our memory issues have also been with helix cache. we are working with them to fix that.

Thanks. Yes we are experimenting with 16G of max heap with our most busy cluster with ~100 tables.

It also depends on the number of segments of tables because metrics like above tracks read and writes to zk.

We would also love to follow up with the helix cache fix if there is any Pinot/Helix github issues on it. @User^^

if/when such a fix comes in, we will definitely move forward eith it in open source

Hi, I am trying to connect from pinot to kafka to try and ingest a stream. I run quick realtime script and then try to submit my schema and add table

I am seeing the following. Any idea of what I should do?

Untitled

here is the table config i tired @User

Yep

Hi all, I just shared the design doc for DISTINCT on the mailing list. Please take some time to review. Thank you

Hi Sidd, looks like you haven't subscribed to the mailing list

I approved this email for now

Hey Kishore, I subscribed immediately after sending the email. Thanks for approving

All, please let me know if you are not able to view/edit the doc and spreadsheet I had shared

Anyone from Factual here?

Hello friends. Do we have any teams that run pinot in cloud environment? specifically i am looking for cases where team would not have control over hostnames.

me 😉

i’m running Pinot over kubernetes