Maybe I am too late to the party but I started a project for the Apache Pinot Kubernetes Operator 🚀 You can find it here: https://github.com/spaghettifunk/pinot-operator If you are interested in contributing or simply give feedback, it would be great 😄 Unless there is already a project like this and I failed to find it 😆

Welcome new Pinot community users! 👋 🍷 @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User!! Would love to know what brought you here! 😄

I remember I was told that transform of post-aggregation (eg

DIV(SUM(subtotal), DISTINCTCOUNT(category_id))

) would be supported in 0.6.0 but couldn’t find it in the release doc. Does anyone know that?

Hi team, can tenants share hosts and servers? or each should have an exclusive set.

Hi everyone, minutes aggregation using Pinot is awesome, but is there anyway to perform rolling sum / minute instead?

Hi, while using

upsert

in the real-time table, which requires partitioned replica group assignment, can I use a different segment assignment in the offline table?

Hi team, when using upsert with realtime table, can we do segments compaction or merge for committed segments? Like merge multiple small segments into one large segment. If not, how should we deal with too many small segments when using upsert? @User @User

Do star-tree index aggregations span segments, or are the aggregates computed per segment?

@User , mostly here to be a part of community and contribute to the project

Hey guys, I'm a software engineer in MyCujoo, which is a company that offers streaming services for what we call the long tail of football, empowering teams in minor leagues all over the world and letting them be seen. We're making out streaming platform work as a service and we were in the search of any good technology to support our analytics services, that's how we got into pinot

Hello everyone! I'm a software engineer and looking for an OLAP tool.. exploring few options - came across Pinot (recommended by a colleague) - spending time learning about it and getting it deployed!

Hi 🙂 I’m a software developer employee on a team; My colleagues are prototyping a solution with Druid. I like Pinot because it looks very polished and it is easy to install on Kubernetes: there’s a Helm Chart, there’s documentation, I’ve completed the quickstart yesterday, and it works. Getting Druid running on Kubernetes seems much messier. They recently deprecated+deleted the main Helm chart; they have a Druid Operator, but it’s still rough, we’ve gotten errors and concluded it’s not ready for use yet. I’m curious if others can share reasons to choose Druid vs Pinot vs something else.

Hi, i am using addtable command to upload a new schema etc to docker pinot. how do i update table, eg add/ remove columns after i have added the table? Thanks

Hello, this is Lee Rhodes of Apache DataSketches. I would be interested in some feedback as to how you are using our library … what sketches are you using, what you feel works well, and constructive feedback on what could work better, or what problems you would like sketches to address?

"Intro to Pinot" session by @User - starting now. https://www.meetup.com/apache-pinot/events/275704309/

is there any example controller.conf file or a generator, all I really want to do is change the port number but now I have a conf file it keeps asking for more and more properties e.g zookeeper address

Hi. I see that theres this "Segment Merge" that we can configure for pinot table. https://docs.google.com/document/d/1zoklHjbli-HIy0JAiBITABNsBOuC2jFthcjwGqsFVFQ/edit# Is the feature ready? Thanks.

The diagram in the dictionary-encoded forward index seems to be wrong - entries with docId 3 and 4 (last two entries) point to "113" for the colA but contain dictId of 2 and 3 respectively. Shouldn't they be set to '2' just like the entries pointing to 112?

Is my understanding correct? If yes, how can fix/update the diagram?

Hello, could someone please share Pinot document on Kinesis integration? Thanks.

@User thanks for the warm welcome, I'm Daniel from Berlin and I'm building the world's first data engineering bootcamp. https://www.dataengineering.academy

@here - I am evaluating Pinot for my organization.. We will be ingesting Real-time data from Kafka. What schema formats does Pinot support to ingest from Kafka? We have protobuf serialized data, is there a support for that with pinot? Thanks in advance for your help! (I am sure I will have more questions)

@here was wondering whether you might be able to point out to some rule-of-thumb compression benchmarks using Pinot? Cheers

Hi. I see that theres this "Segment Merge" that we can configure for pinot table. https://docs.google.com/document/d/1zoklHjbli-HIy0JAiBITABNsBOuC2jFthcjwGqsFVFQ/edit# Is the feature ready? Thanks. Thread in Slack Conversation

https://zoom.us/meeting/register/tJ0kf--rqzstGNBYC4epV69xNyoDwb4UTZ1m Using Apache Kafka and Pinot for User-Facing Real-Time Analytics -- Happening now

I have some high-level questions. I'm looking at the architectural page in the docs: https://docs.pinot.apache.org/basics/architecture 1. Regarding the terminology, is segment store the same as deep store? 2. I assume that if we didn't explicitly configure any storage (i.e., s3, hdfs), it will use the node's local disk? 3. What does Load Segment mean in the figure (specifically, what does the Server do)? Does it copy the segment to a server's local disk? Does it load it in memory? 4. Let say we have a Real Time table ingesting from Kafka and the Pinot cluster is not configured with any specific disk storage (i.e., just using local disk). Once the current consuming segment completes, the server commits it and then writes it to the segment store (i.e., local disk). Will this completed segment be loaded by another server (and start being served by another server)? Or will it just continue to be served by the same server? Are there any policies/settings/config related to this?

Ayyo! 👋 Welcome new 🍷 Pinot members! How did you find us Pinot? What kind of projects are you working on? @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User

Hi. Is there any plan to have a Grafana plugin for Pinot.

Does the S3PinotFS support server-side encryption via KMS or does it require implementing a custom PinotFS?

Hi, I have setup pinot on EKS, how can I start with batch upload from s3 ? how can I set configs,? create tables ? I have started the incubator .