Happy Monday everyone! We have an online meetup scheduled for tomorrow if anyone is interested. 🙂 https://www.meetup.com/apache-pinot/events/277817566/

Hi Everyone, Is there any way to change the default schema name. I.e. I have some tables in Pinot , and have a connection b/w presto and pinot, at presto side when I am doing show schemas so it's showing schema name as default and in this default schema my pinot tables are available.How can I change this default schema name.

Is there an ETA for a newer pinot release than 0.7.1? The documentation already refers to new data types: https://docs.pinot.apache.org/basics/components/schema#data-types that do not seems to be supported in pinot 0.7.1.

I’ve got some questions about the DateTimeFieldSpecs defined here: https://docs.pinot.apache.org/basics/components/schema 1. Are there any benefits (either way) to configuring a LONG column as a dateTimeFieldSpec if it’s being used in GROUP BY clauses of a query? 2. If I have a epoch time that I’ve divided into 15 minutes segments… it seems like the right way to configure format would be to use 15MINUTESEPOCH. But what does granularity do? The first sample where format is 1MILLISECONDSEPOCH but granularity is 15:MINUTES doesn’t make sense to me.

Hey all! 🎉 We have a great meetup next week and would love for you to join! 😃 TALK 1: Pinot vs Elasticsearch, a Tale of Two PoCs by @User TALK 2: Forward index reader performance improvement by @User https://www.meetup.com/apache-pinot/events/277817649/

Is table storage quota enforced for Realtime tables? If so, are consuming segments counted towards the quota or only on disk flushed segments are considered?

We have 4 billion events coming in everyday and average size is 1kb. We want to use apache pinot with s3 as deep storage. 1. Which is a good storage option? local disk or s3? 2. We use presto and need support for about 1000 concurrent queries which query about 10 collections in each query. Can this support be expected over pinot with presto?

3. If using s3, how will the realtime analytics use case happen? Is there a hot shard maintained on the local disk?

Deep storage is only for backup. Servers maintain a local copy of the data on local attached disk

Starting in 15 minutes! 🙂 https://www.meetup.com/apache-pinot/events/277817649/ @User @User

Hi all! Would love for you to join us tomorrow for this event! We have some great speakers from LinkedIn, DoorDash, Confluent, Microsoft, Decodable, Stripe and more! https://www.startree.ai/launch-event-register/

Hey, if I want to see the execution plan of a query. Where can I see that? Is the trace info the execution plan? I have a query like this

select * from table where DATETRUNC('day',updatedAt,'MILLISECONDS')>=1624492800000

and I want to know whether it use the range index or not on the updatedAt column.

"traceInfo": {
    "pinot-server-1.pinot-server-headless.analytics-prod.svc.cluster.local": "[{\"0\":[{\"SelectionOnlyCombineOperator Time\":1},{\"InstanceResponseOperator Time\":1}]},{\"0_0\":[{\"ExpressionFilterOperator Time\":0},{\"RangeDocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"SelectionOnlyOperator Time\":0},{\"ExpressionFilterOperator Time\":0},{\"RangeDocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"SelectionOnlyOperator Time\":1}]}]",
    "pinot-server-0.pinot-server-headless.analytics-prod.svc.cluster.local": "[{\"0\":[{\"SelectionOnlyCombineOperator Time\":3},{\"InstanceResponseOperator Time\":3}]},{\"0_0\":[{\"ExpressionFilterOperator Time\":0},{\"RangeDocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"SelectionOnlyOperator Time\":1},{\"ExpressionFilterOperator Time\":0},{\"RangeDocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"SelectionOnlyOperator Time\":0},{\"ExpressionFilterOperator Time\":0},{\"RangeDocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"DocIdSetOperator Time\":0},{\"ProjectionOperator Time\":0},{\"PassThroughTransformOperator Time\":0},{\"SelectionOnlyOperator Time\":0}]}]"

Hi, we have a table planned to migrate to Pinot, it has only dimension fields and one datetimefield, the data volume is nearly constant, we mainly use it as a look up table, is this a good candidate to migrate to Pinot to gain any query latency benefit from Pinot for this table?

Big hello and welcome to all our newest community members! 🍷 @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User Would love to. know who you are and what brought you here! 😃 Also, don't miss our event TODAY with presentations and panels from experts from companies like LinkedIn, Microsoft, DoorDash, Stripe, Confluent, Decodable and more! Starts in 2 hours! There will also be a great demo of StarTree Cloud! 🎉 🥳 We'd also love your support, please share this event with any of your friends who are also obsessed with SPEED and REAL-TIME-ANALYTICS! ⚡ https://startree.brand.live/c/launch

Hi Karin, we are building a highly responsive website and need sub-second query performance on OLAP data. We are studying aspects of the different technologies and Pinot seems like a very strong candidate on paper

In case you missed it! 🙂 🍷 Here is the recording from StarTree's launch event last week. Pushing the Boundaries of Real-Time Analytics We had an amazing line-up of speakers from companies like LinkedIn, Stripe, DoorDash, Decodable, Conlfuent, Microsoft and more! Big shoutout to our dedicated community members that attended and answered questioned, shared thoughts, etc. We were really happy to see those familiar faces. ❤️ https://startree.brand.live/c/launch

👋

Hi Everyone 👋

Hey all! Join us for this meetup today! Starting in 1.5 hours! Presentations by @User and @User https://www.meetup.com/apache-pinot/events/277818848/

In case you missed the meetup, you can watch it here! Slides from @User 's presentation are also available in the description!

Don't miss these 3 awesome meetups next week: Presenters: @User @User @User @User and Gunnar Morling! https://www.meetup.com/apache-pinot/events/277818929/

Posted in troubleshooting as well

Hi folks, I have a question regarding upserts

Let’s imagine we have this: • Table Schema: field1, field2, field3 • first event: key XYZ, field1, field2 • second event: key XYZ, field3

Docs are not clear whether the end state with upserts enabled will contain all the fields (field1, field2, field3), or only fields from most recent event (field3)

what you are looking for is partial upsert

without partial upsert it override all fields.. in your case you will only see values from second event