set the channel topic: Welcome to the community! Please show your support by starring our project repository on GitHub https://github.com/apache/pinot

Meetup next week! 😄 By LinkedIn engineers on the Apache Pinot team! @User (PMC) and @User https://www.meetup.com/apache-pinot/events/280616089/

Dear Pinot Fans! This is a remainder OSA Con is coming up fast and the call for proposals is closing soon. If you have an idea for a talk on open source analytics (apps, projects, or both), we would love to hear from you. Head over to the conference page to submit your talk. We'll be announcing sessions and opening up attendee signup next week. https://altinity.com/osa-con-2021/

Join LinkedIn engineering team members in 15 minutes for this meetup! @User @User See you there! 🙂 https://www.meetup.com/apache-pinot/events/280616089/

Good session! 👍

In case you missed it 🍷 Presentation by @User (Senior Software Engineer @ LinkedIn and Apache Pinot PMC) @User (Software Engineer @ LinkedIn):

Hi, is batch ingestion the only way to deal with late data arrivals in a hybrid table at the moment? Are the any plans to add some sort of out-of-window ingestion support?

I’ve always been fascinated by sketch algorithms. Here’s a paper that talks about a related issue: how do you estimate the result of a query, when the filter condition (the predicate) is very expensive to compute? Their solution is to create disjoint groups (stratification) using a (faster) proxy for the expensive predicate.

Hi Team, I, along with my team mate @User and @User we have documented our understanding around how to handle duplicate data in Pinot in this medium blog. Please review it and let us know if there are any feedback

🔈 DeveloperWeek is looking for speakers!!! Interested in presenting about what you're doing with Pinot? Submit here: https://www.developerweek.com/conference/call-for-speakers/

DevNexus also looking for speakers! Let's get you all in the developer speaker circuit! 😄 https://devnexus.com/

SIGMOD is another one! All about databases. Get your talk proposals submitted! https://2022.sigmod.org/

seems select is ok, but failed in order by

As suggested by you, please find the output of debug endpoint

Does Pinot have parcels?, so we can add into cloudera manager(cdp) and installed?

HI Team, We are doing pinot poc for offline ingestions currently. Currently facing an issue , while ingesting segment from s3 to pinot.

2021/11/03 08:29:22.109 INFO [SegmentFetcherFactory] [HelixTaskExecutor-message_handle_thread] Segment fetcher is not configured for protocol: s3, using default
2021/11/03 08:29:22.109 WARN [PinotFSSegmentFetcher] [HelixTaskExecutor-message_handle_thread] Caught exception while fetching segment from: <s3://pinot-db/pinot-ingestion/mytable/mytable_OFFLINE_2021091800_2021091800_0.tar.gz> to: /tmp/data/pinotSegments/mytable_OFFLINE/tmp-mytable_OFFLINE_2021091800_2021091800_0-90b8d75e-b2e8-4e4f-b115-36e5528c37cf/mytable_OFFLINE_2021091800_2021091800_0.enc
java.lang.IllegalStateException: PinotFS for scheme: s3 has not been initialized
        at shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:518) ~[pinot-all-0.8.0-jar-with-dependencies.jar:0.8.0-c4ceff06d21fc1c1b88469a8dbae742a4b609808]
        at org.apache.pinot.spi.filesystem.PinotFSFactory.create(PinotFSFactory.java:78) ~[pinot-all-0.8.0-jar-with-dependencies.jar:0.8.0-c4ceff06d21fc1c1b88469a8dbae742a4b609808]

Following are our conf: Server conf: pinot.server.instance.enable.split.commit=true pinot.server.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS pinot.server.storage.factory.s3.region=us-east-1 pinot.server.segment.fetcher.protocols=s3 pinot.server.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher Controller conf: controller.data.dir=s3://pinot-db/ controller.local.temp.dir=/tmp/pinot/ controller.enable.split.commit=true pinot.controller.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS pinot.controller.storage.factory.s3.region=us-east-1 pinot.controller.segment.fetcher.protocols=file,http,s3 pinot.controller.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher

🔈 This conference is still accepting speaker submissions!!! Should be a good one. If you have a good story about your Apache Pinot use case, please submit here 🔈 https://devnexus.com/

🎵 If you like Pinot and you know it, tell your friends 🎶 🤣 In all seriousness, check out this great set of blog posts by @User and @User on "_What Makes Apache Pinot so Fast._" They worked pretty hard on this, so if you love Pinot (and love our PMCs and committers) feel free to share with your network! 💕 https://www.startree.ai/blogs/what-makes-apache-pinot-fast-chapter-1/ https://www.startree.ai/blogs/what-makes-apache-pinot-fast-chapter-ii/

@User has left the channel

Meetup tomorrow (intro level), if anyone wants to join. Feel free to invite friends as well if you think they would benefit from Apache Pinot. ❤️ https://twitter.com/startreedata/status/1457788774360903682

🔈 Reminder - this conference has an open CFP (looking for speakers). CFP deadline is this week. Be a conference speaker and share your story about Apache Pinot https://devnexus.com/

Live in 5 min if anyone wants to join (Intro level on Apache Pinot)  https://twitter.com/startreedata/status/1458475789675073544

👋 Hello newbie Pinot community members! 🍷 🥳 We're happy to have you here! Curious on what you're working on and how you found Apache Pinot! Please introduce yourselves here in this thread! 😃 @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User @User

Kafka Summit London looking for presenters! 🙂 CFP closes Feb 2022. Submit your talk now! 🥳 https://www.kafka-summit.org/events/kafka-summit-london-2022/about

Hello Community, We are pleased to announce that Apache Pinot 0.9.0 is released! Apache Pinot is a distributed columnar storage engine that can ingest data in real-time and serve analytical queries at low latency. The release can be downloaded at https://pinot.apache.org/download The release note is available at https://docs.pinot.apache.org/basics/releases/0.9.0 Additional resources - Project website: https://pinot.apache.org Getting started: https://docs.pinot.apache.org/getting-started Pinot developer blogs: https://medium.com/apache-pinot-developer-blog Intro to Pinot Video:  Join Pinot Community - Twitter: https://twitter.com/ApachePinot Meetup: https://www.meetup.com/apache-pinot/ Slack channel: https://communityinviter.com/apps/apache-pinot/apache-pinot Best Regards, Apache Pinot Team

Hi Team, Few questions regarding Realtime Table. Cluster setup: 2 servers, 1 controller, 1 Broker. 1) If one of the server is down, consumption is completely stopped. why other server doesn't start consuming? 2) if consumer.prop.auto.offset.reset is set to smallest, from which offset it will start consuming? from last committed offset in zk? 3) If i change table config , does it reflect straight away ? Thanks.

I tried to upload a segment with data sorted by a multi valued string column but pinot server shows isSorted = false in the segment metadata.