This is the last call to submit your questions before we go live on Monday! 🔔 < Office Hours | Resources and Tools to Get Started with Apache Pinot > ⬇️ What to expect in this session ⬇️ ► How to run Apache Pinot™ locally ►Quick start ►AP 101 and 201 training courses ►How to run AP in Cloud ►How to connect apps to AP (Java, Python) ►Define the “range”/borders of the theme, what they can ask about in the form that’s related to the topic Sign up and submit it here, and get your questions addressed in a small forum by @Vik Gamov! 🍷

Happy Monday everyone! 🏃🏻‍♀️ This is the last chance to submit your questions and join @Vik Gamov today for an office hour to ask anything about Resources and tools to get started with Apache Pinot ☝🏻 Questions can also be submitted on #C07EUNCHERJ (pls tag me!)

Office hours, starting now, Join here>

https://x.com/gAmUssA/status/1820502371870412893 <- list of the resources I shared during office hours call.

It’s time for the Summer Challenge! 🍦 ☀️ Showcase your Pinot project and get a chance to win 100$ for Swag! Check out the summer challenge specs here; it is limited in time!

Hi All Recognizing the growing need for real-time decision-making, we published a blog highlighting the necessity of real-time OLAP (Online Analytical Processing) capabilities for organizations to stay competitive and responsive in today's fast-paced business environment. https://aws.amazon.com/blogs/big-data/build-a-real-time-analytics-solution-with-apache-pinot-on-aws/ Key highlights: - Ingesting streaming data into Pinot from Amazon Kinesis Data streams - Enabling low-latency analytical queries on real-time data - Scaling Apache Pinot clusters on AWS for high throughput - Visualize streaming data from Pinot using business intelligence tool

🍷 StarTree has been recognized as a Leader and Fast Mover in the 2024 Sonar Report for Real-Time Analytical Databases! 🎉!!! https://www.linkedin.com/posts/startreedata_realtimeanalytics-bigdata-innovation-acti[…]295430524929-TB4g?utm_source=share&amp;utm_medium=member_desktop

Hey everyone! Here are your updates for August ⬇️ 📢 Major News • StarTree Named Leader in Real-Time Analytics by GigaOm 💥 Access Report • Advanced Analytics with Apache Pinot 201 📼 🥂 Community • Meet your community peers: @Elon Azoulay (here) + @SiddTeotia (here) • Pinot Summer Swag Challenge | Showcase your project and get a chance to win 100$☀️ Learn More 🗓️ Events 📌 Mountain View, CA - August 22nd - Road to Current 2024 📌 Virtual - August 24th - GigaOM: Sonar Report for Real-Time Analytical Databases 📌 Bengaluru, India - August 24th - India Open Source Data Infrastructure Meetup 📌 Sunnyvale, CA - August 27th - Uber x Apache Pinot Meetup 🔴 Virtual - September 5th - Community Office Hours | Data Ingestion with Pinot

Join our upcoming Office Hours- Data Ingestion in Apache Pinot with @Kartik Khare and @Manish 📆 Sept 4TH, 3 PM IST/ 10:30 AM GMT Sign up here>

Check out this event with the MASTER @Jackie!! 💥 RSVP>

Hello all, quick question. Why does it say here Introduction (docs) that Microsoft Power BI is supported?

I'm evaluating Pinot for university and I came to the conclusion that Power BI isn't supported in any way, that's why I'm so confused by this.

Got questions about ingestion with Apache Pinot? 🍷 Submit your questions in advance, then join us to walk through them with our experts, @Kartik Khare and @Manish Don’t miss out on the opportunity to get hands-on with anything related to ingestion, all in a small group setting! Sign up here>

Hi, I have deployed Pinot on Kubernetes and it’s working great. The controller UI has been set through Kubernetes ingress but the issue is that it’s wide open. I am looking to add some kind of login UI authentication/authorization backed by Azure AD(user/group RBAC). Is it possible in open source pinot? Or any options in startree managed?

Hi all! I was wondering if anyone with some experience using the mutli-stage query engine in a production environment has any anecdotal or testing data on the scaling characteristics of it. How does it hold up under load? Are there any gotchas about the types of query patterns to either avoid or encourage? Thanks!

discrepancy between the counts from Pinot real-time and offline tables compared to the actual table count after minion job

For eaxmple,Above issue is when we query real time table it shows 10 records and query offline table it shows 20 records and when we query the table without real time and offiline tag part of table it is showing me 10 records.our expectation should be realtime table and offline table count i.e 30 records

Hey All, We have a use-case where we need to do ingestion aggregation at 15 mins level. We want to count the total occurrence of a particular metric. However, during query we have observed multiple entries for same timestamp and metric. [metric_id, timestamp, metric_value, count] [799,1723341600,“AT_LEAST_1080#107#CC#GB”,10],[799,1723340700,“AT_LEAST_1080#107#CC#GB”,13],[799,1723340700,“AT_LEAST_1080#107#CC#GB”,2] Is this a known issue in ingestion aggregation? Does anyone have any idea on where this aggregation is going wrong. Can pinot perform ingestion aggregation across servers as same metric_id can be polled by multiple servers?

We are getting Data loss On Realtime table when we set 3 hours as retention period .Our minion job is running every night which transfer the data to offline table. how can we avoid this situation.

📣 Apache Pinot 1.2.0 is now available 🚀 This release brings a ton of improvements to the Multistage Engine (MSE), Upserts and Compaction. We also have a bunch of improvements to JSON and Text Index Support, along with many other performance improvements and bug fixes. To highlight some: • MSE has support for new window functions, support for logical databases, better support for Multi-Value and Array Functions, and many more improvements. • Minions now support resource isolation via instance tags (aka tenants) and greedy upsert compaction scheduling for better compaction throughput. Additionally, we have made a bunch of reliability improvements to Minions. • Upserts have a consistent table view support which fixes rare under-count/over-count issues. There are multiple consistency modes available: NONE, SYNC, and SNAPSHOT. • Tons of JSON Support Improvements: ability to use JSON Index for evaluating range and regex predicates, contextual array filters and multi-value JSON field extraction support in jsonExtractIndex, isJson UDF for more options to handle invalid JSONs, etc. • Tons of Lucene and Text Search improvements: segment build time for lucene indexes reduced by 40-60%, faster phrase search support and many bug fixes. • New Funnel Functions: funnelMaxStep and funnelCompleteCount. • Support for SQL formatting in the Pinot Controller Query Editor. • New hash function for UUID Upsert Primary keys which can save 30-35% in old gen memory usage. Links: • Project website: https://pinot.apache.org/ • The Release can be downloaded here: https://pinot.apache.org/download/ • Full Release Notes: https://github.com/apache/pinot/releases/tag/release-1.2.0 Pinot Community: • X: https://x.com/apachepinot • Meetup: https://meetup.com/apache-pinot

I was looking for the Pinot 1.2.0 docker image. This command is NOT working for me:

docker pull apachepinot/pinot:1.2.0

This command works for version 1.1.0

docker pull apachepinot/pinot:1.1.0

@Ankit Sultana ☝️

Hello Team

we have hybrid table ,and we have not set any retention on realtime table..can you please let me know what happen once minion job finish.Will data remove from online table after move to offline table?

In support of the 1.2 release, @Vik Gamov wrote up this awesome blog to highlight all of the innovation added to Apache Pinot by the community! https://startree.ai/blog/highlighting-innovation-in-apache-pinot-1-2

“After evaluating several alternatives, including Elasticsearch and Druid, Apache Pinot emerged as the clear winner. Here’s why I chose Apache Pinot>” Great blog by @Unais Siddiqui ! Read here>

Hello everyone i wanted to know how do we implement the moving algorithm in the pinot as i was using the below query :AVG(B1 - B2) OVER (ORDER BY "TimeStamp" ROWS BETWEEN 14 PRECEDING AND CURRENT ROW) AS rolling_mean, where pinot do not support 14 or specific preceding rows and they use UNBOUND PRECEDING which refers to all the preceding rows so is their any alternative way of doing this ??

how to move late arrival data by minion job to offline table and reclaim disk space in pinot

Hello everyone. is there any documentation on Apache Pinot and Power BI integration? Please share.