Is update query possible using pinot

@User I think Slack creates this by default. we dont really discuss anything random here

Starting in 20minutes! 💻 https://www.meetup.com/apache-pinot/events/285662605/

Hi Team, How I can parse kafka header(in bytes) and filter based on specific field ? any code samples?

Hey folks, PrestoCon Day is this Thursday! We’re excited to host a fully virtual, free community event by the Presto community for the Presto community. We have a great agenda including sessions from Uber, Tencent, Meta, Bytedance, Blinkit, Apache Hudi and many more. You’ll learn more about the open source Presto project and how companies of all sizes are using Presto for fast and reliable SQL for data analytics and the open lakehouse. Registration is FREE: https://events.linuxfoundation.org/prestocon-day/register/

Hi everyone, this is reminder that OSA Con 2022 is coming up on Tuesday November 15. It's a community cornucopia of open source analytics with talks on everything from Arrow to ClickHouse to Iceberg and HTAP. The conference is virtual and free. Everyone is welcome, so come on down and sign up! https://altinity.com/osa-con/

I made a short video of running Pulsar to Pinot

Hi guys, I have created an open-source repository in order to install Apache Pinot cluster using Ansible (docker based installation). I would be thankful if you could review this repository and contribute. https://github.com/mostafaghadimi/pinot-dockerized-ansible

is some one using pinot in production

LinkedIn, Uber, Stripe, etc.

is Tiered Storage available in opensource Pinot?

Im curious if anyone can share more context on how Pinot compares to Streaming Databases like RisingWave

have a look at Hubert’s post - https://hubertdulay.substack.com/p/stream-processing-vs-real-time-olap

and Dunith’s - https://medium.com/event-driven-utopia/comparing-stateful-stream-processing-and-streaming-databases-c8c670f3f4bb

Anyone know the pinot SQL to do something like this? “find records in last 1 day”

"select * from table_1 where time > now() - 1 day" OR select * from table_1 where time > now() - 86400

Why some of the parameters are case sensitive while some are not? It would be more convenient if we can standardize them.

Hi team pinot! My team just launched Openlayer — the easiest way to test, debug and collaborate on all sorts of ML models. Here’s some more info about us: https://www.producthunt.com/posts/openlayer Would love to hear your thoughts!

I have a question about realtime to offline job for upsert table. In this recipe, it says

Now let’s run the job to replace the offline segment, which has the name

orders_1632463351000_1632467070000_0

In production, how do we know which segment we need to replace?

Hi Everyone ! I have written a blog post + tutorial, on how pinot control plane can auto detect schema changes and initiate reload of segments based on state transitions on kubernetes. Feedback welcome 🙂 https://www.datainfra.io/blog/dynamic-data-schemas-pinot-schema-evolution-state-management-on-kubernetes

Hi developers who are interested in data security, Cisco and Altinity are meeting over a LIVE webinar tomorrow to showcase their collaborative project on deploying Clickhouse in FedRAMP using Altinity’s FIPS-compatible stable builds. Date and Time: June 20, 10 AM PDT Speakers: • Pauline Yeung, Data Engineer & SecDevOps at Cisco Umbrella and • Robert Hodges, CEO at Altinity Tune in LIVE to learn more about: • What is Cisco Umbrella and how does it use ClickHouse? • What are the challenges of bringing up ClickHouse in a FedRAMP environment? • How are Cisco Umbrella and Altinity working together to deploy FIPS-compatible analytics? • What lessons can we share with other users on the same path? RSVP your free seat here: https://hubs.la/Q01T8qyb0

Hi Team, we’re using same schema in Pinot-production for 2 use-cases and one of them is external customer facing and hence critical. While the other one is internal, it faces frequent addition of attributes and history loads to schema and hence data refresh happens every month. During data refresh, data gets loaded into older partitions and if other use-case is referring those partitions, it can create issues even though use-case1 doesn’t require those new attributes or history. How can we ensure the uninterrupted data availability (without downtime) to use-case1 when data refresh happens for use-case2 ? any read replica possible ?

Tomorrow, Olga Silyutina from Sumsub will show you how Sumsub uses a schema-agnostic approach to transform different event types with ClickHouse materialized views into a flattened form that’s convenient for analysis. Tune in at 10 AM CET, Thursday 20th to join the live discussion on Zoom. Get your free ticket now: https://hubs.la/Q01Ws9SW0

As many of you have noticed ClickHouse and Kubernetes work great together. It’s easy to stand up toy applications, but what about building an entire analytic service based on ClickHouse? This coming Thursday Robert will show you how to build the full stack using Kubernetes, ArgoCD, and open source software. There’s even a GitHub project under Apache 2.0 license with the code to do it yourself. Please join us to learn more: https://hubs.la/Q01WsbS_0

For the first time, a data loading library in python in the open source, built for data people, with simple declarative interface, auto schema inference and evolution and ecosystem integrations like dbt runner and airflow 1 line deploy. Now we support parquet too https://dlthub.com/docs/dlt-ecosystem/file-formats/parquet (and with it Pinot) try our verified sources: https://dlthub.com/docs/dlt-ecosystem/verified-sources/ or build your own pipeline (dlt is really for building pipelines ad hoc) https://dlthub.com/docs/getting-started/build-a-data-pipeline or try the colab demos (yeah it runs in colab with duckdb too and other micro envs cos it's a sensible, sane library, not a heavy framework) - bring your own orchestrator tho. https://dlthub.com/docs/getting-started/try-in-colab we build with open source pricples, no half shares or take backs https://dlthub.com/product/ join our slack and give us feedback if you like it

Hey all.. we open sourced LLMStack, a low code platform to build LLM apps, chatbots and agents. LLMStack includes everything out of the box needed to build AI apps using your own data. The enterprise edition of LLMStack uses pinot for aggregated metrics. Please check it out at https://github.com/trypromptly/LLMStack. Would love to hear if you have any feedback on the project or feature requests.

why is pinot's logo a glass of wine? 😬

@Mayank @Jackie Can anyone please share the link or info related to helm charts for pinot version 1.0

Hi! Try to ran joins against two offline tables. Join as simple as: SELECT FROM table1 JOIN table2 ON table1.some_int_field = table2.some_int_field; Got an error from broker with code = 200, from request handler. Could anybody help with it? @Mayank

Hi ! is Apache Pinot Server stateless in Kubernetes deployment ? As far as my understanding if server pod is restarted then the segments on that pod will copied to other pods and that metadata should be stored in ZK and whenever new pod comes up it again get's connected with ZK for future operations. Can anyone please correct me if my understanding is wrong. @Mayank @Jackie