Yes it can be set in the table config. See this https://docs.pinot.apache.org/configuration-reference/table#segments-config

Thanks

Is this something I should be concerned about, or can I safely ignore it?

Another awesome blog by @User! This time on plugging your own UDFs for queries into Pinot 🍷 https://twitter.com/ApachePinot/status/1310967887008497664

Good afternoon folks! Our chapter 2 in the Pinot Segment Lifecycle series is out! Please enjoy this video on Segment Assignment in Realtime. This also gives an overview on Pinot’s realtime ingestion architecture.

And in case you missed chapter 1 on Segment Assignment in OFFLINE tables, here’s the link to that. Cheers!

We have a new blog out, on Monitoring Pinot with JMX, Prometheus and Grafana. I’m sure this will be very useful for everyone here. Thanks @User for this great piece! https://medium.com/apache-pinot-developer-blog/monitoring-apache-pinot-99034050c1a5

<!here> hello dear pinot community, I'm evaluating pinot in our POC and comparing to druid. One thing that I couldn't find in the doc is the rest api to trigger a batch ingestion. Is CLI the only way of submitting batch ingestion job?

@User on the controller, there is a

POST

api:

<http://controllerhostname>:whateverport/segments

Taking a moment to pause and celebrate that the Apache Pinot community crossed 600 members this month! Big shoutout to everyone in the community! https://twitter.com/ApachePinot/status/1318232173917343746

And, excited to announce our next virtual Apache Pinot meetup in exactly 10 days, hosted by the awesome folks at Uber! We have a superb speaker lineup consisting of folks from Uber, City Storage Systems and Confluera. RSVP 👇 https://twitter.com/ApachePinot/status/1318293974990397441

Please check out our recent blog on how we operate pinot at Uber scale. We are glad to share our learnings with the community. https://eng.uber.com/operating-apache-pinot/

<!here>

Please join us for a Virtual Meetup hosted by Uber and Apache Pinot community on October 29th at 5:00PM PDT!! Please RSVP at https://www.meetup.com/apache-pinot/events/274044806/.

for those interested, there's a Pinot subreddit https://www.reddit.com/r/ApachePinot

Hey folks, @User and @User from the community will be speaking at Flink Forward, which is happening today! They will talk about how Apache Pinot and Apache Flink power realtime analytics at Uber. Their session is at 12.50pm PDT. Please tune in if you’re around! https://twitter.com/ChinmaySoman/status/1317172229990330368

@User - can you please share the recording of the session once its available ?

@User @User Are you guys planning to work on https://github.com/apache/incubator-pinot/issues/6189 ?

If not, we want to take this particular item

time pruning on the broker side

Totally cool with someone else doing it. It’s just something my company would need if we’re going to start using Pinot, and would implement if we decided to move forward with it.

@User ^^

Quick reminder about tomorrow's meetup at 5 PM PST. We have amazing talks lined up. @User - Uber, @User - Confluera @User @User from City Storage Systems. https://www.meetup.com/apache-pinot/events/274044806/

https://uber.zoom.us/j/92749891719?pwd=a2daZUs0cjRXZmJOcnFPc1RXamFlZz09 Passcode: 541532

<!here> meetup happening now ^^

Hi

👋 Hi everyoneeeee!!! I'm new to the Pinot community and I want to shape some of the Pinot community programs for the future. 🍷 I'm looking to get to know some people in the community (newbies and/or seasoned). Anyone here open to having a 30 min video conversation with me? Pleaaaassssseeeee!!!! 🤩 (I only look creepy in the pic of me hiding in corn fields... I'm not that bad). Send a DM or just schedule a time to chat: calendly.com/karin-wolok

Dear Community, looking for some help with

scala-maven-plugin

and gpg signing;

We are trying to cut the next release for Pinot, and running into the following error.
This seems to be due to the fact that `maven-source-plugin` and `scala-maven-plugin` have separate lifecycles, and the gpg files created by the former get deleted by the latter.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-artifacts) on project pinot-spark-connector: Exit code: 2 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-artifacts) on project pinot-spark-connector: Exit code: 2

Our question is if this PR is the right way to address the problem, or there are more elegant ways? Note that this PR would require that we specify additional targets for signing artifacts.

What’s the Pinot philosophy on dimension tables with time series datasets? Say time series

a

joins to time series

b

on some primary key. i.e.

SELECT *
FROM a
JOIN b on a.id = b.id
WHERE a.timestamp BETWEEN a and b

If you were to do a traditional join, you potentially scan all of the data in time series

b

. With bloom filters, you could maybe scan fewer columns. You can also specify a time bound on

b.timestamp

to make it slightly less bad. Alternatively, you could create a pre-aggregation before Pinot that joins them together and puts the important metrics from time series

b

on to time series

a

. What’s the preferred way to solve this problem in the Pinot world?