Is there any way to get a query plan from Pinot, such as indexes used to pull data, or is this something that you can only really interpret from the query response stats on the Controller's query console?

Yes, we are trying to benchmark, Presto-Pinot combination for 10GB to 200GB TPCH data. We want good query performance with fast loading capability. When compared with other OLAP dbs, Pinot seems to be taking long time for loading data. One observation is, the standalone job is using a single CPU(not sure how many threads) for a single upload job, even when there are having multiple files in import folder. Other OLAP dbs seem to load data using more than one CPU. Is there any setting to make Pinot import job use more than one CPU? Using HDFS ,S3 or GC is not in the scope of bench marking, We want to minimize the dependency on Hadoop or other Big systems because the data sizes we are targeting are not truly BigData terriroty.

Does it support any exactly once guarantees like Druid ?

Is there support for Piont with existing parquet files and if it supports, can it build it's own indexing on top of that ?

Hi, question about ServiceManager.. is it a feature or bug to have multiple instances of the same role in bootstrap services? eg. 2 minions

Is there a tool / utility to read segment data . modify it and upload . I was looking to generate bulk data for query performance testing pinot . Thanks ..

Hi there 👋

hi. wondering about composite health endpoint for servicemanager

mind pushing a snapshot here? I can use docker to get latest, but I want to play with using dependencies as flat classpath is so much faster to boot https://repository.apache.org/content/repositories/snapshots/org/apache/pinot/ I can use jitpack but it is awkward and doesn't work to get dep tree due to this: "<version>${revision}${sha1}</version>"

I am new to this project, so I just wanted to know on how to build this project in debug mode?

Curious: Are there any plans of introducing per doc TTL?

hi, what's the general advice to bootstrap pinot as it comes up (thinking servicemanager where you have everything you need in one container)

Hi Community, friendly reminder of the Pinot virtual meetup today (9/2 6pm). Hope to see you all there: https://linkedinpinotmeetup.splashthat.com/

I see that 0.5.0 rc1 is pushed to docker. this is exciting. Is it also pushed to a stable maven repo?

any way to get rid of this error nicely besides hushing the logger in a new cluster (with ServiceManager)

pinot              | 08:53:07,679 WARN [HelixController-pipeline-default-hypertrace-views-(f3017057_DEFAULT)] controller.LogUtil (LogUtil.java:11) - Event f3017057_DEFAULT : Cannot confirm top state missing start time. Use the current system time as the start time.

It seems that controller.data.dir pinot.server.instance.dataDir and pinot.server.instance.segmentTarDir need to be absolute paths. this is a little annoying. can this be relative?

@User Hey Pinot Team, that was great virtual meetup last night. Got to see some amazing developments going on with pinot. Could you please share the link to session recording if it has been uploaded somewhere?

question.. is there a side-effect on ZooKeeper from running

bin/pinot-admin.sh AddTable

or is it contained to pinot's data directory

It seems PinotAdministrator doesn't return exit code 1 on failure.. this makes use in shell scripts difficult

Hello guys! Super-new here. Qq - do we still use Zookeeper inspector for local installation? I couldn’t find its reference in here but saw it mentioned as a pre-requisite in the video. I’m asking this because I have minor thing to add in the documentation.

And the slides can be found here: https://www.slideshare.net/MayankShrivastava25/apache-pinot-meetup-sept02-2020

Is it possible to get JVM metrics like gc time, number of threads , heap / non heap memory etc from pinot-server ? I looked at 1. https://github.com/apache/incubator-pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerMeter.java 2. https://github.com/apache/incubator-pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java but did not find much other than

REALTIME_OFFHEAP_MEMORY_USED

I don’t think this is possible. I am getting

java.lang.IllegalStateException: Cannot read single-value from Collection

in the server logs.

How does Pinot's real-time ingestion handle out-of-orderness for the event timestamps? eg: if we have event timestamps that may have 30s out-of-order, how does Pinot address that? What about longer time-ranges (1min, 10min?) For context: we have publisher time timestamping that naively allows publishers to specify a "created" timestamp, but we force it to be the current timestamp if it's >30s from the current timestamp. This leads to some out-of-orderness.

Follow up question from (^): since segments are not time-partitioned, do hybrid table which need to combine star-indexes use the time-granularity to divide the star-index (ie: allowing for star-index on online to still be efficient?)

Also, does the offline data generation intentionally try to shuggle and generate segment data so segments will contain only one time-interval (ie: generate well-partitioned segments)?

i was hoping that setting up deep store using GCS will save me from perpetually having to expand disk

or another question, is there a TTL for data in a table?