Are we good with this PR functionally? https://github.com/apache/pinot/pull/8314

Is there a way to set the number of segments required when creating a test data set for an integration test?

hey there, can I get someone to review this PR? https://github.com/apache/pinot/pull/8625 I'll need this to finish implementing timestamp and json support in trino connector. I've left a comment with regards to timestamp and time zones, I'll raise separate issue for that if there isn't one yet

@Rong R Hi Rong. Sorry to bother you. I'm trying to build Pinot on branch 'pr-query-integration'. I followed instructions from this Conversation(https://github.com/apache/pinot/pull/8662). But when I try to query on broker, I encountered some problems. When I query like this "SELECT * FROM baseballStats_OFFLINE", it will return some results. But if I query like this "SELECT playerId FROM baseballStats_OFFLINE", it will take almost 10 seconds, and return an empty result. It's same when I use 'inner join', doesn't report any exception or error, but just return empty result. I started Pinot like this 'bin/pinot-admin.sh StartServiceManager -bootstrapConfigPaths conf/pinot-controller.conf conf/pinot-broker.conf conf/pinot-server.conf'(I has already started Zookeeper), and all components on the same machine(1 controller,1 broker,1 server) And now I don't know what happened and want to know how to locate problems, could you please give me some advices? Thanks.

Hi 🙂 Working on integration testing for the purgeProcess implementation and looking for a way to set

SegmentZkMetada

my

lastPurgeTime

during the segment build using something like what have been done in

MergeRollupMinionClusterIntegrationTest

in the

buildSegmentsFromAvroWithPostfix

method. Is there any way to put a metadata to the generated segment in order to validate my code ?

In Pinot, segments belonging to a particular dataset may have different indexes. Is this correct? If so, what is the benefit, aside from not having to reload all segments to apply index changes?

Hello, I'm reading the freshness metrics design document which is something we are thinking of using in one of our uses cases. However the freshness timestamp returned by pinot always seems to be the pinot indexing time. Reading through the code it seems there isn't a row metadata implementation for kafka. I'd like to confirm this is the case and if so I'd like to contribute the code changes to get this working as per the design document. I can't see an open issue in github related to this

I thought it used timestamp from row metadata if it’s available

Yup, it does choose indexing timestamp if row metadata is not available. Doesn't seem like any of the stream ingestion plugins returns row metadata at the moment

The default implementation returns null

I'm going to raise an issue in github and open a PR, I'll post the link here if further discussion is needed

👍

This is the PR, I didn't write a lot of details on it nor in the linked github issue. Apologies for the rush https://github.com/apache/pinot/pull/8774

Is anyone else getting a dependency convergence failure when building from master? Details in thread…

Hi all, this may be a silly question, but are server tags and tenants synonymous? Just noticed when I created my own server tags they also show up under tenants.

Yes.. server tags are super set of tenants..

Tags can be used for an anything.. we use tags to implement tenant functionality

Excellent, that makes sense, thanks!

We use tags for tiering, node pools etc

Tags are similar to k8s labels

If we are overriding server tags on our tables is it good practice to also keep the default ones there?

It basically means that if some creates a table without tenant name, then these servers will be picked

So will there be potential problems if I remove those tags in favor of my own?

Yes but you can run rebalance anytime

Another question. We are building a realtime table with tiered storage. The base storage will be on a realtime server and we have the tiered storage configured to move to an offline server after 7 days. So it makes me thing a realtime table can have data in an offline server. Is that correct?

We do have an offline version of this same table configured, but I don't think that table will get any actual data (so probably not needed) but me just moving the realtime data to the offline servers.

My tables are immutable_events_realtime and immutable_events_offline. My question in other words is if I query immutable_events_offline directly I will not have any data. It all exists in immutable_events_realtime, the first 7 days in realtime server segments and the older data in offline server segments.

And if that's true, I'm not sure I even need my offline table

Sorry all, I think I've been asking my questions in the wrong channel, my apologies.