Hi, @Yash Mayya is working on enhancing the JSON index to support nested exclusive predicates (PR). Currently the exclusive predicate semantic is inconsistent with inclusive predicate (well explained in the description) which prevents us from supporting nested predicates. We plan to change its semantic to be consistent with inclusive predicate. Please let us know if your use cases rely on the old behavior (shouldn't be common).

hey folks, can somebody help by quickly merging a hotfix for upsert-compaction -- https://github.com/apache/pinot/pull/13157 cc @Seunghyun @Jackie

Hi team, in Uber, we want to increase the Pinot's SQL parser identifier max length (i.e. table column name length) to more than the default 128 characters. This is can be done by setting the identifierMaxLength in the Calcite SqlParser config. The code which set up the configs in Pinot is in

src/main/java/org/apache/pinot/query/parser/ParserUtils.java

. Obviously, we can hard-code a larger max length which is fine with us. For OSS, is there any recommendation on the mechiansm to pass in this max length as a config property?

Ready for review: https://github.com/apache/pinot/pull/13222

Hi, When do we usually release a new helm chart version? There are new changes for the helm chart recently merged to master branch. I need to deploy these new changes but I would rather want to deploy helm chart from version tag rather from the master branch directly.

Hi everyone, can someone guide me to integrate basic auth while running pinot using docker-compose.yml file

We have run into an issue with the version 1.1.0 upgrade because there is a couple of places that construct an HttpClient using

HttpClients.custom()

(like here and here) which does not default to the default

SSLSocketFactory

but instead creates a default here. That means that requests are failing between the different nodes. I can add a fix for this by either enabling the use of system properties (with useSystemProperties) or manually define the SSLContext. What are we thinking?

It is possible to use custom Scalar function while doing ingestion transformation, from the documentation i can see we can use built-in Scalar function. Can someone help on this ?

Hey folks, I wanted to solicit opinions on adding "collaborators" to the Apache Pinot project via asf.yaml that assigns the GitHub "triage" role to contributors. The exact permissions are documented here, but essentially it'll allow them to assign, edit, and close issues and PRs (without giving write-access to the code) and also add / remove labels, request PR reviewers etc. The ASF allows 10 collaborators per repository by default and in the Apache Kafka project we adopted a policy to choose the top 10 non-committer contributors in the last year (https://kafka.apache.org/contributing). This list is supposed to be updated every release to account for changes in contributors and to account for existing collaborators potentially becoming committers. The collaborator policy was introduced to the Kafka project over a year ago and has been fairly successful. This PR is a proposal to introduce the same policy to the Pinot project as well.

Thanks @Yash Mayya for taking the initiative. +1 from my side

Hi team, do we not support inverted-index along with text-index? Ref: https://docs.pinot.apache.org/basics/indexing/text-search-support#:~:text=Using%20a%20text%20index%20in%20coexistence%20with%20other%20Pinot%20indexes%20is%20not%20supported

Hi all, This is my first PR for a fix for helm chart https://github.com/apache/pinot/pull/13450 Requesting review for it. Thanks!

Hi folks, Just wanted to call out that we will be cutting the Pinot 1.2 Release from HEAD on June 30. Commits merged before EOD on June 30 (PST) will be part of the release. Thanks

👋 this PR needs review:

migrate from Apache HttpClient 4 to Apache HttpClient 5

https://github.com/apache/pinot/pull/13222

👋 👋 this PR needs review

migrate from Apache HttpClient 4 to Apache HttpClient 5

https://github.com/apache/pinot/pull/13222

Hi, Can someone check this quick PR in pinot python client? Our developers need access to detailed query response returned by pinot api which will be available with this PR. We are particularly interested in numGroupLimitReached to figure out if the query result is accurate or not. https://github.com/python-pinot-dbapi/pinot-dbapi/issues/103

hi I am Amit I want to start contributing to this project can someone please help me how can i start?

Check this out! And congrats on taking the first steps to contribute! https://docs.pinot.apache.org/developers/developers-and-contributors/contribution-guidelines

Also, @amit raj, you might want to check out open github issues to see what's already desired by the community.

sure @Peter Corless

I came across the the .gitignore file and wondering about this line. Ref: https://github.com/apache/pinot/blob/master/.gitignore#L48

#build symlink directory
build*

Should it be

#build symlink directory
build

or

#build symlink directory
build/*

Looking for feedback:

S3 storage class config

https://github.com/apache/pinot/pull/13613

Hi All, Do we have connector to connect metabase with Pinot? @Mayank @Kishore G

draft PR --- I am looking for feedback https://github.com/apache/pinot/pull/13678

I am working with Pinot to deploy a UDF function, which requires me spinning up an AWS Client to interface with another service. What would be the best approach for initializing the client such that when the UDF gets invoked, the client is already pre-initialized? Does Pinot offer any hooks to be implemented for UDFs for pre-initialization? How can I pass on the environment variables for initializing the clients used as a part of the UDFs?

Hi. Can someone review this? https://github.com/apache/pinot/pull/12537

I've been working on a new explain mode for multi-stage queries that is able to include physical information (aka whether indexes are being used or not). It is still a draft, but you can start to look at it if interested: https://github.com/apache/pinot/pull/13733

hey folks, did anything change with respect to the preferred

mvn

build command for building locally? In the docs[1], I can see that the build command listed is:

$mvn install package -DskipTests -Pbin-dist -DdownloadSources -DdownloadJavadocs

However, when running this, I see the following warning and do not end up with a

build

directory which I did in previous executions:

[WARNING] The requested profile "bin-dist" could not be activated because it does not exist.

Any thoughts? Have there been intentional changes to the build process that aren’t yet documented? Trying to narrow down whether it’s my env or not [1] https://docs.pinot.apache.org/developers/developers-and-contributors/code-setup#maven

Hi team, can anyone explain to me what is the error mean in

Pinot Compatibility Regression Testing

CI workflow?

20:23:53.164 INFO [SegmentOp] [main] Successfully created segment: FeatureTest1_Segment5 at directory: /tmp/pinot-compat-test-segment-op-a499654e-d79a-40cb-9ec7-6e3057b2c4c4/output/FeatureTest1_Segment5
20:23:53.205 INFO [SegmentOp] [main] Tarring segment from: /tmp/pinot-compat-test-segment-op-a499654e-d79a-40cb-9ec7-6e3057b2c4c4/output/FeatureTest1_Segment5 to: /tmp/pinot-compat-test-segment-op-a499654e-d79a-40cb-9ec7-6e3057b2c4c4/output/FeatureTest1_Segment5.tar.gz
20:23:53.482 INFO [HttpClient] [main] Sending request: /v2/segments?tableName=FeatureTest1_OFFLINE to controller: <http://fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net|fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net>, version: 1.3.0-SNAPSHOT-01910b8468ce6e8b01c0a031610e3262d2c44619
20:23:53.533 INFO [HttpClient] [main] Sending request: /tables/FeatureTest1_OFFLINE/externalview to controller: <http://fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net|fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net>, version: 1.3.0-SNAPSHOT-01910b8468ce6e8b01c0a031610e3262d2c44619
20:23:53.551 INFO [HttpClient] [main] Sending request: /tables/FeatureTest1_OFFLINE/externalview to controller: <http://fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net|fv-az1432-390.2onsmhel0beebdhu3wzfcxcjxc.cx.internal.cloudapp.net>, version: 1.3.0-SNAPSHOT-01910b8468ce6e8b01c0a031610e3262d2c44619
20:23:53.552 INFO [SegmentOp] [main] Successfully verified segment FeatureTest1_Segment5 and its current status is ONLINE.
20:23:53.737 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:23:54.742 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:23:55.747 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
...
20:25:45.149 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:46.153 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:47.156 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:48.159 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:49.162 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:50.165 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:51.168 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:52.171 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:53.175 WARN [SegmentOp] [main] Routing table has not been updated yet, will retry after 1000 ms.
20:25:54.178 ERROR [SegmentOp] [main] Upload segment verification failed, routing table has not been updated after max wait time 120000 ms.

Is routing table some external service? Is there a way to run the unit test locally to debug?

Hi @Ankit Sultana @Neha Pawar , Can you help me get this reviewd : https://github.com/apache/pinot/pull/13645 Its been open since a week