Hey all! FYI, we’re launching a Pinot OSS YouTube channel soon! 📺 Before we go live, we want to gather all the must-have Pinot videos ; tutorials, meetup recordings, use cases, or anything valuable for the community. Got a video in mind? Maybe one that helped you understand Pinot better? pls drop it in this thread or share it on #C08GH2MAVT4! Thank you ❤️

Hey there 👋 I'd appreciate a PR review 🙏 https://github.com/apache/pinot/pull/15015 It's done for this issue The PR expands

/ingestFromURI

endpoint, which is not meant for production, and the PR includes tests

Hi, anyone looking for a helping hand on issue they are working upon...coding, unit tests etc? I have bandwidth for few days.

Here's the list of currently-open "beginner-task" marked Github issues. Consider starting here, and if you have more time, there's meatier work ahead for sure! https://github.com/apache/pinot/labels/beginner-task

Here's also a bunch of "help-wanted" tasks, if you want to roll your sleeves up: https://github.com/apache/pinot/issues?q=is%3Aopen+label%3A%22help+wanted%22

https://apache-pinot.slack.com/archives/C011C9JHN7R/p1741911243212059 cross-posting for visibility 😆

Created a PR https://github.com/apache/pinot/pull/15377 May someone have a look?

Hey all! The Apache Pinot Contributor call recording is available here #C08H9R5MHTN

Hi, while generating response to a SQL request, which layer in codebase writes to the response?

Watch on-demand: •

Apache Pinot Contributor Call Kickoff | April 3rd, 2025▾

Then tell us what you think: • Apache Pinot Contributor Call Survey Slack Conversation

Hi all, we have started facing node volume affinity issue in our pinot pods after we did an AMI version update to our EKS node groups on which our Pinot is hosted. We had only a single pod(for controller and zookeeper) and since PVCs are AZ bound and the EKS on AMI update changed the AZ in which the node is present we faced this issue. We are planning on having one nodegroup per every AZ(total 3) we have. 3 node groups of controller and zookeeper. Are there any best practices we can follow to avoid this?

Hi All , as part of vulnerability fix avro version didn't changed from 1.11.3 to 1.11.4 . In pinot pom.xml we are seeing that version is updated to 1.11.4 but in the dependencies jar we are not seeing the updated version . could you please let us know on this

Some basic questions. When an index is created, like for null values,what component decides that a given value corresponds to null value. For eg, for a float column, -infinity value is mapped to null. Same for NaN and +infinity.

Basically I am looking into population of null bit maps.

I see few files getting created in the process...columns.psf, floatfield.bitmap.nullvalue.

But couldnt trace where exactly population happens.

@Xiang Fu What’s the difference between Minion and Minion Stateless?

Hi folks, I saw there was a recent change aimed at detecting backwards incompat api changes for pinot-spi - I found this makes it pretty difficult to add more args for index config constructors. Eg., trying to add a new arg for JsonIndexConfig, we now see the error:

Error:  Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.23.1:cmp (default) on project pinot-spi: There is at least one incompatibility: org.apache.pinot.spi.config.table.JsonIndexConfig.JsonIndexConfig(java.lang.Boolean,int,boolean,boolean,java.util.Set,java.util.Set,java.util.Set,java.util.Set,int,boolean):CONSTRUCTOR_REMOVED -> [Help 1]

Any concerns with excluding some config classes (or parent config package) from this check? or should we be creating a new constructor for every config addition going forward?

Apache Pinot + Azure Event Hub Integration: Metadata Fetch Timeout Hey team, I’m integrating Apache Pinot (Dockerized) with Azure Event Hub (Kafka-compatible endpoint) for real-time ingestion. The connection works perfectly from a standalone Python Kafka client using confluent_kafka, but Pinot fails with: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata ✅ Confirmed Working: Pinot Server can curl and TLS handshake with icpeventhubns.servicebus.windows.net:9093 Python consumer successfully consumes data from ICPtrafficdataeventhub topic Using correct sasl.jaas.config format with $ConnectionString and EntityPath streamConfigs use stream.kafka.consumer.prop.* correctly ❌ Still Seeing: Pinot throws a metadata fetch timeout when creating the real-time table, even though connectivity and credentials are confirmed. Stream Config: For your reference ---------------------------------------- "streamConfigs": { "streamType": "kafka", "stream.kafka.topic.name": "icptrafficdataeventhub", "stream.kafka.broker.list": "icpeventhubns.servicebus.windows.net:9093", "stream.kafka.consumer.type": "simple", "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory", "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder", "stream.kafka.consumer.prop.bootstrap.servers": "XXXns.servicebus.windows.net:9093", "stream.kafka.consumer.prop.security.protocol": "SASL_SSL", "stream.kafka.consumer.prop.sasl.mechanism": "PLAIN", "stream.kafka.consumer.prop.ssl.endpoint.identification.algorithm": "", "stream.kafka.consumer.prop.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"$ConnectionString\\\" password=\\\"Endpoint=sb://XXXns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=My acess key=;EntityPath=icptrafficdataeventhub\\\";", "realtime.segment.flush.threshold.rows": "50000", "realtime.segment.flush.threshold.time": "24h" } #C013WKLT5T7 #C03N1JNHXLY

Hi All , Unable to connect to SSL Enabled Zookeeper getting the following error 2025/04/24 095621.256 WARN [ClientCnxnSocketNetty] [zkNetty-EpollEventLoopGroup-3-1] future isn't success. io.netty.channel.StacklessClosedChannelException: null at io.netty.channel.AbstractChannel$AbstractUnsafe.ensureOpen(ChannelPromise)(Unknown Source) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] 2025/04/24 095622.259 INFO [ClientCnxn] [main-SendThread(localhost:2191)] Opening socket connection to server localhost/127.0.0.1:2191. 2025/04/24 095622.260 INFO [ClientCnxn] [main-SendThread(localhost:2191)] SASL config status: Will not attempt to authenticate using SASL (unknown error) 2025/04/24 095622.262 WARN [ChannelInitializer] [zkNetty-EpollEventLoopGroup-3-1] Failed to initialize a channel. Closing: [id: 0xd0530de0] org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /nfmt/instance/certificates/External/truststore.ks at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:612) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.common.ClientX509Util.getTrustManager(ClientX509Util.java:198) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.common.ClientX509Util.createNettySslContextForClient(ClientX509Util.java:77) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initSSL(ClientCnxnSocketNetty.java:449) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:438) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:424) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:1130) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:558) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:45) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1410) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1064) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:599) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:513) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:428) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:485) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /nfmt/instance/certificates/External/truststore.ks

is there any way secure the zookeeper logs with ssl

please guide us to solve this where we are unable to make zookeeper as ssl and zookeeper data logs is storing without encryption

Hi all, I am using pinot-offline ingestion to upload data from Hdfs Cluster A into Pinot backed by Hdfs cluster B using spark. The hadoop config is read from the cluster on which the spark job is running. Is there a way to pass Hdfs config of both the clusters together... Or any other approach that you suggest??

Hi @Jackie , @Xiang Fu - Kindly review this : https://github.com/apache/pinot/pull/15844 https://docs.google.com/document/d/1M5OLUkW0eqn6ZnRYH1gg7lh7VrrJyu43gubix8xHm-s/edit?tab=t.0

A quick question,

cluster config setting will take priority or client setting will take priority in pinot ?

i.e: am setting pinot.broker.timeoutMs = 60000 in broker and server config file itself, but if some clients want to override this setting, can they set at client level a new setting like pinot.broker.timeoutMs = 1200000 and get the query executed ?

Created a PR https://github.com/apache/pinot/pull/16038/files

Hi all, I was setting up SSL on controller UI using below configs, is there any way to encrypt the password in controller conf file

# SSL Enablement
controller.tls.keystore.path=/etc/security/certificates/keystore.jks
controller.tls.keystore.password=Hadoop@123
controller.tls.truststore.path=/etc/security/certificates/truststore.jks
controller.tls.truststore.password=Hadoop@123
controller.access.protocols=https
controller.access.protocols.https.port=9443

Hi all , we are seeing this Added support to pause and resume ingestion based on resource utilization (Added support to pause and resume ingestion based on resource utilization by rajagopr · Pull Request #15008 · apache/pinot · GitHub) got merged on feb 17 and we are seeing the code is not there in the 1.3.0 source code . which adds the resource utilization code . could you please let us know on this which apache pinot version will get added

Hi Everyone, We are working on Pinot 1.4 release! I am planning to cut the Pinot 1.4 Release from HEAD on July 7th. All the commits merged into master branch before EOD on July 6 will be part of the release. Thanks cc: @Mayank @Xiang Fu @Ting Chen @Yupeng Fu