https://pinot.apache.org/ logo
Join Slack
Powered by
# pinot-dev
  • n

    N R Neha

    02/25/2025, 6:59 PM
    Hi , I get this error when running pinot on local (Mac OS)
    Copy code
    2025/02/25 10:55:02.231 ERROR [PinotQueryResource] [grizzly-http-server-13] Caught exception while processing post request
    java.net.ConnectException: Operation timed out
    	at java.base/sun.nio.ch.Net.connect0(Native Method) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:579) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:568) ~[?:?]
    	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:576) ~[?:?]
    	at java.base/java.net.Socket.connect(Socket.java:666) ~[?:?]
    	at java.base/java.net.Socket.connect(Socket.java:600) ~[?:?]
    	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183) ~[?:?]
    Copy code
    2025/02/25 10:55:02.230 ERROR [PinotQueryResource] [grizzly-http-server-13] Caught exception in sendQueryRaw
    java.net.ConnectException: Operation timed out
    	at java.base/sun.nio.ch.Net.connect0(Native Method) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:579) ~[?:?]
    There are no errors in the server logs. Can you provide any pointers to unblock from this? Thanks
    a
    • 2
    • 1
  • r

    robert zych

    02/27/2025, 4:17 PM

    https://youtu.be/yRMhAL_6f4o?si=4TaNGP1CtrQTjVsF▾

  • y

    Yarden Rokach

    03/06/2025, 3:46 PM
    Hey all! FYI, we’re launching a Pinot OSS YouTube channel soon! 📺 Before we go live, we want to gather all the must-have Pinot videos ; tutorials, meetup recordings, use cases, or anything valuable for the community. Got a video in mind? Maybe one that helped you understand Pinot better? pls drop it in this thread or share it on #C08GH2MAVT4! Thank you ❤️
  • d

    Daniil

    03/07/2025, 1:43 PM
    Hey there 👋 I'd appreciate a PR review 🙏 https://github.com/apache/pinot/pull/15015 It's done for this issue The PR expands
    /ingestFromURI
    endpoint, which is not meant for production, and the PR includes tests
  • m

    Manish G

    03/11/2025, 12:41 PM
    Hi, anyone looking for a helping hand on issue they are working upon...coding, unit tests etc? I have bandwidth for few days.
  • p

    Peter Corless

    03/11/2025, 3:42 PM
    Here's the list of currently-open "beginner-task" marked Github issues. Consider starting here, and if you have more time, there's meatier work ahead for sure! https://github.com/apache/pinot/labels/beginner-task
  • p

    Peter Corless

    03/11/2025, 3:43 PM
    Here's also a bunch of "help-wanted" tasks, if you want to roll your sleeves up: https://github.com/apache/pinot/issues?q=is%3Aopen+label%3A%22help+wanted%22
  • b

    benson yuan

    03/14/2025, 12:34 AM
    https://apache-pinot.slack.com/archives/C011C9JHN7R/p1741911243212059 cross-posting for visibility 😆
  • m

    Manish G

    03/27/2025, 12:38 PM
    Created a PR https://github.com/apache/pinot/pull/15377 May someone have a look?
  • y

    Yarden Rokach

    04/03/2025, 5:44 PM
    Hey all! The Apache Pinot Contributor call recording is available here #C08H9R5MHTN
    r
    • 2
    • 2
  • m

    Manish G

    04/05/2025, 8:26 AM
    Hi, while generating response to a SQL request, which layer in codebase writes to the response?
    x
    • 2
    • 1
  • p

    Peter Corless

    04/07/2025, 6:31 PM
    Watch on-demand: •

    Apache Pinot Contributor Call Kickoff | April 3rd, 2025▾

    Then tell us what you think: • Apache Pinot Contributor Call Survey Slack Conversation
  • i

    Indira Vashisth

    04/23/2025, 3:02 AM
    Hi all, we have started facing node volume affinity issue in our pinot pods after we did an AMI version update to our EKS node groups on which our Pinot is hosted. We had only a single pod(for controller and zookeeper) and since PVCs are AZ bound and the EKS on AMI update changed the AZ in which the node is present we faced this issue. We are planning on having one nodegroup per every AZ(total 3) we have. 3 node groups of controller and zookeeper. Are there any best practices we can follow to avoid this?
    x
    • 2
    • 4
  • v

    vinod kumar naidu

    04/24/2025, 6:52 AM
    Hi All , as part of vulnerability fix avro version didn't changed from 1.11.3 to 1.11.4 . In pinot pom.xml we are seeing that version is updated to 1.11.4 but in the dependencies jar we are not seeing the updated version . could you please let us know on this
    x
    • 2
    • 2
  • m

    Manish G

    05/01/2025, 11:21 AM
    Some basic questions. When an index is created, like for null values,what component decides that a given value corresponds to null value. For eg, for a float column, -infinity value is mapped to null. Same for NaN and +infinity.
    x
    • 2
    • 3
  • m

    Manish G

    05/01/2025, 11:23 AM
    Basically I am looking into population of null bit maps.
  • m

    Manish G

    05/01/2025, 11:26 AM
    I see few files getting created in the process...columns.psf, floatfield.bitmap.nullvalue.
  • m

    Manish G

    05/01/2025, 11:27 AM
    But couldnt trace where exactly population happens.
  • r

    robert zych

    05/01/2025, 1:57 PM
    @Xiang Fu What’s the difference between Minion and Minion Stateless?
    x
    • 2
    • 9
  • c

    Chris Peck

    05/05/2025, 7:51 PM
    Hi folks, I saw there was a recent change aimed at detecting backwards incompat api changes for pinot-spi - I found this makes it pretty difficult to add more args for index config constructors. Eg., trying to add a new arg for JsonIndexConfig, we now see the error:
    Copy code
    Error:  Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.23.1:cmp (default) on project pinot-spi: There is at least one incompatibility: org.apache.pinot.spi.config.table.JsonIndexConfig.JsonIndexConfig(java.lang.Boolean,int,boolean,boolean,java.util.Set,java.util.Set,java.util.Set,java.util.Set,int,boolean):CONSTRUCTOR_REMOVED -> [Help 1]
    Any concerns with excluding some config classes (or parent config package) from this check? or should we be creating a new constructor for every config addition going forward?
    s
    x
    • 3
    • 12
  • n

    Nithinjith Pushpakaran

    05/07/2025, 9:07 AM
    Apache Pinot + Azure Event Hub Integration: Metadata Fetch Timeout Hey team, I’m integrating Apache Pinot (Dockerized) with Azure Event Hub (Kafka-compatible endpoint) for real-time ingestion. The connection works perfectly from a standalone Python Kafka client using confluent_kafka, but Pinot fails with: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata ✅ Confirmed Working: Pinot Server can curl and TLS handshake with icpeventhubns.servicebus.windows.net:9093 Python consumer successfully consumes data from ICPtrafficdataeventhub topic Using correct sasl.jaas.config format with $ConnectionString and EntityPath streamConfigs use stream.kafka.consumer.prop.* correctly ❌ Still Seeing: Pinot throws a metadata fetch timeout when creating the real-time table, even though connectivity and credentials are confirmed. Stream Config: For your reference ---------------------------------------- "streamConfigs": { "streamType": "kafka", "stream.kafka.topic.name": "icptrafficdataeventhub", "stream.kafka.broker.list": "icpeventhubns.servicebus.windows.net:9093", "stream.kafka.consumer.type": "simple", "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory", "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder", "stream.kafka.consumer.prop.bootstrap.servers": "XXXns.servicebus.windows.net:9093", "stream.kafka.consumer.prop.security.protocol": "SASL_SSL", "stream.kafka.consumer.prop.sasl.mechanism": "PLAIN", "stream.kafka.consumer.prop.ssl.endpoint.identification.algorithm": "", "stream.kafka.consumer.prop.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"$ConnectionString\\\" password=\\\"Endpoint=sb://XXXns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=My acess key=;EntityPath=icptrafficdataeventhub\\\";", "realtime.segment.flush.threshold.rows": "50000", "realtime.segment.flush.threshold.time": "24h" } #C013WKLT5T7 #C03N1JNHXLY
    p
    • 2
    • 1
  • v

    vinod kumar naidu

    05/12/2025, 9:44 AM
    Hi All , Unable to connect to SSL Enabled Zookeeper getting the following error 2025/04/24 095621.256 WARN [ClientCnxnSocketNetty] [zkNetty-EpollEventLoopGroup-3-1] future isn't success. io.netty.channel.StacklessClosedChannelException: null at io.netty.channel.AbstractChannel$AbstractUnsafe.ensureOpen(ChannelPromise)(Unknown Source) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] 2025/04/24 095622.259 INFO [ClientCnxn] [main-SendThread(localhost:2191)] Opening socket connection to server localhost/127.0.0.1:2191. 2025/04/24 095622.260 INFO [ClientCnxn] [main-SendThread(localhost:2191)] SASL config status: Will not attempt to authenticate using SASL (unknown error) 2025/04/24 095622.262 WARN [ChannelInitializer] [zkNetty-EpollEventLoopGroup-3-1] Failed to initialize a channel. Closing: [id: 0xd0530de0] org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /nfmt/instance/certificates/External/truststore.ks at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:612) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.common.ClientX509Util.getTrustManager(ClientX509Util.java:198) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.common.ClientX509Util.createNettySslContextForClient(ClientX509Util.java:77) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initSSL(ClientCnxnSocketNetty.java:449) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:438) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:424) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:1130) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:558) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:45) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1410) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1064) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:599) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:513) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:428) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:485) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /nfmt/instance/certificates/External/truststore.ks
  • v

    vinod kumar naidu

    05/12/2025, 9:44 AM
    is there any way secure the zookeeper logs with ssl
  • v

    vinod kumar naidu

    05/12/2025, 9:45 AM
    please guide us to solve this where we are unable to make zookeeper as ssl and zookeeper data logs is storing without encryption
  • r

    Raunak Binani

    05/19/2025, 12:03 PM
    Hi all, I am using pinot-offline ingestion to upload data from Hdfs Cluster A into Pinot backed by Hdfs cluster B using spark. The hadoop config is read from the cluster on which the spark job is running. Is there a way to pass Hdfs config of both the clusters together... Or any other approach that you suggest??
  • a

    Anand Kr Shaw

    05/20/2025, 9:47 AM
    Hi @Jackie , @Xiang Fu - Kindly review this : https://github.com/apache/pinot/pull/15844 https://docs.google.com/document/d/1M5OLUkW0eqn6ZnRYH1gg7lh7VrrJyu43gubix8xHm-s/edit?tab=t.0
    • 1
    • 1
  • m

    Mannoj

    06/03/2025, 7:44 AM
    A quick question,
    cluster config setting will take priority or client setting will take priority in pinot ?
    i.e: am setting pinot.broker.timeoutMs = 60000 in broker and server config file itself, but if some clients want to override this setting, can they set at client level a new setting like pinot.broker.timeoutMs = 1200000 and get the query executed ?
    m
    • 2
    • 1
  • s

    Soupam Mandal

    06/08/2025, 1:07 PM
    Created a PR https://github.com/apache/pinot/pull/16038/files
    x
    • 2
    • 2
  • h

    Himanshu Verma

    06/11/2025, 5:21 PM
    Hi all, I was setting up SSL on controller UI using below configs, is there any way to encrypt the password in controller conf file
    Copy code
    # SSL Enablement
    controller.tls.keystore.path=/etc/security/certificates/keystore.jks
    controller.tls.keystore.password=Hadoop@123
    controller.tls.truststore.path=/etc/security/certificates/truststore.jks
    controller.tls.truststore.password=Hadoop@123
    controller.access.protocols=https
    controller.access.protocols.https.port=9443
    • 1
    • 1
  • v

    vinod kumar naidu

    06/13/2025, 1:29 AM
    Hi all , we are seeing this Added support to pause and resume ingestion based on resource utilization (Added support to pause and resume ingestion based on resource utilization by rajagopr · Pull Request #15008 · apache/pinot · GitHub) got merged on feb 17 and we are seeing the code is not there in the 1.3.0 source code . which adds the resource utilization code . could you please let us know on this which apache pinot version will get added
    m
    x
    • 3
    • 2