https://pinot.apache.org/ logo
Join Slack
Powered by
# pinot-dev
  • a

    Anand Kr Shaw

    12/16/2024, 9:08 AM
    Hi Team , There was a minor issue we were facing while adding a new table via UI : https://github.com/apache/pinot/pull/14663 One of the UI engineer from Our team has raised the fix , can you please check
  • a

    Abhishek Sharma

    12/31/2024, 1:07 AM
    FYI, Pinot-1.3 Release soon.
  • a

    Abhishek Sharma

    01/10/2025, 2:50 AM
    Reminder, tomorrow EOD, I will cut the 1.3.0 version release candidate. Slack Conversation
  • a

    Abhishek Sharma

    01/12/2025, 11:03 PM
    FYI.
  • a

    Abhishek Sharma

    02/02/2025, 1:07 PM
    FYI
  • m

    Mannoj

    02/03/2025, 9:26 AM
    Hi All, We are exploring Pinot. And have few questions, hope this is a right group to put across if not kindly let me know the right group. ( Question ). Can pinot work with deep storage, we are looking for inverted index and columnar which pretty much solved in pinot servers itself. Should we need deep storage? like hdfs? (Answer we find in ) https://docs.pinot.apache.org/release-0.4.0/developers/tutorials/store-data . Yes you need hdfs or s3 to have persistence. (Overall Question is ) : Does it mean one need hdfs or s3 for persistent storage and you can't make pinot server itself as a persistent layer as incase of ALL pinot server shutdown and started, data will be gone?
  • a

    Anand Kr Shaw

    02/06/2025, 4:35 AM
    Hi Team , Is anyone working on this : https://docs.pinot.apache.org/developers/advanced/ingestion-level-transformations#one-record-into-many This is a good to have native support feature. @Jackie @Mayank
  • j

    Jackie

    02/06/2025, 7:56 PM
    @Anand Kr Shaw I think https://docs.pinot.apache.org/basics/data-import/pinot-input-formats/complex-type was introduced for this
    a
    • 2
    • 1
  • a

    Arman

    02/19/2025, 9:25 AM
    Hi everyone! There is connectivity issue with ZooKeeper in the latest release of Java client lib (1.2.0) , and causes any call to
    ConnectionFactory.fromZookeeper(zkServers)
    to block forever. Apparently, the issue is due to the latest version of zkClient library (0.11). I tested older version of Pinot client (0.9.3) relying on previous versions of zkClient (0.7) and everything worked fine. I have also opened an issue on GH https://github.com/apache/pinot/issues/15074 . A couple of workaround comes to my mind and I wanted to know your perspective on this and how we can tackle it. First, this repo https://github.com/sgroschupf/zkclient seems abandoned and we may not expect any fix for this issue from their side. We have the option to use the implementation provided by Apache Helix - zkClient . Second, we can remove the dependency to this lib and write a compatible wrapper using latest official zookeeper client.
  • n

    N R Neha

    02/25/2025, 6:59 PM
    Hi , I get this error when running pinot on local (Mac OS)
    Copy code
    2025/02/25 10:55:02.231 ERROR [PinotQueryResource] [grizzly-http-server-13] Caught exception while processing post request
    java.net.ConnectException: Operation timed out
    	at java.base/sun.nio.ch.Net.connect0(Native Method) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:579) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:568) ~[?:?]
    	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:576) ~[?:?]
    	at java.base/java.net.Socket.connect(Socket.java:666) ~[?:?]
    	at java.base/java.net.Socket.connect(Socket.java:600) ~[?:?]
    	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183) ~[?:?]
    Copy code
    2025/02/25 10:55:02.230 ERROR [PinotQueryResource] [grizzly-http-server-13] Caught exception in sendQueryRaw
    java.net.ConnectException: Operation timed out
    	at java.base/sun.nio.ch.Net.connect0(Native Method) ~[?:?]
    	at java.base/sun.nio.ch.Net.connect(Net.java:579) ~[?:?]
    There are no errors in the server logs. Can you provide any pointers to unblock from this? Thanks
    a
    • 2
    • 1
  • r

    robert zych

    02/27/2025, 4:17 PM

    https://youtu.be/yRMhAL_6f4o?si=4TaNGP1CtrQTjVsF▾

  • y

    Yarden Rokach

    03/06/2025, 3:46 PM
    Hey all! FYI, we’re launching a Pinot OSS YouTube channel soon! 📺 Before we go live, we want to gather all the must-have Pinot videos ; tutorials, meetup recordings, use cases, or anything valuable for the community. Got a video in mind? Maybe one that helped you understand Pinot better? pls drop it in this thread or share it on #C08GH2MAVT4! Thank you ❤️
  • d

    Daniil

    03/07/2025, 1:43 PM
    Hey there 👋 I'd appreciate a PR review 🙏 https://github.com/apache/pinot/pull/15015 It's done for this issue The PR expands
    /ingestFromURI
    endpoint, which is not meant for production, and the PR includes tests
  • m

    Manish G

    03/11/2025, 12:41 PM
    Hi, anyone looking for a helping hand on issue they are working upon...coding, unit tests etc? I have bandwidth for few days.
  • p

    Peter Corless

    03/11/2025, 3:42 PM
    Here's the list of currently-open "beginner-task" marked Github issues. Consider starting here, and if you have more time, there's meatier work ahead for sure! https://github.com/apache/pinot/labels/beginner-task
  • p

    Peter Corless

    03/11/2025, 3:43 PM
    Here's also a bunch of "help-wanted" tasks, if you want to roll your sleeves up: https://github.com/apache/pinot/issues?q=is%3Aopen+label%3A%22help+wanted%22
  • b

    benson yuan

    03/14/2025, 12:34 AM
    https://apache-pinot.slack.com/archives/C011C9JHN7R/p1741911243212059 cross-posting for visibility 😆
  • m

    Manish G

    03/27/2025, 12:38 PM
    Created a PR https://github.com/apache/pinot/pull/15377 May someone have a look?
  • y

    Yarden Rokach

    04/03/2025, 5:44 PM
    Hey all! The Apache Pinot Contributor call recording is available here #C08H9R5MHTN
    r
    • 2
    • 2
  • m

    Manish G

    04/05/2025, 8:26 AM
    Hi, while generating response to a SQL request, which layer in codebase writes to the response?
    x
    • 2
    • 1
  • p

    Peter Corless

    04/07/2025, 6:31 PM
    Watch on-demand: •

    Apache Pinot Contributor Call Kickoff | April 3rd, 2025▾

    Then tell us what you think: • Apache Pinot Contributor Call Survey Slack Conversation
  • i

    Indira Vashisth

    04/23/2025, 3:02 AM
    Hi all, we have started facing node volume affinity issue in our pinot pods after we did an AMI version update to our EKS node groups on which our Pinot is hosted. We had only a single pod(for controller and zookeeper) and since PVCs are AZ bound and the EKS on AMI update changed the AZ in which the node is present we faced this issue. We are planning on having one nodegroup per every AZ(total 3) we have. 3 node groups of controller and zookeeper. Are there any best practices we can follow to avoid this?
    x
    • 2
    • 4
  • v

    vinod kumar naidu

    04/24/2025, 6:52 AM
    Hi All , as part of vulnerability fix avro version didn't changed from 1.11.3 to 1.11.4 . In pinot pom.xml we are seeing that version is updated to 1.11.4 but in the dependencies jar we are not seeing the updated version . could you please let us know on this
    x
    • 2
    • 2
  • m

    Manish G

    05/01/2025, 11:21 AM
    Some basic questions. When an index is created, like for null values,what component decides that a given value corresponds to null value. For eg, for a float column, -infinity value is mapped to null. Same for NaN and +infinity.
    x
    • 2
    • 3
  • m

    Manish G

    05/01/2025, 11:23 AM
    Basically I am looking into population of null bit maps.
  • m

    Manish G

    05/01/2025, 11:26 AM
    I see few files getting created in the process...columns.psf, floatfield.bitmap.nullvalue.
  • m

    Manish G

    05/01/2025, 11:27 AM
    But couldnt trace where exactly population happens.
  • r

    robert zych

    05/01/2025, 1:57 PM
    @Xiang Fu What’s the difference between Minion and Minion Stateless?
    x
    • 2
    • 9
  • c

    Chris Peck

    05/05/2025, 7:51 PM
    Hi folks, I saw there was a recent change aimed at detecting backwards incompat api changes for pinot-spi - I found this makes it pretty difficult to add more args for index config constructors. Eg., trying to add a new arg for JsonIndexConfig, we now see the error:
    Copy code
    Error:  Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.23.1:cmp (default) on project pinot-spi: There is at least one incompatibility: org.apache.pinot.spi.config.table.JsonIndexConfig.JsonIndexConfig(java.lang.Boolean,int,boolean,boolean,java.util.Set,java.util.Set,java.util.Set,java.util.Set,int,boolean):CONSTRUCTOR_REMOVED -> [Help 1]
    Any concerns with excluding some config classes (or parent config package) from this check? or should we be creating a new constructor for every config addition going forward?
    s
    x
    • 3
    • 8
  • n

    Nithinjith Pushpakaran

    05/07/2025, 9:07 AM
    Apache Pinot + Azure Event Hub Integration: Metadata Fetch Timeout Hey team, I’m integrating Apache Pinot (Dockerized) with Azure Event Hub (Kafka-compatible endpoint) for real-time ingestion. The connection works perfectly from a standalone Python Kafka client using confluent_kafka, but Pinot fails with: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata ✅ Confirmed Working: Pinot Server can curl and TLS handshake with icpeventhubns.servicebus.windows.net:9093 Python consumer successfully consumes data from ICPtrafficdataeventhub topic Using correct sasl.jaas.config format with $ConnectionString and EntityPath streamConfigs use stream.kafka.consumer.prop.* correctly ❌ Still Seeing: Pinot throws a metadata fetch timeout when creating the real-time table, even though connectivity and credentials are confirmed. Stream Config: For your reference ---------------------------------------- "streamConfigs": { "streamType": "kafka", "stream.kafka.topic.name": "icptrafficdataeventhub", "stream.kafka.broker.list": "icpeventhubns.servicebus.windows.net:9093", "stream.kafka.consumer.type": "simple", "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory", "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder", "stream.kafka.consumer.prop.bootstrap.servers": "XXXns.servicebus.windows.net:9093", "stream.kafka.consumer.prop.security.protocol": "SASL_SSL", "stream.kafka.consumer.prop.sasl.mechanism": "PLAIN", "stream.kafka.consumer.prop.ssl.endpoint.identification.algorithm": "", "stream.kafka.consumer.prop.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"$ConnectionString\\\" password=\\\"Endpoint=sb://XXXns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=My acess key=;EntityPath=icptrafficdataeventhub\\\";", "realtime.segment.flush.threshold.rows": "50000", "realtime.segment.flush.threshold.time": "24h" } #C013WKLT5T7 #C03N1JNHXLY
    p
    • 2
    • 1