We have upgraded Python DB-API and SQLAlchemy dialect for Pinot ( https://pypi.org/project/pinotdb/ ) to use new SQL API. The latest library version is

0.3.2

, and it’s compatible with Pinot Version

0.3.0+

.

Hi All,

Hi, We have a spring boot application getting data from apache pinot, what is the best way to query data from pinot...is it java client or using rest api...

@User I can help you with that. We do have a Java client. I’m working on building a new JDK REST client for Spring Boot... with auto-configuration etc. Would you be interested in trying that out? If so let’s move to DM and we can sort out the details.

Right now we have a JDK client that is not released as a Maven artifact, but a part of our Pinot source. https://docs.pinot.apache.org/users/clients/java

Hi, Assume that we have a hybrid table. Offline table of this hybrid table has some data, and realtime table of this hybrid table does not have data. in this case, if i want to get

routing table for realtime table

of this hybrid table, will it return

404

not found or

empty

result(that means returned code is 200 ok)? Note: with using

/debug/routingTable(getRoutingTableForQuery method)

of

PinotBrokerDebug

API

We just released a new blog post on implementing text indexing by the one and only @User. This is an awesome engineering article that articulates the design for text analytics and search in Pinot. Twitter (to get the word out): https://twitter.com/ApachePinot/status/1270384752210706437 Medium (direct link to blog post): https://medium.com/apache-pinot-developer-blog/text-analytics-on-apache-pinot-cbf5c45d282c

QQ, I am trying out a test pinot cluster setup, I have some questions (I maybe missing to find it in the documentation), could someone help me: • Is it possible to change config of an already existing table, for example I want to update

tagsOverridenConfig

so the segments get moved to the machine tagged as OFFLINE. • For segments to be able to move across machines, does deep storage need to be enabled? • I am consuming events from kafka, I am trying to observe the lag, wondering is it exposed somewhere? ConsumerGroup name created seems to be empty is there a way to pass that as a parameter? • Does the segments size get exposed as part of metrics somewhere? • curious, does pinot inherently handle duplicate rows based on some column?

Is it possible to change config of an already existing table, for example I want to update 

tagsOverridenConfig

 so the segments get moved to the machine tagged as OFFLINE. • Yes, you will have to invoke the rebalance API

• For segments to be able to move across machines, does deep storage need to be enabled? Yes, Uber is working on this https://cwiki.apache.org/confluence/display/PINOT/By-passing+deep-store+requirement+for+Realtime+segment+completion which can potential allow rebalancing to happen without going to deep store

thanks, that makes sense, yeah saw this link was not sure whether it was already implemented

let me try rebalance

• I am consuming events from kafka, I am trying to observe the lag, wondering is it exposed somewhere? ConsumerGroup name created seems to be empty is there a way to pass that as a parameter? This is hard to measure accurately. there are some proxies. Are you using HLC or LLC

• Does the segments size get exposed as part of metrics somewhere? There is an API for this, not sure if there is a metric. This is a good thing to add

I am using LLC

I am trying to get segmentsize from /tables/{tableName}/size, returns empty data. Is this api not for realtime tables?

I think it does not return while consuming

lets move to #C011C9JHN7R ?

sure

hey, folks, regarding this note in the 0.2.0 release notes, can someone clarify why it’s required to upgrade to 0.2.0 before moving to newer versions? some pointer to the related PRs would be even better

You must upgrade to this release before moving onto newer versions of Pinot
release. The protocol between Pinot-broker and Pinot-server has been changed
and this release has the code to retain compatibility moving forward.
Skipping this release may (depending on your environment) cause query errors
if brokers are upgraded and servers are in the process of being upgraded.

Hi the main website (http://pinot.apache.org/) still shows 0.3.0 release but I think the latest release is 0.4.0

can someone update that

will do

Updated

Hello community, We are pleased to announce that Apache Pinot (incubating) 0.4.0 is released! Apache Pinot (incubating) is a distributed columnar storage engine that can ingest data in realtime and serve analytical queries at low latency. The release can be downloaded at: https://pinot.apache.org/download The release note is available at: https://docs.pinot.apache.org/releases/0.4.0 Additional resources - Project website: https://pinot.apache.org Getting started: https://docs.pinot.apache.org/getting-started Mailing list: dev@pinot.apache.org Slack channel: https://communityinviter.com/apps/apache-pinot/apache-pinot Twitter: https://twitter.com/ApachePinot Best Regards, Apache Pinot (incubating) Team

Also pinot 0.4.0 docker image is pushed at: https://hub.docker.com/layers/apachepinot/pinot/0.4.0/images/sha256-74e1f3b59c2b5223e185476219f9d26e92021b9625d08eec4dc42ee2f1276c87?context=repo

you can try with

docker pull apachepinot/pinot:0.4.0

or

docker pull apachepinot/pinot:release-0.4.0

We are trying to simplify the getting started section and the commands needed to start/stop Pinot components. Would love to hear your thoughts/inputs

hey everyone! today was the first day i've heard about Pinot and i must say i'm really impressed, great to join the Slack community! one thing that i noticed is that there's no community group on LinkedIn for updates, learnings or job opportunities so i've created one here: https://www.linkedin.com/groups/12429326/. please feel free to join and spread the word, looking forward to get into some good discussions here! 😃