hello, I am not able to load controller UI and after inspect i can see api call to get tables is failing. any pointers? Controller logs:

Server error: 
java.lang.NullPointerException: null
 at org.apache.pinot.controller.api.resources.PinotTenantRestletResource.getTablesServedFromTenant(PinotTenantRestletResource.java:249) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.apache.pinot.controller.api.resources.PinotTenantRestletResource.getTablesOnTenant(PinotTenantRestletResource.java:240) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
 at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
 at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
 at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]
 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[pinot-all-0.11.0-SNAPSHOT-jar-with-dependencies.jar:0.11.0-SNAPSHOT-bbec0a8070ce1d689a382195b291d68ca945c040]

Hi, I am facing a strange error while querying on pinot. I am using pinot release v0.12.0. My query was of the pattern:

Select * from table where id=12;

I have followed the steps given in the doc to enable multi-query engine. I noticed that when I use v2 Engine, query runs fine but when I do not use it, I get "No records found". Does it have something to do with enablic multi-query engine? This particularly happens when a certain record is updated and queried. For other records, it runs fine but for fresh record, there seems to be an issue. Please help.

general question: when you delete a table (eg via the controller web UI) how do the segments actually get "off" the servers, ie stop taking up disk usage?

Hi team, we have our pinot cluster deployed in kubernetes and we noticed that every time when we restart a server pod (for rollout, upgrade or other operations), it seems to take some time for the segments from that server to come back online, but I think the data in pvc will not lose and I would expect the segment become available immediately after pods come back. I wonder what is happening here during the server restart?

Hi Team, I am testing pinot on EKS currently. I have provided S3 acces to the service accounts of pinot controller, broker, server and minion statefulsets via IRSA roles. I can see segments are being pushed to S3 However with task RealtimeToOfflineSegmentTask, I am getting S3 NoSuchKeyException. Does anyone have any hint for me?

howdy! I have a realtime table that doesn't seem to be consuming (see the pic) I tried to /tables/{tableName}/pauseConsumption on it to see if I could fix it, but then I get this message:

No force commit message was sent for table: ec91e00d-4a56-4e4b-b792-8b5e85babdd0-testing_REALTIME segments: [ec91e00d-4a56-4e4b-b792-8b5e85babdd0-testing__0__0__20230210T0156Z]

I might be doing something silly, but seeing something strange: why would adding

OFFLINE

process 1 more segment?

Hello Team, Can we submit Pinot data ingestion job from AWS Glue? If anyone have tried this could you please share steps/sample script for the same.

howdy! somehow, we have some "bad" data in one of our OFFLINE tables: this data can't be correct, since that

null

is never present in our data and our last segment has an end time before the time shown here. notably, this is only happening on a single segment: how do I find which segment this is so I can try reloading it or replacing it? I tried enabling tracing but it doesn't appear to have per segment info

Hey all! 👋 My team is working on migrating our initial Pinot deployment on EC2 to kubernetes. We were hoping to use the open-source pinot docker images and have started with the use of the

apachepinot/pinot:0.12.0

image. However, when building, this image is getting flagged for multiple critical/high level security vulnerabilities, I believe due to the fact that it is built off of a deprecated

openjdk11

base image. Have folks run into similar issues here? Any good workarounds besides forking the repo and re-writing the `Dockerfile`s from scratch using a compliant base image?

what would be the best way to replace a single segment if I know the parquet file?

Hi, I am unable to make RealtimeToOfflineSegmentsTask work for this demo recepie. https://github.com/startreedata/pinot-recipes/tree/main/recipes/managed-offline-flow-automatic-scheduling My bucketTimePeriod is 5m and bufferTimePeriod is 1m and I ran the event generator for more than 15 minutes. The scheduler runs every minute as expected and triggers this task. However it is skipped every time with the following error in controller logs:

Window data overflows into CONSUMING segments for partition of segment: events__0__443__20230212T0051Z. Skipping task generation: RealtimeToOfflineSegmentsTask

What does this mean and how can I debug this? I also tried to manually trigger the task via controller api but got the same error

tableName="events_REALTIME"

curl -X POST "<http://localhost:9000/tasks/schedule?taskType=RealtimeToOfflineSegmentsTask&tableName=${tableName}>" \

-H "accept: application/json" 2>/dev/null | jq '.'

My kafa CDC payload has after and before values after editing rows, not sure if the fields from payload should match the hierarchy within kafka payload but somehow my Pinot tables is getting events but the data is null. Kafka payload has

{
  "schema": {
    "type": "struct",
    "fields": [
      {
        "type": "struct",
        "fields": [
          {
            "type": "int32",
            "optional": false,
            "field": "user_community_id"
          },
          {
            "type": "string",
            "optional": true,
            "field": "name"
          },
          ...
          {
            "type": "string",
            "optional": false,
            "name": "io.debezium.time.ZonedTimestamp",
            "version": 1,
            "default": "1970-01-01T00:00:00Z",
            "field": "updated_at"
          }
        ],
        "optional": true,
        "name": "mysql.communication.user_community.Value",
        "field": "after"
      }
  },
  "payload": {
    "before": {
      "user_community_id": 2,
      "name": "Test",
      ...
      "updated_at": "2023-02-12T05:38:26Z"
    },
    "after": {
      "user_community_id": 2,
      "name": "Test 1",
      ...
    }

Hey folks, COALESCE method is not working as expected for me i.e. COLASCE(colA, colB) is returning result of colA even though its value is null and colB is not null I have

"nullHandlingEnabled": true

in the table config. Anything i am missing here? TIA

noob question; can I have a sortedColumn and a invertedIndexColumns in one table that point to 2 different fields? this is for a realtime table

Hi,

Hi, my PInot cluster 0.11 worked well, after upgrade to Tag release-0.12.0, we can not login to Pinot from WebUI by user: user, but with user: admin it is pretty good. Do you have any idea? Thanks.

howdy! I'm trying to ingest some .zstd compressed parquet files but getting:

Caused by: java.lang.RuntimeException: native zStandard library not available: this version of libhadoop was built without zstd support.

on my minions (full trace in thread) Looks like minions can do .snappy compressed parquet files just fine; is there a list of supported compression codecs somewhere, or has anybody tried actually getting minions to ingest .zstd compressed files?

Hey all, I'd like to get some clarification on the chapter in https://docs.pinot.apache.org/basics/components/controller

For redundancy, there can be multiple instances of Pinot controllers. Pinot expects that all controllers are configured with the same back-end storage system so that they have a common view of the segments (e.g. NFS). Pinot can use other storage systems such as HDFS or ADLS.

My cluster is running in K8S, with 3 controller instances, should the three instances share one volume, or assign one volume for each? Sometimes I see metrics reported by the 3 instances are not the same, wondering if they're related.

Hey Everyone! I'm a beginner and using pinot for the first time! Can anyone help me with how to run pinot with kafka on docker-compose? just a basic tutorial maybe!will be much helpful!

Hi Everyone, I see that the status of my tables is in “Updating” state for almost 5days. Also attaching the reload status, estimatedTimeRemainingInMinutes is almost 16316 and it keeps on increasing. Am I missing anything?

Hello 👋 I'd like to improve our logging of cron jobs which are using the

pinot-admin.sh

script in our context. I doubt it should be possible to change the SLF4J log level to ERROR only in JAVA_OPTS. Has anyone done this before?

Hi all I have a kafka topic providing an empty array [] and when data is queried after RT ingestion Pinot has turned into a null value: is there any config to control this conversion?

Anybody tried this pinot-spark-connector and have a

spark-shell

command for testing Pinot reads

Will this kind of error (hadoop cluster crash) cause the pinot service to be unavailable ?

Hey everyone 👋 I ingested a relatively small number of events into a Pinot table (4741) but whenever I query for the total count (

SELECT COUNT(*) FROM table

) then I get a result of 2409. I am using realtime ingestion with Kafka. I checked the logs of all components and cannot find anything there. What could be causing the events not being accessible? Thanks for any help thankyou Adding the query result JSON in the 🧵

Hellos, question on graphana Json shared here, can someone explain the rationale on why the Table QPS is calculated based on following with

10m

as time granularity

sum by (table) (rate(pinot_broker_queries_Count[10m]))

Upgraded from 0.11 to 0.12 in my Dev env, now am seeing a lot of zk exceptions. Looks like either the znode size limit has charged or I have some large data somewhere in zk. Any suggestions on how to fix other than increasing the zk limit?

java.io.ioexception: packet len 1250829 is out of range!

Hey teams, I have a some problems with Pinot-0.12 when upgraded from pinot-0.11. Probleme-1: in the browser WebUI, Admin can login to the pinot cluster but user (READ) not. See below image Nothing happened in the log when I input user and password in the browser WebUI. Problem-2: I found also tableau desktop (2021.4, 2022.4) do not work with Pinot-0.12.0 using driver *_0.12.jar like Pinot-jdbc-client (java)-0.12.jar. but works well with pinot-jdbc (java) -client-0.11.jar, We want to upgrade pinot-jdbc(java)-0.11.jar to XXX_0.12.0.jar because of critical vulnerabilities. Here it is tableau connection log, it indicates empty user and password, but we did give them. Hope you will find how these problmes come from soon.

last black capture related to docker log (controller and broker), it is silent.