Hi Everyone, I am trying run the realtime provisioning helper, getting this exception can anyone help us with this. Command used : ./pinot-admin.sh RealtimeProvisioningHelper -tableConfigFile /opt/pinot/data/table.json -sampleCompletedSegmentDir /data/cs_fact_v2_ablaus__1__5__20230426T1640Z -numPartitions 16 -pushFrequency null -numHosts 8,6,10 -numHours 6,12,18,24 -ingestionRate 2000 -maxUsableHostMemory 100G -retentionHours 1 Exception : at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:159) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-f5dba86d5cb19eeee9c8eb7a7cff81d92fe9958b] at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-f5dba86d5cb19eeee9c8eb7a7cff81d92fe9958b] sh-4.2# ./pinot-admin.sh RealtimeProvisioningHelper -tableConfigFile /opt/pinot/data/table.json -sampleCompletedSegmentDir /data/cs_fact_v2_ablaus__1__5__20230426T1640Z -numPartitions 16 -pushFrequency null -numHosts 8,6,10 -numHours 6,12,18,24 -ingestionRate 2000 -maxUsableHostMemory 100G -retentionHours 1 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jarfile/opt/pinot/lib/pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jarfile/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-0.13.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar) to method java.lang.Object.finalize() WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 2023/05/02 141852.107 INFO [RealtimeProvisioningHelperCommand] [main] Executing command: RealtimeProvisioningHelper -tableConfigFile /opt/pinot/data/table.json -numPartitions 16 -pushFrequency null -numHosts 8,6,10 -numHours 6,12,18,24 -sampleCompletedSegmentDir /data/cs_fact_v2_ablaus__1__5__20230426T1640Z -ingestionRate 2000 -maxUsableHostMemory 100G -retentionHours 1 java.lang.NullPointerException: Name is null at java.base/java.lang.Enum.valueOf(Enum.java:238) at java.base/java.util.concurrent.TimeUnit.valueOf(TimeUnit.java:75) at org.apache.pinot.tools.admin.command.RealtimeProvisioningHelperCommand.execute(RealtimeProvisioningHelperCommand.java:225) at org.apache.pinot.tools.Command.call(Command.java:33) at org.apache.pinot.tools.Command.call(Command.java:29) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:171) at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:202)

Hi Pinot community, We have added a offline table for already existing Realtime table in pinot, From Pinot and Presto i can get data from online and offline table but in trino ,i can get data only from real-time table. I can get aggregated count for offline data in trino but SELECT * is not working for offline data Any help would be appreciated

Hi - I have a DATETIMECONVERT issue, below is the tranform functions, one is working and another not working. Working - { "columnName": "timestamp_epoch_long", "transformFunction": "DATETIMECONVERT(Replace(Replace(timestamp_tdt, 'Z', ''), 'T', ' '), '1SECONDSSIMPLE_DATE_FORMAT:yyyy-MM-dd HHmmss', '1SECONDSEPOCH', '1:SECONDS')" }. Not Working - { "columnName": "timestamp_epoch_tdt", "transformFunction": "DATETIMECONVERT(Replace(Replace(timestamp_tdt, 'Z', ''), 'T', ' '), '1SECONDSSIMPLE_DATE_FORMAT:yyyy-MM-dd HHmmss', '1SECONDSEPOCH', '1:SECONDS')" }. Input Date timestamp_tdt is - 2022-02-05T153906Z , timestamp_epoch_long: 1644075546 , but the timestamp_epoch_tdt is coming as 1970-01-20 004115.546 , what is happening with the last one ? timestamp_tdt is of Timestamp datatype

Hi Team, we are facing an error when performing a LOOKUP between a singleValue from a Dimension table and a multiValue field from a fact table. Is this not supported in PInot ? If yes then will there be plans to support this ?

JoinValue argument must be a single value expression\n\tat org.apache.pinot.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)\n\tat org.apache.pinot.core.operator.transform.function.LookupTransformFunction.init(LookupTransformFunction.java:124)\n\tat org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:307)\n\t... 21 more"

cc: @Zhengfei

Hi, I’m using upsert tables. I read official documents and it shows

... all segments of the same partition must be served from the same server ...

(https://docs.pinot.apache.org/basics/data-import/upsert#use-strictreplicagroup-for-routing) However, when I set kafka partition as 2 and kafka replica as 2, the pinot upsert table used 3 servers to consume the topic. As I understand, should pinot only uses 2 server to consume the topic?

Hi team , I have a doubt in totalDocs and count(*) as both will need to be same as expected Why it differs , Need the reason to know why? If I do Separate queries like OFFLINE or REALTIME and takes upon an addition , the expected result comes

If I already have segments in S3, but I had to recreate a table / schema with no change, is it possible to load these directly?

Without going through the process of segment generation / overwriting?

Hi community currently i'm trying to run the mergeRollup feature with the CLI. All segments are successfully loaded, the process started and failed on SegmentProcessingFramework. Any suggestions about what i've missed ?

When doing a select * on a pinot table i am seeing below error . We have a multivalue column having 12000 values in one row. Can you please suggest what is causing this error and how to increase this limit.

{
    "message": "QueryExecutionError:\njava.lang.Exception: java.lang.OutOfMemoryError: Requested array size exceeds VM limit\n\tat org.apache.pinot.core.transport.InstanceRequestHandler$1.onFailure(InstanceRequestHandler.java:160)\n\tat org.apache.pinot.shaded.com.google.common.util.concurrent.Futures$4.run(Futures.java:1123)\n\tat org.apache.pinot.shaded.com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435)\n\tat org.apache.pinot.shaded.com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:141)\n...\nCaused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit\n\tat java.base/java.util.Arrays.copyOf(Arrays.java:3745)\n\tat java.base/java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)\n\tat java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)\n\tat java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:137)",
    "errorCode": 200
  },

cc: @Elon

I am seeing this error in the server logs..I tried to restart the server .. But the error did not go away.This seems to be a Zookeeper error.The Queries are returning results

│ Waiting for lock to refresh : workflowEvents__1__63__20220110T0100Z, queue-length: 429                                                                                       │
│ Waiting for lock to refresh : workflowEvents__1__87__20220203T0106Z, queue-length: 430                                                                                       │
│ Waiting for lock to refresh : workflowEvents__1__390__20221130T2041Z, queue-length: 431                                                                                      │
│ Waiting for lock to refresh : workflowEvents__1__41__20211219T0053Z, queue-length: 432                                                                                       │
│ Waiting for lock to refresh : workflowEvents__1__46__20211224T0055Z, queue-length: 432                                                                                       │
│ Waiting for lock to refresh : workflowEvents__1__106__20220222T0112Z, queue-length: 432                                                                                      │
│ Waiting for lock to refresh : workflowEvents__1__76__20220123T0103Z, queue-length: 432                                                                                       │
│ Waiting for lock to refresh : workflowEvents__1__113__20220301T0114Z, queue-length: 432                                                                                      │
│ Waiting for lock to refresh : workflowEvents__1__179__20220506T0132Z, queue-length: 432                                                                                      │
│

Hi - I have GCP cluster with 6 nodes and each with a 95 G disk. When index/ingest bunch of files, I get disk full. When ssh into each of the node/host, I see only 7% used in the 95G. I would like to understand how the disk space is allocated in the controllers for the segments. Any pointers to this and the configuration for more disk space/memory will be helpful

Hi , Upsertion is inserting duplicate rows in my Pinot table, Can you help on why this might be happening and how to resolve this

Hi team, found this https://github.com/apache/pinot/pull/5608 In the description of this PR

A future commit will enable properties loaded from PINOT_X environment variables.

This functionality is of use to us. Can anybody confirm if this change has been rolled out or not?

Hi team - I have an issue with superset pulling the data from Pinot.The query runs fine in pinot sql query, superset sqlLab but when I exported as data set and explore as chart in superset , it is throwing an pinot sql parsing exception. How to debug this ?

Hi, I was trying to run a query on pinot where the column is a time string. I am getting the error :

Cannot compute min for non-numeric type: STRING

Query:

select min(time_col) from table;

Is there a workaround for this as I am trying to run this query from presto and transformation functions can only be applied on pinot. Else, there would be an issue in pushing down group by and aggregation queries to pinot.

java.io.IOException: Packet len 1049422 is out of range!
        at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:121) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-856503256fa55ec5ff7e48f3beea9fea88a9a406]
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:84) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-856503256fa55ec5ff7e48f3beea9fea88a9a406]
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-856503256fa55ec5ff7e48f3beea9fea88a9a406]
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-856503256fa55ec5ff7e48f3beea9fea88a9a406]
2023/05/08 17:25:53.670 INFO [ZkClient] [main-EventThread] zkclient 9, zookeeper state changed ( Disconnected )
2023/05/08 17:25:53.670 WARN [ZKHelixManager] [ZkClient-EventThread-125-localhost:2191] KeeperState:Disconnected, SessionId: 1000284ac7e000d, instance: Controller_172.20.59.198_9000, type: PARTICIPANT
2023/05/08 17:25:53.688 INFO [LLCSegmentCompletionHandlers] [grizzly-http-server-3] Processing segmentConsumed:Offset: -1,Segment name: pinot_5411001__0__0__20230508T1149Z,Instance Id: Server_172.20.59.198_8098,Reason: timeLimit,NumRows: 5247,BuildTimeMillis: -1,WaitTimeMillis: -1,ExtraTimeSec: -1,SegmentLocation: null,MemoryUsedBytes: 1232884,SegmentSizeBytes: -1,StreamPartitionMsgOffset: 5247
2023/05/08 17:25:53.688 INFO [LLCSegmentCompletionHandlers] [grizzly-http-server-3] Response to segmentConsumed for segment:pinot_5411001__0__0__20230508T1149Z is :{"offset":-1,"status":"NOT_LEADER","isSplitCommitType":false,"streamPartitionMsgOffset":null,"buildTimeSec":-1}

Why this is happenning?

Hi, I setup a new cluster on VM with 2 controller 2 broker and 2 servers. but when I am querying newly added table getting error:

BrokerResourceMissingError

. Also I don’t see anything in EXTERNALVIEW of zk browser. I checked all logs but don’t see any error.

Is the following behavior a bug where Pinot doesn't respect the value of

schemaName

in

SegmentsConfig

within the table config if there's already a schema with the same name as table name? For example: • you have 2 schemas:

pinot_1

and

pinot_2

• you have a table with table name

pinot_1

Even if you specify

pinot_2

as the schema for table with the name

pinot_1

, the table

pinot_1

will always use

pinot_1

schema despite

pinot_2

specified in the config.

I'm trying to batch ingest into pinot using the Insert Into in Pinot SQL. However I keep getting this error: Can anyone help me with, what am I missing. I have already added to the config- "task": { "taskTypeConfigsMap": { "SegmentGenerationAndPushTask": {} } } Been following this blog: https://startree.ai/blog/apache-pinot-0-11-inserts-from-sql

Hi - when trying to do a basic group by query via the Query Console, I receive an errorCode 427.

select count(COL) from TABLE group by COL limit 10

[
  {
    "errorCode": 427,
    "message": "3 servers [10.193.3.30_O, 10.193.2.88_O, 10.193.4.249_O] not responded"
  }
]

Anyone know why the servers may not be responding?

Hi，If use timestamp to query the table, will the corresponding segment be located according to this time range?

Hi, following table configs are not documented may I know what they do exactly? Specifically

createInvertedIndexDuringSegmentGeneration

which improves the performance for queries dramatically.

"autoGeneratedInvertedIndex": false,
"createInvertedIndexDuringSegmentGeneration":false

Hi Everyone, We deployed our pinot on kubernetes. When we trigger some large scale query from trino we are getting

Unexpected response status: 503 for request for broker

, And the broker is getting restarted with below error.

ls_636), sum(remaining_avails_637), sum(remaining_avails_638), sum(remaining_avails_639)  from offlinebookingwide_main where country = 'United Kingdom' option(timeoutMs=60000)
requestId=29,table=offlinebookingwide_main,timeMs=231360,docs=724301/4773376,entries=4980587/2172903,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/unavailable):80/80/80/0/0/0/0,consumingFreshnessTimeMs=0,servers=9/9,groupLimitReached=false,brokerReduceTimeMs=4391,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs,RequestSentDelayMs);pinot-adsforecast-dev-server-3_O=0,226965,1098563882,905,-1;pinot-adsforecast-dev-server-6_O=0,188165,1094491757,989,-1;pinot-adsforecast-dev-server-1_O=0,209146,1096388587,908,-1;pinot-adsforecast-dev-server-5_O=0,226965,1084725198,858,-1;pinot-adsforecast-dev-server-8_O=0,163120,968666214,1200,-1;pinot-adsforecast-dev-server-4_O=0,219490,1095107844,897,-1;pinot-adsforecast-dev-server-0_O=0,210310,1093962024,982,1;pinot-adsforecast-dev-server-7_O=0,199838,1091794081,1063,1;pinot-adsforecast-dev-server-2_O=0,212512,1100748319,916,1,offlineThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,realtimeThreadCpuTimeNs(total/thread/sysActivity/resSer):0/0/0/0,clientIp=unknown,query=SELECT "trc_content_ratings", "iab_content_categories", "dmp_segments" FROM offlinebookingwide_main WHERE AND(regexp_like("trc_content_ratings", '^tv14$'), ("dmp_segments") = '164898') LIMIT 5000000

Any suggetion on this

Hi, any idea what might be the reason behind

Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentMetadataPushJobRunner
...
Caused by: java.lang.NullPointerException: uri authority is null

I will write more in the thread

Hi, It looks like i have some invalid json record in data file i am trying to ingest into Pinot, is there way to ignore those bad records

Caused by: org.apache.pinot.shaded.com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0x20

Are subqueries supported? i.e.

select * from table where column in (select * from table2 where column=x')

Hi team, the current behavior of

text_match

"phrase search" where it fails ignore the stop words a bug right?

Attempting to use the v2 query engine and attempting to increase the timeout through the UI. Ideas on what this error message means?

Why would Pinot show

DEAD

servers even though my pods / instances are still running?