Deepak Mishra
03/30/2022, 6:12 AMMathew Pallan
03/30/2022, 2:18 PMElon
03/30/2022, 6:06 PMnpm install
and npm run build
?dmitry H
03/30/2022, 6:29 PMDaniel Lavoie
03/30/2022, 6:47 PMzookeeper.x
valuesLuis Fernandez
03/30/2022, 7:03 PM"message": "2 servers [pinot-server-1_O, pinot-server-0_O] not responded"
I just changed the version of my broker trying to downgrade the entire system to 0.9.3
kaushal aggarwal
03/31/2022, 4:09 AMArpita Bajpai
03/31/2022, 7:33 AMfrancoisa
03/31/2022, 12:54 PM"ingestionConfig": {
"transformConfigs": [
{
"columnName": "type",
"transformFunction": "JSONPATHSTRING(data,'$.type')"
}
],
"complexTypeConfig": {
"fieldsToUnnest": [
"data.attributes.actualExpenses"
],
"delimiter": "."
}
},
Schema is already defined with type STRINGLuis Fernandez
03/31/2022, 2:48 PMDiana Arnos
03/31/2022, 4:53 PMFailed to find servers hosting segment: <segment> for table: <tableName>_REALTIME (all ONLINE/CONSUMING instances: [] and OFFLINE instances: [] are disabled, counting segment as unavailable)
How can I:
1- Make the brokers find the segments?
2- If 1 is not possible, how can I make the servers download or fetch all the missing segments?abhinav wagle
03/31/2022, 10:28 PMjava -jar pinot-tool-launcher-jar-with-dependencies.jar QueryRunner -mode singleThread -queryFile test.q -numTimesToRunQueries 0 -numIntervalsToReportAndClearStatistics 5 -brokerHost <host-name>
Grace Lu
04/01/2022, 2:48 AMShailee Mehta
04/01/2022, 5:23 AM{
"tableName": "lineorder_star_OFFLINE",
"tableType": "OFFLINE",
"segmentsConfig": {
"timeColumnName": "LO_ORDERDATE", //date field with day-granularity
"timeType": "DAYS",
"replication": "1",
"schemaName": "lineorder"
},
"tenants": {
"broker": "DefaultTenant",
"server": "DefaultTenant"
},
"metadata": {
"customConfigs": {}
},
"tableIndexConfig": {
"starTreeIndexConfigs": [
{
"dimensionsSplitOrder": [
"LO_ORDERDATE", //date
"LO_SUPPKEY", // dim field
"LO_PARTKEY", // dim field
"LO_DISCOUNT", // measure
"LO_QUANTITY", //measure
"LO_REVENUE", // dim field
"LO_ORDERPRIORITY" //measure
],
"skipStarNodeCreationForDimensions": [],
"functionColumnPairs": [
"SUM__LO_QUANTITY",
"COUNT__LO_ORDERKEY",
"SUM__LO_REVENUE"
]
}
]
}
}
Diogo Baeder
04/01/2022, 9:31 PMDiogo Baeder
04/01/2022, 9:42 PMtimeColumn
? I was looking at using DAYS
as the type for this, but it doesn't seem correct - I want to use real dates and not "days since Epoch".ahsen m
04/02/2022, 1:47 AMOpening socket connection to server kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181. Will not attempt to authenticate using SASL (unknown error)
53
Socket connection established, initiating session, client: /10.48.12.58:54236, server: kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181
52
Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
51
Opening socket connection to server kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181. Will not attempt to authenticate using SASL (unknown error)
50
Socket connection established, initiating session, client: /10.48.12.58:54248, server: kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181
49
Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
48
Opening socket connection to server kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181. Will not attempt to authenticate using SASL (unknown error)
47
Socket connection established, initiating session, client: /10.48.12.58:54268, server: kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181
46
Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
45
Opening socket connection to server kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181. Will not attempt to authenticate using SASL (unknown error)
44
Socket connection established, initiating session, client: /10.48.12.58:54274, server: kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181
43
Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
42
Opening socket connection to server kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181. Will not attempt to authenticate using SASL (unknown error)
41
Socket connection established, initiating session, client: /10.48.12.58:54278, server: kafka-cluster-zookeeper-client.kafka-cluster.svc.cluster.local/10.52.8.240:2181
40
Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
39
Failed to initialize Pinot Broker Starter
38
java.lang.NullPointerException: null
37
at org.apache.helix.manager.zk.client.ZkConnectionManager.cleanupInactiveWatchers(ZkConnectionManager.java:112) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
36
at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:95) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
35
at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:91) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
34
at org.apache.helix.manager.zk.zookeeper.ZkClient.connect(ZkClient.java:1620) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
33
at org.apache.helix.manager.zk.zookeeper.ZkClient.<init>(ZkClient.java:186) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
32
at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:87) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
31
at org.apache.helix.manager.zk.client.ZkConnectionManager.<init>(ZkConnectionManager.java:41) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
30
at org.apache.helix.manager.zk.client.SharedZkClientFactory.getOrCreateZkConnectionNamanger(SharedZkClientFactory.java:60) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
29
at org.apache.helix.manager.zk.client.SharedZkClientFactory.buildZkClient(SharedZkClientFactory.java:40) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
28
at org.apache.pinot.common.utils.ServiceStartableUtils.applyClusterConfig(ServiceStartableUtils.java:54) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
27
at org.apache.pinot.broker.broker.helix.BaseBrokerStarter.init(BaseBrokerStarter.java:118) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
26
at org.apache.pinot.tools.service.PinotServiceManager.startBroker(PinotServiceManager.java:137) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
25
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:92) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
24
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:275) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
23
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:301) [pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
22
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:275) [pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
21
Failed to start a Pinot [BROKER] at 31.159 since launch
20
java.lang.NullPointerException: null
19
at org.apache.helix.manager.zk.client.ZkConnectionManager.cleanupInactiveWatchers(ZkConnectionManager.java:112) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
18
at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:95) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
17
at org.apache.helix.manager.zk.client.ZkConnectionManager.close(ZkConnectionManager.java:91) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
16
at org.apache.helix.manager.zk.zookeeper.ZkClient.connect(ZkClient.java:1620) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
15
at org.apache.helix.manager.zk.zookeeper.ZkClient.<init>(ZkClient.java:186) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
14
at org.apache.helix.manager.zk.ZkClient.<init>(ZkClient.java:87) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
13
at org.apache.helix.manager.zk.client.ZkConnectionManager.<init>(ZkConnectionManager.java:41) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
12
at org.apache.helix.manager.zk.client.SharedZkClientFactory.getOrCreateZkConnectionNamanger(SharedZkClientFactory.java:60) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
11
at org.apache.helix.manager.zk.client.SharedZkClientFactory.buildZkClient(SharedZkClientFactory.java:40) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
10
at org.apache.pinot.common.utils.ServiceStartableUtils.applyClusterConfig(ServiceStartableUtils.java:54) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
9
at org.apache.pinot.broker.broker.helix.BaseBrokerStarter.init(BaseBrokerStarter.java:118) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
8
at org.apache.pinot.tools.service.PinotServiceManager.startBroker(PinotServiceManager.java:137) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
7
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:92) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
6
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:275) ~[pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
5
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:301) [pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
4
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:275) [pinot-all-0.10.0-SNAPSHOT-jar-with-dependencies.jar:0.10.0-SNAPSHOT-078c711d35769be2dc4e4b7e235e06744cf0bba7]
3
Shutting down Pinot Service Manager with all running Pinot instances...
2
Shutting down Pinot Service Manager admin application...
1
Deregistering service status handler
Shailesh Jha
04/03/2022, 3:04 PMpinot-zookeeper-0 0/1 CrashLoopBackOff 241 16h
pinot-zookeeper-1 0/1 Running 210 14h
pinot-zookeeper-2 0/1 CrashLoopBackOff 6 12m
zk Pod describe snap:
ThanksShailee Mehta
04/03/2022, 4:54 PM{
"id": "100056e21a30006__lineorder_star_OFFLINE",
"simpleFields": {},
"mapFields": {
"HELIX_ERROR 20220402-222348.000619 STATE_TRANSITION 631528bd-ca2b-4b60-b66e-01cfa58a44ff": {
"AdditionalInfo": "Exception while executing a state transition task lineorder_star_OFFLINE_1992-01-01_1998-08-02_1java.lang.reflect.InvocationTargetException\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.base/java.lang.reflect.Method.invoke(Method.java:566)\n\tat org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404)\n\tat org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331)\n\tat org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97)\n\tat org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: org.apache.pinot.spi.utils.retry.AttemptsExceededException: Operation failed after 3 attempts\n\tat org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:61)\n\tat org.apache.pinot.common.utils.fetcher.BaseSegmentFetcher.fetchSegmentToLocal(BaseSegmentFetcher.java:72)\n\tat org.apache.pinot.common.utils.fetcher.SegmentFetcherFactory.fetchSegmentToLocalInternal(SegmentFetcherFactory.java:148)\n\tat org.apache.pinot.common.utils.fetcher.SegmentFetcherFactory.fetchSegmentToLocal(SegmentFetcherFactory.java:142)\n\tat org.apache.pinot.common.utils.fetcher.SegmentFetcherFactory.fetchAndDecryptSegmentToLocalInternal(SegmentFetcherFactory.java:164)\n\tat org.apache.pinot.common.utils.fetcher.SegmentFetcherFactory.fetchAndDecryptSegmentToLocal(SegmentFetcherFactory.java:158)\n\tat org.apache.pinot.core.data.manager.BaseTableDataManager.downloadAndDecrypt(BaseTableDataManager.java:406)\n\tat org.apache.pinot.core.data.manager.BaseTableDataManager.downloadSegmentFromDeepStore(BaseTableDataManager.java:393)\n\tat org.apache.pinot.core.data.manager.BaseTableDataManager.downloadSegment(BaseTableDataManager.java:385)\n\tat org.apache.pinot.core.data.manager.BaseTableDataManager.addOrReplaceSegment(BaseTableDataManager.java:372)\n\tat org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addOrReplaceSegment(HelixInstanceDataManager.java:355)\n\tat org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:162)\n\t... 12 more\n",
"Class": "class org.apache.helix.messaging.handling.HelixStateTransitionHandler",
"MSG_ID": "4dab2e7f-f693-4459-9cd6-6e837fa32c5b",
"Message state": "READ"
},
"HELIX_ERROR 20220402-222348.000679 STATE_TRANSITION bca29fad-4930-48f8-b626-3013f465b1c8": {
"AdditionalInfo": "Message execution failed. msgId: 4dab2e7f-f693-4459-9cd6-6e837fa32c5b, errorMsg: java.lang.reflect.InvocationTargetException",
"Class": "class org.apache.helix.messaging.handling.HelixStateTransitionHandler",
"MSG_ID": "4dab2e7f-f693-4459-9cd6-6e837fa32c5b",
"Message state": "READ"
}
},
"listFields": {}
}
Jonathan Meyer
04/04/2022, 8:08 AMpinot-admin
step is finished), then querying Pinot, that's where we're seeing stale data
Is there any way to
• Have ./bin/pinot-admin.sh LaunchDataIngestionJob
wait for the data to be fully query-able ?
• Have Pinot somehow notify when data becomes fully query-able ?
NOTE: Job type is SegmentCreationAndTarPush
Alice
04/04/2022, 10:27 AMDiana Arnos
04/05/2022, 7:31 AMFind unloaded segment: <tableName>__0__35__20220404T0749Z, table: <tableName>_REALTIME, expected: ONLINE, actual: CONSUMING
Sleeping 1 second waiting for all segments loaded for partial-upsert table: <tableName>_REALTIME
Which endpoint should I use to try to sort this out? The reload
one does not work, for the segment is still consuming and the reset
always fails, for it can't stop a consuming segment for some reason.
Would it be okay to just delete this segment? Would the Controller know it needs to be consumed again?Lars-Kristian Svenøy
04/05/2022, 10:46 AMSaumya Upadhyay
04/05/2022, 1:36 PMEduardo Cusa
04/05/2022, 1:55 PMingestFromFile
endpoint to ingest data, but after some minutes the table is empty again. Do we need to set up a persistence backend?David Cyze
04/05/2022, 4:01 PMtimestamp
column in the schema needed to be renamed (I chose timestamparoo
), because presto queries interpreted timestamp
as a casting function as opposed to a column
• The timeFieldSpec
field in the table schema needed to change to dateTimeFieldSpec
After making this changes, I could ingest and query (mostly) fine
The docs have since recommended changing to 0.10.0, which I have tried doing.
However, now when I run ./bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile ~/pinot-tutorial/transcript/batch-job-spec.yml
, I get an exception related to the timestamp column:
Exception while collecting stats for column:timestamparoo in row:{
"fieldToValueMap" : {
"studentID" : 200,
"firstName" : "Lucy",
"lastName" : "Smith",
"score" : 3.8,
"gender" : "Female",
"subject" : "Maths",
"timestamparoo" : null
},
"nullValueFields" : [ ]
}
or.collect(LongColumnPreIndexStatsCollector.java:50) ~[pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
at org.apache.pinot.segment.local.segment.creator.impl.stats.SegmentPreIndexStatsCollectorImpl.collectRow(SegmentPreIndexStatsCollectorImpl.java:96) ~[pinot-all-0.10.0-jar-with-dependencies.jar:0.10.0-30c4635bfeee88f88aa9c9f63b93bcd4a650607f]
It seems Pinot isn’t parsing the values for this column from the CSV.
Why would that be?
(More supporting files in thread)Daniel
04/05/2022, 6:58 PMFailed to find servers hosting segment: <segmentName> for table: <tableName> (all ONLINE/CONSUMING instances: [] and OFFLINE instances: [] are disabled, counting segment as unavailable)
Could this be due to an under-provisioned Pinot cluster? Something else?Pankaj Thakkar
04/05/2022, 7:01 PMAlice
04/06/2022, 5:42 AMArpita Bajpai
04/06/2022, 7:33 AM