Pedro Silva
06/08/2021, 5:03 PM#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0x7) at pc=0x00007f4b79052422, pid=8, tid=0x00007f4ae8739700
#
# JRE version: OpenJDK Runtime Environment (8.0_292-b10) (build 1.8.0_292-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.292-b10 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# v ~StubRoutines::jbyte_disjoint_arraycopy
#
# Core dump written. Default location: /opt/pinot/core or core.8
#
[thread 139959708407552 also had an error]
# An error report file with more information is saved as:
# /opt/pinot/hs_err_pid8.log
#
# If you would like to submit a bug report, please visit:
# <http://bugreport.java.com/bugreport/crash.jsp>
#
Aborted (core dumped)
Server 2:
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Start a Pinot [SERVER]-SendThread(pinot-zookeeper:2181)"
2021/06/08 16:35:05.338 ERROR [LLRealtimeSegmentDataManager_HitExecutionView_3mo__1__3__20210608T1552Z] [HitExecutionView_3mo__1__3__20210608T1552Z] Could not build segment
java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1072) ~[?:1.8.0_292]
at org.apache.pinot.segment.local.realtime.converter.RealtimeSegmentConverter.build(RealtimeSegmentConverter.java:132) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentInternal(LLRealtimeSegmentDataManager.java:783) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentForCommit(LLRealtimeSegmentDataManager.java:717) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:628) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
Caused by: java.lang.OutOfMemoryError: Java heap space
AsyncLogger error handling event seq=1, value='null': java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "HitExecutionView_3mo__3__3__20210608T1552Z" java.lang.OutOfMemoryError: Java heap space
2021/06/08 16:35:05.395 ERROR [LLRealtimeSegmentDataManager_HitExecutionView_3mo__7__3__20210608T1553Z] [HitExecutionView_3mo__7__3__20210608T1553Z] Could not build segment
java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1072) ~[?:1.8.0_292]
at org.apache.pinot.segment.local.segment.index.converter.SegmentV1V2ToV3FormatConverter.copyIndexData(SegmentV1V2ToV3FormatConverter.java:160) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.segment.local.segment.index.converter.SegmentV1V2ToV3FormatConverter.convert(SegmentV1V2ToV3FormatConverter.java:86) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.convertFormatIfNecessary(SegmentIndexCreationDriverImpl.java:370) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:303) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:256) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.segment.local.realtime.converter.RealtimeSegmentConverter.build(RealtimeSegmentConverter.java:131) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentInternal(LLRealtimeSegmentDataManager.java:783) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentForCommit(LLRealtimeSegmentDataManager.java:717) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:628) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-f15225f9c8abe8d9efa52c31c00f0d7418b368eb]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
Caused by: java.lang.OutOfMemoryError: Java heap space
Sadim Nadeem
06/09/2021, 7:39 AMMap
06/09/2021, 5:04 PMdocker build
to work although it compiles on my local computer. For example, if I do ./docker-build.sh pinot
under docker/images/pinot/docker-build.sh
, it fails when compiling pinot-controller
, and the error is
#12 241.2 [INFO] --- frontend-maven-plugin:1.1:npm (npm run-script build) @ pinot-controller ---
#12 241.2 [INFO] Running 'npm run-script build' in /opt/pinot-build/pinot-controller/src/main/resources
#12 241.4 [INFO]
#12 241.4 [INFO] > pinot-controller-ui@1.0.0 build /opt/pinot-build/pinot-controller/src/main/resources
#12 241.4 [INFO] > webpack --mode production
#12 241.4 [INFO]
#12 257.6 [ERROR] Killed
#12 257.6 [ERROR] npm ERR! code ELIFECYCLE
#12 257.6 [ERROR] npm ERR! errno 137
#12 257.6 [ERROR] npm ERR! pinot-controller-ui@1.0.0 build: `webpack --mode production`
#12 257.6 [ERROR] npm ERR! Exit status 137
#12 257.6 [ERROR] npm ERR!
#12 257.6 [ERROR] npm ERR! Failed at the pinot-controller-ui@1.0.0 build script.
#12 257.6 [ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
#12 257.6 [ERROR]
#12 257.6 [ERROR] npm ERR! A complete log of this run can be found in:
#12 257.6 [ERROR] npm ERR! /root/.npm/_logs/2021-06-09T16_55_54_192Z-debug.log
...
#12 264.0 [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.1:npm (npm run-script build) on project pinot-controller: Failed to run task: 'npm run-script build' failed. (error code 137) -> [Help 1]
#12 264.0 [ERROR]
#12 264.0 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
#12 264.0 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
#12 264.0 [ERROR]
#12 264.0 [ERROR] For more information about the errors and possible solutions, please read the following articles:
#12 264.0 [ERROR] [Help 1] <http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException>
#12 264.0 [ERROR]
#12 264.0 [ERROR] After correcting the problems, you can resume the build with the command
#12 264.0 [ERROR] mvn <args> -rf :pinot-controller
Am I missing something obvious? Not sure if someone else can reproduce it.Phúc Huỳnh
06/10/2021, 10:57 AMselect count(*) from RuleLogsUAT
But got timeout & difference result per request. Here is the request log
requestId=43,table=RuleLogsUAT,timeMs=10001,docs=210279/210377,entries=0/0,segments(queried/processed/matched/consuming/unavailable):21/18/18/2/0,consumingFreshnessTimeMs=1623319344461,servers=4/5,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);pinot-server-1_R=1,-1,0,0;pinot-server-2_R=2,6,250,0;pinot-server-0_R=2,6,250,0;pinot-server-0_O=1,7,230,0;pinot-server-1_O=2,11,230,0,query=select count(*) from RuleLogsUAT,offlineThreadCpuTimeNs=1548682,realtimeThreadCpuTimeNs=1908620
requestId=44,table=RuleLogsUAT,timeMs=10002,docs=44807/44811,entries=0/0,segments(queried/processed/matched/consuming/unavailable):19/17/17/3/0,consumingFreshnessTimeMs=1623319158004,servers=4/5,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);pinot-server-1_R=1,10,250,0;pinot-server-2_R=1,7,250,0;pinot-server-0_R=1,8,250,0;pinot-server-2_O=1,-1,0,0;pinot-server-1_O=1,11,230,0,query=select count(*) from RuleLogsUAT,offlineThreadCpuTimeNs=1211610,realtimeThreadCpuTimeNs=1412400
requestId=45,table=RuleLogsUAT,timeMs=10002,docs=73633/73731,entries=0/0,segments(queried/processed/matched/consuming/unavailable):23/20/20/3/0,consumingFreshnessTimeMs=1623319158007,servers=4/5,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);pinot-server-1_R=0,5,250,0;pinot-server-2_R=0,8,250,0;pinot-server-0_R=0,8,250,0;pinot-server-0_O=0,7,230,0;pinot-server-1_O=1,-1,0,0,query=select count(*) from RuleLogsUAT,offlineThreadCpuTimeNs=655059,realtimeThreadCpuTimeNs=1141471
requestId=46,table=RuleLogsUAT,timeMs=10002,docs=44807/44811,entries=0/0,segments(queried/processed/matched/consuming/unavailable):19/17/17/3/0,consumingFreshnessTimeMs=1623319158004,servers=4/5,groupLimitReached=false,brokerReduceTimeMs=0,exceptions=0,serverStats=(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);pinot-server-1_R=1,5,250,0;pinot-server-2_R=1,7,250,0;pinot-server-0_R=1,6,250,0;pinot-server-2_O=1,-1,0,0;pinot-server-1_O=1,7,230,0,query=select count(*) from RuleLogsUAT,offlineThreadCpuTimeNs=417430,realtimeThreadCpuTimeNs=1821230
Shailesh Jha
06/10/2021, 12:21 PMRavi R
06/10/2021, 12:32 PMbin/pinot-admin.sh AddTable \
-schemaFile examples/stream/airlineStats/airlineStats_schema.json \
-tableConfigFile examples/stream/airlineStats/airlineStats_realtime_table_config.json \
-exec
Error:
Got Exception to upload Pinot Schema: airlineStats
shaded.org.apache.http.conn.HttpHostConnectException: Connect to 172.26.0.5:9000 [/172.26.0.5] failed: Connection refused (Connection refused)
at shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.common.utils.FileUploadDownloadClient.sendRequest(FileUploadDownloadClient.java:383) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.common.utils.FileUploadDownloadClient.addSchema(FileUploadDownloadClient.java:448) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.command.AddTableCommand.uploadSchema(AddTableCommand.java:135) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.command.AddTableCommand.execute(AddTableCommand.java:170) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:164) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:184) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_282]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_282]
at java.net.Socket.connect(Socket.java:607) ~[?:1.8.0_282]
at shaded.org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
... 15 more
Exception caught:
shaded.org.apache.http.conn.HttpHostConnectException: Connect to 172.26.0.5:9000 [/172.26.0.5] failed: Connection refused (Connection refused)
at shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.common.utils.FileUploadDownloadClient.sendRequest(FileUploadDownloadClient.java:383) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.common.utils.FileUploadDownloadClient.addSchema(FileUploadDownloadClient.java:448) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.command.AddTableCommand.uploadSchema(AddTableCommand.java:135) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.command.AddTableCommand.execute(AddTableCommand.java:170) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:164) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:184) [pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_282]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_282]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_282]
at java.net.Socket.connect(Socket.java:607) ~[?:1.8.0_282]
at shaded.org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
at shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed]
... 15 more
DOCKER_HOST=unix:///var/run/docker.sock docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
545bf572fd0d apachepinot/pinot:release-0.7.1 "./bin/pinot-admin.s…" 2 hours ago Up 2 hours 8096-8097/tcp, 8099/tcp, 9000/tcp, 0.0.0.0:8098->8098/tcp, :::8098->8098/tcp composetest_pinot-server_1
4033365f967e apachepinot/pinot:release-0.7.1 "./bin/pinot-admin.s…" 2 hours ago Up 2 hours 8096-8098/tcp, 9000/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp composetest_pinot-broker_1
1712fba17460 apachepinot/pinot:release-0.7.1 "./bin/pinot-admin.s…" 2 hours ago Up 2 hours 8096-8099/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp composetest_pinot-controller_1
aeac35e847b6 confluentinc/cp-kafka:5.3.0 "/etc/confluent/dock…" 2 days ago Up 2 hours 0.0.0.0:9092->9092/tcp, :::9092->9092/tcp composetest_kafka_1
050bf6511ab9 zookeeper:latest "/docker-entrypoint.…" 2 days ago Up 2 hours 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 8080/tcp composetest_zookeeper_1
Can anyone kindly suggest how to proceed from here ?Aaron Wishnick
06/10/2021, 4:35 PMJonathan Meyer
06/10/2021, 5:33 PMSELECT * FROM table WHERE x in (SELECT ...)
and
2. SELECT * FROM (SELECT ...)
(i.e. working with the output of a subquery)luanmorenomaciel
06/10/2021, 7:14 PMStarting a Pinot [SERVICE_MANAGER] at 0.08s since launch
Registering service status handler
Skip Starting Pinot Service Manager admin application
Started Pinot [SERVICE_MANAGER] instance [ServiceManager_pinot-broker-0.pinot-broker-headless.datastore.svc.cluster.local_-1] at 0.084s since launch
Starting a Pinot [BROKER] at 0.087s since launch
Trying to start Pinot Broker...
Failed to initialize Pinot Broker Starter
java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_292]
at java.lang.Integer.parseInt(Integer.java:592) ~[?:1.8.0_292]
at java.lang.Integer.parseInt(Integer.java:615) ~[?:1.8.0_292]
at org.apache.pinot.core.util.ListenerConfigUtil.buildBrokerConfigs(ListenerConfigUtil.java:107) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.broker.broker.helix.HelixBrokerStarter.<init>(HelixBrokerStarter.java:131) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.service.PinotServiceManager.startBroker(PinotServiceManager.java:133) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:92) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.lambda$run$0(StartServiceManagerCommand.java:259) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:285) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.access$000(StartServiceManagerCommand.java:56) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand$1.run(StartServiceManagerCommand.java:259) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
Failed to start a Pinot [BROKER] at 0.114 since launch
Calvin Karundu
06/11/2021, 9:52 AMPinot schema column is called "properties" with a type of string
# First Record
{
timestamp: '...',
browser: '...'
}
# Second Record
null
# Third Record
{
balance: '...'
}
This data is provided by the end user and can be whatever they’d like (this is by design) … so there’s really no “schema” for the data in the JSON column.Jonathan Meyer
06/13/2021, 10:59 AMSELECT SUM(value) FROM table WHERE user IN LOOKUP('group', 'user', 'groupId', '<groupId>')
? (which isn't valid)
Basically my goal is to fetch a list of 'users' in the dimTable using a 'group' identifier and filtering on those (in the main table)luanmorenomaciel
06/14/2021, 5:59 PM21/06/11 21:08:15.665 ERROR [LLRealtimeSegmentDataManager_realtime_enriched_music_data_users__7__0__20210611T2008Z] [realtime_enriched_music_data_users__7__0__20210611T2008Z] Could not build segment
java.lang.IllegalArgumentException: Invalid format: "2021-06-10 10:42:25" is too short
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:826) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.writeMetadata(SegmentColumnarIndexCreator.java:552) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.seal(SegmentColumnarIndexCreator.java:512) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:284) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:257) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.realtime.converter.RealtimeSegmentConverter.build(RealtimeSegmentConverter.java:131) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentInternal(LLRealtimeSegmentDataManager.java:794) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.buildSegmentForCommit(LLRealtimeSegmentDataManager.java:728) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:634) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
2021/06/11 21:08:15.665 ERROR [LLRealtimeSegmentDataManager_realtime_enriched_music_data_users__4__0__20210611T2008Z] [realtime_enriched_music_data_users__4__0__20210611T2008Z] Could not build segment
java.lang.IllegalArgumentException: Invalid format: "2021-06-10 10:42:25" is too short
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:826) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.writeMetadata(SegmentColumnarIndexCreator.java:552) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.seal(SegmentColumnarIndexCreator.java:512) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.handlePostCreation(SegmentIndexCreationDriverImpl.java:284) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
Elon
06/14/2021, 10:52 PMjava.io.IOException: Input is not in the .gz format
- we saw that the segment was just being written to gcs. Once I deleted the segment on the server and restarted it downloaded it without any issues. Has anyone ever experienced that before? I can create a github issue w some ideas for fixes...khush
06/15/2021, 6:23 AMAzri Jamil
06/15/2021, 2:39 PMSurendra
06/15/2021, 5:38 PM2021/06/10 16:53:02.084 WARN [BaseBrokerRequestHandler] [jersey-server-managed-async-executor-100] Caught exception while updating Column names in Query 13040: Illegal AstNode type for TransformExpressionTree: org.apache.pinot.pql.parsers.pql2.ast.StarExpressionAstNode
and if we update else null
in case clause to else 'null'
it's working in .7.1 , any clues ?
select emp_id ,
id ,
max(timestampMs) as last_updated,
count(DISTINCT case when emp_type = 'admin' then user_count else null end) as admin_counts
from emp_events
where org_id = 987
AND timestampMs >= 1620892156939
AND timestampMs < 1621630122162
group by emp_id, id
limit 10
Jonathan Meyer
06/15/2021, 6:45 PMselect DISTINCT(kpi) from kpis
takes ~6ms (with 100M docs, & numDocsScanned: 100000
) - this query returns 45 strings only
But doing
select DISTINCT(kpi) from kpis ORDER BY kpi
takes >300ms (50 times slower) - It scans every documents (numDocsScanned: 101250000
)
I guess the ORDER BY
breaks some optimizations down
But from the outside it seems like pretty surprising behavior (sorting 45 strings "should not take this long" is what I mean)
Anyway, not here to complain, just wanted to point it out in case it would be considered as something worth investigatingJai Patel
06/15/2021, 8:41 PM{
"name": "created_at_1_week_seconds",
"dataType": "LONG",
"defaultNullValue": 0,
"transformFunction": "dateTrunc('week', created_at, 'MILLISECONDS')",
"format": "1:SECONDS:EPOCH",
"granularity": "1:SECONDS"
},
Error:
Exception in getting arguments for transform function 'dateTrunc('week', created_at, MILLISECONDS)' for column 'created_at_1_week_seconds'"
}
This works:
{
"name": "created_at_1_day_seconds",
"dataType": "LONG",
"defaultNullValue": 0,
"transformFunction": "toEpochSecondsRounded(fromEpochDaysBucket(toEpochDaysBucket(created_at, 1), 1), 1)",
"format": "1:SECONDS:EPOCH",
"granularity": "1:SECONDS"
},
Something about the dateTrunc function doesn’t seem to be compatible. Looking through the docs and the code it looks like dateTrunc may not be listed as a valid transform function in all use cases. The main issue is I want a toEpochDaysBucket(7) that lines up with the calendar week (Sunday or Monday) instead of the epoch week (Thursday).
Any ideas?Surendra
06/15/2021, 11:47 PMEugene Ramirez
06/16/2021, 3:53 AMback up/restore
feature. I appreciate if anyone can help.
Here is my setup.
Kubernetes: EKS 1.20.4
Pinot version: 0.7.1
So I enable S3 as deep storage based from this link.
Then ingested Parquet data from S3 based on this instruction.
Data loaded fine and I can query the expected data from Pinot.
Next I simulated replacing the cluster, by uninstalling all pods and its related volumes(therefore losing all state) but kept the segment files in s3 segment location(therefore backup is intact in deep store).
Next I reinstalled cluster, and reconfigured the tables. I was expecting that the servers would automatically fetch the segments from deep store as mentioned in previous post, but it does not seem to be happening.
Am i missing a step? Thanks in advance.
https://apache-pinot.slack.com/archives/C011C9JHN7R/p1623336369017900?thread_ts=1623327667.015000&cid=C011C9JHN7Rjainendra tarun
06/16/2021, 4:59 PMselect * from pinot.default.mytable where where datekey='2021-04-19 00:00:00' limit 10
Though this query returns empty result in Pinot as expected. The exception in presto is :
java.lang.IllegalStateException: Expected at least one row to be present
at com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at com.facebook.presto.pinot.PinotBrokerPageSourceSql.populateFromQueryResults(PinotBrokerPageSourceSql.java:118)
at com.facebook.presto.pinot.PinotBrokerPageSourceBase.lambda$issueQueryAndPopulate$0(PinotBrokerPageSourceBase.java:327)
at com.facebook.presto.pinot.PinotUtils.doWithRetries(PinotUtils.java:39)
at com.facebook.presto.pinot.PinotBrokerPageSourceBase.issueQueryAndPopulate(PinotBrokerPageSourceBase.java:312)
at com.facebook.presto.pinot.PinotBrokerPageSourceBase.getNextPage(PinotBrokerPageSourceBase.java:222)
at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:252)
at com.facebook.presto.operator.Driver.processInternal(Driver.java:418)
at com.facebook.presto.operator.Driver.lambda$processFor$9(Driver.java:301)
at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:722)
at com.facebook.presto.operator.Driver.processFor(Driver.java:294)
at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:599)
at com.facebook.presto.$gen.Presto_0_256_SNAPSHOT_5059796____20210616_162510_1.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Is this a bug or am I missing some configuration ?RK
06/16/2021, 5:35 PMMateus Oliveira
06/16/2021, 7:53 PMTrying to create instance for class org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
Initializing PinotFS for scheme s3, classname org.apache.pinot.plugin.filesystem.S3PinotFS
Creating an executor service with 1 threads(Job parallelism: 0, available cores: 1.)
Listed 8 files from URI: <s3://landing/bank/>, is recursive: true
Got exception to kick off standalone data ingestion job -
java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:144) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:113) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.execute(LaunchDataIngestionJobCommand.java:132) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:166) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:186) [pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
Caused by: java.lang.IllegalArgumentException
at sun.nio.fs.UnixFileSystem.getPathMatcher(UnixFileSystem.java:288) ~[?:1.8.0_292]
at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.run(SegmentGenerationJobRunner.java:175) ~[pinot-batch-ingestion-standalone-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:142) ~[pinot-all-0.8.0-SNAPSHOT-jar-with-dependencies.jar:0.8.0-SNAPSHOT-2de40fde8051c2c0281416c2da11c179c2190435]
... 4 more
this is my job
executionFrameworkSpec:
name: 'standalone'
segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '<s3://landing/bank/>'
includeFileNamePattern: '*.json'
outputDirURI: '<s3://pinot/>'
overwriteOutput: true
pinotFSSpecs:
- scheme: s3
className: org.apache.pinot.plugin.filesystem.S3PinotFS
configs:
region: 'us-east-1'
endpoint: '<http://10.0.220.205:9000>'
accessKey: 'access'
secretKey: 'key'
recordReaderSpec:
dataFormat: 'json'
className: 'org.apache.pinot.plugin.inputformat.json.JSONRecordReader'
tableSpec:
tableName: 'bank'
pinotClusterSpecs:
- controllerURI: '<http://localhost:9000>'
Jackie
06/17/2021, 1:46 AMDaniel Lavoie
06/17/2021, 2:03 AMLaxman Ch
06/18/2021, 2:50 PMJai Patel
06/18/2021, 4:51 PMAaron Wishnick
06/18/2021, 8:58 PMSUBSTR(foo, ..., ...) = bar
I still see the query taking more than 10 secondsZsolt Takacs
06/21/2021, 7:08 AMJonathan Meyer
06/21/2021, 7:25 AM"message": "MergeResponseError: Data schema mismatch between merged block:
after a REALTIME table backward-compatible schema change ? (new column)
I've tried Reload All Segments
Is that because I haven't set pinot.server.instance.reload.consumingSegment
?