https://pinot.apache.org/ logo
Join Slack
Powered by
# troubleshooting
  • h
    Any idea if we can create table from more than 1 topic (on same broker)??? Can we list topics with “,” and Pinot can read all topic to create a single table??
    m
    • 2
    • 1
  • d
    Hi Pinot Experts. After Pinot ingests realtime events from Kafka, is there any support to emit the original event/change event to downstream after the event can be queried in pinot? Currently, I am faced with query-after-write consistency problem
    m
    k
    m
    • 4
    • 10
  • y
    Hi, Where can I find documentation about Pinot ingestion job types?
    m
    • 2
    • 2
  • g
    hi all, got an issue after ingesting a segment where the segment ends up in an ERROR state due to
    Copy code
    java.lang.IllegalStateException: Forward index disabled column name must have a dictionary
    i do see in the metadata.properties that this column does have 
    hasDictionary = true
    though, any idea of where else i can look at to fix this issue?
    c
    • 2
    • 3
  • b
    Hi. How can I get a connectionfactory for pinot zookeeper? I'm referring to this documentation https://docs.pinot.apache.org/users/clients/java A screenshot where I'm trying to get the connection factory.
    m
    • 2
    • 9
  • s
    Hi Team, I get the SQLParsing error when I use perform join with v2 engine enabled. Can anyone help me with this.
    x
    • 2
    • 2
  • d
    Hi all, I am seeing this exception for every realtime table I have created at five minute interval, please help me understand this exception and resolve this. 
    javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=PartitionGroupMetadataFetcher-
    x
    • 2
    • 6
  • f
    Hi, is upsert possible in OFFLINE tables (ingestion job) in Apache Pinot?
    m
    • 2
    • 1
  • k
    Hi, all I am using pinot 0.12.1. Problem • Kinesis consumer is trying to fetch messages from wrong shard (
    000000000000
    ) and offset (
    49611548018071132552656657372575220481442610790216499202
    ) ◦ Currently the kinesis stream has 2 shards 
    000000000001
    AND 
    000000000002
    ◦ and the most recent offset is greater than that (
    49611548018071132552656657372575220481442610790216499202
    ) • I tried 
    LATEST
    config for shardIterator But no luck. ◦ Also tried 
    "stream.kinesis.consumer.prop.auto.offset.reset": "largest",
    but It does not work too. • Seems consumer is failure to get shard information Here are few messages regard to the consumer.
    Cannot fetch latest stream offset for clientId {KINESIS_STREAM_NAME}-0 and partitionGroupId 0 with maxWaitTime 5000
    …
    Starting consumption loop start offset {“shardId-000000000000”:“49611548018071132552656657372575220481442610790216499202"}, finalOffset null
    …
    Consumed 0 events from (rate:0.0/s), currentOffset={“shardId-000000000000”:“49611548018071132552656657372575220481442610790216499202"}, numRowsConsumedSoFar=0, numRowsIndexedSoFar=0
    Question Q1. Is there any way to reset kinesis stream offset? Q2. The problematic kinesis stream was created at 2020-09-01. Can’t I use a Kinesis Stream created a long time ago with Pinot? Is there anything I can refer? Thanks.
    n
    • 2
    • 12
  • a
    Hello, I am trying to query Pinot via Trino and running into following Error in Pinot Server logs.
    Copy code
    Stream Error
io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:167)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.cancel(DefaultHttp2RemoteFlowController.java:481)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$1.onStreamClosed(DefaultHttp2RemoteFlowController.java:105)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:356)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1000)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:956)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:512)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.close(DefaultHttp2Connection.java:152)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$BaseDecoder.channelInactive(Http2ConnectionHandler.java:209)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.channelInactive(Http2ConnectionHandler.java:417)
	at io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.channelInactive(NettyServerHandler.java:601)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)
	at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)
	at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)
	at io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
	at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)
    x
    • 2
    • 13
  • r
    Copy code
    AsyncLogger error handling event seq=0, value='null': Exception in thread "HttpServer-0" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
    Does Hi team, yesterday we got this error in our pinot-server while running query. Is there anyone configuration to clear heap space without restarting server and how can we configure in helm charts?
    x
    • 2
    • 8
  • d
    Hi have an issue with quite a few segments' on out uat and prod servers where the segments are reported as missing (ERROR) or (OFFLINE) , but when you look at the servers them selves the data is there , have tried segment reload and segment reset and server rebalance with bootstrap ... see pick and server listings
    m
    • 2
    • 1
  • d
    image.png
  • d
    Copy code
    <http://my-server-ptctlr01:9000/segments/location_density> -> /data/pinot/data/controller/location_density
-rw-r--r--. 1 pinot pinot 4098 Feb 13 13:05 location_density__0__111__20230212T1305Z

[root@my-server-ptctlr02 location_density]$ ll location_density__0__111__20230212T1305Z
-rw-r--r--. 1 pinot pinot 4098 Feb 13 13:05 location_density__0__111__20230212T1305Z


[root@my-server-ptsvr01 location_density_REALTIME]$ pwd
/data/pinot/data/index/location_density_REALTIME

[root@my-server-ptsvr01 location_density_REALTIME]$ ls -l location_density__0__111__20230212T1305Z/*
-rw-r--r--. 1 pinot pinot    0 Mar 27 09:33 location_density__0__111__20230212T1305Z/location_timestamp.bloom.inprogress

location_density__0__111__20230212T1305Z/v3:
total 20
-rw-r--r--. 1 pinot pinot 4534 Mar 27 09:33 columns.psf
-rw-r--r--. 1 pinot pinot   16 Mar 27 09:33 creation.meta
-rw-r--r--. 1 pinot pinot 1260 Mar 27 09:33 index_map
-rw-r--r--. 1 pinot pinot 3906 Mar 27 09:33 metadata.properties

[root@my-server-ptsvr02 location_density_REALTIME]$  ls -l location_density__0__111__20230212T1305Z/*
-rw-r--r--. 1 pinot pinot    0 Mar 27 08:41 location_density__0__111__20230212T1305Z/location_timestamp.bloom.inprogress

location_density__0__111__20230212T1305Z/v3:
total 20
-rw-r--r--. 1 pinot pinot 4534 Mar 27 08:41 columns.psf
-rw-r--r--. 1 pinot pinot   16 Mar 27 08:41 creation.meta
-rw-r--r--. 1 pinot pinot 1260 Mar 27 08:41 index_map
-rw-r--r--. 1 pinot pinot 3906 Mar 27 08:41 metadata.properties

[root@my-server-ptsvr03 location_density_REALTIME]$ ls  -l location_density__0__111__20230212T1305Z/*
-rw-r--r--. 1 pinot pinot    0 Mar 27 09:32 location_density__0__111__20230212T1305Z/location_timestamp.bloom.inprogress

location_density__0__111__20230212T1305Z/v3:
total 20
-rw-r--r--. 1 pinot pinot 4534 Mar 27 09:32 columns.psf
-rw-r--r--. 1 pinot pinot   16 Mar 27 09:32 creation.meta
-rw-r--r--. 1 pinot pinot 1260 Mar 27 09:32 index_map
-rw-r--r--. 1 pinot pinot 3906 Mar 27 09:32 metadata.properties
  • d
    Hi, I am trying to filter records using 
    filterConfig
    but not quite sure why its not really working out. Attaching the schema and filter function.
    d
    p
    • 3
    • 6
  • d
    Hello, I am running into a OOMKilled on my pinot servers. This happened with resource request of 8gb memory and limit of 24gb memory Does anyone have any suggestions with any tools or commands on how to analyze what is consuming all the memory on the pinot server to cause the crash? We are using realtime tables
    m
    • 2
    • 3
  • d
    Additionally, we are using prometheus with grafana and ever since the crash, we are not seeing and JVM usage data but the CPU shows
  • a
    I have been trying out the pinot-admin 
    UploadSegment
    and every time I upload a new segment, instead of adding to the existing segments, it replaces it with what I provide it. Is there a way I can specify it to append to existing segments. I see an option existing in batched ingestions but cannot find one for ad hoc processes like this. This is the exact command 
    bin/pinot-admin.sh UploadSegment -controllerHost localhost -controllerPort 9000 -segmentDir ./outputDir
    Thank you in advance for helping out with this noob problem. 🙂
    • 1
    • 1
  • p
    I getting this in my server logs, can anyone say what this is :
    Copy code
    Controller response {"streamPartitionMsgOffset":"100000","buildTimeSec":126,"isSplitCommitType":true,"controllerVipUrl":"<http://pinot-na1-controller-1.pinot-na1-controller-headless.revenue-reporting.svc.cluster.local:9000>","statu │
│ Sending request: <http://pinot-na1-controller-1.pinot-na1-controller-headless.revenue-reporting.svc.cluster.local:9000/segmentCommitStart?segmentSizeBytes=85637857&buildTimeMillis=3509&streamPartitionMsgOffset=100000&instance=Serv> │
│ Controller response {"streamPartitionMsgOffset":null,"buildTimeSec":-1,"isSplitCommitType":false,"status":"COMMIT_CONTINUE","offset":-1} for <http://pinot-na1-controller-1.pinot-na1-controller-headless.revenue-reporting.svc.cluste> │
│ Sending request: <http://pinot-na1-controller-1.pinot-na1-controller-headless.revenue-reporting.svc.cluster.local:9000/segmentUpload?segmentSizeBytes=85637857&buildTimeMillis=3509&streamPartitionMsgOffset=100000&instance=Server_pi> │
│ Controller response {"streamPartitionMsgOffset":null,"buildTimeSec":-1,"isSplitCommitType":false,"status":"FAILED","offset":-1} for <http://pinot-na1-controller-1.pinot-na1-controller-headless.revenue-reporting.svc.cluster.local:9> │
│ Could not commit segment. Retrying after hold
    m
    • 2
    • 4
  • p
    Hi All!! I see that RT tables support MergeRollupTask based on this . will the steps mentioned here work on the RT tables ? Any example / documentation I can check for the same? I am trying to implement it right now. Thanks !!!
    l
    s
    +2
    • 5
    • 20
  • s
    For those who may not be aware of it this utility has been very helpful: http://java-decompiler.github.io/ Using it I can easily examine the contents of the jars from the build. It has been especially helpful in verifying the contents of the shaded jars.
  • a
    I am trying to query ~1TB data size in Pinot and run into the following IO exception right away when I query using trino. When I check the node health it is fine, but get this exception. Any pointers. Is there IO network limit I am hitting.
    Copy code
    2023-03-30T01:36:32.639Z	ERROR	SplitRunner-138-61	io.trino.execution.executor.TaskExecutor	Error processing Split 20230330_013630_00009_paj2g.4.0.0-9 PinotSplit{splitType=SEGMENT, segments=[<redacted>__8__2333__20230328T2149Z], segmentHost=Optional[<redacted>]} (start = 3.1565860028796E7, wall = 988 ms, cpu = 0 ms, wait = 0 ms, calls = 1)
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
	at io.grpc.Status.asRuntimeException(Status.java:535)
	at io.grpc.stub.ClientCalls$BlockingResponseStream.hasNext(ClientCalls.java:648)
	at io.trino.plugin.pinot.client.PinotGrpcDataFetcher$PinotGrpcServerQueryClient$ResponseIterator.computeNext(PinotGrpcDataFetcher.java:274)
	at io.trino.plugin.pinot.client.PinotGrpcDataFetcher$PinotGrpcServerQueryClient$ResponseIterator.computeNext(PinotGrpcDataFetcher.java:261)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
	at io.trino.plugin.pinot.client.PinotGrpcDataFetcher.endOfData(PinotGrpcDataFetcher.java:85)
	at io.trino.plugin.pinot.PinotSegmentPageSource.getNextPage(PinotSegmentPageSource.java:124)
	at io.trino.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:400)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:182)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.lambda$flatten$7(WorkProcessorUtils.java:296)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:338)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$3(WorkProcessorUtils.java:219)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils.lambda$finishWhen$4(WorkProcessorUtils.java:234)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:150)
	at io.trino.operator.Driver.processInternal(Driver.java:411)
	at io.trino.operator.Driver.lambda$process$10(Driver.java:314)
	at io.trino.operator.Driver.tryWithLock(Driver.java:706)
	at io.trino.operator.Driver.process(Driver.java:306)
	at io.trino.operator.Driver.processForDuration(Driver.java:277)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:736)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:164)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:515)
	at io.trino.$gen.Trino_400____20230330_004502_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
    g
    s
    x
    • 4
    • 17
  • p
    Hi Team, Could you please help with following queries. 1. How Can I explode below JSON, specially the "Results" Object so that , each value in delta/gamma is in separate row. 
    {
    "positionid" : 123, "securityId" : 345, "Results" : { delta : [123, 456, 789], gamma : [0.0, 1.2, 4.5] } } 2. How can we achieve ARRAY JOIN feature provided in clickhouse in pinot.
    k
    c
    • 3
    • 4
  • l
    HI Team, may i ask what is the pinot.query.server.port and runner port when enabling the v2 engine ? Do i have to open those ports ? i am facing some issue when using the v2 engine.
    Copy code
    [
  {
    "errorCode": 200,
    "message": "QueryExecutionError:\nio.grpc.StatusRuntimeException: UNAVAILABLE: io exception\n\tat io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)\n\tat io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)\n\tat io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)\n\tat org.apache.pinot.common.proto.PinotQueryWorkerGrpc$PinotQueryWorkerBlockingStub.submit(PinotQueryWorkerGrpc.java:166)\n...\nCaused by: io.grpc.netty.shaded.io.netty.channel.ConnectTimeoutException: connection timed out: ip-10-110-219-144.ap-southeast-1.compute.internal/10.110.219.144:8421\n\tat io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261)\n\tat io.grpc.netty.shaded.io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)\n\tat io.grpc.netty.shaded.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)\n\tat io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)"
  }
]
    x
    r
    • 3
    • 12
  • s
    Hi everyone, I am using default kafka partitioning strategy in my kafka stream and have set requried config in pinot RT table config i.e. functionName= murmur. But I am observing that I am not able to get the result on querying pinot on applying the PK filter for a consuming segment. Query: 
    select * from table where id=xyz;
    Result: No record(s) found. This doesn't happen for a PK which resides in an online segment. Now when I remove the segmentPrunerType= partition, I start getting result for the consuming segment as well. Can somebody enlighten why am I facing this with the consuming segments only?
    m
    n
    h
    • 4
    • 32
  • l
    Hi Team, If the table of segments in one pinot-server does not match the table of indexes, can i remove the extra table from the index?
    x
    p
    • 3
    • 4
  • p
    Hello! I am having an issue with a realtime table freezing at different number of docs ingested. They always seem to stop be an event number of docs like (600000). Any idea what can cause this?
    m
    • 2
    • 12
  • e
    Hello, I'm trying to build Pinot from source (the 
    master
    branch) and I'm getting a 
    MojoFailureException
    when trying to build 
    pinot-broker
    (Error details below):
    Copy code
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project pinot-broker: Compilation failure -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project pinot-broker: Compilation failure
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:306)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:211)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:165)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:157)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:121)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:127)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1219)
    at org.apache.maven.plugin.compiler.TestCompilerMojo.execute (TestCompilerMojo.java:181)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:301)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:211)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:165)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:157)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:121)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:127)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] <http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException>
    My current Java environment is:
    Copy code
    ➜  pinot git:(master) javac -version
javac 17.0.6
➜  pinot git:(master) java -version
openjdk version "17.0.6" 2023-01-17
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.10-1.fc37) (build 17.0.6+10)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.10-1.fc37) (build 17.0.6+10, mixed mode, sharing)
    m
    • 2
    • 3
  • t
    Hey, i have setup a pinot cluster and i am trying to Batch-ingestion data, using command
    Copy code
    apache-pinot-0.9.0-bin/bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile jobspec.yml
    i specified some outputURI in spec that it is not able to create.. is any extra param required in this class or is that output Dir supposed to be specified at time of cluster creation.. error
    Copy code
    Caused by: java.io.IOException: Unable to create directory /var/pinot/server/data/segment
	at org.apache.commons.io.FileUtils.forceMkdir(FileUtils.java:2384)
	at org.apache.pinot.spi.filesystem.LocalPinotFS.mkdir(LocalPinotFS.java:50)
	at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.init(SegmentGenerationJobRunner.java:126)
  • t
    jobspec
    Copy code
    executionFrameworkSpec:
  name: 'standalone'
  segmentGenerationJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner'
  segmentTarPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentTarPushJobRunner'
  segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'
jobType: SegmentCreationAndTarPush
inputDirURI: '.'
includeFileNamePattern: 'glob:**/export_json.json'
outputDirURI: '/var/pinot/server/data/segment'
overwriteOutput: true
pinotFSSpecs:
  - scheme: hdfs
    className: org.apache.pinot.plugin.filesystem.HadoopPinotFS
  - scheme: file
    className: org.apache.pinot.spi.filesystem.LocalPinotFS
recordReaderSpec:
  dataFormat: 'json'
  className: 'org.apache.pinot.plugin.inputformat.json.JSONRecordReader'
  configClassName: 'org.apache.pinot.plugin.inputformat.json.JSONRecordReaderConfig'
tableSpec:
  tableName: 'reporting_benchmark_REALTIME'
pinotClusterSpecs:
  - controllerURI: '<http://localhost:9000>'
1...757677...166Latest