Hi @Mayank @Xiang Fu while loading json data to OFFLINE table , am getting below error and segment state showing as ERROR.

java.lang.reflect.InvocationTargetException
        at jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350)
        at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278)
        at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97)
        at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.OutOfMemoryError: Cannot reserve 132127 bytes of direct buffer memory (allocated: 64424466414, limit: 64424509440)

am actually using spark3 to push data to pinot OFFLINE table. any help here?

looks like you are running out of heap memory, u can maybe try increasing your jvm heap allocation (ensure u have sufficient ram) also try seeing if there are a lot of segments that are being pushed by the spark job at once, or if there are a lot of small segments (that may add some overhead to the metadata manager)

try out more xmx from your jvm settings