Hi All,
I am using pinot libraries to load a segment and modify the value of a column in a table to add additional encrypted data for other workflow purpose. As part of this task, i am using pinot libraries and its API's. Once the column values are modified, recreate the segment as csv file format and convert it back to segments again. The updated segments, i would be using PINOT REST APIs to upload the same in PINOT DB.
Below are the libraries used.
implementation 'org.apache.pinot
pinot common1.3.0'
implementation 'org.apache.pinot
pinot segment spi1.3.0'
implementation 'org.apache.pinot
pinot segment local1.3.0'
implementation 'org.apache.pinot
pinot core1.3.0'
implementation 'org.apache.pinot
pinot spi1.3.0'
implementation("org.reflections
reflections0.10.2")
IndexSegment indexSegment = ImmutableSegmentLoader.load(segmentFileToLoad, ReadMode.mmap; / IndexSegment indexSegment = ImmutableSegmentLoader.load(segmentFileToLoad, ReadMode.heap);
For the above API has been invoked from Spring 6, SPRING Boot Microservices Simple Test Program. The output of above method is not returning the indexSegment. Further debugged the pinot code, i am getting the error in below lines...
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/PinotDataBuffer.java.
/**
* Memory maps a file into a buffer.
*
pNOTE: If the file gets extended, the contents of the extended portion of the file are not defined.
*/
public static *PinotDataBuffer mapFile*(File file, boolean readOnly, long offset, long size, ByteOrder byteOrder,
/**
* Allocates a buffer using direct memory and loads a file into the buffer.
*/
public static PinotDataBuffer loadFile(File file, long offset, long size, ByteOrder byteOrder,
@Nullable String description)
throws IOException {
PinotDataBuffer buffer;
Above line is invoked from "*pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/SingleFileIndexDirectory.java*"
Code is stuck in loadFile(...), mapFile(....) methods.
private void mapAndSliceFile(SortedMap
Long, IndexEntry startOffsets, List
Long offsetAccum, long endOffset)
throws IOException {
Preconditions.checkNotNull(startOffsets);
Preconditions.checkNotNull(offsetAccum);
Preconditions.checkArgument(!offsetAccum.isEmpty());
long fromFilePos = offsetAccum.get(0);
long size = endOffset - fromFilePos;
String context = allocationContext(_indexFile,
"
single_file_index.rw." + "." + String.valueOf(fromFilePos) + "." + String.valueOf(size));
// Backward-compatible: index file is always big-endian
PinotDataBuffer buffer;
if (_readMode == ReadMode.heap) {
buffer = *PinotDataBuffer.loadFile(_*indexFile, fromFilePos, size, ByteOrder.BIG_ENDIAN, context);
} else {
buffer = *PinotDataBuffer.mapFile*(_indexFile, true, fromFilePos, size, ByteOrder.BIG_ENDIAN, context);
}
The same program works fine in Spring framework 3, 4 and only
spring framework 6, spring 3
it is not working. Need some inputs on spring framework 6 the same jars with API's are not working?
Added below add-opens JVM as well in spring framework 6....
--add-opens java.base/java.lang=ALL-UNNAMED \
--add-opens java.base/java.util=ALL-UNNAMED \
--add-opens java.base/java.nio=ALL-UNNAMED \
--add-opens java.base/java.io=ALL-UNNAMED \
--add-opens java.base/java.security=ALL-UNNAMED \
--add-opens java.base/sun.nio.ch=ALL-UNNAMED \
--add-opens java.base/java.lang.reflect=ALL-UNNAMED \
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED