Is there a tool / utility to read segment data . m...
# general
s
Is there a tool / utility to read segment data . modify it and upload . I was looking to generate bulk data for query performance testing pinot . Thanks ..
I was able to figure out
pinot-tools
and use
./pinot-admin.sh ConvertPinotSegment
to get data from segment. however when I try to create a segment from same json file which was generate by ``ConvertPinotSegment`` I get below exception:
Copy code
2020/08/26 18:11:02.352 ERROR [PinotAdministrator] [main] Exception caught: 
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Value: vNGrzPyoR5w= is not Hex encoded
	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
	at org.apache.pinot.tools.admin.command.CreateSegmentCommand.execute(CreateSegmentCommand.java:264) ~[classes/:?]
	at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:150) [classes/:?]
	at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:162) [classes/:?]
Caused by: java.lang.IllegalArgumentException: Value: vNGrzPyoR5w= is not Hex encoded
	at org.apache.pinot.spi.utils.BytesUtils.toBytes(BytesUtils.java:39) ~[classes/:?]
	at org.apache.pinot.common.utils.PinotDataType$9.toBytes(PinotDataType.java:344) ~[classes/:?]
	at org.apache.pinot.common.utils.PinotDataType$10.convert(PinotDataType.java:386) ~[classes/:?]
I used
./pinot-admin.sh CreateSegment
cc @Jackie looking at commits in file, you might have some context. thanks
issue seems to be happening for
JSON
format, and works fine for
AVRO
k
@samarth Can you please file a PR to fix the JSON format
j
@samarth Are you using the same schema to generate the segment? Can you please share a sample json record?
s
@Jackie sent you in DM