I'm not 100% sure those docs are right - I think y...
# troubleshooting
m
I'm not 100% sure those docs are right - I think you do need to explicitly specify the
sortedColumn
in your table config. Or have you already tried setting it?
x
I haven't, but will try tomorrow 🙂
It does automatically set for some columns (e.g. with all values as 0, another properly sorted)
So i took the docs at face value
sorry, this was my bad - the data type of the column was wrong (specified as STRING while it was an INT), so the ingestion job did not correctly derive it as sorted
m
oh, cool
I'm quite curious now though. If you call the segment metadata endpoint and pass in the name of that column, doe it show that it has a sorted index? http://localhost:9000/help#/Segment/getServerMetadata https://docs.pinot.apache.org/basics/getting-started/frequent-questions/ingestion-faq#how-to-apply-an-inverted-index-to-existing-segments
x
im not sure what segment metadata endpoint is that, i just downloaded the segment directly and inspected the
metadata.properties
m
@xtrntr I am wrong about needing to explicitly state the sorted column. For offline ingestion a sorted index should be auto-added to any columns where the data is sorted.
x
okay thanks for the clarification. i’ve figured out that the issue is that spark is not writing out sorted data..