Resharing my below query here to wider users. Please let me know if anyone here was successful in setting up atomic data load using the spark ingestion https://docs.pinot.apache.org/release-1.4.0/manage-data/data-import/batch-ingestion/spark. Thank you. https://apache-pinot.slack.com/archives/C011C9JHN7R/p1763605148332299 cc: @@Priyen Patel, @Jackie

this is definitely working as we’re using it internally

can you share me your template table config file or how did u set this up.. thanks

it’s what you have already

"consistentDataPush": true

yes i was expecting segments in pinot to be created with appended timestamp as per documentation. Which is not happening

how are you specifying the pinot dependency in your spark application? Is there a chance you’re using some older version of Pinot that does not have this?

I was using 1.2 version. Let me try again with 1.4 version then

nice, that should hopefully solve it 🤞

It worked for me in 1.4 version. Thank you.

awesome, ty for following up