Emerson Lesage
08/06/2025, 2:15 PMstartReplaceSegments and endReplaceSegments endpoints the correct and most robust approach for doing atomic transactions for offline tables?
For example, if my table currently contains segments s1, s2, and s3, and I need to do the following in an all or nothing transaction:
• Insert new segments s4 and s5.
• Replace existing segment s3 with a new segment s6.Mayank
Manish
08/06/2025, 2:54 PMEmerson Lesage
08/06/2025, 3:31 PMEmerson Lesage
08/06/2025, 4:35 PMManish
08/07/2025, 2:10 AMEmerson Lesage
08/07/2025, 12:41 PMstartReplaceSegments to swap s1 and s2 with s3 and s4, and something fails during the ingestion of s3 or s4, do we need to manually call revertReplaceSegments?Emerson Lesage
08/11/2025, 8:11 PMManish
08/24/2025, 11:38 AMstartReplaceSegments to swap s1 and s2 with s3 and s4" - We will do the ingestion of s3, s4 and the metadata push will happen after startReplaceSegments is called. Could you look at the forceCleanUp up param of ``startReplaceSegments` . That'll auto clean up failed lineages incase the segment replacement fails, when startReplaceSegments is called subsequently.
2. For consistent push, we do need unique segment names (unique uuid). Could you look at this SegmentMetadataPushJobRunner (pr reference), that gives an example of how stand alone job uses consistent push feature.