https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
c

Connor Lough

05/04/2022, 10:55 PM
Hey all, with S3 as a destination, all the S3 Path Format variables reference the time "that the sync was writing output" Is there any way to get the timestamp for the entire sync start? I'd like to make an S3 folder for the whole sync (currently 120 files) and then reference the most recent sync as opposed to the most recent file from the sync
m

Marcos Marx (Airbyte)

05/05/2022, 1:39 AM
Not today, there is an issue requesting to add the sync_id column see here: https://github.com/airbytehq/airbyte/issues/4025 feel free to open an issue in the Github project to implement this in the feature
c

Connor Lough

05/05/2022, 1:51 PM
Hey @Marcos Marx (Airbyte), my issue is a little different than this. The linked request is to add a column to the dataset. I'm looking for a new variable to be available to the S3 Path Format. For example: ${SYNC_START_TIME} ${NAMESPACE}/${STREAM_NAME}/*${SYNC_START_TIME}*
2 Views