Hello, a bit of a newb question... but for S3 destinations, what's the best practice for mapping folders? So far I've had to create a new destination for each folder I am directing my data to. This is inefficient for obvious reasons (especially when we go to rotate our AWS keys 😅). I haven't been able to find any solution here or online.

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2570 to ensure timely resolution!

You unfortunately need to have a destination for each table (stream). Basically each table will need to be in its own folder, inside that you can create subfolders, for example with each folder containing the data that from each day, and airbyte will read from that no problem.

You can use the

custom_format

to re-use the destination having different sub-folders from the root path.