Hello! We've written a custom incremental HTTP source and are using the S3 destination. We are running in incremental append mode. The current behavior is that, when we run a sync, all of the new records from that sync are written into a single new file in S3. Is there some way to instead write each record into its own file in S3? Thanks!

You want to separate the records based in some field information from your source?

Yep - for example, each individual record might have a created_at field which could be used to create a unique filename for it.

No, using the default connector. The ELT paradigm will transfer your data from source to destinatino with minimal transformation. If you want to achieve this probably is better execute a post-trasformation.

Okay, thanks for the info!