Alessandro Pietrobon

02/07/2022, 10:21 AM
hi team, first of all, we love our airbyte tests so far. Took a bit to setup but we're good now. Quick question: we are looking for a way to scrape through a directory (on s3 or google drive) which contains a series of csv files. Instead of connecting to each one, we'd want to scrape the directory and combine all the results on the same table (each file has consistent headers, they are versions of the same report). Is this something that is you guys think is achievable with a custom connector or is it outside the scope of airbyte entirely? Thanks
Hey hi. Good to hear from you. You can look at this

Naveen Sai Patnana

02/07/2022, 1:54 PM
ah, fantastic. thanks