I am currently exploring data engineering tools. it would be really great if you could resolve the query (use case given below) • Do airbyte supports data ingestion part of ETL/ELT? if yes how? 1. let's say we have mysql prod db and i want to run bunch of airbyte piplines (using spark for larger data processing) and dump data in partition parquet format in one of the s3 bucket. can i do this using airbyte..? 2. cause usually in past experience what i did basically enable bin logs and reading cdc sequence we were reading all new rows or updates rows using nifi and spark on emr applications were reading this data. to dump in s3. all new data/ updated data can airbyte read this and dump in s3 using running some spark applications? Pls let me know, Thanks

loading...

1. no, you can create a mysql connector to s3 bucket. But it’s possbile to connect to a spark cluster to execute it