Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi All,
I'm trying to find a way to load data from our microsoft dynamics data lake into snowflake.
The data is stored in csv files without a header and an accompanying manifest json file that holds the information about the table.
In spark I can use `spark.read.format("com.microsoft.cdm")` to read the format, in python I had to make my own parser to read the manifest and create a table based on this.
Would it be possible to use airbyte to read the manifest to create a table, and check for schema drift, and to read the csv files when updated?

Hmm, I don't know enough to give you a confident answer yet. However, you can refer to the Airbyte documentation for more information on how to work with different sources and destinations: <https://docs.airbyte.com/>