Hello everyone! I want to build custom connectors but don't know how to start with it. Any reference for the same?

Hi Jay! Are you looking to build Sources or Destinations?

We offer template generation for sources. Here is some general documentation to start with

Feel free to reach out if you have any questions!

Hello @Davin Chia (Airbyte), Actually i need both source as well as destination for mariaDB.

I see. We don't have great documentation for Java connectors, but there are good existing examples on how to implement one

Here is the Postgres Destination and a Redshift Source as examples. You'll be able to take advantage of the

AbstractJdbcDestination

and

AbstractJdbcSource

to easily implement the MariaDB equivalent

I'm also happy to set up a call and walk you through what needs to be done 🙂

Hi @Davin Chia (Airbyte), sorry for delay

do i need to use CDK to implement this connector?

Nope. Since these are Java

But before you try to build a source, I suspect our MySQL source and destinations might be MariaDb compatible. Do you want to give those a shot first?

i already did that.

is there any chance if we can store exact formatted table in destination?

can we store table in destination with same structure as source table?

The normalization should produce a table that’s very similar

Did you turn it on?

no

any link?

it should be an option when setting up the connection

ah! it's because we don't offer normalisation on MySql yet

this is being worked on this sprint

okay thanks.

one last question, why airByte stores whole row as json format in destination table?

good question

this helps us decouple deliverying the data from normalising it

some users don't like or normalisation rules and prefer to use DBT and transform the data in different ways

okay, got it.