With Meltano’s recent announcement of their formalized Singer Spec, the introduction of their Singer SDK, and their recent fundraising, it seems like most of Airbyte’s critiques of Singer either no longer apply or are much less applicable (save for the python-only limitations). I would love to hear from the airbyte team what they think of these recent efforts by Meltano with regards to the future of Singer and airbyte-singer compatability.

We will continue to make it possible use Singer taps (not targets) within Airbyte. But we want to encourage the community to build using Airbyte CDK as it will make the maintenance of these connectors a lot simpler for the community and it will make them compatible with the extensions we will add to the protocol (Streaming, Sharding...).

@Sawyer Waugh can you explain your list of reasons, I'm interested to known because I'm using booth tools.

Is there a reason why Singer taps over targets makes more sense? Doesn’t that mean that only half of the connection use cases are supported?

The problem with singer is that the taps and target are very rarely pairwise compatible. People often think that you can use any tap with any target, but it is not true. The targets have custom behaviors that are triggered by taps (wether you want it or not). This is because Singer is more of a guideline than a spec. When we started Airbyte we thought of it as an actual protocol. It means that now a system can make real assumptions about how sources & destinations are working and that allows us to build powerful features without having to think about all these customs edge cases that were inserted into tap and target.

There are always different assumptions on what a “spec” vs “protocol” vs “framework” is. For instance, HTTP to me is a protocol whereas Apache Spark and Flink is a framework. How does Airbyte define a “spec”? If any spec is in production, wouldn’t that mean it’s an actual framework now?