https://linen.dev logo
Join Slack
Powered by
the terminology we’re using is misaligned between ...
# contributing-to-airbyte
s
the terminology we’re using is misaligned between the product/frontend and the backend. What do you think about the following renames on the data model: 
Source
--> 
SourceConnector
Destination
--> 
DestinationConnector
SourceImplementation
--> 
Source
DestinationImplementation
--> 
Destination
u
also 
Integration
-> 
Connector
u
time for a glossary
u
could we do this instead? 
Source
 --> 
SourceDefinition
Destination
 --> 
DestinationDefinition
SourceImplementation
 --> 
Source
DestinationImplementation
 --> 
Destination
u
i don’t know what a 
SourceConnector
is, and I don’t want to find out 😛. sounds like a whole new thing.
u
my goal is to have the UI/product terminology align with our domain language in the backend
u
Connector
is a ubiquitous term in the industry
u
we’re already calling these things 
Connector
in conversations and in the UI
u
so if we want to use 
SourceDefinition
then I would say we should also use it on the frontend instead of 
Connector
u
it doesn’t really make sense that the implementation of a 
SourceConnector
is a 
Source
. It would be a 
SourceConnectorImplementation
. which is a bad name because it’s too long as we found out with the first iteration.
u
kind of agree with Charles
u
Source
and 
Destination
are types of 
Connector
u
so 
SourceConnector
is kind of redundant
u
something like 
SourceTemplate
or similar works too. I don't love 
*Definition
u
Spec is kind of overloaded and confusing here
u
i am ambivalent on this point.
u
yeah I don't hate it either
u
I dont follow — what is the list of terminology that we would use on the UI ?
u
would we use sourceImplementation for what we currently call a “source” in the UI?
u
u
I think this is what is being proposed ^
u
with the possibility that the word Definition be replaced by some other word like Template or something.
u
what’s wrong with the word connector?
u
vs definition?
u
because Source already means it's a connector
u
those things are not synonyms.
u
template / definition is a description of how to build something
u
we want to say template/class/definition/etc creates an instance
u
a source is just a type of connector, so appending connector adds no greater understanding.
u
and is kinda confusing.
u
jared and i are so in sync right now. scary stuff.
u
😂
u
I don't think this conflicts with the UI either
u
It treats all sources and destinations as connectors, which is accurate
u
and if you're adding a source (and if we're calling instances sources)
u
the user is never explicitly introduced to the concept of a definition
u
they just see options for sources they can create, and then they end up with a source
u
really those options are all definitions but users don't need to know the backend name for that
u
Ah
u
u
so we keep the term connector in the UI, and in the domain model language, source and destination are ‘subclasses’ of that
u
yep
u
that’s not exactly the case though. the 
add new connector
button in v0.2 is actually adding a new definition
u
no it isn't
u
oh
u
add new connector not source
u
yeah
u
hmm
u
if we summarize we want to decide if 
source
destination
represent the instance and `source{Connector,Definition}`/`destination{Connector,Definition}` represent the class right?
u
that's adding a connector definition and the image used for the connectors themselves
u
and right now the term 
connector
is abiguous right?
u
it could mean either a class or an instance whereas 
definition
is obvious that it is something used to build something real
u
I came into this conversation thinking it’s (
Connector
) a class. Jared and Charles think of it as an instance.
u
Well in the UI we have a list of sources, which are all instances. That's how I've been thinking about it.
u
where in the UI?
u
the top left button, says 
Sources
which are all instances
u
they are yes
u
^
u
so whenever we say "add a new source" in the context of a user, we are always referring to an instance
u
I think what’s tripping me up is in the documentation, if we use the word 
Definition
, we’ll have a section called 
Definitions
under which the two sections will be 
Sources
and 
Destinations
. But 
Definition
is not a replacement for 
Connector
(fivetran terminology) or 
Integration
(stitch terminology) EDIT: actually nvm, both FT and Stitch use 
Integration
to mean a “Definition”. In FT a 
Connector
is what we call a 
Source
.
u
xxx{Connector,Definition}
is what will be displayed in Admin & in the dropdown when you create a new source
u
is 
Connector
ambiguous wether it is a Class or an Instance?
u
yes.
u
yeah
u
so is integration imo
u
wait
u
lt's go 1 by 1
u
@s?
u
Connector is ambiguous
u
great
u
so we don't use connector in our data model
u
is 
Integration
ambiguous? (yes 👍 / no 👎 )
u
(use a reaction)
u
i think all of these things are going to be ambigous in a vacuum. you need to present them as pairs.
u
SourceTemplate and Source are not as ambiguous in the context of each other. Same for Source and SourceImplementation.
u
But obviously source by itself is almost entirely meaningless.
u
I was assuming we were already aligned on `source`/`destination` being the instance
u
Cool. I’m fine with that.
u
yeah, I think Charles was just illustrating his point
u
SourceConnector and Source is still ambiguous to me.
u
I agree
u
we've already selected source/destination as instance as the anchor for the model
u
yeah those two aren't clear
u
I think we're also missing what we call the actual implementation (plugin? image?) that represents the integration/connector/source
u
can we solve one thing at a time?
u
👍 kk. i think i understand what you’re after now. i’ll respond with emojis from here on out to proposed options.
u
so: 
source
destination
: instance model in our database (yes 👍 / no because ambiguous 👎)
u
OK, anchored then
u
xxxConnector
for the class
u
xxxIntegration
 for the class
u
xxxTemplate
 for the class
u
xxxDefinition
 for the class
u
shrif do you have other suggestions?
u
what is ambiguous/not properly captured by either 
SourceTemplate
or 
SourceDefinition
?
u
(can't wait to write a documentation page for the terminology 🙂 )
u
actually definition is fine
u
I'm actually more template than definition
u
🤣 🤣 🤣 🤣
u
fuck
u
hahaha
u
I feel like definition overlaps with whatever we call the actual image while template feels like it doesn't at all
u
I don't know how to express that concern in order @Michel
u
xxxConfiguration
u
Well, I think we have 2 good candidates for the Class. Let see which one works best with the actual code implementation
u
sorry to flip everything here, but I think 
Source
to mean a “class” and 
Connector
to mean instance is more intuitive. if someone is asking which “sources” do we support, and we say “postgres and biquery”, that is a natural use of the word. If someone is asking which “definitions” or templates we support, that’s a bit of a stretch of the word’s meaning
u
a 
Connector
is just the process/pipe/instance connecting airbyte to a source
u
same for destination
u
So you’re suggesting? class - 
Source
instance - 
SourceConnector
u
yes
u
same for destination
u
SourceConnector
to me is like saying 
AppleFruit
.
u
I could be convinced of 
Source
as a class if we have a very solid and intuitive way of referring to an instance.
u
SourceConnector
isn't.
u
Fivetran's terminology has 
Connector
as a class while "source" refers to like an actual database providing data
u
But that gets really confusing if you're talking about sources and destinations because then you have to scope connector by source or destination.
u
I think 
SourceConnector
is not like 
AppleFruit
because 
Connector
specifically means that it is something bridging the source (e.g postgres) and airbyte. 
Source
is just the generic “postgres”
u
u
I don't see them really split that terminology for users
u
to your previous point I think 
SourceConnector
is just awful to say. if we go in that direction maybe we should just keep tap/target
u
haha what’s awful? just that it feels redundant?
u
it's very difficult to use that kind of terminology correctly in a discussion
u
Source
 is just the generic “postgres”
I don’t think this is true. This is the configuration required to make the connector / integration / data mover to work. The credentials required to connect to the postgres are a subset of that configuration. But a Source is not exclusively a generic postgres.
u
I stand by my 
AppleFruit
assertion.
u
Source - SourceConfiguration - Tap - ConfiguredTap
u
I feel like we need to anchor something and derived from it
u
I don't think we will completely remove the ambiguity because we are not the only one thinking of terminology
u
but what is important is that for every term that we define we have a clear definition
u
so we can ensure we talk about the same thing when talking with someone who doesn't know our terminology
u
u
terrific
u
maybe we should anchor on their definition of connector, sources, and destinations
u
they must have faced the same discussion we have right now
u
we use connection right now
u
not completely
u
I would honestly prefer pipeline since it is something internal to our app whereas connector is more overloaded toward external services
u
pipeline instead of connection?
u
we are not solving our current problem right now by trying to define yet another term
u
we need to stay focused on: object, class, code_implementation
u
for both inbound and outbound
u
Can we do the vote on if we agree with fivetran's definitions of source, destination, and connector/connection/pipeline?
u
source
u
destination
u
connector (or connection/pipeline)
u
I can't vote sorry. I don't understand their definition of source and destination, and connectors match to our data model
u
A source is any database, application, file storage service, event tracking service, or function from which you wish to sync your data.
u
it's purely the underlying database
u
they make no distinction between 
Source
the instance and the class. Like is 
Source
a particular schema/username/pw in postgres, or is it the general concept of postgres?
u
what if we call it 
SourceClass
and 
Source
(for the instance). It’s kind of how we already talk about it
u
what I understand is that what we want to call an object is a mix between a source and a half a connector (the other half for the destination) for them
u
from the definition it sounds like the former
u
ok suggestion. we want to focus on the datamodel for now. let's not take any risk: • {Source,Destination}Implementation • {Source,Destination}Template • {Source,Destination}Artifact (image...)
u
longer but non-ambiguous
u
we can always translate what people talk about to these terms
u
Can we tweak to the following to match how we usually talk about it: {Source,Destination}Instance {Source,Destination}Template {Source,Destination}Image
u
so the idea is to not have exact alignment between the UI and the code?
u
I don't think we can
u
we will evolve but at least our datamodel will be clear for contributors
u
I think the main value I was seeking in bringing up the OP is in having a Ubiquitous Language that is the same in the backend and the frontend, because it eliminates the confusion of “when you say X, do you mean Y” in discussion, which happened a lot in our discussions yesterday. To this end, I’d take an aligned glossary that I don’t 100% agree with (like 
SourceDefinition
or template over 
Connector
) rather than two glossaries, one for BE and one for FE. So with that said, if 
SourceConnector
etc.. is vetoed, I’m happy to take 
Source
and 
SourceDefinition
provided that this language is aligned across the frontend and backend domain model
u
u
I still prefer template but would be happy to accept this
u
Time is a flat circle… 😉
u
let's do it
u
one last suggestion: SourceType instead of SourceDefinition ?
u
(same for destination)
u
I like SourceType because it’s a more natural use of the word (“we support the following sourcetypes” vs. “we support the following source definitions”) but I suspect it is more likely to be overloaded in the future in case we have different “types” of connectors like a batch or stream (or any way of having different “kinds” of source connectors)
u
you don't like having a type of SourceType 😛
u
🤯 🤯 🤯 🤯
Open in Slack
PreviousNext
Powered by