https://linen.dev logo
s

s

10/02/2020, 8:59 PM
the terminology we’re using is misaligned between the product/frontend and the backend. What do you think about the following renames on the data model:
Source
-->
SourceConnector
Destination
-->
DestinationConnector
SourceImplementation
-->
Source
DestinationImplementation
-->
Destination
u

user

10/02/2020, 9:03 PM
also
Integration
->
Connector
u

user

10/02/2020, 9:04 PM
time for a glossary
u

user

10/02/2020, 9:09 PM
could we do this instead?
Source
 --> 
SourceDefinition
Destination
 --> 
DestinationDefinition
SourceImplementation
 --> 
Source
DestinationImplementation
 --> 
Destination
u

user

10/02/2020, 9:09 PM
i don’t know what a
SourceConnector
is, and I don’t want to find out 😛. sounds like a whole new thing.
u

user

10/02/2020, 9:09 PM
my goal is to have the UI/product terminology align with our domain language in the backend
u

user

10/02/2020, 9:10 PM
Connector
is a ubiquitous term in the industry
u

user

10/02/2020, 9:10 PM
we’re already calling these things
Connector
in conversations and in the UI
u

user

10/02/2020, 9:11 PM
so if we want to use
SourceDefinition
then I would say we should also use it on the frontend instead of
Connector
u

user

10/02/2020, 9:11 PM
it doesn’t really make sense that the implementation of a
SourceConnector
is a
Source
. It would be a
SourceConnectorImplementation
. which is a bad name because it’s too long as we found out with the first iteration.
u

user

10/02/2020, 9:11 PM
kind of agree with Charles
u

user

10/02/2020, 9:12 PM
Source
and
Destination
are types of
Connector
u

user

10/02/2020, 9:12 PM
so
SourceConnector
is kind of redundant
u

user

10/02/2020, 9:13 PM
something like
SourceTemplate
or similar works too. I don't love
*Definition
u

user

10/02/2020, 9:13 PM
Spec is kind of overloaded and confusing here
u

user

10/02/2020, 9:14 PM
i am ambivalent on this point.
u

user

10/02/2020, 9:14 PM
yeah I don't hate it either
u

user

10/02/2020, 9:20 PM
I dont follow — what is the list of terminology that we would use on the UI ?
u

user

10/02/2020, 9:22 PM
would we use sourceImplementation for what we currently call a “source” in the UI?
u

user

10/02/2020, 9:23 PM
I think this is what is being proposed ^
u

user

10/02/2020, 9:23 PM
with the possibility that the word Definition be replaced by some other word like Template or something.
u

user

10/02/2020, 9:27 PM
what’s wrong with the word connector?
u

user

10/02/2020, 9:27 PM
vs definition?
u

user

10/02/2020, 9:27 PM
because Source already means it's a connector
u

user

10/02/2020, 9:27 PM
those things are not synonyms.
u

user

10/02/2020, 9:28 PM
template / definition is a description of how to build something
u

user

10/02/2020, 9:28 PM
we want to say template/class/definition/etc creates an instance
u

user

10/02/2020, 9:28 PM
a source is just a type of connector, so appending connector adds no greater understanding.
u

user

10/02/2020, 9:28 PM
and is kinda confusing.
u

user

10/02/2020, 9:28 PM
jared and i are so in sync right now. scary stuff.
u

user

10/02/2020, 9:28 PM
😂
u

user

10/02/2020, 9:29 PM
I don't think this conflicts with the UI either
u

user

10/02/2020, 9:29 PM
It treats all sources and destinations as connectors, which is accurate
u

user

10/02/2020, 9:29 PM
and if you're adding a source (and if we're calling instances sources)
u

user

10/02/2020, 9:30 PM
the user is never explicitly introduced to the concept of a definition
u

user

10/02/2020, 9:30 PM
they just see options for sources they can create, and then they end up with a source
u

user

10/02/2020, 9:30 PM
really those options are all definitions but users don't need to know the backend name for that
u

user

10/02/2020, 9:31 PM
Ah
u

user

10/02/2020, 9:32 PM
so we keep the term connector in the UI, and in the domain model language, source and destination are ‘subclasses’ of that
u

user

10/02/2020, 9:32 PM
yep
u

user

10/02/2020, 9:33 PM
that’s not exactly the case though. the
add new connector
button in v0.2 is actually adding a new definition
u

user

10/02/2020, 9:33 PM
no it isn't
u

user

10/02/2020, 9:33 PM
oh
u

user

10/02/2020, 9:33 PM
add new connector not source
u

user

10/02/2020, 9:33 PM
yeah
u

user

10/02/2020, 9:33 PM
hmm
u

user

10/02/2020, 9:35 PM
if we summarize we want to decide if
source
destination
represent the instance and `source{Connector,Definition}`/`destination{Connector,Definition}` represent the class right?
u

user

10/02/2020, 9:35 PM
that's adding a connector definition and the image used for the connectors themselves
u

user

10/02/2020, 9:35 PM
and right now the term
connector
is abiguous right?
u

user

10/02/2020, 9:35 PM
it could mean either a class or an instance whereas
definition
is obvious that it is something used to build something real
u

user

10/02/2020, 9:35 PM
I came into this conversation thinking it’s (
Connector
) a class. Jared and Charles think of it as an instance.
u

user

10/02/2020, 9:36 PM
Well in the UI we have a list of sources, which are all instances. That's how I've been thinking about it.
u

user

10/02/2020, 9:36 PM
where in the UI?
u

user

10/02/2020, 9:37 PM
the top left button, says
Sources
which are all instances
u

user

10/02/2020, 9:37 PM
they are yes
u

user

10/02/2020, 9:37 PM
^
u

user

10/02/2020, 9:37 PM
so whenever we say "add a new source" in the context of a user, we are always referring to an instance
u

user

10/02/2020, 9:38 PM
I think what’s tripping me up is in the documentation, if we use the word
Definition
, we’ll have a section called
Definitions
under which the two sections will be
Sources
and
Destinations
. But
Definition
is not a replacement for
Connector
(fivetran terminology) or
Integration
(stitch terminology) EDIT: actually nvm, both FT and Stitch use
Integration
to mean a “Definition”. In FT a
Connector
is what we call a
Source
.
u

user

10/02/2020, 9:38 PM
xxx{Connector,Definition}
is what will be displayed in Admin & in the dropdown when you create a new source
u

user

10/02/2020, 9:41 PM
is
Connector
ambiguous wether it is a Class or an Instance?
u

user

10/02/2020, 9:42 PM
yes.
u

user

10/02/2020, 9:42 PM
yeah
u

user

10/02/2020, 9:42 PM
so is integration imo
u

user

10/02/2020, 9:42 PM
wait
u

user

10/02/2020, 9:42 PM
lt's go 1 by 1
u

user

10/02/2020, 9:42 PM
@s?
u

user

10/02/2020, 9:42 PM
Connector is ambiguous
u

user

10/02/2020, 9:42 PM
great
u

user

10/02/2020, 9:42 PM
so we don't use connector in our data model
u

user

10/02/2020, 9:43 PM
is
Integration
ambiguous? (yes 👍 / no 👎 )
u

user

10/02/2020, 9:43 PM
(use a reaction)
u

user

10/02/2020, 9:43 PM
i think all of these things are going to be ambigous in a vacuum. you need to present them as pairs.
u

user

10/02/2020, 9:44 PM
SourceTemplate and Source are not as ambiguous in the context of each other. Same for Source and SourceImplementation.
u

user

10/02/2020, 9:44 PM
But obviously source by itself is almost entirely meaningless.
u

user

10/02/2020, 9:44 PM
I was assuming we were already aligned on `source`/`destination` being the instance
u

user

10/02/2020, 9:45 PM
Cool. I’m fine with that.
u

user

10/02/2020, 9:45 PM
yeah, I think Charles was just illustrating his point
u

user

10/02/2020, 9:45 PM
SourceConnector and Source is still ambiguous to me.
u

user

10/02/2020, 9:45 PM
I agree
u

user

10/02/2020, 9:45 PM
we've already selected source/destination as instance as the anchor for the model
u

user

10/02/2020, 9:45 PM
yeah those two aren't clear
u

user

10/02/2020, 9:46 PM
I think we're also missing what we call the actual implementation (plugin? image?) that represents the integration/connector/source
u

user

10/02/2020, 9:47 PM
can we solve one thing at a time?
u

user

10/02/2020, 9:49 PM
👍 kk. i think i understand what you’re after now. i’ll respond with emojis from here on out to proposed options.
u

user

10/02/2020, 9:50 PM
so:
source
destination
: instance model in our database (yes 👍 / no because ambiguous 👎)
u

user

10/02/2020, 9:51 PM
OK, anchored then
u

user

10/02/2020, 9:52 PM
xxxConnector
for the class
u

user

10/02/2020, 9:52 PM
xxxIntegration
 for the class
u

user

10/02/2020, 9:52 PM
xxxTemplate
 for the class
u

user

10/02/2020, 9:53 PM
xxxDefinition
 for the class
u

user

10/02/2020, 9:54 PM
shrif do you have other suggestions?
u

user

10/02/2020, 9:55 PM
what is ambiguous/not properly captured by either
SourceTemplate
or
SourceDefinition
?
u

user

10/02/2020, 9:55 PM
(can't wait to write a documentation page for the terminology 🙂 )
u

user

10/02/2020, 9:57 PM
actually definition is fine
u

user

10/02/2020, 9:57 PM
I'm actually more template than definition
u

user

10/02/2020, 9:57 PM
🤣 🤣 🤣 🤣
u

user

10/02/2020, 9:57 PM
fuck
u

user

10/02/2020, 9:57 PM
hahaha
u

user

10/02/2020, 9:57 PM
I feel like definition overlaps with whatever we call the actual image while template feels like it doesn't at all
u

user

10/02/2020, 9:58 PM
I don't know how to express that concern in order @Michel
u

user

10/02/2020, 9:58 PM
xxxConfiguration
u

user

10/02/2020, 10:00 PM
Well, I think we have 2 good candidates for the Class. Let see which one works best with the actual code implementation
u

user

10/02/2020, 10:00 PM
sorry to flip everything here, but I think
Source
to mean a “class” and
Connector
to mean instance is more intuitive. if someone is asking which “sources” do we support, and we say “postgres and biquery”, that is a natural use of the word. If someone is asking which “definitions” or templates we support, that’s a bit of a stretch of the word’s meaning
u

user

10/02/2020, 10:01 PM
a
Connector
is just the process/pipe/instance connecting airbyte to a source
u

user

10/02/2020, 10:01 PM
same for destination
u

user

10/02/2020, 10:02 PM
So you’re suggesting? class -
Source
instance -
SourceConnector
u

user

10/02/2020, 10:02 PM
yes
u

user

10/02/2020, 10:02 PM
same for destination
u

user

10/02/2020, 10:03 PM
SourceConnector
to me is like saying
AppleFruit
.
u

user

10/02/2020, 10:03 PM
I could be convinced of
Source
as a class if we have a very solid and intuitive way of referring to an instance.
u

user

10/02/2020, 10:03 PM
SourceConnector
isn't.
u

user

10/02/2020, 10:05 PM
Fivetran's terminology has
Connector
as a class while "source" refers to like an actual database providing data
u

user

10/02/2020, 10:05 PM
But that gets really confusing if you're talking about sources and destinations because then you have to scope connector by source or destination.
u

user

10/02/2020, 10:07 PM
I think
SourceConnector
is not like
AppleFruit
because
Connector
specifically means that it is something bridging the source (e.g postgres) and airbyte.
Source
is just the generic “postgres”
u
u

user

10/02/2020, 10:09 PM
I don't see them really split that terminology for users
u

user

10/02/2020, 10:09 PM
to your previous point I think
SourceConnector
is just awful to say. if we go in that direction maybe we should just keep tap/target
u

user

10/02/2020, 10:10 PM
haha what’s awful? just that it feels redundant?
u

user

10/02/2020, 10:10 PM
it's very difficult to use that kind of terminology correctly in a discussion
u

user

10/02/2020, 10:12 PM
Source
 is just the generic “postgres”
I don’t think this is true. This is the configuration required to make the connector / integration / data mover to work. The credentials required to connect to the postgres are a subset of that configuration. But a Source is not exclusively a generic postgres.
u

user

10/02/2020, 10:12 PM
I stand by my
AppleFruit
assertion.
u

user

10/02/2020, 10:13 PM
Source - SourceConfiguration - Tap - ConfiguredTap
u

user

10/02/2020, 10:15 PM
I feel like we need to anchor something and derived from it
u

user

10/02/2020, 10:15 PM
I don't think we will completely remove the ambiguity because we are not the only one thinking of terminology
u

user

10/02/2020, 10:16 PM
but what is important is that for every term that we define we have a clear definition
u

user

10/02/2020, 10:16 PM
so we can ensure we talk about the same thing when talking with someone who doesn't know our terminology
u

user

10/02/2020, 10:17 PM
terrific
u

user

10/02/2020, 10:18 PM
maybe we should anchor on their definition of connector, sources, and destinations
u

user

10/02/2020, 10:18 PM
they must have faced the same discussion we have right now
u

user

10/02/2020, 10:19 PM
we use connection right now
u

user

10/02/2020, 10:20 PM
not completely
u

user

10/02/2020, 10:20 PM
I would honestly prefer pipeline since it is something internal to our app whereas connector is more overloaded toward external services
u

user

10/02/2020, 10:21 PM
pipeline instead of connection?
u

user

10/02/2020, 10:21 PM
we are not solving our current problem right now by trying to define yet another term
u

user

10/02/2020, 10:22 PM
we need to stay focused on: object, class, code_implementation
u

user

10/02/2020, 10:23 PM
for both inbound and outbound
u

user

10/02/2020, 10:23 PM
Can we do the vote on if we agree with fivetran's definitions of source, destination, and connector/connection/pipeline?
u

user

10/02/2020, 10:23 PM
source
u

user

10/02/2020, 10:23 PM
destination
u

user

10/02/2020, 10:23 PM
connector (or connection/pipeline)
u

user

10/02/2020, 10:23 PM
I can't vote sorry. I don't understand their definition of source and destination, and connectors match to our data model
u

user

10/02/2020, 10:23 PM
A source is any database, application, file storage service, event tracking service, or function from which you wish to sync your data.
u

user

10/02/2020, 10:24 PM
it's purely the underlying database
u

user

10/02/2020, 10:24 PM
they make no distinction between
Source
the instance and the class. Like is
Source
a particular schema/username/pw in postgres, or is it the general concept of postgres?
u

user

10/02/2020, 10:25 PM
what if we call it
SourceClass
and
Source
(for the instance). It’s kind of how we already talk about it
u

user

10/02/2020, 10:25 PM
what I understand is that what we want to call an object is a mix between a source and a half a connector (the other half for the destination) for them
u

user

10/02/2020, 10:25 PM
from the definition it sounds like the former
u

user

10/02/2020, 10:29 PM
ok suggestion. we want to focus on the datamodel for now. let's not take any risk: • {Source,Destination}Implementation • {Source,Destination}Template • {Source,Destination}Artifact (image...)
u

user

10/02/2020, 10:30 PM
longer but non-ambiguous
u

user

10/02/2020, 10:30 PM
we can always translate what people talk about to these terms
u

user

10/02/2020, 10:30 PM
Can we tweak to the following to match how we usually talk about it: {Source,Destination}Instance {Source,Destination}Template {Source,Destination}Image
u

user

10/02/2020, 10:30 PM
so the idea is to not have exact alignment between the UI and the code?
u

user

10/02/2020, 10:32 PM
I don't think we can
u

user

10/02/2020, 10:34 PM
we will evolve but at least our datamodel will be clear for contributors
u

user

10/02/2020, 10:48 PM
I think the main value I was seeking in bringing up the OP is in having a Ubiquitous Language that is the same in the backend and the frontend, because it eliminates the confusion of “when you say X, do you mean Y” in discussion, which happened a lot in our discussions yesterday. To this end, I’d take an aligned glossary that I don’t 100% agree with (like
SourceDefinition
or template over
Connector
) rather than two glossaries, one for BE and one for FE. So with that said, if
SourceConnector
etc.. is vetoed, I’m happy to take
Source
and
SourceDefinition
provided that this language is aligned across the frontend and backend domain model
u

user

10/02/2020, 10:50 PM
u

user

10/02/2020, 10:51 PM
I still prefer template but would be happy to accept this
u

user

10/02/2020, 11:17 PM
Time is a flat circle… 😉
u

user

10/05/2020, 1:01 AM
let's do it
u

user

10/05/2020, 5:14 PM
one last suggestion: SourceType instead of SourceDefinition ?
u

user

10/05/2020, 5:14 PM
(same for destination)
u

user

10/05/2020, 5:40 PM
I like SourceType because it’s a more natural use of the word (“we support the following sourcetypes” vs. “we support the following source definitions”) but I suspect it is more likely to be overloaded in the future in case we have different “types” of connectors like a batch or stream (or any way of having different “kinds” of source connectors)
u

user

10/05/2020, 5:41 PM
you don't like having a type of SourceType 😛
u

user

10/05/2020, 5:42 PM
🤯 🤯 🤯 🤯