Channels
s
s
10/02/2020, 8:59 PM
the terminology we’re using is misaligned between the product/frontend and the backend. What do you think about the following renames on the data model:
SourceSourceConnectorDestinationDestinationConnectorSourceImplementationSourceDestinationImplementationDestinationu
user
10/02/2020, 9:03 PMalso
IntegrationConnectoru
user
10/02/2020, 9:04 PMtime for a glossary
u
user
10/02/2020, 9:09 PMcould we do this instead?
SourceSourceDefinitionDestinationDestinationDefinitionSourceImplementationSourceDestinationImplementationDestinationu
user
10/02/2020, 9:09 PMi don’t know what a
SourceConnectoru
user
10/02/2020, 9:09 PMmy goal is to have the UI/product terminology align with our domain language in the backend
u
user
10/02/2020, 9:10 PMConnectoru
user
10/02/2020, 9:10 PMwe’re already calling these things
Connectoru
user
10/02/2020, 9:11 PMso if we want to use
SourceDefinitionConnectoru
user
10/02/2020, 9:11 PMit doesn’t really make sense that the implementation of a
SourceConnectorSourceSourceConnectorImplementationu
user
10/02/2020, 9:11 PMkind of agree with Charles
u
user
10/02/2020, 9:12 PMSourceDestinationConnectoru
user
10/02/2020, 9:12 PMso
SourceConnectoru
user
10/02/2020, 9:13 PMsomething like
SourceTemplate*Definitionu
user
10/02/2020, 9:13 PMSpec is kind of overloaded and confusing here
u
user
10/02/2020, 9:14 PMi am ambivalent on this point.
u
user
10/02/2020, 9:14 PMyeah I don't hate it either
u
user
10/02/2020, 9:20 PMI dont follow — what is the list of terminology that we would use on the UI ?
u
user
10/02/2020, 9:22 PMwould we use sourceImplementation for what we currently call a “source” in the UI?
u
user
10/02/2020, 9:23 PMI think this is what is being proposed ^
u
user
10/02/2020, 9:23 PMwith the possibility that the word Definition be replaced by some other word like Template or something.
u
user
10/02/2020, 9:27 PMwhat’s wrong with the word connector?
u
user
10/02/2020, 9:27 PMvs definition?
u
user
10/02/2020, 9:27 PMbecause Source already means it's a connector
u
user
10/02/2020, 9:27 PMthose things are not synonyms.
u
user
10/02/2020, 9:28 PMtemplate / definition is a description of how to build something
u
user
10/02/2020, 9:28 PMwe want to say template/class/definition/etc creates an instance
u
user
10/02/2020, 9:28 PMa source is just a type of connector, so appending connector adds no greater understanding.
u
user
10/02/2020, 9:28 PMand is kinda confusing.
u
user
10/02/2020, 9:28 PMjared and i are so in sync right now. scary stuff.
u
user
10/02/2020, 9:28 PM😂
u
user
10/02/2020, 9:29 PMI don't think this conflicts with the UI either
u
user
10/02/2020, 9:29 PMIt treats all sources and destinations as connectors, which is accurate
u
user
10/02/2020, 9:29 PMand if you're adding a source (and if we're calling instances sources)
u
user
10/02/2020, 9:30 PMthe user is never explicitly introduced to the concept of a definition
u
user
10/02/2020, 9:30 PMthey just see options for sources they can create, and then they end up with a source
u
user
10/02/2020, 9:30 PMreally those options are all definitions but users don't need to know the backend name for that
u
user
10/02/2020, 9:31 PMAh
u
user
10/02/2020, 9:31 PMhttps://airbytehq.slack.com/archives/C019WEENQRM/p1601674208067700?thread_ts=1601672342.057100&cid=C019WEENQRM I think this is the piece I was missing
u
user
10/02/2020, 9:32 PMso we keep the term connector in the UI, and in the domain model language, source and destination are ‘subclasses’ of that
u
user
10/02/2020, 9:32 PMyep
u
user
10/02/2020, 9:33 PMthat’s not exactly the case though. the
add new connectoru
user
10/02/2020, 9:33 PMno it isn't
u
user
10/02/2020, 9:33 PMoh
u
user
10/02/2020, 9:33 PMadd new connector not source
u
user
10/02/2020, 9:33 PMyeah
u
user
10/02/2020, 9:33 PMhmm
u
user
10/02/2020, 9:35 PMif we summarize we want to decide if
sourcedestinationu
user
10/02/2020, 9:35 PMthat's adding a connector definition and the image used for the connectors themselves
u
user
10/02/2020, 9:35 PMand right now the term
connectoru
user
10/02/2020, 9:35 PMit could mean either a class or an instance whereas
definitionu
user
10/02/2020, 9:35 PMI came into this conversation thinking it’s (
Connectoru
user
10/02/2020, 9:36 PMWell in the UI we have a list of sources, which are all instances. That's how I've been thinking about it.
u
user
10/02/2020, 9:36 PMwhere in the UI?
u
user
10/02/2020, 9:37 PMthe top left button, says
Sourcesu
user
10/02/2020, 9:37 PMthey are yes
u
user
10/02/2020, 9:37 PM^
u
user
10/02/2020, 9:37 PMso whenever we say "add a new source" in the context of a user, we are always referring to an instance
u
user
10/02/2020, 9:38 PMI think what’s tripping me up is in the documentation, if we use the word
DefinitionDefinitionsSourcesDestinationsDefinitionConnectorIntegrationIntegrationConnectorSourceu
user
10/02/2020, 9:38 PMxxx{Connector,Definition}u
user
10/02/2020, 9:41 PMis
Connectoru
user
10/02/2020, 9:42 PMyes.
u
user
10/02/2020, 9:42 PMyeah
u
user
10/02/2020, 9:42 PMso is integration imo
u
user
10/02/2020, 9:42 PMwait
u
user
10/02/2020, 9:42 PMlt's go 1 by 1
u
user
10/02/2020, 9:42 PM@s?
u
user
10/02/2020, 9:42 PMConnector is ambiguous
u
user
10/02/2020, 9:42 PMgreat
u
user
10/02/2020, 9:42 PMso we don't use connector in our data model
u
user
10/02/2020, 9:43 PMis
Integrationu
user
10/02/2020, 9:43 PM(use a reaction)
u
user
10/02/2020, 9:43 PMi think all of these things are going to be ambigous in a vacuum. you need to present them as pairs.
u
user
10/02/2020, 9:44 PMSourceTemplate and Source are not as ambiguous in the context of each other. Same for Source and SourceImplementation.
u
user
10/02/2020, 9:44 PMBut obviously source by itself is almost entirely meaningless.
u
user
10/02/2020, 9:44 PMI was assuming we were already aligned on `source`/`destination` being the instance
u
user
10/02/2020, 9:45 PMCool. I’m fine with that.
u
user
10/02/2020, 9:45 PMyeah, I think Charles was just illustrating his point
u
user
10/02/2020, 9:45 PMSourceConnector and Source is still ambiguous to me.
u
user
10/02/2020, 9:45 PMI agree
u
user
10/02/2020, 9:45 PMwe've already selected source/destination as instance as the anchor for the model
u
user
10/02/2020, 9:45 PMyeah those two aren't clear
u
user
10/02/2020, 9:46 PMI think we're also missing what we call the actual implementation (plugin? image?) that represents the integration/connector/source
u
user
10/02/2020, 9:47 PMcan we solve one thing at a time?
u
user
10/02/2020, 9:49 PM👍 kk. i think i understand what you’re after now. i’ll respond with emojis from here on out to proposed options.
u
user
10/02/2020, 9:50 PMso:
sourcedestinationu
user
10/02/2020, 9:51 PMOK, anchored then
u
user
10/02/2020, 9:52 PMxxxConnectoru
user
10/02/2020, 9:52 PMxxxIntegrationu
user
10/02/2020, 9:52 PMxxxTemplateu
user
10/02/2020, 9:53 PMxxxDefinitionu
user
10/02/2020, 9:54 PMshrif do you have other suggestions?
u
user
10/02/2020, 9:55 PMwhat is ambiguous/not properly captured by either
SourceTemplateSourceDefinitionu
user
10/02/2020, 9:55 PM(can't wait to write a documentation page for the terminology 🙂 )
u
user
10/02/2020, 9:57 PMactually definition is fine
u
user
10/02/2020, 9:57 PMI'm actually more template than definition
u
user
10/02/2020, 9:57 PM🤣 🤣 🤣 🤣
u
user
10/02/2020, 9:57 PMfuck
u
user
10/02/2020, 9:57 PMhahaha
u
user
10/02/2020, 9:57 PMI feel like definition overlaps with whatever we call the actual image while template feels like it doesn't at all
u
user
10/02/2020, 9:58 PMI don't know how to express that concern in order @Michel
u
user
10/02/2020, 9:58 PMxxxConfigurationu
user
10/02/2020, 10:00 PMWell, I think we have 2 good candidates for the Class. Let see which one works best with the actual code implementation
u
user
10/02/2020, 10:00 PMsorry to flip everything here, but I think
SourceConnectoru
user
10/02/2020, 10:01 PMa
Connectoru
user
10/02/2020, 10:01 PMsame for destination
u
user
10/02/2020, 10:02 PMSo you’re suggesting?
class -
SourceSourceConnectoru
user
10/02/2020, 10:02 PMyes
u
user
10/02/2020, 10:02 PMsame for destination
u
user
10/02/2020, 10:03 PMSourceConnectorAppleFruitu
user
10/02/2020, 10:03 PMI could be convinced of
Sourceu
user
10/02/2020, 10:03 PMSourceConnectoru
user
10/02/2020, 10:05 PMFivetran's terminology has
Connectoru
user
10/02/2020, 10:05 PMBut that gets really confusing if you're talking about sources and destinations because then you have to scope connector by source or destination.
u
user
10/02/2020, 10:07 PMI think
SourceConnectorAppleFruitConnectorSourceu
user
10/02/2020, 10:08 PMhttps://airbytehq.slack.com/archives/C019WEENQRM/p1601676339088800?thread_ts=1601672342.057100&cid=C019WEENQRM I thought connector is what they call an instance?
u
user
10/02/2020, 10:09 PMI don't see them really split that terminology for users
u
user
10/02/2020, 10:09 PMto your previous point I think
SourceConnectoru
user
10/02/2020, 10:10 PMhaha what’s awful? just that it feels redundant?
u
user
10/02/2020, 10:10 PMit's very difficult to use that kind of terminology correctly in a discussion
u
user
10/02/2020, 10:12 PMI don’t think this is true. This is the configuration required to make the connector / integration / data mover to work. The credentials required to connect to the postgres are a subset of that configuration. But a Source is not exclusively a generic postgres.is just the generic “postgres”Source
u
user
10/02/2020, 10:12 PMI stand by my
AppleFruitu
user
10/02/2020, 10:13 PMu
user
10/02/2020, 10:15 PMI feel like we need to anchor something and derived from it
u
user
10/02/2020, 10:15 PMI don't think we will completely remove the ambiguity because we are not the only one thinking of terminology
u
user
10/02/2020, 10:16 PMbut what is important is that for every term that we define we have a clear definition
u
user
10/02/2020, 10:16 PMso we can ensure we talk about the same thing when talking with someone who doesn't know our terminology
u
user
10/02/2020, 10:16 PMu
user
10/02/2020, 10:17 PMterrific
u
user
10/02/2020, 10:18 PMmaybe we should anchor on their definition of connector, sources, and destinations
u
user
10/02/2020, 10:18 PMthey must have faced the same discussion we have right now
u
user
10/02/2020, 10:19 PMwe use connection right now
u
user
10/02/2020, 10:20 PMnot completely
u
user
10/02/2020, 10:20 PMI would honestly prefer pipeline since it is something internal to our app whereas connector is more overloaded toward external services
u
user
10/02/2020, 10:21 PMpipeline instead of connection?
u
user
10/02/2020, 10:21 PMwe are not solving our current problem right now by trying to define yet another term
u
user
10/02/2020, 10:22 PMwe need to stay focused on: object, class, code_implementation
u
user
10/02/2020, 10:23 PMfor both inbound and outbound
u
user
10/02/2020, 10:23 PMCan we do the vote on if we agree with fivetran's definitions of source, destination, and connector/connection/pipeline?
u
user
10/02/2020, 10:23 PMsource
u
user
10/02/2020, 10:23 PMdestination
u
user
10/02/2020, 10:23 PMconnector (or connection/pipeline)
u
user
10/02/2020, 10:23 PMI can't vote sorry. I don't understand their definition of source and destination, and connectors match to our data model
u
user
10/02/2020, 10:23 PMA source is any database, application, file storage service, event tracking service, or function from which you wish to sync your data.
u
user
10/02/2020, 10:24 PMit's purely the underlying database
u
user
10/02/2020, 10:24 PMthey make no distinction between
SourceSourceu
user
10/02/2020, 10:25 PMwhat if we call it
SourceClassSourceu
user
10/02/2020, 10:25 PMwhat I understand is that what we want to call an object is a mix between a source and a half a connector (the other half for the destination) for them
u
user
10/02/2020, 10:25 PMfrom the definition it sounds like the former
u
user
10/02/2020, 10:29 PMok suggestion. we want to focus on the datamodel for now. let's not take any risk:
• {Source,Destination}Implementation
• {Source,Destination}Template
• {Source,Destination}Artifact (image...)
u
user
10/02/2020, 10:30 PMlonger but non-ambiguous
u
user
10/02/2020, 10:30 PMwe can always translate what people talk about to these terms
u
user
10/02/2020, 10:30 PMCan we tweak to the following to match how we usually talk about it:
{Source,Destination}Instance
{Source,Destination}Template
{Source,Destination}Image
u
user
10/02/2020, 10:30 PMso the idea is to not have exact alignment between the UI and the code?
u
user
10/02/2020, 10:32 PMI don't think we can
u
user
10/02/2020, 10:34 PMwe will evolve but at least our datamodel will be clear for contributors
u
user
10/02/2020, 10:48 PMI think the main value I was seeking in bringing up the OP is in having a Ubiquitous Language that is the same in the backend and the frontend, because it eliminates the confusion of “when you say X, do you mean Y” in discussion, which happened a lot in our discussions yesterday. To this end, I’d take an aligned glossary that I don’t 100% agree with (like
SourceDefinitionConnectorSourceConnectorSourceSourceDefinitionu
user
10/02/2020, 10:50 PMSo how do you guys feel about the exact suggestion Charles made here: https://airbytehq.slack.com/archives/C019WEENQRM/p1601672952058200?thread_ts=1601672342.057100&cid=C019WEENQRM
u
user
10/02/2020, 10:51 PMI still prefer template but would be happy to accept this
u
user
10/02/2020, 11:17 PMTime is a flat circle… 😉
u
user
10/05/2020, 1:01 AMlet's do it
u
user
10/05/2020, 5:14 PMone last suggestion: SourceType instead of SourceDefinition ?
u
user
10/05/2020, 5:14 PM(same for destination)
u
user
10/05/2020, 5:40 PMI like SourceType because it’s a more natural use of the word (“we support the following sourcetypes” vs. “we support the following source definitions”) but I suspect it is more likely to be overloaded in the future in case we have different “types” of connectors like a batch or stream (or any way of having different “kinds” of source connectors)
u
user
10/05/2020, 5:41 PMyou don't like having a type of SourceType 😛
u
user
10/05/2020, 5:42 PM🤯 🤯 🤯 🤯