https://linen.dev logo
t

tharaka prabath

10/06/2021, 5:39 AM
Hi team, I need to have multiple connections with the same source. ex: multiple accounts in Shopify but one database, when using like this there is no way to identify which source data in the database, I'm trying to add an additional parameter with MySQL insert query. I but haven't found a way to get source details in the destination application. any helps ??
u

user

10/06/2021, 5:50 AM
Hi, couple of solutions here: 1) use same database but different destination variants of destination connectors pointing to the different schemas (2 source, 2 destination connectors looking at same database, pointing to different db schemas) 2) use the prefix for your table in the connections between Source and Destination (2 sources, 1 destination connector)
u

user

10/06/2021, 6:27 AM
1 cannot go with this solution
u

user

10/06/2021, 6:29 AM
do you know how this application works ?? how can i pass source details to connector ???
u

user

10/06/2021, 7:11 AM
Why you cannot go with 1?
t

tharaka prabath

10/06/2021, 7:26 AM
my application is a customer portal. every customer has to configure their Shopify app. when customers add their credentials, automatically connect those apps to our airbyte. I develop like this. in this situation we have created multiple databases for each customer. that's not possible
u

user

10/06/2021, 8:58 AM
for example, I have one application that uses only one database schema. I want to insert data from various Shopify clients into that database. also, I want to keep some references to where that data came from. for that, I want to get the source name in the destination with other records
u

user

10/06/2021, 9:00 AM
@Davin Chia (Airbyte) do we have the opportunity to satisfy this requirements?
u

user

10/06/2021, 9:02 AM
Currently we have the possibility to pass the additional “prefix” for the table per each connection, have you try to leverage this?
u

user

10/06/2021, 9:04 AM
thank you. let me try this and let you know
u

user

10/06/2021, 9:14 AM
yeap I’d use prefixes
u

user

10/06/2021, 9:29 AM
Yes, if you combine: • using prefixes (or custom namespace if you prefer) to dispatch your source data per account in the destination (shopify_account1, shopify_account2, etc) • using a custom SQL operations at the end of the sync to union all dispatched tables back into a single shopify table but with an added column denoting which account it is coming from https://docs.airbyte.io/understanding-airbyte/connections#destination-stream-name
t

tharaka prabath

10/06/2021, 9:29 AM
Thanks guys)
5 Views