Hi <@U04S8PLJWMT>, is it possible to setup several...
# replication-ask-ai
c
Hi @kapa.ai, is it possible to setup several connections targetting the same destination schema in Snowflake, but different streams and end-tables ?
k
Yes, this is possible and is actually a common pattern in Airbyte. Here's what you need to know: Multiple Connections to the Same Snowflake Destination Schema You can set up several connections targeting the same Snowflake destination schema, as long as each connection writes to unique stream/table names. Airbyte now requires stream identifiers to be unique across connections sharing the same destination. [Stream uniqueness] How to ensure unique table names There are two main approaches: 1. Use a unique Stream Prefix per connection — In each connection's schema settings, set a different Stream prefix. This prepends text to every stream name, resulting in different destination table names (e.g.,
source1_orders
,
source2_orders
). [Configuring schemas] 2. Use a unique Namespace per connection — Configure each connection to write to a different schema (namespace) within Snowflake. [Namespaces] Important constraint You cannot configure two connections to sync to the exact same destination table. Doing so risks data loss and job failures, and is no longer permitted in Airbyte. [Airbyte 1.6 release] If you want data in a single table If your goal is to eventually merge data from multiple connections into one table, the recommended approach is to let Airbyte write to separate tables (using prefixes or namespaces) and then combine them downstream using SQL views or dbt
UNION ALL
models. [Configuring schemas] In summary: multiple connections → same Snowflake destination schema → different streams/tables = ✅ fully supported, as long as table names don't collide. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
c
@kapa.ai OK assuming I had one connector that populates table in one schema. Now I want to create 2 separate connectors containing streams with different frequencies. First connector stream A, second connector stream B. What should I do to prevent issues ?
k
Great scenario! Since your two new connectors have different streams (Stream A and Stream B), you're already in a good position — there's no table name collision by default. However, here are the key steps to ensure everything works cleanly: Steps to Prevent Issues 1. Ensure stream names are unique across connections Since Stream A and Stream B are different streams going to the same destination schema, they will naturally write to different tables. This is perfectly fine. [Stream uniqueness] 2. Use a unique Namespace or Stream Prefix (as a safeguard) Even if your stream names are different today, it's good practice to either: • Set a unique stream prefix per connection (e.g.,
conn1_
and
conn2_
), or • Use a unique namespace per connection This protects you if stream names ever overlap in the future. [Configuring schemas] 3. Do not sync the same stream from both connections Make sure Stream A is only enabled in Connection 1 and Stream B is only enabled in Connection 2. If the same stream were enabled in both connections writing to the same schema, Airbyte would reject this configuration. [Airbyte 1.6 release] 4. Disable any overlapping streams from the old connector If your original connector was already syncing Stream A or Stream B, make sure to disable those streams in the old connection before enabling them in the new ones. Running the same stream from two connections to the same destination is not permitted. [Configuring schemas] In summary, your setup should work without issues as long as each stream is only synced by one connection at a time to the same destination schema. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.