Hi Team - We have been struggling quite a while wi...
# replication-troubleshooting
h
Hi Team - We have been struggling quite a while with stabilizing our Airbyte env. We are trying to sync a massive postgres to Snowflake. The Postgres is huge in terms of number of tables. The structure is tenant based which means tables are replicated for each tenant. We are using Octavia to setup sources and connections. We have split the connections up into tenant+subset of tables meaning we have 200+ connections with 10-20 tables in each connection. This creates a very unresponsive interface no matter the size of EC2 instance we are running on. Looking at the EC2 usage we are quite low on CPU and memory consumption however the interface is still unresponsive and we keep getting "Oops! Something went wrong…" messages. This means we actual don't know if airbyte is working or not. Looking at the target tables we don't see any updates. Currently moving to kubernetes based setups are not an option. Do you have any suggestions on addressing this issue?
🔼 1
u
Hello Heine Bodekaer, it's been a while without an update from us. Are you still having problems or did you find a solution?
h
We didn't find a good way to resolve this on a single instance EC2.
u
Sorry to hear you couldn't resolve this Heine, we've had other users in the past report similar issues with massive tables with a tenant based structure and it was difficult to configure and generally a mess to debug, which seems to be what your team was experiencing. We hope to improve the Postgres connector both in performance and in capability as best as we can but unfortunately, I don't have an update for you at the moment.