I'm trying to replicate connections in the config ...
# ask-community-for-troubleshooting
I'm trying to replicate connections in the config files and then upload it.. but I am having troubles getting it to recognize the added connections. I have ensured they all have unique identifiers. Is there something else I am missing?
Hi @Justin Cole, as I suggested in our previous discussion this quite an hacky approach 😄 Would you be ready to share the config file (with obfuscated credentials) + server logs after an import attempt (you can download it from the settings page in the UI)?
Everything in the entire config works except the default2 connection
I do understand it is not the ideal way to accomplish the task. The problem I am encountering is that we want 5 tables from a large db. When I try to set up the connection through the web interface it times out and never completes... the tables are the same as those in a smaller db so I am trying to clone the connection for just those 5 tables. If there is another way around the time-out I would be happy to explore it 🙂
Anything obvious in those files?
Hey this looks like inside the dump there is no VERSION file
can you confirm this ?
Sorry. I just now saw this.
Version is 0.34.2-alpha
Hi @Justin Cole, what's the parameter you want to change for each cloned connection? Looks
targets a different sourceID than
The source is the thing that we want to change. I did, eventually, get it to work but it seems like there should be a much easier way to do it
So adding the
file in the archive solved your original problem?
We have, essentially, a bunch of databases that are identical in structure; we want to extract 5 tables from each of them and then, in another step afterward, combine the individual tables so that we have 5 end result tables that span all the data.... so - db1_table1 db2_table1 dbx_table1 all get smashed into full_table1 so we need to clone the sources and connections to expand it outward each time a new db is added.
All right, I get this, so a button in the UI to clone connection would be a solution for you?
Very much so, yes
I think this is covered by this open issue: https://github.com/airbytehq/airbyte/issues/9296 , feel free to subscribe to it to receive updates on this topic
We could then edit it to adjust the source (or any other one factor) and it would make things so much faster and prevent the time out (for example.... one of the dbs is MUCH larger than one of the others... so I had to make the connection in the smaller db and then make it again and manually alter the sourceID because trying to build it in the larger db timed out while trying to parse the db structure... at least that's what it looks like is happening)
Cool. That does look like it is the same request I have. Thanks!
👍 1