https://linen.dev logo
Join Slack
Powered by
Hi Airbyte Team, I've tried looking through the b...
# feedback-and-requests
b
Hi Airbyte Team, I've tried looking through the backlog of issues and can't quite find what I'm looking for so thought I would try here before creating an additional issue. In the UI, I have done the tedious work on selecting and configuring each table for an MS SQL server source in a connection. With that, if I need to add an additional table I have to perform the entire sequence again of selecting and configuring everything. Is there an issue already out for just saving the configuration settings that are done for any of the sources in Airbyte?
This appears to be an issue to that effect. Is there a timeline on when this might be completed? https://github.com/airbytehq/airbyte/issues/3206 I would contribute to this as this is halting some of our development and adoption of Airbyte internally at my company.
u
@Artem Astapenko this issue will be solve in front-end?
a
@[DEPRECATED] Marcos Marx Actually it is a backend issue - we are just setting what is returned from backend. If we need to keep old settings - then probably 
Update latest source schema
should send correct form. Probably we should add some sort of checkbox that will either reset schema or just refresh the schema but store old values
b
How big is the lift for this? Is this something I as a first-time community contributor might be able to work on?
I'd be curious what others think, but honestly the default should be that the schema settings are saved and the additional step should be for resetting the configuration. It's more difficult to setup the configuration the first time than it is to just reset it
a
First of all, before working on this issue, I believe it is required to decide what is the expected behavior and where we should apply changes ( front or back ). Just to clarify: you changed schema, saved it and then pressed update latest source schema and instead of your values you see new fresh stream settings?
b
Yes, so for example, I want to add an additional table to a connection that exists currently. When I select 
Update latest source schema
this will erase all of the existing settings and repopulate the table selections with the default configuration for each table
Hey @Artem Astapenko, I was looking into this a bit more and it appears the configuration for those that were configured before is working, but the last piece of this is that the checkbox states are erased in the process (i.e. the checkboxes are all selected again during the refresh rather than only keeping selected those that were previously select.)
So in that case, most of the configuration is getting stored in the process and all that appears left to store is the state of the checkbox when the refresh button was selected for each table (those that were there already and potentially just default to unselected for new tables that have appeared in the interim)
The last additional piece (which might be another issue entirely) with this is that the logical flow for adding additional tables forces a full reset of all the tables managed by a connection. Are connection configurations managed at the table level or the connection level in Airbyte? I'm assuming it is managed at the table level. If that is the case, is it possible to allow new tables to be added to a connection, but NOT reset the entire connection?
e
Blake there is an epic to implement the schema evolution (add/remove tables without reseting the data) https://github.com/airbytehq/airbyte/issues/6912. Probably the feature your discussion here also need the dependency from schema evoltuoin task that is the new protocol 🙂
So, it’s not a easy implementation
b
Hmmm...I was hopefully I might be able to take this on myself, but suspected this might be the case in the process. Thanks for the information here Marcos!
Open in Slack
PreviousNext
Powered by