I have a few transformation tasks that I’m hoping ...
# feedback-and-requests
I have a few transformation tasks that I’m hoping Airbyte can handle and I’m looking for advice on how to approach things. Our current stack is mysql, dbt, and snowflake. (I’m trying to extract wordpress multisite data into a sane format.) 1. I have 1000s of tables with the same schema named like this wp_10_reg, wp_11_reg, wp_12_reg, that I want to combine into one table ‘reg’. 2. I have some data stored as PHP serialized format, I can use a python script to unserialize these. Would I leverage dbt for this? how? 3. I want to join table data together - I see Airbyte has basic transform tools, but then leans on dbt for the harder stuff. How would I accomplish all this with Airbyte and dbt? I’m guessing I’ll need to hire a consultant to help do this or give advice on how to do this. Any suggestions on how to find one?
When joining tables together, do you need to track extra metadata per row like where it came from for example? (Wp_10_reg vs wp_11_reg, etc)
we need there to be a site_id in every row which in this case would be 10 and 11. it would be great if that could be done in dbt or airbyte using just the table names.