Channels
v
Vitor Avancini
04/28/2021, 12:53 PMis there a way to run all integration tests?
u
user
04/28/2021, 12:57 PMintegration tests are relying on remote api or data warehouses so you need credentials
u
user
04/28/2021, 12:59 PMah allright
u
user
04/28/2021, 12:59 PMive submitted a pr
u
user
04/28/2021, 12:59 PMdidnt check the box of integration tests
u
user
04/28/2021, 1:11 PMThanks for your contribution!
u
user
04/28/2021, 1:16 PMI have a PR in progress that includes a similar patch that what you propose, I’ll link it once I push it too
u
user
04/28/2021, 1:16 PMNice
u
user
04/28/2021, 1:17 PMit’s fixing some other bugs and introduce a more complete integration tests
u
user
04/28/2021, 1:17 PMIve added integration tests for the nested array case
u
user
04/28/2021, 1:17 PMthe point of code ive changed I wasnt 100% sure
u
user
04/28/2021, 1:17 PMYes i saw!
u
user
04/28/2021, 1:18 PMit works, but it feels a bit hacky
u
user
04/28/2021, 1:18 PMim getting a unquoted field and quoting it
u
user
04/28/2021, 1:18 PMcause I cant add a prefix to the already qouted one
u
user
04/28/2021, 1:18 PMwe can actually rename the nested column to whatever name we want
u
user
04/28/2021, 1:19 PMfor example
_airbyte_nested_datau
user
04/28/2021, 1:20 PMah yeah, that is right
u
user
04/28/2021, 1:20 PMbut I think the main issue is selecting from this field
u
user
04/28/2021, 1:20 PMin the projection
u
user
04/28/2021, 1:21 PMwhere there is json_extract_path(json_field, col_name)
u
user
04/28/2021, 1:21 PMsomething like this
u
user
04/28/2021, 1:21 PMI can be very wrong though
u
user
04/28/2021, 2:21 PMi am solving this as part of a bigger PR around nesting issues in redshift / postgres here too: https://github.com/airbytehq/airbyte/pull/3110
u
user
04/28/2021, 5:35 PMnice
u
user
04/28/2021, 5:35 PMshould I close my pr in favor of this one?
u
user
04/28/2021, 5:36 PMyes
u
user
04/28/2021, 5:40 PMdone
u
user
04/28/2021, 5:40 PMthanks Chris!
u
user
04/28/2021, 5:40 PMwhen it gets merged Ill pull it and test my source
u
user
04/28/2021, 5:42 PMwe’d need to do a release of airbyte though since it’s tied to normalization code
u
user
04/28/2021, 5:43 PM(not a connector specific release)
u
user
04/28/2021, 5:51 PMIll build it locally for testing
u
user
04/28/2021, 5:51 PMare you planning on merging it soon?
u
user
04/28/2021, 5:52 PMin that case, can you checkout that branch and build it from there instead?
u
user
04/28/2021, 5:55 PMyeah sure
u
user
04/28/2021, 5:55 PMwhichever goes on first, I get back to it or you merging it
u
user
04/29/2021, 2:02 PMI have deployed your fix from the branch 2 hours ago
u
user
04/29/2021, 2:02 PMjust saw it was merged 1 hour ago