https://linen.dev logo
v

Vitor Avancini

04/28/2021, 12:53 PM
is there a way to run all integration tests?
u

user

04/28/2021, 12:57 PM
integration tests are relying on remote api or data warehouses so you need credentials
u

user

04/28/2021, 12:59 PM
ah allright
u

user

04/28/2021, 12:59 PM
ive submitted a pr
u

user

04/28/2021, 12:59 PM
didnt check the box of integration tests
u

user

04/28/2021, 1:11 PM
Thanks for your contribution!
u

user

04/28/2021, 1:16 PM
I have a PR in progress that includes a similar patch that what you propose, I’ll link it once I push it too
u

user

04/28/2021, 1:16 PM
Nice
u

user

04/28/2021, 1:17 PM
it’s fixing some other bugs and introduce a more complete integration tests
u

user

04/28/2021, 1:17 PM
Ive added integration tests for the nested array case
u

user

04/28/2021, 1:17 PM
the point of code ive changed I wasnt 100% sure
u

user

04/28/2021, 1:17 PM
Yes i saw!
u

user

04/28/2021, 1:18 PM
it works, but it feels a bit hacky
u

user

04/28/2021, 1:18 PM
im getting a unquoted field and quoting it
u

user

04/28/2021, 1:18 PM
cause I cant add a prefix to the already qouted one
u

user

04/28/2021, 1:18 PM
we can actually rename the nested column to whatever name we want
u

user

04/28/2021, 1:19 PM
for example
_airbyte_nested_data
since it’s only use internally to the model
u

user

04/28/2021, 1:20 PM
ah yeah, that is right
u

user

04/28/2021, 1:20 PM
but I think the main issue is selecting from this field
u

user

04/28/2021, 1:20 PM
in the projection
u

user

04/28/2021, 1:21 PM
where there is json_extract_path(json_field, col_name)
u

user

04/28/2021, 1:21 PM
something like this
u

user

04/28/2021, 1:21 PM
I can be very wrong though
u

user

04/28/2021, 2:21 PM
i am solving this as part of a bigger PR around nesting issues in redshift / postgres here too: https://github.com/airbytehq/airbyte/pull/3110
u

user

04/28/2021, 5:35 PM
nice
u

user

04/28/2021, 5:35 PM
should I close my pr in favor of this one?
u

user

04/28/2021, 5:36 PM
yes
u

user

04/28/2021, 5:40 PM
done
u

user

04/28/2021, 5:40 PM
thanks Chris!
u

user

04/28/2021, 5:40 PM
when it gets merged Ill pull it and test my source
u

user

04/28/2021, 5:42 PM
we’d need to do a release of airbyte though since it’s tied to normalization code
u

user

04/28/2021, 5:43 PM
(not a connector specific release)
u

user

04/28/2021, 5:51 PM
Ill build it locally for testing
u

user

04/28/2021, 5:51 PM
are you planning on merging it soon?
u

user

04/28/2021, 5:52 PM
in that case, can you checkout that branch and build it from there instead?
u

user

04/28/2021, 5:55 PM
yeah sure
u

user

04/28/2021, 5:55 PM
whichever goes on first, I get back to it or you merging it
u

user

04/29/2021, 2:02 PM
I have deployed your fix from the branch 2 hours ago
u

user

04/29/2021, 2:02 PM
just saw it was merged 1 hour ago