https://linen.dev logo
Channels
Powered by
a

Angie Marable

01/13/2022, 7:45 PM
Hi - what is the status of this issue and a solution (for postgres)? It's a business impacting issue for our company and we'll need to replace airbyte with another solution within the next couple of weeks without a target date for a fix. https://github.com/airbytehq/airbyte/issues/8280
u

[DEPRECATED] Marcos Marx

01/13/2022, 8:49 PM
@Alex Sheheda [GL] can you give your opinion here? 😄
o

Omar Ghalawinji

01/14/2022, 11:15 AM
@[DEPRECATED] Marcos Marx currently we are working on solution for MySQL, so we can work on the solution for Postgres during this sprint, will create a separate ticket to track the activity
@Alex Sheheda [GL] - is that a (2) week sprint? cc: @Jabari Long fyi
@Angie Marable yes, I will let you know as soon as we get any results
s

Sadik Bakiu

01/31/2022, 2:58 PM
good morning @Alex Sheheda [GL]! any updates on this issue?
good morning @Jabari Long we finished with a research and posted updates to the comment https://github.com/airbytehq/airbyte/issues/9337#issuecomment-1023203660 right now we are working on the issue https://github.com/airbytehq/airbyte/issues/7944 extending airbyte JSONSchema to pass additional parameters that will indicate whether timestamp/time timezone aware or not this will allow to create correct data-type on the destination side could you possibly describe what is your business case? cc @Angie Marable
j

Jefferson da Silva Martins

01/31/2022, 3:42 PM
Okay, thanks for the update @Alex Sheheda [GL]. Our business case is that we use a postgres instance as our transaction system and timezones are important. When replicating that transaction data into Snowflake we essentially lose fidelity because of this issue. Running the same query in both environment yields different results when sorting/ranking by timestamp-timezone.
@Jabari Long could you possibly share a few examples of the sorting results from Postgres and Snowflake?
@Alex Sheheda [GL] Here is an example where we are trying to rank 
category_id
by the max 
updated_at
column. The 1st screenshot is the source(postgres) and the 2nd screenshot is the target(snowflake). You'll notice that on the target, the timezone info has been truncated and the same ranking query returns the records in a completely different order.
I see, it’s a different issue https://github.com/airbytehq/airbyte/issues/8904 @Andy Yeo (Airbyte) can we prioritise the fix for Postgres to the current sprint? cc @Sherif Nada
m

Mayank Malu

02/16/2022, 3:45 PM
@Alex Sheheda [GL] any progress to share on this issue?
hi @Jabari Long the fix is under Airbyte review https://github.com/airbytehq/airbyte/pull/10242 cc @Liren Tu (Airbyte) @Edward Gao (Airbyte)
Powered by