Hi everyone,
I'm testing Postgres to Postgres syn...
# announcementsc
Carlos González
01/20/2021, 8:16 PMHi everyone,
I'm testing Postgres to Postgres syncing with Incremental - Append using docker-compose on my computer: https://docs.airbyte.io/deploying-airbyte/on-your-workstation. I'm getting the following error on every sync attempt (full log attached):
Copy code
2021-01-20 19:55:26 ERROR (/tmp/workspace/7/0) LineGobbler(voidCall):69 - Exception in thread "main" java.lang.NullPointerExceptions
s
01/20/2021, 8:31 PM
hey carlos, what did you set as the cursor field for the incremental sync on your tables?
s
01/20/2021, 8:32 PM
and is that field populated in every record in your table?
c
charles
01/20/2021, 8:55 PM
looks like this happens when the selected cursor field isn't present in a record.
charles
01/20/2021, 8:56 PM
i can fix the NPE.
charles
01/20/2021, 8:57 PM
but ultimately this incremental sync is unlikely to work the way you expect it to.
charles
01/20/2021, 8:58 PM
i don't know what field you're using as a cursor but let's assume it's updated_at. so if a record is written and doesn't have an updated_at value the behavior around what records get sync is weird.
charles
01/20/2021, 8:58 PM
lmk if any of this doesn't make sense.
charles
01/20/2021, 8:58 PM
also if you think we're missing a use case here, i'm happy to discuss!
charles
01/20/2021, 9:02 PM
think this should fix it: https://github.com/airbytehq/airbyte/pull/1746
c
Carlos González
01/20/2021, 9:06 PMI’m using a
timestamptzc
charles
01/20/2021, 9:08 PM
can we sanity check that to just remove it as a possibility?
Copy code
SELECT COUNT(*) FROM <table-name> WHERE <column-name> IS NULL;c
Carlos González
01/20/2021, 9:10 PMThat’s exactly the test I ran 😉
c
charles
01/20/2021, 9:13 PM
huh. weird!
charles
01/20/2021, 9:15 PM
i'm not sure then. i'm happy to take a closer look, but probably can't do so until later today or tomorrow morning.
2 Views