https://linen.dev logo
#troubleshooting
Title
# troubleshooting
f

Filipe Araújo

03/14/2022, 5:43 PM
Hi everyone! Airbyte Version: 0.35.53-alpha Source name/version: Hubspot 0.1.45 Destination name/version: Postgres 0.3.15 Step: Running Full Refresh Hubspot Description: By accessing the Hubspot dashboard I can see that I have around 362k engagement (activities), I run my connection on Saturday afternoon and till now its still importing with 32 Million (and counting) rows for the engagements. Can you help me try to understand what is happening? Since version 0.1.43 I can’t seem to get this running probably, with that version I got the 362k engagement. Thanks!
hi @[DEPRECATED] Marcos Marx, unfortunately we still getting a weird behavior, since yesterday we have more then 5M rows
the thing is, by looking into the data I’m getting duplicates
check the id inside
_airbyte_data
the
_airbyte_emitted_at
its close to each other
@[DEPRECATED] Marcos Marx can you help me with this ? how can I debug this issue ? thanks
@[DEPRECATED] Marcos Marx one of the reasons I updated to the new version was the new deals structure which I need to import
concerning the data, I’m assuming the id that is inside the json element
"id": 8391227241
its unique, I will check the content
its unique, the only different fields are the airbyte ones
@[DEPRECATED] Marcos Marx ˆ
m

Marcos Marx (Airbyte)

03/15/2022, 11:28 PM
The content is the same or is unique inside the json object, is not clear.
f

Filipe Araújo

03/16/2022, 9:14 AM
@[DEPRECATED] Marcos Marx I think it's unique, later I will compare one by one, however should I spend time doing that instead of trying to debug ?
m

Marcos Marx (Airbyte)

03/17/2022, 1:24 AM
@Filipe Araújo the best would return to a version that was working to you and open a Github issue to team take a look.
f

Filipe Araújo

03/17/2022, 7:23 PM
hi @[DEPRECATED] Marcos Marx thanks for the tip, fyi, I notice there are a lot of
Copy code
2022-03-16 23:56:55 [44msource[0m > Backing off get(...) for 0.0s (source_hubspot.errors.HubspotRateLimited: 429 Rate Limit Exceeded: API rate-limit has been reached until None seconds. See <https://developers.hubspot.com/docs/api/usage-details>)`
or
Copy code
2022-03-17 02:16:06 [44msource[0m > Backing off _send(...) for 5.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException)
2022-03-17 02:16:11 [44msource[0m > Caught retryable error '' after 1 tries. Waiting 5 seconds then retrying...
I don’t know if this can request again and replicate the data
I will a different approach like, export everything except the
engagements
and see how it goes
it seems its really an issue
I think there are already a fix to it, https://github.com/airbytehq/airbyte/pull/11266
hope it can be deployed as quick as possible, thanks!
m

Marcos Marx (Airbyte)

03/23/2022, 2:18 AM
It was merged, can you update to latest version?
f

Filipe Araújo

03/23/2022, 4:03 PM
@[DEPRECATED] Marcos Marx yes, already tested and it worked! thanks!
4 Views