I upgraded from version `0.39.39` to `0.40.11` an...
# ask-community-for-troubleshooting
c
I upgraded from version
0.39.39
to
0.40.11
and the GA connector is starting to fail with this message:
Copy code
Additional Failure Information: message='Job ran during migration from Legacy State to Per Stream State. One of the streams that did not have state is: io.airbyte.protocol.models.StreamDescriptor@3dc7762a[name=primary,namespace=<null>,additionalProperties={}]. Job must be retried in order to properly store state.', type='java.lang.IllegalStateException', nonRetryable=false
I tried resetting the connector but that did not make a difference. The sync mode being used is
Incremental Deduped + History
. EDIT: I was able to resolve this issue by manually deleting the state record from the app db.
✍️ 1
u
@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2616 to ensure timely resolution!
m
Please share the complete log file to debug the error.
f
Hello, i got the exact same issue
I can see there's a NEED_STATE_VALIDATION env. variable that can be set to False. What would be the implication to do so ?
udpate : I just updated to version 1.40.14, but nothing changed
j
I have the same issue with the Stripe Source when I upgraded to 0.1.39.
m
@Jonas Meier did the sync failed or just output this warning?
š
We get the same issue (airbyte upgrade from 0.36. to 0.40.14) on connection from postgres to bigquery. @Marcos Marx (Airbyte) the sync stucks at the warning and eventually fails.
j
@Marcos Marx (Airbyte) It fails. I have downgraded now to 0.1.32 which we used previously and there everything works again.
f
@Marcos Marx (Airbyte): I got the same issue so I can answer for @Jonas Meier: the sync eventually fails, and no data is ingested
m
The log file shared doesn’t have any error message, so I can’t help debugging it. @Šimon Appelt can you provide one log with the error message?
f
@Marcos Marx (Airbyte) please allow me to share one of mine
Hi there, did you find smthg on this topic ? @Marcos Marx (Airbyte) ? Maybe can I help in debugging in some way ? Feel free to ask ! This is a serious issue on my side as data are refreshed anymore since monday
s
I am having a similar issue. I recently updated to 0.40.16 and some of my Facebook connectors now fail with
Job ran during migration from Legacy State to Per Stream State. One of the streams that did not have state is
I tried deleting the table from my destination (BigQuery) and even reset my data, but nothing helped. What exactly is meant by
Copy code
"EDIT: I was able to resolve this issue by manually deleting the state record from the app"
m
@Sebastian Brickel can you share the complete log?
s
So in this case it was the
ad_sets
stream that causes the error, when removing it from syncing it then is
ads
and afterwards a few other streams that have the wrong state. I found out that changing the sync mode from
Incremental | Deduped + history
to
Full refresh | Append
seems to work, but I don’t no why
m
Sorry the missing reply here Sebastian, are you still having issues?
s
Hard to say, I got into contact with Christian Lam, who explained to me that deleting the state record from the Airbyte application database worked for him. So I checked for the connector states in my VM and delete the ones that caused issues. I then reverted to sync mode
Incremental | Deduped + history
but unfortunately now run into facebooks limit for call to ad-accounts. The strange thing is that his only happens to the three facebook ad acount where I had this issue, all other facebook ad accounts that I sync have no problem with these limits. It is still a bit strange! Did you find something in my logs?
m
Besides the Facebook side error:
Copy code
RuntimeError: Batch request failed with response: {'error': {'message': '(#80004) There have been too many calls to this ad-account. Wait a bit and try again. For more info, please refer to <https://developers.facebook.com/docs/graph-api/overview/rate-limiting.>', 'type': 'OAuthException', 'code': 80004, 'error_subcode': 2446079, 'fbtrace_id': 'ANX17jXpw22W_9wKVVIIco_'}}
There isn’t anything meaningful. Are you using same crdentials to run 3 accounts or they use different credentials?
s
Actually I am using the same facebook app to fetch data from around a dozen accounts
The strange thing with the
There have been too many calls to this ad-account
message is that it also appeared on Monday after I turned of the connector for the weekend
m
@Sebastian Brickel the issue continues?
s
Unfortunately yes, I tried to stack the different Facebook connectors in order to avoid Facebooks rate limit but that did not help either
u
Unfortunately I don't have any better inputs to help you. You can create a Github issue to raise the discussion to other engineers too
s
ok, I will do that. Thank you @Marcos Marx (Airbyte)
The solution was to ask for advanced access to Ads Management Standard Access, as described in the Setup Guide. However I am really certain that this was not in there when I setup these connectors in August