I upgraded from version

0.39.39

to

0.40.11

and the GA connector is starting to fail with this message:

Additional Failure Information: message='Job ran during migration from Legacy State to Per Stream State. One of the streams that did not have state is: io.airbyte.protocol.models.StreamDescriptor@3dc7762a[name=primary,namespace=<null>,additionalProperties={}]. Job must be retried in order to properly store state.', type='java.lang.IllegalStateException', nonRetryable=false

I tried resetting the connector but that did not make a difference. The sync mode being used is

Incremental Deduped + History

. EDIT: I was able to resolve this issue by manually deleting the state record from the app db.

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2616 to ensure timely resolution!

Please share the complete log file to debug the error.

Hello, i got the exact same issue

I have the same issue with the Stripe Source when I upgraded to 0.1.39.

@Jonas Meier did the sync failed or just output this warning?

We get the same issue (airbyte upgrade from 0.36. to 0.40.14) on connection from postgres to bigquery. @Marcos Marx (Airbyte) the sync stucks at the warning and eventually fails.

@Marcos Marx (Airbyte) It fails. I have downgraded now to 0.1.32 which we used previously and there everything works again.

@Marcos Marx (Airbyte): I got the same issue so I can answer for @Jonas Meier: the sync eventually fails, and no data is ingested

The log file shared doesn’t have any error message, so I can’t help debugging it. @Šimon Appelt can you provide one log with the error message?

@Marcos Marx (Airbyte) please allow me to share one of mine

I am having a similar issue. I recently updated to 0.40.16 and some of my Facebook connectors now fail with

Job ran during migration from Legacy State to Per Stream State. One of the streams that did not have state is

I tried deleting the table from my destination (BigQuery) and even reset my data, but nothing helped. What exactly is meant by

"EDIT: I was able to resolve this issue by manually deleting the state record from the app"

@Sebastian Brickel can you share the complete log?

So in this case it was the

ad_sets

stream that causes the error, when removing it from syncing it then is

ads

and afterwards a few other streams that have the wrong state. I found out that changing the sync mode from

Incremental | Deduped + history

to

Full refresh | Append

seems to work, but I don’t no why

Sorry the missing reply here Sebastian, are you still having issues?

Hard to say, I got into contact with Christian Lam, who explained to me that deleting the state record from the Airbyte application database worked for him. So I checked for the connector states in my VM and delete the ones that caused issues. I then reverted to sync mode

Incremental | Deduped + history

but unfortunately now run into facebooks limit for call to ad-accounts. The strange thing is that his only happens to the three facebook ad acount where I had this issue, all other facebook ad accounts that I sync have no problem with these limits. It is still a bit strange! Did you find something in my logs?

Besides the Facebook side error:

RuntimeError: Batch request failed with response: {'error': {'message': '(#80004) There have been too many calls to this ad-account. Wait a bit and try again. For more info, please refer to <https://developers.facebook.com/docs/graph-api/overview/rate-limiting.>', 'type': 'OAuthException', 'code': 80004, 'error_subcode': 2446079, 'fbtrace_id': 'ANX17jXpw22W_9wKVVIIco_'}}

There isn’t anything meaningful. Are you using same crdentials to run 3 accounts or they use different credentials?

Actually I am using the same facebook app to fetch data from around a dozen accounts

@Sebastian Brickel the issue continues?

Unfortunately yes, I tried to stack the different Facebook connectors in order to avoid Facebooks rate limit but that did not help either

Unfortunately I don't have any better inputs to help you. You can create a Github issue to raise the discussion to other engineers too

ok, I will do that. Thank you @Marcos Marx (Airbyte)