Hello team! I'm having problems with the airbyte p...
# replication-troubleshooting
m
Hello team! I'm having problems with the airbyte platform (version 0.40.28) for a while now. The main bug is that there are containers that are not deleted when source replication fails. The error I get from airbyte is "Something went wrong within the airbyte platform". Any idea how to fix it?
m
I didn’t see this before, can you share the complete log file?
m
Yes of course!
default_logs_22613_txt.txt
m
I found this error:
Copy code
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 409 Conflict
POST https://www.googleapis.com/bigquery/v2/projects/valenciacf/datasets/legend/tables?prettyPrint=false
{
  "code" : 409,
  "errors" : [ {
    "domain" : "global",
    "message" : "Already Exists: Table valenciacf:legend._airbyte_raw_defaultdb_geo_place_types",
    "reason" : "duplicate"
  } ],
  "message" : "Already Exists: Table valenciacf:legend._airbyte_raw_defaultdb_geo_place_types",
  "status" : "ALREADY_EXISTS"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.create(HttpBigQueryRpc.java:185)
The problem is happening to all connections or only one?
Did you deploy Airbyte using k8s or Docker?
m
This problem is happening to other connections too
In Mailchimp connection too
I am deploying it locally on Docker
m
Can you upgrade to latest version and test again? Also, would be possible to test other destination?
m
It only happens with this sources and I do not know why
The destination I have for all my connections is BigQuery and it works with most of my connections
But I don't know why it fails in only some of them
I attach you another file that may help you with this error.
I have updated to the latest version of the platform and the issue still happen. What I think it might be is a bug with the connector or with the worker... What do you think?
m
Well.. https://github.com/airbytehq/airbyte/pull/22497 fix the issue. Can you upgrade to latest version? Sorry I didn’t see this PR before!
m
Is there any possibility that this error could affect on the disk usage of the computer? I discovered that the main reason why airbyte crashed was because disk usage was 100%! I tried with the new version on a connection that was failing and it has not crashed.
u
How much resources your server has? (memory and disk)?
m
I have got 24 GB RAM (used 50% when Airbyte runs more or less) and 143 GB free space on the disk (SSD)
I attach some more images of the problem in case it helps
u
Sorry the delay Miguel, I'll take a look tomorrow again in your issue!
m
Thank you so much Marcos 🤗
Hi @Marcos Marx (Airbyte), any idea of this? Do you need more information from my side?
m
Hello Miguel sorry the long delay to return to you. Would you mind upgrade to 0.41.0 which solves some bugs?