Hello, maybe someone could help me :disappointed: ...
# replication-troubleshooting
в
Hello, maybe someone could help me 😞 Description in discuss — https://discuss.airbyte.io/t/failed-bigquery-clickhouse-sync/3217 I can give an additional information if someone tell me what kind of
✅ 1
m
Hello @Валерий Аржанов did you try to sync only one small table from Bigquery to Clickhouse? I’m trying to create some hypothesis and see where the error happens. If a small table fails probably is a general error btw BQ->CH but if it works probably the error is because of resource limitation, maybe memory. One indication of this is from your logs:
Copy code
Suppressed: io.airbyte.workers.exception.WorkerException: Source process exit with code 137. This warning is normal if the job was cancelled.
Exit code 137 means a container or pod is trying to use more memory than it’s allowed.
в
In this try I’ve just tried to sync one table not the whole data from BQ I don’t sure that table is enough small — ~800.000 rows and ~700 mb I’ll try to create smaller table and sync it, in BigQuery it not so easy to create new tables Thank you for you idea, it may be very helpful, I’ll also try to run it on more RAM
👍 1
I foung in BQ enough small table — ~300 rows. And now I have a bit different problem: Sync without normalization ends successfully but with normalization it says me «Failure Origin: normalization, Message: Normalization failed during the dbt run. This may indicate a problem with the data itself.» Please help me ;( Logs in file
m
Copy code
2022-11-23 15:54:14 [42mnormalization[0m >   File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 742, in get_adapter
2022-11-23 15:54:14 [42mnormalization[0m >     raise InvalidSchema("No connection adapters were found for {!r}".format(url))
2022-11-23 15:54:14 [42mnormalization[0m > requests.exceptions.InvalidSchema: No connection adapters were found for '<http://http//142.93.228.191:8123>'
Looks there is a problem dbt connecting to your Clickhouse instance. As you can see the the address is wrong, maybe try to edit the host removing the http protocol
в
Thank you very much! That was the problem! Now it works!
🙏 1
😀 1