Hi everyone! We are having issues to replicate da...
# replication-troubleshooting
g
Hi everyone! We are having issues to replicate data from a ClickHouse source to a ClickHouse destination. Regardless of the amount of memory the Clickhouse instance(dest) has, the sync process always hangs with “Memory limit (total) exceeded” error. And this happens during the normalization step:
Copy code
2023-01-26 14:57:07 normalization > 6 of 9 ERROR creating sql incremental model postgres.user_chapters_scd ................................................. [ERROR in 28.23s]
Copy code
2023-01-26 14:57:07 normalization > Database Error in model user_chapters_scd (models/generated/airbyte_incremental/scd/postgres/user_chapters_scd.sql)
2023-01-26 14:57:07 normalization >   :HTTPDriver for <http://ip-10-0-11-80.ec2.internal:8123> returned response code 500)
2023-01-26 14:57:07 normalization >    Code: 241. DB::Exception: Memory limit (total) exceeded: would use 27.82 GiB (attempt to allocate chunk of 4259816 bytes), maximum: 27.69 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (avg_value_size_hin
2023-01-26 14:57:07 normalization > Done. PASS=7 WARN=0 ERROR=1 SKIP=1 TOTAL=9
2023-01-26 14:57:07 normalization > Code: 241. DB::Exception: Memory limit (total) exceeded: would use 27.82 GiB (attempt to allocate chunk of 4259816 bytes), maximum: 27.69 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (avg_value_size_hint = 311.873046875, avg_chars_size = 364.64765625, limit = 8192): (while reading column _airbyte_data): (while reading from part /var/lib/clickhouse/store/1b7/1b7bc2ee-08fa-4d83-a701-ae65cbb607a1/all_35_66_2/ from mark 440 with max_rows_to_read = 8192): While executing MergeTreeThread. (MEMORY_LIMIT_EXCEEDED) (version 22.12.3.5 (official build))
2023-01-26 14:57:07 normalization >
Can anyone help us please? 🙏
🙏 1
m
I never saw this before but looks you need to increase memory in your Clickhouse instance to run the queries created by normalization.
Github issue related to the problem: https://github.com/airbytehq/airbyte/issues/15352
👀 1