Dmytro Vorotyntsev
08/21/2022, 5:57 PM85.24 GB | 271,975,505 emitted records | 271,975,505 committed records | 25h 2m 5s | Sync
For the subsequent sync operation I expected to take less, because it should use publication slot to get only new entries for tables where Incremental | Deduped + History is selected, and rewrite completely those entries where Full refresh | Overwrite is chosen,
But I don’t understand (missed the doc?) why the subsequent sync is taking even longer (still in progress since yesterday) and re-scanning all data from the source.
UPDATE: The issue seems to be in another dimensions
We hit Airbyte performance limit, so we have increased to more powerful CPU and increased Network throughput
Also the Redhisft initial single-node has 100% storage full, so we are switching to multi-node setup with automatic storage scalingDmytro Vorotyntsev
08/21/2022, 6:00 PMDmytro Vorotyntsev
08/21/2022, 6:00 PMDmytro Vorotyntsev
08/21/2022, 6:01 PMDmytro Vorotyntsev
08/21/2022, 6:01 PMDmytro Vorotyntsev
08/21/2022, 6:07 PM