It's very hard to see what Airbyte is doing exactly. The syncs are still "syncing" but the amount of records loaded don't go up and the logs are incomprehensible to fully understand what is happening or why. Postgres to postgres, so shouldn't be a very uncommon situation. It would be very helpful to have more granular and detailed information on what is happening in the UI.

Thanks for the feedback, Sam. Are there any areas in the logs that you think could be improved or where you have suggestions? 🪄 @Natalie (Airbyte) as we’re discussing about log readability.

I literally don't know what they mean. I have no idea why my sync is not progressing or what I can do to speed it up. E.g. this is what they are constantly saying, repeating over and over for all of my syncs:

2024-08-06 17:06:43 destination > INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 307.2 MB, allocated: 10 MB (10.0 MB), %% used: 0.03255208335354433 | State Manager memory usage: Allocated: 10 MB, Used: 0 bytes, percentage Used 0.0
2024-08-06 17:06:43 destination > INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0
2024-08-06 17:07:43 destination > INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):94 [ASYNC QUEUE INFO] Global: max: 307.2 MB, allocated: 10 MB (10.0 MB), %% used: 0.03255208335354433 | State Manager memory usage: Allocated: 10 MB, Used: 0 bytes, percentage Used 0.0
2024-08-06 17:07:43 destination > INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):127 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0

It would be good if the logs would clearly output what they are doing and why, maybe with links to the docs.