Jian Zhu
05/27/2024, 6:52 AM2024-05-27 06:50:13 destination > INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):128 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0kapa.ai
05/27/2024, 6:52 AMshell 2024-05-27 06:50:13 destination > INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):128 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0 This issue seems to be related to the destination-redshift connector, where the sync process gets stuck in the printWorkerInfo method. Similar issues have been reported in the following GitHub issues: 1. [Issue #37835](https://github.com/airbytehq/airbyte/issues/37835): The sync process gets stuck while getting table information from Redshift after a version update. 2. [Issue #34620](https://github.com/airbytehq/airbyte/issues/34620): The sync process gets stuck in the printQueueInfo method, showing similar log patterns with no active threads and minimal memory usage. In both cases, the logs indicate that the pool queue size is 0 and there are no active threads, which suggests that the process is not progressing as expected. This could be due to compatibility issues with the Redshift version or a problem with the connector's handling of the async worker threads. ### Steps to Resolve 1. Check for Updates: Ensure that you are using the latest version of the destination-redshift connector. Compatibility issues are often resolved in newer releases. 2. Rollback Version: If the issue started after a recent update, consider rolling back to a previous version of the Redshift connector that was working correctly. This was a temporary solution mentioned in [Issue #37835](https://github.com/airbytehq/airbyte/issues/37835). 3. Review Logs: Examine the full logs for any additional error messages or warnings that might provide more context about why the process is stuck. 4. Report the Issue: If the problem persists, consider reporting it on the Airbyte GitHub repository with detailed logs and steps to reproduce the issue. This will help the maintainers to diagnose and fix the problem. ### Example Log Output Here is an example of the log output that indicates the job is stuck: shell 2024-05-04 21:00:03 destination > INFO pool-6-thread-1 i.a.c.i.d.a.FlushWorkers(printWorkerInfo):128 [ASYNC WORKER INFO] Pool queue size: 0, Active threads: 0 2024-05-04 21:01:03 destination > INFO pool-3-thread-1 i.a.c.i.d.a.b.BufferManager(printQueueInfo):89 [ASYNC QUEUE INFO] Global: max: 1.05 GB, allocated: 10 MB (10.0 MB), %% used: 0.009300595239745114 | State Manager memory usage: Allocated: 10 MB, Used: 0 bytes, percentage Used 0.0 For more details, you can refer to the related GitHub issues: - [Issue #37835](https://github.com/airbytehq/airbyte/issues/37835) - [Issue #34620](https://github.com/airbytehq/airbyte/issues/34620) If you need further assistance, you can also check the [Airbyte documentation](https://docs.airbyte.com/) or reach out to the community for support.Jian Zhu
05/27/2024, 6:57 AMkapa.ai
05/27/2024, 6:57 AMkapa.ai
05/27/2024, 6:57 AMdestination-postgres connector. However, there are several issues related to the destination-postgres connector that might be relevant to your situation: 1. Sync Hangs Indefinitely: - [Issue #38534](https://github.com/airbytehq/airbyte/issues/38534): This issue reports that the sync process hangs indefinitely with no error. The recommendation is to provide detailed information about the connector when reporting such problems. 2. Column Name Inconsistency: - [Issue #25911](https://github.com/airbytehq/airbyte/issues/25911): This issue discusses column name inconsistencies, which might cause unexpected behavior during the sync process. 3. Significant Log Warnings: - [Issue #31878](https://github.com/airbytehq/airbyte/issues/31878): This issue mentions significant log warnings generated during syncs. Upgrading to the latest version of the connector was suggested as a solution. ### Steps to Resolve 1. Check for Updates: Ensure that you are using the latest version of the destination-postgres connector. Compatibility issues and bugs are often resolved in newer releases. 2. Review Logs: Examine the full logs for any additional error messages or warnings that might provide more context about why the process is stuck. 3. Provide Detailed Information: If the problem persists, consider reporting it on the Airbyte GitHub repository with detailed logs and steps to reproduce the issue. This will help the maintainers to diagnose and fix the problem. ### Example Log Output Here is an example of a related log output that indicates a problem during the sync process: shell java.lang.RuntimeException: org.postgresql.util.PSQLException: ERROR: numeric field overflow Detail: A field with precision 38, scale 9 must round to an absolute value less than 10^29. For more details, you can refer to the related GitHub issues: - [Issue #38534](https://github.com/airbytehq/airbyte/issues/38534) - [Issue #25911](https://github.com/airbytehq/airbyte/issues/25911) - [Issue #31878](https://github.com/airbytehq/airbyte/issues/31878) If you need further assistance, you can also check the [Airbyte documentation](https://docs.airbyte.com/) or reach out to the community for support.