You are using Airbyte to sync data from HubSpot to...
# ask-ai
l
You are using Airbyte to sync data from HubSpot to BigQuery, and while most tables sync correctly, the
line_items
table is getting stuck during incremental sync. Key Observations:Full sync works fine, but incremental sync never progresses. • Other HubSpot tables sync correctly, so it's not a permission, connectivity, or API quota issue. • The logs show no explicit errors, but the process remains idle, showing messages like: ◦ Pool queue size: 0, Active threads: 0
k
Based on the information provided in the knowledge sources, I can offer some insights and potential solutions for your issue with the HubSpot
line_items
table getting stuck during incremental sync in Airbyte. 1. Known Limitation: There's a known issue with HubSpot's incremental sync for certain streams, including
line_items
. This is mentioned in the [HubSpot Source Connector documentation](https://docs.airbyte.com/integrations/sources/hubspot#supported-streams): "The current behavior of Incremental is not able to handle source schema changes yet, for example, when a column is added, renamed or deleted from an existing table etc. It is recommended to trigger a [Full refresh - Overwrite](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/full-refresh-overwrite) to correctly replicate the data to the destination with the new schema changes." 2. Potential Workaround: As a temporary solution, you might want to consider using Full Refresh sync mode for the
line_items
table, even if it's not ideal for large datasets. 3. Check for Updates: Ensure you're using the latest version of the HubSpot source connector. Some issues might have been resolved in newer versions. 4. Investigate Cursor Field: The issue might be related to the cursor field used for incremental syncs. As mentioned in the [Incremental Sync documentation](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append#known-limitations): "Due to the use of a cursor column, if modifications to the underlying records are made without properly updating the cursor field, then the updated records won't be picked up by the Incremental sync as expected." 5. Check for Schema Changes: If there have been any recent changes to the
line_items
table schema in HubSpot, this could be causing issues with incremental sync. 6. Monitor Resource Usage: While your logs don't show explicit errors, it's worth monitoring the resource usage of your Airbyte instance during sync attempts. High resource utilization might cause syncs to appear stuck. 7. Consider Splitting the Sync: If the
line_items
table is particularly large, you might want to consider splitting it into a separate connection with its own sync schedule, as suggested in a [GitHub issue comment](https://github.com/airbytehq/airbyte/issues/30073): "SubscriptionItems and InvoiceLineItems represented 99% of our billed rows per sync, so we had to separate them into separate connections to sync on different schedules than the incremental tables." If these suggestions don't resolve the issue, you may want to open a support ticket with Airbyte or contribute to the existing GitHub issues related to HubSpot incremental sync problems. The Airbyte team is actively working on improving these connectors, so your feedback could be valuable for future updates.