hey <@U04S8PLJWMT> i have upgrade postgres desinta...
# replication-ask-ai
i
hey @kapa.ai i have upgrade postgres desintaton connector from 2.x to 3.x to a new direct load architecture. Can i delete my airbyte_internal tables now ?
k
Yes, you can safely delete the old raw tables in the
airbyte_internal
schema after upgrading to version 3.0.0, but you should follow a few verification steps first. Steps before deleting 1. Verify your syncs are completing successfully with the new Direct Load architecture. 2. Confirm that any downstream processes (dbt models, SQL queries, etc.) have been updated to use final tables instead of raw tables — specifically, remove any references to the old
_airbyte_loaded_at
column. 3. Once confirmed, you can safely drop the old raw tables from the
airbyte_internal
schema (or your configured raw table schema). [Postgres Migration Guide] > ⚠️ Important: This action is irreversible. Only remove raw tables after you have verified that all your workflows are working correctly with the new architecture. [Postgres Migration Guide] What changed in 3.0.0 • Data is now written directly to final tables — no more intermediate raw tables by default. • New metadata columns in final tables:
_airbyte_meta
(JSONB) and
_airbyte_generation_id
(BIGINT), replacing
_airbyte_loaded_at
. • Existing raw tables are not automatically deleted during the upgrade, so they remain until you manually remove them. [Postgres Migration Guide] If you still need raw tables for some reason (e.g., a downstream process depends on them), you can enable the "Raw tables only" option in the connector configuration instead of deleting them. [Postgres Destination] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai i am getting this error now Warning from destination: ERROR: column "_airbyte_generation_id" of relation "Eck_Collection_Camp_all" does not exist Position: 4600 after upgrading to 3.0.6
k
The knowledge sources do not contain enough information to give a definitive answer specific to this Postgres 3.0.6 error. However, based on what is available, here is what I can tell you: What's happening The error suggests that the
_airbyte_generation_id
column is missing from your existing table
Eck_Collection_Camp_all
. This is a known pattern when upgrading to a new connector version that introduces this column — the table was created before the column existed, and the connector is trying to reference it. [Postgres Migration Guide] A similar issue was documented and fixed for the BigQuery destination connector, where legacy tables created before DV2/v3.0 lacked the
_airbyte_generation_id
column, causing crashes during sync. [BigQuery fix] What you can try 1. Run a Refresh/Reset on the affected connection. This will rebuild the final table with the correct schema including
_airbyte_generation_id
. You can do this from the connection's settings in the Airbyte UI. 2. Manually add the missing column to the affected table:ALTER TABLE "Eck_Collection_Camp_all" ADD COLUMN IF NOT EXISTS _airbyte_generation_id BIGINT;The expected type for Postgres is
BIGINT
. [Postgres Schema Map] > ⚠️ I'd recommend trying the Refresh approach first, as it is the officially supported path. The manual
ALTER TABLE
is a workaround — use it only if you understand the implications for your data. If the issue persists, it may be worth checking the Airbyte GitHub issues or opening a support ticket, as this could be a bug specific to the Postgres 3.0.x upgrade path. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai i did a refresh on the connection but i am still getting that error
k
I'm sorry, but the knowledge sources do not contain enough information to provide a confident solution for this specific case — where the error persists even after a refresh on the Postgres 3.0.6 destination. What I can suggest Since the refresh didn't resolve it, you could try the manual column addition as a workaround:
Copy code
ALTER TABLE "Eck_Collection_Camp_all" ADD COLUMN IF NOT EXISTS _airbyte_generation_id BIGINT;
Then trigger a new sync and see if the error clears. [Postgres Schema Map] Recommended next steps Since this appears to be a potential bug in the 3.0.x upgrade path, I'd recommend: 1. Check the Airbyte GitHub issues for similar reports — a comparable issue was found and fixed for Snowflake (PR #80781) and BigQuery (PR #76970), where missing meta columns after upgrades caused similar errors. A fix may be pending or already available for Postgres. 2. Open a GitHub issue at airbytehq/airbyte with your connector version and error details. 3. If you're on Airbyte Cloud, open a support ticket at support.airbyte.com referencing the error. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai seeing this error now org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
k
I'm sorry, but the knowledge sources do not contain enough information to provide a confident answer about this specific
PSQLException: An I/O error occurred while sending to the backend
error. What this error generally indicates This is a network/connection-level error between the Airbyte connector and your PostgreSQL server — it typically points to: • A dropped or unstable network connection during the sync • A connection timeout • The PostgreSQL server closing the connection unexpectedly What you can try 1. Check your PostgreSQL server logs for any connection drops or errors around the time of the sync. 2. Verify network stability between Airbyte and your Postgres instance. 3. Check PostgreSQL connection limits — if
max_connections
is being hit, the server may be dropping connections. 4. Review your SSL/tunnel configuration in the destination connector settings, as SSL mismatches can cause I/O errors. Recommended next steps Given that you've now encountered multiple errors since upgrading to 3.0.x, I'd strongly recommend: • Opening a GitHub issue at airbytehq/airbyte with the full sequence of errors and your connector version. • If on Airbyte Cloud, opening a support ticket at support.airbyte.com with the full logs from the failing sync. The combination of errors you're seeing may be related to the 3.0.x upgrade, and the Airbyte team would be best placed to investigate with full log access. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.