Hi, after upgrading the Postgres destination to v3...
# replication-troubleshooting
s
Hi, after upgrading the Postgres destination to v3.0.0, Airbyte stopped writing to my old table and started creating a new table instead, even though the sync mode is still Incremental | Append. No schema changes were made. Is this expected in v3.0, and how can I make Airbyte continue appending to my original table? Also, is there a migration step I’m missing? Thanks!
k
This behavior is likely related to the architectural change introduced in Postgres destination v3.0.0, which switches the connector to Direct Load and changes how tables are created and managed. [[Postgres dest](https://docs.airbyte.com/integrations/destinations/postgres); [Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] From the docs: - v3.0.0 no longer uses raw tables by default and writes directly to final tables. [[Postgres dest](https://docs.airbyte.com/integrations/destinations/postgres); [Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] - Final tables now include new metadata columns (
_airbyte_meta
,
_airbyte_generation_id
) and no longer use
_airbyte_loaded_at
. [[Postgres schema](https://docs.airbyte.com/integrations/destinations/postgres#schema-map); [Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] The knowledge sources do not explicitly say that v3.0.0 will always create a new table instead of appending to an existing one, but they do indicate that the table structure and metadata columns change with Direct Load. If your pre‑3.0 table doesn’t match the new expected schema (e.g., missing
_airbyte_meta
/
_airbyte_generation_id
), the connector may create a new table that conforms to the new format. ### Is this expected? Based on the docs, it is expected that: - The table format changes in v3.0.0. - Existing tables and raw tables are **not automatically migrated**; there is no automated upgrade process for Postgres v3.0.0. [[Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] The docs do not clearly state whether the connector will always reuse the old final table vs. create a new one when upgrading, so I can’t say with certainty that the new-table behavior is fully intended, only that schema changes and non-automatic migration are expected. ### How to keep appending to your “original” table Given the documented behavior, you effectively have two options: 1. Adopt the new Direct Load table as the “official” one - Update downstream models/queries to use the new table and its new metadata columns (
_airbyte_meta
,
_airbyte_generation_id
). [[Postgres schema](https://docs.airbyte.com/integrations/destinations/postgres#schema-map); [Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] - Optionally backfill or copy data from the old table into the new one using SQL. 2. If you truly need the old layout / raw tables behavior - Enable “Raw tables only” in the Postgres destination configuration before or as part of the upgrade, which preserves the legacy raw-table workflow. [[Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] - However, note that with v3.0.0, the connector no longer supports writing both raw and final tables in a single connection; you’d need separate destinations if you want both. [[Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] The docs do not describe a supported way to force v3.0.0 to keep writing into an old final table with the old schema; the recommended path is to: - Let v3.0.0 create/use the new-format final table, and - Migrate your downstream logic to that table (and optionally drop the old one once you’re satisfied). [[Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#removing-old-raw-tables)] ### Migration steps you might be missing For Postgres v3.0.0, the documented migration steps are: [[Postgres migration](https://docs.airbyte.com/integrations/destinations/postgres-migrations#upgrading-to-300)] - If you only use final tables: no special action; just update downstream models to the new metadata columns if needed. - If you depend on raw tables: enable “Raw tables only” before upgrading. - If you depend on both raw and final tables: create two destinations (one with, one without “Raw tables only”) and dual connections. There is no mention of a step that would “bind” the new connector version to an existing final table and keep appending to it with the old schema. --- If you can share the exact old table name and the new table name pattern you’re seeing, you can compare their schemas; if they differ in the metadata columns described above, that would confirm this is the Direct Load migration effect rather than a sync-mode issue.