Hi all, we've deployed the following version on an EC2 machine using external RDS: Chart Version: 2.0.19 App Version: 2.0.1 Now if we try to create connection from Redshift to Redshift the only working sync mode is

Full refresh | Overwrite

, if we try to do a

Incremental | Append + Deduped

our connection throws the following error:

LEGACY states are deprecated.

{
  "failureOrigin": "replication",
  "internalMessage": "LEGACY states are deprecated.",
  "externalMessage": "Something went wrong during replication",
  "metadata": {
    "attemptNumber": 4,
    "jobId": 1
  },
  "stacktrace": "java.lang.IllegalArgumentException: LEGACY states are deprecated.\n\tat io.airbyte.container.orchestrator.bookkeeping.ParallelStreamStatsTracker.getEmittedCountForCurrentState(ParallelStreamStatsTracker.kt:193)\n\tat io.airbyte.container.orchestrator.worker.state.StateEnricher.enrich(StateEnricher.kt:38)\n\tat io.airbyte.container.orchestrator.worker.ReplicationWorkerHelper.processMessageFromSource(ReplicationWorkerHelper.kt:324)\n\tat io.airbyte.container.orchestrator.worker.MessageProcessor.run(ReplicationTask.kt:158)\n\tat io.airbyte.container.orchestrator.worker.MessageProcessor$run$1.invokeSuspend(ReplicationTask.kt)\n\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)\n\tat kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n",
  "timestamp": 1762164405299
}

The error message

LEGACY states are deprecated.

indicates that your connection is using the deprecated "LEGACY" state type for checkpointing and state management during syncs. Recent versions of Airbyte have moved away from the LEGACY state type in favor of the STREAM and GLOBAL state types, which provide better isolation and flexibility for stream management and are required for advanced sync modes like Incremental | Append + Deduped. Why is this happening? - The LEGACY state type is no longer supported for certain sync modes, especially those that require per-stream state management (like Incremental | Append + Deduped). - The error is thrown because the system expects the source and destination to emit and handle state messages using the new protocol (STREAM or GLOBAL), but instead, it receives a LEGACY state message, which is now explicitly rejected in recent Airbyte versions [[State Types - Airbyte Protocol](https://docs.airbyte.com/platform/understanding-airbyte/airbyte-protocol#state-types)]. What can you do? - Ensure both your Redshift source and destination connectors are updated to the latest versions that support the new state types (STREAM or GLOBAL). - If you have existing connections that were created with older versions of Airbyte, try deleting and recreating the connection. This can reset the state type and force the use of the new protocol, as noted in a related forum post where deleting and recreating the connection resolved a similar issue [[Forum: SFTP Failing to Redshift](https://discuss.airbyte.io/t/413)]. - Make sure to refresh the source schema after making changes to the connection or upgrading connectors, as mismatches between the source and destination schemas can also cause sync failures. Summary: The error is due to the use of the deprecated LEGACY state type, which is not compatible with Incremental | Append + Deduped sync mode in recent Airbyte versions. Update your connectors and recreate the connection to ensure the new state protocol is used [[State Types - Airbyte Protocol](https://docs.airbyte.com/platform/understanding-airbyte/airbyte-protocol#state-types)]. If you continue to experience issues after updating and recreating the connection, please provide additional details about the connector versions in use.

@kapa.ai So the problem is that the Redshift source connector that works properly is with version 0.3.9, but from the official documentation we see that

0.5.0 - Remove LEGACY state

. We cannot bump to the latest version because we cannot create a connection with redshift source connector >0.3.9, it throws the following error during the connection creation.

java.lang.NullPointerException: null value in entry: isNullable=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33)
	at com.google.common.collect.ImmutableMapEntry.<init>(ImmutableMapEntry.java:54)
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:339)
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:449)
	at io.airbyte.cdk.integrations.source.jdbc.AbstractJdbcSource.getColumnMetadata(AbstractJdbcSource.java:248)
	at io.airbyte.cdk.db.jdbc.JdbcDatabase$1.tryAdvance(JdbcDatabase.java:83)
	at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at io.airbyte.cdk.db.jdbc.DefaultJdbcDatabase.bufferedResultSetQuery(DefaultJdbcDatabase.java:56)
	at io.airbyte.cdk.integrations.source.jdbc.AbstractJdbcSource.discoverInternal(AbstractJdbcSource.java:171)
	at io.airbyte.cdk.integrations.source.jdbc.AbstractJdbcSource.discoverInternal(AbstractJdbcSource.java:258)
	at io.airbyte.integrations.source.redshift.RedshiftSource.discoverInternal(RedshiftSource.java:102)
	at io.airbyte.integrations.source.redshift.RedshiftSource.discoverInternal(RedshiftSource.java:30)
	at io.airbyte.cdk.integrations.source.relationaldb.AbstractDbSource.discoverWithoutSystemTables(AbstractDbSource.java:268)
	at io.airbyte.cdk.integrations.source.relationaldb.AbstractDbSource.discover(AbstractDbSource.java:126)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:159)
	at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)

A message from kapa.ai

Confirming that the no-schema-binding views were the problem. We managed to fix this by creating a separate view in the source which we are using for replication. Only this schema is listed in the source connector settings.