<@U04S8PLJWMT> I am getting this error when trying...
# replication-ask-ai
a
@kapa.ai I am getting this error when trying to run sync for one connection why?
Copy code
Failed to initialize connector operation
k
The
Failed to initialize connector operation
error can have several causes in Airbyte. Based on the knowledge sources, here are the most common reasons: Common Causes 1. Invalid Cursor Field in Stream Schema If you're using Incremental | Append + Deduped sync mode, the cursor field configured for a stream may not actually exist in the stream's schema. For example, the
VendorOrdersStatus
stream in the Amazon Seller Partner connector fails because its default cursor (
createdBefore
) is an API query parameter, not a response field. Workaround: Disable the problematic stream, or change the cursor field to one that actually exists in the schema. [GitHub Issue] 2. Misconfigured Connector (Custom Connector Issues) Adding an official connector as a custom connector can trigger this error due to internal initialization failures (e.g.,
NullPointerException
during bean instantiation). [GitHub Issue] 3. Invalid Configuration Parameters Certain connector versions may pass malformed values (e.g., an invalid timeout string in
source-mssql
) that cause initialization to fail even when the connection check passes. [GitHub Issue] 4. SSL/SSH Configuration Issues For connectors like
source-postgres
on Airbyte Cloud, insufficient SSL mode without an SSH tunnel can cause this error during initialization. [PR #76474] How to Diagnose 1. Check the full logs for a more specific
Caused by:
message beneath the generic
Failed to initialize connector operation
error. 2. Identify the stream causing the issue — often only one stream is affected, and disabling it allows the rest to sync. 3. Review your sync mode settings — if using Incremental + Deduped, verify the cursor field exists in the stream schema. 4. Check connector version — a recent upgrade may have introduced a regression; consider pinning to a previous version if needed. Could you share the full error logs or the connector name/version? That would help narrow down the exact cause. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
a
@kapa.ai I am using custom connector and my sync mode is full refresh | Append
k
Based on your description — custom connector with Full Refresh | Append sync mode — this matches a known bug documented in the knowledge sources. Root Cause When using a custom connector, the Airbyte platform does not populate the
generationId
,
minimumGenerationId
, and
syncId
fields on the
ConfiguredAirbyteStream
. Bulk CDK-based destinations expect these as non-nullable
Long
values, so when they are
null
, Kotlin's auto-unboxing throws a
NullPointerException
, which surfaces as:
Copy code
ConfigErrorException: Failed to initialize connector operation
You can confirm this is your issue by checking your logs for:
Copy code
generationId=<null>, minimumGenerationId=<null>, syncId=<null>
[GitHub Issue] Affected Connectors This affects all bulk CDK-based destinations (e.g., S3, ClickHouse, HubSpot, MSSQL, and others) when used as custom connectors. [GitHub PR] Fix A fix has been submitted via PR #75211 which adds null-coalescing (
?: 0L
) for these fields in
DestinationStreamFactory.make()
, defaulting them to
0
when not provided by the platform. This fix is targeted for bulk CDK version 1.0.7. [GitHub PR] Workaround (Until Fix is Released) • If your end goal is to build a custom connector, wait for the CDK fix to be released and use the updated CDK version (
1.0.7+
) as your base. • If you are using an official connector as a custom connector, switch back to using it as an official connector instead, as the issue does not occur in that path. [GitHub Issue] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
a
@kapa.ai this is the logs
Copy code
2026-06-04 09:35:14,672 [io-executor-thread-4]	INFO	i.a.c.s.h.ConnectionsHandler(applySchemaChange):2055 - Sending notification of schema auto propagation for connectionId: 'class ConnectionRead {
    connectionId: e95d41e7-9719-4ee7-8969-fad7e5085efd
    name: salesforce_bigQuery → Salesforce Destination
    namespaceDefinition: destination
    namespaceFormat: null
    prefix:
    sourceId: aec68df5-4107-4c3c-bb52-b90980f71d6b
    destinationId: 5f83d12b-a48f-489f-bae2-de43525c668a
    operationIds: []
    syncCatalog: class AirbyteCatalog {
        streams: [class AirbyteStreamAndConfiguration {
            stream: class AirbyteStream {
                name: Contract_Product_Usage__c
                jsonSchema: {"type":"object","properties":{"App_Token__c":{"type":"string"},"Data_MUCs__c":{"type":"number","airbyte_type":"integer"}}}
                supportedSyncModes: [full_refresh, incremental]
                sourceDefinedCursor: false
                defaultCursorField: []
                sourceDefinedPrimaryKey: []
                namespace: gold_salesforce_ksa_prod
                isResumable: null
                isFileBased: null
            }
            config: class AirbyteStreamConfiguration {
                syncMode: full_refresh
                cursorField: []
                destinationSyncMode: append
                namespace: null
                primaryKey: []
                aliasName: Contract_Product_Usage__c
                selected: true
                suggested: false
                destinationObjectName: null
                includeFiles: false
                fieldSelectionEnabled: false
                selectedFields: []
                hashedFields: []
                mappers: []
                minimumGenerationId: null
                generationId: null
                syncId: null
            }
        }]
    }
    schedule: class ConnectionSchedule {
        units: 24
        timeUnit: hours
    }
    scheduleType: basic
    scheduleData: class ConnectionScheduleData {
        basicSchedule: class ConnectionScheduleDataBasicSchedule {
            timeUnit: hours
            units: 24
        }
        cron: null
    }
    status: active
    resourceRequirements: null
    sourceCatalogId: 04af1f51-86ac-4e34-a6e6-61fa150fe5ac
    destinationCatalogId: null
    breakingChange: false
    notifySchemaChanges: true
    notifySchemaChangesByEmail: false
    nonBreakingChangesPreference: propagate_columns
    createdAt: 1780565685
    backfillPreference: disabled
    workspaceId: null
    dataplaneGroupId: null
    tags: []
}.connectionId'
2026-06-04 09:35:14,672 [io-executor-thread-4]	INFO	i.a.c.s.h.h.NotificationHelper(getSchemaUpdateNotification):47 - No diff to report for connection: 'class ConnectionRead {
    connectionId: e95d41e7-9719-4ee7-8969-fad7e5085efd
    name: salesforce_bigQuery → Salesforce Destination
    namespaceDefinition: destination
    namespaceFormat: null
    prefix:
    sourceId: aec68df5-4107-4c3c-bb52-b90980f71d6b
    destinationId: 5f83d12b-a48f-489f-bae2-de43525c668a
    operationIds: []
    syncCatalog: class AirbyteCatalog {
        streams: [class AirbyteStreamAndConfiguration {
            stream: class AirbyteStream {
                name: Contract_Product_Usage__c
                jsonSchema: {"type":"object","properties":{"App_Token__c":{"type":"string"},"Data_MUCs__c":{"type":"number","airbyte_type":"integer"}}}
                supportedSyncModes: [full_refresh, incremental]
                sourceDefinedCursor: false
                defaultCursorField: []
                sourceDefinedPrimaryKey: []
                namespace: gold_salesforce_ksa_prod
                isResumable: null
                isFileBased: null
            }
            config: class AirbyteStreamConfiguration {
                syncMode: full_refresh
                cursorField: []
                destinationSyncMode: append
                namespace: null
                primaryKey: []
                aliasName: Contract_Product_Usage__c
                selected: true
                suggested: false
                destinationObjectName: null
                includeFiles: false
                fieldSelectionEnabled: false
                selectedFields: []
                hashedFields: []
                mappers: []
                minimumGenerationId: null
                generationId: null
                syncId: null
            }
        }]
    }
    schedule: class ConnectionSchedule {
        units: 24
        timeUnit: hours
    }
    scheduleType: basic
    scheduleData: class ConnectionScheduleData {
        basicSchedule: class ConnectionScheduleDataBasicSchedule {
            timeUnit: hours
            units: 24
        }
        cron: null
    }
    status: active
    resourceRequirements: null
    sourceCatalogId: 04af1f51-86ac-4e34-a6e6-61fa150fe5ac
    destinationCatalogId: null
    breakingChange: false
    notifySchemaChanges: true
    notifySchemaChangesByEmail: false
    nonBreakingChangesPreference: propagate_columns
    createdAt: 1780565685
    backfillPreference: disabled
    workspaceId: null
    dataplaneGroupId: null
    tags: []
}.connectionId'; skipping notification.
2026-06-04 09:35:41,395 [io-executor-thread-3]	INFO	i.a.n.CustomerioNotificationClient(sendNotifyRequest):397 - <http://Customer.io|Customer.io> API token is empty. Skipping email notification. workspaceId: 0cd90a42-1041-45d9-bf68-499a0719d5e0
2026-06-04 09:35:41,395 [io-executor-thread-3]	WARN	i.a.p.j.JobNotifier(sendNotification):251 - Failed to successfully notify workspaceId {}: 0cd90a42-1041-45d9-bf68-499a0719d5e0, notificationItem
2026-06-04 09:35:41,873 [io-executor-thread-3]	INFO	i.a.n.SlackNotificationClient(notifyJson):417 - Successful notification (200): {ok}
2026-06-04 09:35:41,884 [io-executor-thread-3]	INFO	i.a.p.j.e.JobErrorReporter(reportSyncJobFailure$lambda$6):63 - 2 failures incoming for jobId '13194' connectionId 'e95d41e7-9719-4ee7-8969-fad7e5085efd'
2026-06-04 09:35:41,893 [io-executor-thread-3]	INFO	i.a.p.j.e.JobErrorReporter(reportSyncJobFailure$lambda$6):81 - 1 failures to report for jobId '13194' connectionId 'e95d41e7-9719-4ee7-8969-fad7e5085efd'
2026-06-04 09:35:41,893 [io-executor-thread-3]	INFO	i.a.p.j.e.JobErrorReporter(reportSyncJobFailure$lambda$6):84 - Reporting failure for jobId '13194' connectionId 'e95d41e7-9719-4ee7-8969-fad7e5085efd' origin 'destination'
2026-06-04 09:35:41,903 [io-executor-thread-3]	INFO	i.a.c.s.h.JobsHandler(reportIfLastFailedAttempt):141 - Successfully reported failure for job id '13194' connectionId: 'e95d41e7-9719-4ee7-8969-fad7e5085efd'
2026-06-04 09:35:48,509 [io-executor-thread-1]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:35:48,509 [io-executor-thread-1]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:42:10,377 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:42:10,377 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:45:50,990 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:45:50,990 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:45:52,467 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
2026-06-04 09:45:52,468 [io-executor-thread-2]	WARN	i.a.c.s.h.h.CatalogConverter(ensureCompatibleDestinationSyncMode):532 - Default sync modes are incompatible, so falling back to full_refresh | append
k
Your logs confirm the exact issue discussed earlier. The key evidence is:
Copy code
minimumGenerationId: null
generationId: null
syncId: null
This confirms the platform is not populating these required fields for your custom connector, which causes the
NullPointerException
and
Failed to initialize connector operation
error. [GitHub Issue] Additionally, your logs show:
Copy code
Default sync modes are incompatible, so falling back to full_refresh | append
This means your destination connector doesn't natively advertise
full_refresh | append
as a supported mode, and Airbyte is forcing a fallback to it — which may compound the initialization issue. What You Can Do Now 1. Use the official connector instead of the custom connector — the same image works fine when registered as an official catalog connector, as the platform correctly populates `generationId`/`syncId` in that path. [GitHub Issue] 2. Wait for the CDK fix — PR #75211 adds a null-safe default (
0L
) for these fields in
DestinationStreamFactory.make()
, targeting bulk CDK version 1.0.7. Once released, destinations built on the updated CDK will no longer crash when these fields are null. [GitHub PR] 3. Check your destination connector's supported sync modes — the
Default sync modes are incompatible
warning suggests your custom destination may not be declaring
full_refresh | append
as a supported mode. Verify the connector's
spec
output includes this mode. > āš ļø The knowledge sources do not contain enough information to confirm whether the CDK fix (PR #75211) has already been merged and released. Check the PR status directly to see if version 1.0.7 is available. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
a
@kapa.ai can I create my own custom connector and pass the `generationId`/`syncId` correctly?
k
The knowledge sources do not contain enough information to provide a complete guide on how to correctly pass `generationId`/`syncId` in a custom connector implementation. However, based on what the knowledge sources do tell us: What We Know The root cause is a platform-side issue — when a connector is registered as a custom connector, the Airbyte platform simply does not populate
generationId
,
minimumGenerationId
, and
syncId
on the
ConfiguredAirbyteStream
before passing it to the destination. This happens regardless of what your connector does — it's the platform that sets these fields, not the connector itself. [GitHub Issue] So even if you build your own custom connector correctly, the platform will still pass
null
for these fields in the custom connector path. What the Fix Looks Like (CDK Side) The proper CDK-side fix (PR #75211) is to handle nulls defensively in
DestinationStreamFactory.make()
using null-coalescing:
Copy code
generationId = stream.generationId ?: 0L
minimumGenerationId = stream.minimumGenerationId ?: 0L
syncId = stream.syncId ?: 0L
This means if you are building a bulk CDK-based destination, you should use CDK version 1.0.7+ (once released) which includes this fix. [GitHub PR] Recommendation • If you're building a custom destination connector, base it on the updated bulk CDK (1.0.7+) once the fix is released. • The underlying platform issue (not populating these fields for custom connectors) is a separate problem that may require a platform-level fix beyond the CDK workaround. I'd recommend monitoring PR #75211 and issue #69351 for updates. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
a
@kapa.ai dose this mean that custom connectors will not work?
k
Not necessarily. Based on the knowledge sources: Custom Connectors Can Work, With Limitations The issue is specific to bulk CDK-based destination connectors. Custom connectors that are not built on the bulk CDK are not affected by this particular `generationId`/`syncId` NPE bug. Evidence from the knowledge sources shows that custom connectors do work in some cases: • S3 Destination
1.6.0
worked fine on OSS platform
1.8.2
, while
1.7.0+
(which switched to the bulk CDK) started failing. [GitHub Issue] • One user reported that bootstrapping a connection with an older version and then upgrading to
1.9.7
also worked. [GitHub Issue] Summary Bottom Line If you are building a source connector, this issue does not affect you at all. If you are building a destination connector using the bulk CDK, you need to wait for CDK version 1.0.7+ to be released with the null-safety fix. [GitHub PR] > āš ļø The knowledge sources do not confirm whether CDK 1.0.7 has been officially released yet. Monitor PR #75211 for the latest status. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements