<@U04S8PLJWMT> i have this The source or destinati...
# replication-ask-ai
g
@kapa.ai i have this The source or destination used in this connection does not support file transfers, so they cannot be enabled. when i create the connection. destination is S3 based, and source file based airbyte python cdk
k
"Does not support file transfers" Error This error typically occurs when either the source or destination connector doesn't have file transfer support properly configured. Here's what to check: Supported Connectors for File Transfers Not all connectors support file transfers. According to Airbyte's documentation, only specific connectors have
supportsFileTransfer: true
in their metadata. [Sync files and records] Sources that support file transfers: • Zendesk Support • S3 • Sharepoint • Google Drive • SFTP Bulk Destination that supports file transfers: • S3 Common Causes 1. Your file-based source connector (Python CDK) may not have file transfer support enabled. Custom or community connectors built with the Python CDK do not automatically support file transfers — the connector must explicitly implement the file transfer capability (e.g., the
upload
method in the stream reader). [Sync files and records] 2. Delivery method not set correctly on the source. For file-based sources, you must select Copy Raw Files as the delivery method when configuring the source connector. [Delivery methods] 3. The connector metadata is missing
supportsFileTransfer: true
.
If you built a custom Python CDK connector, this flag needs to be set in the connector's metadata for Airbyte to recognize it as file-transfer capable. Steps to Resolve 1. When setting up your file-based source, under Delivery Method, choose Copy Raw Files. 2. Ensure your source connector is one of the officially supported ones listed above, or that your custom connector has properly implemented file transfer support. 3. Confirm your S3 destination is version 1.4.0 or later (required for file transfer support). [Delivery methods] If Using a Custom Python CDK Source The knowledge sources do not contain enough detail about enabling
supportsFileTransfer
for a custom Python CDK connector. If you've built a custom connector, you would need to ensure it implements the file transfer interface (similar to how the S3 source implements the
upload
method in its stream reader). I'd recommend checking the Airbyte Python CDK documentation or opening a GitHub discussion for guidance on enabling this in a custom connector. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai airbyte version 2.0.1, s3 destination version 1.9.7, it worked, and existing connection works, but at the connection creation, i have this error The source or destination used in this connection does not support file transfers, so they cannot be enabled. when i create the connection. destination is S3 based, and source file based airbyte python cdk
k
I'm sorry, but the knowledge sources do not contain enough information to definitively explain why a new connection creation fails with the file transfer error while existing connections continue to work fine on Airbyte 2.0.1 with S3 destination 1.9.7. However, based on what the knowledge sources do tell us, here are some things worth checking: What We Know • File transfer support requires
supportsFileTransfer: true
in the connector's metadata. Airbyte checks this flag at connection creation time to determine if file transfers can be enabled. [Sync files and records] • Your custom Python CDK source likely does not have this metadata flag set, even if it functionally transfers files. Most Likely Cause The error at connection creation is almost certainly coming from your custom Python CDK source connector, not the S3 destination. Airbyte validates both sides of the connection for
supportsFileTransfer: true
before allowing file transfers to be enabled. A custom connector won't have this flag unless you explicitly added it to its
metadata.yaml
. What to Check 1. Inspect your source connector's
metadata.yaml
— confirm it containssupportsFileTransfer true 2. Delivery method on the source — ensure you selected Copy Raw Files when configuring the source. 3. Why existing connections still work — they were likely created before this validation was enforced, or were created when the connector metadata was different. Recommendation Since the knowledge sources don't cover the specifics of enabling
supportsFileTransfer
in a custom Python CDK connector in detail, I'd recommend: • Opening a GitHub Discussion with the Airbyte team • Referencing the existing connection that works as a comparison point for the metadata differences 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai i have this in the server container 2026-05-29 150023,239 [scheduler-executor-thread-23] ERROR i.a.c.s.e.p.AbstractThrowableProblemHandler(handle):31 - Throwable Problem Handler caught exception: io.airbyte.api.problems.throwable.generated.ConnectionDoesNotSupportFileTransfersProblem: Detail: class ConnectionDoesNotSupportFileTransfersProblemResponse { status: 400 type: error:connection-validation/file-transfer/connection-unsupported title: Connection does not support file transfers detail: The source or destination used in this connection does not support file transfers, so they cannot be enabled. documentationUrl: null data: class ProblemConnectionUnsupportedFileTransfersData { streams: [class ProblemStreamDataItem { streamName: ITEX streamNamespace: null }] } }.detail, Data: class ProblemConnectionUnsupportedFileTransfersData { streams: [class ProblemStreamDataItem { streamName: ITEX streamNamespace: null }] } at io.airbyte.commons.server.handlers.ConnectionsHandler.validateCatalogIncludeFiles(ConnectionsHandler.kt:1008) at io.airbyte.commons.server.handlers.ConnectionsHandler.createConnection(ConnectionsHandler.kt:574) at io.airbyte.commons.server.handlers.WebBackendConnectionsHandler.webBackendCreateConnection(WebBackendConnectionsHandler.kt:697) at io.airbyte.server.apis.controllers.WebBackendApiController.webBackendCreateConnection$lambda$2(WebBackendApiController.kt:88) at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.kt:32) at io.airbyte.server.apis.controllers.WebBackendApiController.webBackendCreateConnection(WebBackendApiController.kt:83) at io.airbyte.server.apis.controllers.$WebBackendApiController$Definition$Intercepted.$$access$$webBackendCreateConnection(Unknown Source) at io.airbyte.server.apis.controllers.$WebBackendApiController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456) at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:134) at io.airbyte.audit.logging.AuditLoggingInterceptor.intercept(AuditLoggingInterceptor.kt:79) at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143) at io.airbyte.server.apis.controllers.$WebBackendApiController$Definition$Intercepted.webBackendCreateConnection(Unknown Source) at io.airbyte.server.apis.controllers.$WebBackendApiController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)
k
The stack trace makes the root cause very clear:
Copy code
at io.airbyte.commons.server.handlers.ConnectionsHandler.validateCatalogIncludeFiles(ConnectionsHandler.kt:1008)
Airbyte's server is calling
validateCatalogIncludeFiles
during connection creation and checking whether the stream
ITEX
supports file transfers. It fails because your custom Python CDK source connector is not recognized as supporting file transfers. What's Happening The server-side validation checks
supportsFileTransfer
on both the source and destination connectors before allowing any stream to be synced as a file stream. Your S3 destination (1.9.7) supports it, but your custom Python CDK source does not have
supportsFileTransfer: true
in its metadata. [Sync files and records] How to Fix It Add
supportsFileTransfer: true
to your custom source connector's
metadata.yaml
file. It should look something like:
Copy code
data:
  connectorType: source
  ...
  supportsFileTransfer: true
Then rebuild and redeploy your custom connector image so Airbyte picks up the updated metadata. Why Existing Connections Still Work Existing connections were created before this validation ran against your connector, so they are not subject to the same creation-time check. The knowledge sources do not contain the exact
metadata.yaml
schema for
supportsFileTransfer
in a custom Python CDK connector, so I'd recommend cross-referencing an officially supported file-transfer source (like the S3 or SFTP Bulk connector's
metadata.yaml
in the Airbyte GitHub repo) to ensure your metadata structure is correct. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai we AIRBYTE_ENABLE_UNSAFE_CODE
k
Based on the knowledge sources, here is what you need to know about enabling `AIRBYTE_ENABLE_UNSAFE_CODE`: What It Does This environment variable enables Custom Components in the Connector Builder. It is considered UNSAFE and EXPERIMENTAL — it allows execution of arbitrary code in your Airbyte environment. [Custom components] ⚠️ Important: Documentation Bug for Airbyte 2.x The current docs show
connectorBuilderServer
as the key, but this is wrong for Airbyte 2.x (including your version 2.0.1). In Helm chart V2,
connector-builder-server
was renamed to
manifest-server
. Using
connectorBuilderServer
is silently ignored by Helm. [GitHub PR #77810] Correct values.yaml for Airbyte 2.x
Copy code
workloadLauncher:
  extraEnv:
    - name: AIRBYTE_ENABLE_UNSAFE_CODE
      value: "true"
manifestServer:
  extraEnv:
    - name: AIRBYTE_ENABLE_UNSAFE_CODE
      value: "true"
Apply with abctl
Copy code
abctl local install --values values.yaml
Apply with Helm
Copy code
helm upgrade airbyte airbyte/airbyte \
  --namespace airbyte \
  --values ./values.yaml \
  --version 2.x.x
Verify It Worked
Copy code
kubectl -n airbyte-abctl get pods
kubectl -n airbyte-abctl exec deploy/airbyte-abctl-manifest-server -- env | grep AIRBYTE_ENABLE_UNSAFE_CODE
kubectl -n airbyte-abctl exec deploy/airbyte-abctl-workload-launcher -- env | grep AIRBYTE_ENABLE_UNSAFE_CODE
[GitHub Issue #77809] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.