@kapa.ai, i have made the custom connector for clickhouse replication and it is working but the custom connector uses Using data channel medium STDIO and the original connector Using data channel medium SOCKET which is responsible for the speedy inserts. is there a way the platform passes the same config to connectors as i see the default is STDIO.

The knowledge sources do not contain enough detailed information about exactly how the platform passes the

SOCKET

data channel medium configuration to connectors at runtime. However, based on what is available, here's what I can tell you: How the Platform Controls Data Channel Medium The data channel medium (

STDIO

vs

SOCKET

) is set by the platform at runtime — it's not something the connector hardcodes. Your

application-connector.yml

sets the default, but the platform can override it via environment variables. [Getting Started] Enabling SOCKET (Speed Mode) for Your Custom Connector For the official ClickHouse connector, speed mode (SOCKET) support was enabled by adding

connectorIPCOptions

to the connector's

metadata.yaml

. This configures the data channel with both JSONL/PROTOBUF serialization and SOCKET/STDIO transport options. [Redshift Speed Mode PR] The same approach should apply to your custom connector. You need to add

connectorIPCOptions

to your connector's

metadata.yaml

to signal to the platform that your connector supports SOCKET mode. What to Check in Your Custom Connector 1.

metadata.yaml

: Add

connectorIPCOptions

similar to what was done for destination-redshift and the official ClickHouse connector to declare SOCKET support. 2.

application-connector.yml

: Ensure your config supports both modesairbyte destination: core: data-channel: medium: STDIO # Default; platform overrides this at runtime format: JSONL[Getting Started] 3.

BeanFactory

: Ensure your

AggregatePublishingConfig

handles both

STDIO

and

SOCKET

modes, including the

maxBufferedAggregates

parameter for SOCKET mode. [Write Infrastructure] The key takeaway is: the platform only enables SOCKET mode for connectors that declare support for it via

connectorIPCOptions

in

metadata.yaml

. Without that declaration, the platform defaults to STDIO. I'd recommend examining the official ClickHouse connector's

metadata.yaml

as a reference for what to add to your custom connector. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai, this is not working

I'm sorry, but the knowledge sources don't contain enough detailed information to help troubleshoot why adding

connectorIPCOptions

to your custom connector's

metadata.yaml

isn't enabling SOCKET mode. To get more targeted help, I'd suggest: 1. Opening a GitHub issue in the airbytehq/airbyte repository describing your custom connector setup and what you've tried. 2. Joining the Airbyte Community Slack where the connector maintainers are active and can provide more hands-on guidance — especially since the ClickHouse connector team monitors it closely. [ClickHouse Beta Discussion] If you can share more details about what you've configured and what errors or logs you're seeing, that would also help narrow down the issue. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai, how does the custom connector and original connector makes it different for platform to inject different values

I'm sorry, but the knowledge sources do not contain enough information to explain exactly how the platform differentiates between a custom connector and an official connector when injecting runtime values like the data channel medium (

STDIO

vs

SOCKET

). However, based on what the knowledge sources do reveal, there is one relevant clue: What We Know A GitHub issue shows that when using an official connector as a custom connector (same image, just registered as custom), syncs fail with errors like

syncId

being null — suggesting the platform does treat custom connectors differently at the protocol/catalog level, even with identical images. [GitHub Issue] What's Likely Happening The platform likely uses the connector's registry entry (from

oss_registry.json

or

cloud_registry.json

) to determine capabilities like SOCKET support. Custom connectors are not part of the registry, so the platform may simply not know to inject SOCKET-related configuration for them. This is controlled via the

metadata.yaml

→

registryOverrides

mechanism for official connectors. [Connector Metadata] Recommendation For a definitive answer on this, I'd strongly recommend: • Opening a GitHub issue at airbytehq/airbyte • Or asking in the Airbyte Community Slack, where platform engineers can clarify how runtime injection differs for custom vs. registered connectors 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.