Abhishek Batra
12/30/2025, 10:36 AMkapa.ai
12/30/2025, 10:36 AMairbyte-ci to build your modified connector. This will typically tag the image as dev on your local Docker host. bash airbyte-ci connectors --name source-<source-name> build [[Developing Locally (2.0)](https://docs.airbyte.com/platform/2.0/contributing-to-airbyte/developing-locally#connector-contributions)] * Load the Image (if using abctl): If you are running Airbyte via abctl, you must load the local image into the cluster: bash kind load docker-image airbyte/source-<source-name>:dev -n airbyte-abctl [[Developing Locally (2.0)](https://docs.airbyte.com/platform/2.0/contributing-to-airbyte/developing-locally#connector-contributions)] * Update in UI: Navigate to Workspace settings > Sources (or Destinations), find the original connector, and click the edit icon. Change the version tag to dev (or your custom tag) and click Change to save. This action also refreshes the connector specification cache, which is necessary if your catalog enrichment logic has changed. [[Developing Locally (2.0)](https://docs.airbyte.com/platform/2.0/contributing-to-airbyte/developing-locally#connector-contributions); [Custom or New Connector](https://docs.airbyte.com/integrations/custom-connectors)] ### 2. Adding a New Custom Docker Connector If you have built a separate image and want to add it as a new option: 1. Go to Workspace settings > Sources (or Destinations). 2. Click New connector and select Add a new Docker connector. 3. Enter a display name, the Docker full image name, and the Docker image tag. 4. Once added, you can select this custom version when setting up a new connection. [[Using Custom Connectors (2.0)](https://docs.airbyte.com/platform/2.0/operator-guides/using-custom-connectors#4-use-your-custom-docker-connector-in-airbyte)] ### Addressing Catalog Enrichment Differences If the catalog enrichment is different between your custom connector and the original, the knowledge sources suggest that Airbyte Server caches these specifications. To ensure your custom catalog logic is recognized: * In the UI settings for that connector, re-saving the version (even if it's still dev) triggers a refresh of the connector's specification. [[Developing Locally (2.0)](https://docs.airbyte.com/platform/2.0/contributing-to-airbyte/developing-locally#connector-contributions)] * If you are using the Connector Builder, you can implement Custom Components in Python to handle specialized transformations or complex API behaviors that standard components cannot address. [[Custom components for the Connector Builder](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/custom-components)]Abhishek Batra
12/30/2025, 11:27 AMkapa.ai
12/30/2025, 11:27 AMkapa.ai
12/30/2025, 11:27 AMsync_id or generation_id to a connector's catalog definition. In Airbyte, these fields are typically system-level metadata added by the platform at sync time rather than fields defined within the connector's discovered catalog. Based on the documentation provided: * _airbyte_generation_id and _airbyte_meta (which contains sync_id) are described as fields that "Always exist" in the destination output schema (e.g., in S3 CSV destinations) rather than in the source's discovered catalog. [[S3 Destination CSV Schema](https://docs.airbyte.com/integrations/destinations/s3#csv)] * The stream_generation table in the internal configuration database tracks these IDs, linking a connection_id to a generation_id and a start_job_id. [[Config Database stream_generation](https://docs.airbyte.com/platform/2.0/understanding-airbyte/database-data-catalog#stream_generation)] * When defining a connector's catalog via the discover method, you typically define the json_schema for the data and the supported_sync_modes. [[Custom Python Connector Discover](https://docs.airbyte.com/platform/1.8/connector-development/tutorials/custom-python-connector/discover)] If your custom connector requires specific sync modes or cursor fields to be present in the catalog for the platform to manage generations and syncs correctly, you may need to ensure your AirbyteStream object includes: python supported_sync_modes=['full_refresh', 'incremental'] source_defined_cursor=True default_cursor_field=['your_field'] [[Generated Python Source Connector Error](https://discuss.airbyte.io/t/2940)] If you are looking for these fields during the "Discover" phase in the UI, they generally will not appear because they are internal platform metadata injected during the actual sync process.