@kapa.ai if airbyte-ci is deprecated, whats the new way to my connector image

Since

airbyte-ci

is deprecated, the recommended replacement for building connector images is Poe the Poet (

poe

). [airbyte-ci deprecated] You can discover available build-related tasks by running

poe --help

from your connector directory. [airbyte-ci deprecated] Steps to get set up 1. Install

uv

(used to manage Python CLI apps):brew install uv 2. Install Poe the Poet:brew tap nat-n/poethepoet brew install nat-n/poethepoet/poethepoet 3. Run tasks from your connector directory — navigate to your connector folder and run

poe

to see available tasks (e.g.,

build

,

lint

,

check-all

). [local connector development] If you need a custom Dockerfile If you're building a forked/patched connector and want to create a Docker image manually (without any CLI tooling), you can use a Dockerfile like this:

FROM airbyte/python-connector-base:1.1.0

COPY . ./airbyte/integration_code
RUN pip install ./airbyte/integration_code

ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

Then build and push it to your own Docker registry (e.g., DockerHub, ECR, GCR). [forked connector Docker] Once your image is hosted, you can register it in the Airbyte UI under Admin → New Connector, providing the Docker repository name, tag, and documentation URL. [custom connectors] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai there is no build task

You're right that

poe

doesn't have a generic "build" task — the available tasks vary by connector and connector type. Here are the alternatives supported by the knowledge sources: Option 1: airbyte-cdk image build (Python/all connector types) From your connector directory, you can use the

airbyte-cdk

CLI:

cd airbyte-integrations/connectors/source-<name>
airbyte-cdk image build

This method supports all connector types and automatically builds both

arm64

and

amd64

images. [Docker Images README] Option 2: Gradle (Java/Kotlin connectors) For Java or Kotlin connectors, use Gradle from the Airbyte project root:

# Build the Docker image only (faster)
./gradlew :airbyte-integrations:connectors:<connector-name>:assemble

# Run unit tests only
./gradlew :airbyte-integrations:connectors:<connector-name>:test

# Full build (unit tests + integration tests + assemble)
./gradlew :airbyte-integrations:connectors:<connector-name>:build

[Docker Images README] Checking what poe tasks are available To see what tasks are available for your specific connector, run

poe

on its own from the connector directory:

cd airbyte-integrations/connectors/<your-connector>
poe

The available commands vary by connector type (Java vs Python vs manifest-only). [local connector development] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai failing with Running command: docker --version Docker version 28.4.0, build d8eb465 Building Image for Connector: airbyte/source-samba:dev Downloading Dockerfile and .dockerignore templates from GitHub... Building image: airbyte/source-samba:dev-amd64 (ArchEnum.AMD64) Running command: docker buildx build --platform linux/amd64 --file /home/guillaume.fesquet@texa.frsi/TBOX/WORK/airbyte-connectors/airbyte-integrations/connectors/source-samba/build/docker/Dockerfile --label io.airbyte.version=2.1.1 --label io.airbyte.name=airbyte/source-samba --build-arg BASE_IMAGE=docker.io/airbyte/python-connector-base:4.0.2@sha256:9fdb1888c4264cf6fee473649ecb593f56f58e5d0096a87ee0b231777e2e3e73 --build-arg CONNECTOR_NAME=source-samba --build-arg EXTRA_BUILD_SCRIPT= -t airbyte/source-samba:dev-amd64 /home/guillaume.fesquet@texa.frsi/TBOX/WORK/airbyte-connectors/airbyte-integrations/connectors/source-samba Tagging image 'airbyte/source-samba:dev-amd64' as: airbyte/source-samba:dev Running command: docker tag airbyte/source-samba:dev-amd64 airbyte/source-samba:dev Running command: docker run --rm airbyte/source-samba:dev spec {"type":"LOG","log":{"level":"ERROR","message":"Failed to run airbyte command: docker run --rm airbyte/source-samba:dev spec\nAirbyteErrorTraceMessage(message='Something went wrong in the connector. See the logs for more details.', internal_message=\"Command '['docker', 'run', '--rm', 'airbyte/source-samba:dev', 'spec']' returned non-zero exit status 1.\", stack_trace=\"subprocess.CalledProcessError: Command '['docker', 'run', '--rm', 'airbyte/source-samba:dev', 'spec']' returned non-zero exit status 1.\n\", failure_type=<FailureType.system_error: 'system_error'>, stream_descriptor=None)"}} Built image failed verification: airbyte/source-samba:dev Error was:Failed to run airbyte command: docker run --rm airbyte/source-samba:dev spec AirbyteErrorTraceMessage(message='Something went wrong in the connector. See the logs for more details.', internal_message="Command '['docker', 'run', '--rm', 'airbyte/source-samba:dev', 'spec']' returned non-zero exit status 1.", stack_trace="subprocess.CalledProcessError: Command '['docker', 'run', '--rm', 'airbyte/source-samba:dev', 'spec']' returned non-zero exit status 1. ", failure_type=<FailureType.system_error: 'system_error'>, stream_descriptor=None)

The build itself succeeded, but the verification step (

docker run --rm airbyte/source-samba:dev spec

) failed with exit status 1. This means the image was built but the connector's

spec

command is crashing at runtime. To diagnose the root cause, run the spec command directly and capture the full output:

docker run --rm airbyte/source-samba:dev spec

Based on similar issues reported by other users, the most common causes are: 1. Missing Python dependencies The

spec

command fails because a required module is not installed inside the image. Check if you see a

ModuleNotFoundError

in the output. [Get Spec job failed] [Python source build] Fix: Make sure all your dependencies are listed in

setup.py

(or

pyproject.toml

). For example:

install_requires=[
    "airbyte-cdk",
    "your-dependency==x.y.z",
    # ...
]

2. Dockerfile entrypoint issues The

AIRBYTE_ENTRYPOINT

and

ENTRYPOINT

lines may be commented out or missing. [Internal Server Error] Make sure your Dockerfile (or the generated one in

build/docker/Dockerfile

) includes:

ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

3. Architecture mismatch If you're on an ARM machine (e.g., Apple M1/M2), the image built for

amd64

may behave differently. However, since the log shows

--platform linux/amd64

was explicitly used, this is less likely here. [architecture issue] Next step Run the spec command manually and share the full error output — it will pinpoint the exact failure (missing module, import error, etc.). 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai ModuleNotFoundError: No module named 'psutil'

The error

ModuleNotFoundError: No module named 'psutil'

means

psutil

is not declared as a dependency in your connector's package configuration, so it doesn't get installed inside the Docker image. Fix: Add

psutil

explicitly to your connector's

setup.py

(or

pyproject.toml

) in the

install_requires

/

MAIN_REQUIREMENTS

list. For example, in `setup.py`:

MAIN_REQUIREMENTS = [
    "airbyte-cdk",
    "psutil>=5.8,<7",
    # ... other dependencies
]

Or in `pyproject.toml`:

[tool.poetry.dependencies]
psutil = ">=5.8,<7"

This is a known pattern — connectors sometimes rely on a package being available as a transitive dependency (pulled in by another library), but it needs to be declared explicitly to guarantee it's installed. [psutil fix] [missing dependencies] After updating, rebuild your image:

airbyte-cdk image build

Then verify the spec command works:

docker run --rm airbyte/source-samba:dev spec

👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai psutil has been removed from python 3.11?

No,

psutil

has not been removed from Python 3.11 — it is a third-party library available on PyPI and is not part of the Python standard library at all. It simply needs to be explicitly declared as a dependency in your connector's package configuration so it gets installed inside the Docker image. The knowledge sources do confirm this pattern:

source-s3

had the same issue where

psutil

was being used but relied on as an undeclared transitive dependency rather than an explicit one. The fix was to add it directly: [psutil fix]

psutil = ">=5.8,<7"

So the solution remains the same as in my previous answer — explicitly declare

psutil

in your

setup.py

or

pyproject.toml

. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai how to change image tag/version

Based on the knowledge sources, there are two ways to change the image tag when building your connector: Option 1: Override the tag at build time with airbyte-cdk The

airbyte-cdk image build

command defaults to tagging the image as

dev

. You can retag it afterward using the standard Docker command:

docker image tag airbyte/source-samba:dev airbyte/source-samba:2.1.1

[Docker Images README] Option 2: Update the version in metadata.yaml and pyproject.toml For a proper versioned release, bump the version in two files: •

metadata.yaml

: update the

dockerImageTag

value •

pyproject.toml

: update the

version

value This is the standard approach when publishing a new connector version. [source-shopify README] Option 3: Use the tag in the Airbyte UI Once your image is built and loaded locally, you can change which tag Airbyte uses: 1. Go to

http://localhost:8000/

2. Navigate to Settings → Sources (or Destinations) 3. Update the version number to match your Docker image tag 4. Click Change to save [Connector Contributions] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai i did option 2, but they are not applied

Based on the knowledge sources, updating

metadata.yaml

and

pyproject.toml

is the standard approach for publishing a new version through a Pull Request to the Airbyte repository — it doesn't automatically change the tag used when building locally with

airbyte-cdk image build

. [source-shopify README] The

airbyte-cdk image build

command reads the version from

metadata.yaml

to label the image, but the knowledge sources don't provide enough detail about why your version changes aren't being picked up locally. What you can do in the meantime: After building with

airbyte-cdk image build

(which produces the

dev

tag), manually retag the image with your desired version:

docker image tag airbyte/source-samba:dev airbyte/source-samba:2.1.1

This is the most straightforward way to apply a specific version tag to your locally built image without relying on the build tooling to pick it up automatically. If you need the version to be correctly embedded in the image labels, I'd recommend double-checking that both files are saved correctly and that you're running

airbyte-cdk image build

from the connector directory after the changes. Beyond that, the knowledge sources don't contain enough detail to diagnose why the version isn't being applied. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.