<@U04S8PLJWMT> using connection builder, component...
# replication-ask-ai
b
@kapa.ai using connection builder, component.py, how do i call aws, i cant seem to include boto3
k
Thinking completed I'm sorry, but the provided knowledge sources do not contain specific information or instructions on how to install or import third-party libraries like
boto3
inside the Connector Builder's custom components (
components.py
). Here is what the available documentation confirms regarding custom components and the environment: 1. Custom Components in Connector Builder: ā—¦ Custom components run within the Connector Builder execution environment (specifically the
manifest-server
or
workload-launcher
pods in Core/local deployments) [Unable to Enable Custom Components for Connector Builder; Custom components for the Connector Builder]. ā—¦ Only dependencies already pre-installed in those underlying Airbyte server images are available directly for import; there is currently no documented mechanism in the Connector Builder UI to install additional pip packages like
boto3
on the fly. 1. Alternatives for handling external dependencies like
boto3
: ā—¦ Local Connector Development with Python CDK: If your connector requires custom Python packages such as
boto3
, you can develop the connector locally using the Python CDK or Low-Code CDK outside of the web UI. You can define a custom
setup.py
/
requirements.txt
with
boto3
as a dependency and reference your custom component class in your connector package [Singer connector dependencies conflict with Airbyte CDK; Custom Authenticator and CustomRequestor for API authentication in Low Code Source Connectors]. ā—¦ Custom Docker Image: Package your custom connector code and its dependencies into a custom Docker image and add it as a Custom Connector in your Airbyte workspace [Custom or New Connector]. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
b
what exactly do i have to do to get the local python cdk thingy?
@kapa.ai what exactly do i have to do to get the local python cdk thingy?
k
Thinking completed To set up and use the local Python CDK tooling, follow these steps based on the Airbyte documentation: 1. Install Tooling Prerequisites Airbyte uses
uv
for package management and CLI tools, along with
poe
and
docker
for running connector tasks and containers [Developing Connectors Locally]. 1. Install
uv
(using Homebrew on macOS):brew install uv 2. Install
poe
(Poe the Poet)
:brew tap nat-n/poethepoet brew install nat-n/poethepoet/poethepoet 3. Install Docker: Ensure Docker Desktop or an alternative runtime like Orbstack is installed and running [Developing Connectors Locally]. 2. Install the airbyte-cdk CLI Install the Airbyte CDK command-line tool using
uv
[Developing Connectors Locally]:
Copy code
uv tool install --upgrade 'airbyte-cdk[dev]'
You can verify the installation by checking available commands:
Copy code
airbyte-cdk --help
3. Generate a Connector Template To scaffold a new Python connector directory, run the Airbyte code generator [Documentation for old Python CDK and Source Connector for Async HTTP API with Incremental Sync Modes; How to build a new Webflow connector]:
Copy code
cd airbyte-integrations/connector-templates/generator
./generate.sh
• When prompted, select the Python HTTP API Source option [How to build a new Webflow connector]. • Provide a name for your connector. The generator creates a connector directory under
airbyte-integrations/connectors/source-<connector-name>
[Documentation for old Python CDK and Source Connector for Async HTTP API with Incremental Sync Modes; How to build a new Webflow connector]. 4. Set Up the Local Python Virtual Environment Navigate into your newly generated connector directory and set up a virtual environment [How to build a new Webflow connector]:
Copy code
cd airbyte-integrations/connectors/source-<connector-name>

# Create and activate the virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies (including the Python CDK)
pip install -r requirements.txt
_(Note: You can also use
airbyte_cdk
directly as a dependency in your own Python project if you prefer not to use the scaffolding script [Connector Development Kit].)_ šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.