https://linen.dev logo
Join Slack
Powered by
<@U04SE163WC8> How to add a new connector stored i...
# ask-ai
s
@kapa.ai How to add a new connector stored in our internal docker registry "docker.registry/pool/main/airbyte/source-lnd-rest:dev" ?
k
To add a new connector stored in your internal Docker registry, follow these steps: 1. Ensure Your Airbyte Instance is Authenticated to Your Private Docker Registry: Make sure that your Airbyte instance can access your private Docker registry. This typically involves configuring Docker to use the appropriate credentials for your registry. 2. Pull the Connector Image to Validate Access: You can pull the connector image from your private registry to validate that the previous steps were successful. Run the following command on your Airbyte instance:
Copy code
bash    docker pull docker.registry/pool/main/airbyte/source-lnd-rest:dev
3. Add the Connector via the Airbyte UI: - Visit your Airbyte instance in your browser (e.g., http://localhost:8000). - Log in with your credentials. - Click on 
Settings
in the left-hand sidebar. - Navigate to 
Sources
or 
Destinations
depending on your connector. - Click on 
Add a new Docker connector
. - Fill in the following details: - Connector display name: Give your custom connector a name (e.g., "LND REST Source"). - Docker full image name: 
docker.registry/pool/main/airbyte/source-lnd-rest
- Docker image tag: 
dev
- Connector documentation URL: (Optional) Add a link to the connector's documentation if available. - Click 
Add
to save the configuration. 4. Select Your New Connector When Setting Up a New Connection: You can now select your new connector when setting up a new connection in Airbyte. For more detailed instructions, you can refer to the [Airbyte documentation on using custom Docker connectors](https://docs.airbyte.com/operator-guides/using-custom-connectors#4-use-your-custom-docker-connector-in-airbyte). If you encounter any issues with authentication to your private Docker registry, ensure that your Docker configuration is correctly set up to use the necessary credentials.
s
@kapa.ai Thanks. How to authenticate against our local registry? I have a valid user/password credentials for docker l ogin, but which Airbyte docker container should use it?
k
A message from kapa.ai
To authenticate against your local Docker registry using valid user/password credentials, you need to ensure that both your local/CI environment and your Airbyte instance can access the registry. Here’s how you can do it: ### Step 1: Authenticate Your Local/CI Environment Use the 
docker login
command to authenticate your local or CI environment where you build your connector image. Run the following command:
Copy code
bash docker login <your-docker-registry-url>
You will be prompted to enter your username and password. ### Step 2: Authenticate Your Airbyte Instance For Docker-compose Airbyte deployments, you need to ensure that the Airbyte containers can pull images from your private Docker registry. Here’s how you can do it: 1. Create a Docker Configuration File: Create a 
.docker/config.json
file with your Docker credentials. You can generate this file by running 
docker login
on your local machine and then copying the generated 
config.json
file. 2. Mount the Docker Configuration File in Airbyte Containers: Modify your 
docker-compose.yaml
file to mount the Docker configuration file into the Airbyte containers. Here’s an example of how to do this:
Copy code
yaml    version: '3'    services:      airbyte-server:        image: airbyte/server:latest        volumes:          - type: bind            source: /path/to/your/.docker/config.json            target: /root/.docker/config.json      airbyte-scheduler:        image: airbyte/scheduler:latest        volumes:          - type: bind            source: /path/to/your/.docker/config.json            target: /root/.docker/config.json      airbyte-worker:        image: airbyte/worker:latest        volumes:          - type: bind            source: /path/to/your/.docker/config.json            target: /root/.docker/config.json
### Step 3: Restart Airbyte After modifying the 
docker-compose.yaml
file, restart your Airbyte instance to apply the changes:
Copy code
bash docker-compose down docker-compose up -d
### Additional Information For more details on setting up authentication for your Docker registry, you can refer to the [Airbyte documentation on using custom connectors](https://docs.airbyte.com/operator-guides/using-custom-connectors#self-hosted---open-source-docker-registry). By following these steps, you should be able to authenticate your Airbyte instance against your local Docker registry using your user/password credentials.
s
@kapa.ai Mapping to /root is probably not right when using default airbyte user in containers... ?
k
A message from kapa.ai
New message text here
Open in Slack
PreviousNext
Powered by