https://linen.dev logo
Join Slack
Powered by
Hi <@U04SE163WC8>, how can I get my new custom con...
# replication-ask-ai
s
Hi @kapa.ai, how can I get my new custom connector to appear in my locally running instance of airbyte. I want to verify it works and have built the docker image already
k
Sure, you can add your custom connector to your locally running instance of Airbyte by following these steps: 1. In your local 
airbyte
repository, run the following command:
Copy code
bash ./run-ab-platform
2. Then, build the connector image:
Copy code
bash docker build ./airbyte-integrations/connectors/<connector-name> -t airbyte/<connector-name>:dev
The above connector image is tagged with 
dev
. You can change this to use another tag if you'd like. 3. In your browser, visit http://localhost:8000/ 4. Log in with the default user 
airbyte
and default password 
password
5. Go to 
Settings
(gear icon in lower left corner) 6. Go to 
Sources
or 
Destinations
(depending on which connector you are testing) 7. Update the version number to use your docker image tag (default is 
dev
) 8. Click 
Change
to save the changes Now when you run a sync with that connector, it will use your local docker image. [source](https://docs.airbyte.com/contributing-to-airbyte/resources/developing-locally) If you want to use your connector in your own installation of Airbyte, build the docker image for your container by running 
docker build . -t airbyte/source-python-http-example:dev
. Then, follow the instructions from the [building a Python source tutorial](https://docs.airbyte.com/connector-development/tutorials/building-a-python-source#step-11-add-the-connector-to-the-api-ui) for using the connector in the Airbyte UI, replacing the name as appropriate. [source](https://docs.airbyte.com/connector-development/tutorials/cdk-tutorial-python-http/use-connector-in-airbyte) Note: your built docker image must be accessible to the 
docker
daemon running on the Airbyte node. If you're doing this tutorial locally, these instructions are sufficient. Otherwise you may need to push your Docker image to Dockerhub.
s
I am doing the tutorial locally but I can't see the new custom source in the setting tab
@kapa.ai I'm getting this error 
Internal Server Error: Get Spec job failed.
Are there any docs or posts that go over what it means
k
A message from kapa.ai
The error 
Internal Server Error: Get Spec job failed
typically occurs when there's an issue with the Spec worker in Airbyte. The Spec worker is responsible for retrieving the specification of a connector (the inputs needed to run this connector). This error might occur if the Spec worker is unable to retrieve the specification for a particular connector. This could be due to various reasons such as network issues, incorrect configuration, or issues with the connector itself. Unfortunately, the provided extracts do not contain specific troubleshooting steps or solutions for this error. However, you can try the following general troubleshooting steps: 1. Check your network connection and ensure that the Airbyte instance is able to communicate with the connector. 2. Verify that the inputs to the connector are valid. 3. If you're using a custom connector, ensure that it's correctly implemented and the docker image is accessible. For more specific guidance, you would need to check the logs for more detailed error messages. These logs can provide more context about what's causing the Spec worker to fail. If you're still having trouble, I would recommend reaching out to the Airbyte community on their [forum](https://discuss.airbyte.io) or [Slack](https://airbytehq.slack.com) for more specific assistance.
s
@kapa.ai Ok I've gotten past those previous errors but encountered a new one. I have established a connection that uses my custom source connector. However, when I sync my stream I see that the api call is having an error. Is there a way I can see the exact api calls being made during a sync?
k
A message from kapa.ai
Yes, you can see the exact API calls being made during a sync by enabling debug logging. However, the provided extracts do not contain specific instructions on how to enable debug logging in Airbyte. Generally, you would need to set the log level to 
DEBUG
in your connector's configuration or in the Airbyte server's configuration. This would cause the connector to log detailed information about the API calls it's making, including the request URL, headers, and body. Once debug logging is enabled, you can view the logs through the Airbyte UI or directly on the server where Airbyte is running. Please note that enabling debug logging can significantly increase the amount of log data and may impact performance. Therefore, it's recommended to only enable debug logging when troubleshooting issues. For more specific instructions on how to enable debug logging in Airbyte, you may want to refer to the [Airbyte documentation](https://docs.airbyte.com/) or ask in the [Airbyte community forum](https://discuss.airbyte.io) or [Slack workspace](https://airbytehq.slack.com).
Open in Slack
PreviousNext
Powered by