How do I build a connector for a source that uses self-signed certificates? Our current connector in builder receives a response when testing the connector, but when setting up a new connection in Connections it runs into an SSLError (certificate verify failed). The source is Prometheus and is set up on our machine, with our own certificates

To build a connector for a source that uses self-signed certificates, you need to ensure that the custom certificate authority (CA) is properly configured within the Docker container of your custom connector. Here are the steps you can follow based on the information from the knowledge sources: 1. Add the SSL Certificate to the Dockerfile: You need to add the SSL certificate inside the custom connector Dockerfile. This involves copying the certificate to the appropriate directory and updating the CA certificates. Here is an example of how you might modify your Dockerfile:

dockerfile    FROM airbyte/source-prometheus:latest     # Copy your custom CA certificate to the container    COPY your_custom_ca.crt /usr/local/share/ca-certificates/your_custom_ca.crt     # Update the CA certificates    RUN update-ca-certificates

This will ensure that your custom CA certificate is recognized by the system within the Docker container. 2. Handle SSL Verification in Code: If you are using a library like

requests

in Python, you might need to specify the path to your custom CA certificate in your code. Here is an example:

python    import requests     response = requests.get('<https://your-prometheus-url>', verify='/path/to/your_custom_ca.crt')

3. Testing the Connector: After making these changes, rebuild your Docker image and test the connector again to ensure that it can successfully connect to the Prometheus source using the self-signed certificate. 4. Patching the Docker Image: If you encounter issues with the connector in the Airbyte Connections setup, you might need to patch the Docker image to set the trust store as expected. This is a workaround mentioned in the forum post: > "If you want to overcome this problem at the moment the only workaround would be patching our destination ES docker image to set the trust store as you expect." ([source](https://discuss.airbyte.io/t/983)) By following these steps, you should be able to build and configure a connector that can handle self-signed certificates for your Prometheus source. For more detailed information, you can refer to the related discussions and issues: - [GitHub Discussion on SSL certificate verify failed](https://github.com/airbytehq/airbyte/discussions/33498) - [GitHub Issue on Source ElasticSearch: support for setting a custom certificate authority](https://github.com/airbytehq/airbyte/issues/26413) - [Forum Post on Self Signed Cert / Internal CA and Connectors](https://discuss.airbyte.io/t/983) If you need further assistance, feel free to ask for more help on the Airbyte community forums or GitHub discussions.

@kapa.ai In what Airbyte docker container are the custom connector dockerfiles?

Hmm, I don't know enough to give you a confident answer yet. However, you can refer to the [Airbyte documentation on custom connectors](https://docs.airbyte.com/operator-guides/using-custom-connectors) for more detailed information. If you have specific questions about the setup or usage, feel free to ask!

@kapa.ai And how would I do this for the low-code builder method? There is no airbyte/source-prometheus currently available. Prometheus is not a source, but a connector made in Builder

A message from kapa.ai