Aviraj Singh Gour
02/06/2025, 10:04 AMkapa.ai
02/06/2025, 10:04 AMairbyte-ci connectors --name=$connector test
for testing connectors in your CI checks when creating a GitHub PR, there are a few requirements you need to meet. Based on the provided knowledge sources, here's what you need to know:
1. Install airbyte-ci
CLI:
You need to have the airbyte-ci
CLI tool installed. This tool is used for building and testing connectors. You can install it by running make
from the Airbyte repository root, as mentioned in the [Connector Development guide](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#setting-up-standard-acceptance-tests-for-your-connector).
2. Connector Configuration:
Ensure that your connector has a proper configuration file. For most connectors, this file is named acceptance-test-config.yml
and should be located in the root folder of your connector. This file defines the test suites and configurations for your connector [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#setting-up-standard-acceptance-tests-for-your-connector).
3. Secrets Management:
If your connector requires credentials for testing, you need to manage these secrets properly. For local testing, you can provide connector configuration as a config.json
file in a .secrets
folder in the connector code directory. For CI, you may need to add credentials to GSM (Google Secret Manager) if new credentials are required [Contribute a New Connector](https://docs.airbyte.com/contributing-to-airbyte/submit-new-connector).
4. Python Version:
Make sure you're using Python version 3.10, as mentioned in one of the knowledge sources. Using an incorrect Python version may cause the build to crash [Github Issue](https://github.com/airbytehq/airbyte/issues/39378).
5. Docker:
The test suite runs against the connector's Docker image, so ensure that Docker is available in your CI environment [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#architecture-of-standard-tests).
6. Connector Metadata:
Ensure that your connector has a proper metadata.yaml
file. This file should declare the test suites that will run in CI, including unitTests
, integrationTests
, and acceptanceTests
[Connector Metadata.yaml File](https://docs.airbyte.com/connector-development/connector-metadata-file#the-connectortestsuitesoptions-section).
7. Icon:
Each connector must have an icon available at the root of the connector code directory. It must be an SVG file named icon.svg
and must be a square [QA Checks](https://docs.airbyte.com/contributing-to-airbyte/resources/qa-checks#-assets).kapa.ai
02/06/2025, 10:04 AMairbyte-ci connectors --name=$connector test
for testing connectors in your CI checks when creating a GitHub PR, there are a few requirements you need to meet. Based on the provided knowledge sources, here's what you need to know: 1. Install airbyte-ci
CLI: You need to have the airbyte-ci
CLI tool installed. This tool is used for building and testing connectors. You can install it by running make
from the Airbyte repository root, as mentioned in the [Connector Development guide](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#setting-up-standard-acceptance-tests-for-your-connector). 2. Connector Configuration: Ensure that your connector has a proper configuration file. For most connectors, this file is named acceptance-test-config.yml
and should be located in the root folder of your connector. This file defines the test suites and configurations for your connector [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#setting-up-standard-acceptance-tests-for-your-connector). 3. Secrets Management: If your connector requires credentials for testing, you need to manage these secrets properly. For local testing, you can provide connector configuration as a config.json
file in a .secrets
folder in the connector code directory. For CI, you may need to add credentials to GSM (Google Secret Manager) if new credentials are required [Contribute a New Connector](https://docs.airbyte.com/contributing-to-airbyte/submit-new-connector). 4. Python Version: Make sure you're using Python version 3.10, as mentioned in one of the knowledge sources. Using an incorrect Python version may cause the build to crash [Github Issue](https://github.com/airbytehq/airbyte/issues/39378). 5. Docker: The test suite runs against the connector's Docker image, so ensure that Docker is available in your CI environment [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference#architecture-of-standard-tests). 6. Connector Metadata: Ensure that your connector has a proper metadata.yaml
file. This file should declare the test suites that will run in CI, including unitTests
, integrationTests
, and acceptanceTests
[Connector Metadata.yaml File](https://docs.airbyte.com/connector-development/connector-metadata-file#the-connectortestsuitesoptions-section). 7. Icon: Each connector must have an icon available at the root of the connector code directory. It must be an SVG file named icon.svg
and must be a square [QA Checks](https://docs.airbyte.com/contributing-to-airbyte/resources/qa-checks#-assets). Remember, the exact requirements may vary depending on the specific connector you're working with and your CI setup. Always refer to the most up-to-date documentation and your specific connector's README for the most accurate information.Aviraj Singh Gour
02/06/2025, 10:21 AMkapa.ai
02/06/2025, 10:21 AMkapa.ai
02/06/2025, 10:22 AMbash git clone <https://github.com/airbytehq/airbyte.git> cd airbyte
2. Install airbyte-ci using the following commands: bash make tools.airbyte-ci.install make tools.airbyte-ci-dev.install
These commands will install airbyte-ci and its development dependencies [Github Issue](https://github.com/airbytehq/airbyte/discussions/48706). 3. If you encounter any issues with missing dependencies, you may need to install additional Python packages. For example: bash pip install airbyte langchain langchain-openai qdrant-client python-dotenv langchainhub
This command installs some common libraries used with Airbyte [Airbyte Tutorial](https://airbyte.com/tutorials/building-knowledge-management-system-pyairbyte-vector-databases). 4. Ensure you have Docker and Docker Compose installed on your system, as they are prerequisites for running Airbyte and airbyte-ci [YouTube Tutorial](). 5. If you're working on a specific connector, you may need to install additional dependencies. Navigate to the connector's directory and use Poetry to install its dependencies: bash cd airbyte-integrations/connectors/your-connector-name poetry install
This step is particularly relevant for Python connectors [Connector Development Documentation](https://docs.airbyte.com/connector-development/testing-connectors/). Remember that the exact dependencies may vary depending on your specific use case and the connectors you're working with. Always refer to the README file in the connector's directory or the official Airbyte documentation for the most up-to-date and specific instructions.