<@U04S8PLJWMT>, how can I run the source-snowflake...
# ask-ai
n
@kapa.ai, how can I run the source-snowflake connector locally? this is a java based. Im attempting to run discovery through airbyte-ci if possible
k
To run the source-snowflake connector locally (which is Java-based), you should follow these general steps: 1. Build the Connector: After making any changes or if you want to use a local version, build the connector using Gradle. For example:
Copy code
./gradlew :airbyte-integrations:connectors:source-snowflake:build
This will build the connector and create a Docker image, typically tagged with
dev
by default. 2. Run Airbyte Locally: Start Airbyte locally using Docker Compose:
Copy code
docker compose up
This will bring up the Airbyte UI at [http://localhost:8000](http://localhost:8000). 3. Add the Custom Connector in the UI: In the Airbyte UI, go to Settings → Sources → + New connector. Here, you can add your custom-built connector by specifying the local Docker image you just built. You do not need to publish the image to a registry; as long as it exists locally, Airbyte can use it. 4. Configure and Test: Once added, you can create a new source using your custom Snowflake connector, enter the required configuration (like credentials and connection details), and test the connection and sync as you would with any other connector. This process is confirmed by community discussions and documentation on local connector development and testing in Airbyte. The same approach applies to any Java-based connector, including source-snowflake. For more details on local connector development, see the official documentation: [Developing Connectors Locally](https://docs.airbyte.com/platform/connector-development/local-connector-development) and [Forum discussion on testing local connectors](https://discuss.airbyte.io/t/1812)<source>https://docs.airbyte.com/platform/connector-development/local-connector-development&lt;/source&gt;&lt;source&gt;https://discuss.airbyte.io/t/1812&lt;/source>. If you need to run Gradle tasks, you can also use the
poe
tool for convenience, as described in the documentation: > From within a connector directory you can run
poe gradle tasks
for a list of Gradle tasks that apply to the connector and
poe gradle TASK_NAME
to run a given Gradle task for that connector. [Gradle Tooling](https://docs.airbyte.com/platform/connector-development/local-connector-development#gradle)<source>https://docs.airbyte.com/platform/connector-development/local-connector-development#gradle&lt;/source>. If you need more specific steps or run into issues, please provide more details!
A message from kapa.ai
To run the source-snowflake connector locally and perform discovery using
airbyte-ci
, follow these steps: 1. Build the Connector Image Make sure you have
airbyte-ci
installed (note:
airbyte-ci
is deprecated, but still referenced for these workflows). From the Airbyte repo root, run:
Copy code
airbyte-ci connectors --name source-snowflake build
This will build the Docker image
airbyte/source-snowflake:dev
and make it available on your local Docker host. 2. Run Acceptance Tests (including discovery) You can use
airbyte-ci
to run the acceptance tests, which include the discovery step:
Copy code
airbyte-ci connectors --name=source-snowflake test
This will execute the full suite of tests, including the
discover
command, against your local connector image. > The acceptance tests will require a valid
config.json
with your Snowflake credentials, typically placed in a
.secrets
folder in the connector directory. [Testing Connectors](https://docs.airbyte.com/platform/connector-development/testing-connectors) [Acceptance Tests Reference](https://docs.airbyte.com/platform/connector-development/testing-connectors/connector-acceptance-tests-reference#setting-up-standard-acceptance-tests-for-your-connector) 3. Note on airbyte-ci Deprecation The
airbyte-ci
tool is deprecated and will be phased out. Most commands have equivalents in Poe, but for now, the above process is still valid for running connector tests and discovery locally for Java connectors like source-snowflake [airbyte-ci (deprecated)](https://docs.airbyte.com/platform/connector-development/local-connector-development#airbyte-ci-deprecated). If you want to run only the discovery step, you may need to run the connector container directly with Docker, as there is no documented
airbyte-ci
command for running just discovery outside the test suite:
Copy code
docker run airbyte/source-snowflake:dev discover --config /path/to/config.json
But this is not explicitly documented for Java connectors in the provided sources. Summary: - Build with
airbyte-ci connectors --name source-snowflake build
- Run tests (including discovery) with
airbyte-ci connectors --name=source-snowflake test
- Place your config in
.secrets/config.json
as required by the test suite If you need more granular control (e.g., running only discovery), the knowledge sources do not provide a direct
airbyte-ci
command for that, and you may need to use Docker directly.