I am trying to create a new connector using the UI...
# help-connector-development
b
I am trying to create a new connector using the UI, but the documentation on what to do with the exported yaml seems to be either incorrect or outdated or I am just confused. I am following Connector Builder UI | Airbyte Documentation 1. I start to have issues in the “Exporting the YAML” section, when it talks about overwrite the contents. It references a yaml file with the name of the connector but I don’t get a file like that I get a manifest.yaml. I used the connector generator talked about in this doc: Step 1: Generate the source connector project locally | Airbyte Documentation. I am able to follow step 1 and 2 without issues but I get lost in step 3 when it talks about a yaml file with same name as the connector, again I just have a manifest.yaml where the named file should be. If I take the content of the yaml from the UI builder and put it in manifest.yaml I get a successful connection when I run main.py check. In step 4 it talks about creating schema files, but doesn’t reference how to use the schema built in the UI builder. Is there any documentation for the UI Builder connector process, or anything I am missing?
1
s
@Branson Buchanan i think you’ve discovered out of date docs 😢 we’ll fix them! But there is only ever 1 YAML, the
manifest.yaml
what version of Airbyte OSS are you running?
b
0.40.32
I figured it was outdated since stuff wasn’t lining up. Makes me feel better that I wasn’t just unable to follow the instructions
s
@Branson Buchanan you don’t need to manage any separate files for schema. You can import the schema in the UI
basically once you run “test” the UI detects the schema for you and you can click import. Once you do that it shows up in the “declared schema” tab. Then when you export the schema is bundled into the manifest yaml
b
After I export the yaml from the builder UI. Do I use the template generator and select configuration based source, then replace the content of /airbyte/airbyte-integrations/connectors/source-connector-name/source_connector_name/manifest.yaml with content of the exported yaml?
s
yup!
b
Then do I install dependencies with: python -m venv .venv source .venv/bin/activate pip install -r requirements.txt After the dependencies install do I build the docker with: docker build . -t airbyte/source-<name>:<version>
s
@Branson Buchanan you can actually run
docker build
without the other commands. So if your goal is just to build a docker image you can run
docker build
and nothing else the other commands e.g
pip install
are only needed to iterate locally i.e if you want to run
python …
commands
b
I built the connector using “docker build . -t airbyte/source-keap-infusionsoft:0.1.0” then tried to add it on the sources page under settings using the new connectors button. Here are the settings I used for new connector: Display name: Keap-Infusionsoft Docker repo name: airbyte/source-keap-infusionsoft Docker image tag: 0.1.0 After I click add I get an error that says Fetching connector failed. Try again.
s
are you open to sharing your YAML file?
b
Sure
infusionsoft (1).yaml
s
@Branson Buchanan I think this is a bug in the UI. To fix it: 1. edit line
839
in the YAML you shared so that
documentationUrl
is a real URL like
<https://docs.airbyte.com>
or something 2. run docker build again 3. in the settings --> sources screen, “reload” the connector image by clicking the “change” button next to the connector you added (see screenshot)
recorded this bug here. Thank you for raising it! I also raised a PR to fix the issue with file namings in docs like you pointed out
b
Thanks Sherif it has been working just fine now
s
@Sherif Nada I am getting the same error as below :
I have deployed airbyte on aws ec-2 instance, one for development and another for production. I have created one custom connector along with testing in dev instance. Uploaded that image on dev interface and it worked. I have pushed the same image to aws ecr repository and pulled the same in production instance and on instance when i try to create new connector i'm getting above error as image. Let me know if im missing something. dev version - 0.43.0, prod version - 0.40.32