Hey everyone, I am trying to learn connector deve...
# replication-troubleshooting
j
Hey everyone, I am trying to learn connector development in Python. I have completed the step 3 of the document. But stuck at step 4 when checking connection. This is the document I am following: https://airbyte.gitbook.io/airbyte/connector-development/tutorials/cdk-tutorial-python-http/4-connection-checking When I run the command:
python main.py check --config sample_files/config.json
I get the error saying:
Copy code
{
  "type": "CONNECTION_STATUS",
  "connectionStatus": {
    "status": "FAILED",
    "message": "Config validation error: 'api_key' is a required property"
  }
}
In the document it is mentioned to add api key in `secrets/config.json`:
While developing, we recommend storing configs which contain secrets in
secrets/config.json
because the
secrets
directory is gitignored by default.
I generated the api key from the exchange rates api website, added it in the
secrets/config.json
file, and build the image using this command:
docker build . -t airbyte/source-python-http-example:dev
. But the error still persists. I am contributing towards documentation. It is important for me to understand how connectors work. Here is the Github link of the partially completed example I was trying: https://github.com/Jagrutiti/airbyte/tree/source-python-http-example-connector/airbyte-integrations/connectors/source-python-http-example
e
Hi. the param that is storing the secrets/config.json is "config" so the only key here is to know how did you map the config object and how did you try to work with it since the error is saying that it cant find it in that file
j
Hey @Eli Sigal I am sharing a screenshot with code and file path. File path:
airbyte-integrations\connectors\source-python-http-example\secrets\config.json
File content:
Copy code
{
  "api_key": "PrXYJ4czaWGTYdv7qo8tOxUvKV5atcYJ"
}
In the document there is no instructions to add api key.
e
so based on the file - the only thing that could be wrong is that step 4 cant find the api key. you need to pass it via config['api_key']
j
you need to pass it via config['api_key']
Could you give me the exact command here?
Where should I use
config['api_key']
?
e
Do you have some thing like this? this is from the docs:
Copy code
class ExchangeRates(HttpStream):
    url_base = "<http://api.exchangeratesapi.io/>"

    primary_key = None

    def __init__(self, config: Mapping[str, Any], **kwargs):
        super().__init__()
        self.base = config['base']
        self.access_key = config['access_key']
j
This piece of code is in the following files: 1. `airbyte\airbyte-integrations\connectors\source-python-http-tutorial\source_python_http_tutorial\source.py`| 2.
docs\connector-development\tutorials\cdk-tutorial-python-http\declare-schema.md
3.
docs\connector-development\tutorials\cdk-tutorial-python-http\read-data.md
b
The above is from the tutorial but do you have that in your code? I see all this in the [tutorial ](https://github.com/Jagrutiti/airbyte/blob/source-python-http-example-connector/air[…]urce-python-http-tutorial/source_python_http_tutorial/source.py) but not in your code [here](https://github.com/Jagrutiti/airbyte/blob/source-python-http-example-connector/air[…]source-python-http-example/source_python_http_example/source.py) Also in the example as Eli says, it shows this:
self.access_key = config['access_key']
but you should probably change that to
self.access_key = config['api_key']
?
✅ 1
👀 1
j
Hey @Eli Sigal, @Bish Sinha Thank you for your help. I have added the
api_key
here: https://github.com/Jagrutiti/airbyte/blob/source-python-http-example-connector/air[…]source-python-http-example/source_python_http_example/source.py Now when I run the command:
python main.py check --config sample_files/invalid_config.json
I get an error saying:
Copy code
Traceback (most recent call last):
  File "airbyte\airbyte-integrations\connectors\source-python-http-example\main.py", line 12, in <module>
    source = SourcePythonHttpExample()
TypeError: SourcePythonHttpExample.__init__() missing 1 required positional argument: 'config'
What shall I pass as an argument here? I did not find any instructions for the same.
e
the ___init___ doesnt have config param you need to pass this along
j
Hey @Eli Sigal You are right. What parameter should I pass? Where should I import it from?
e
config is not imported since it is a parameter that is passes along and this parameter has all the data that you input in the UI or in secrets/config.json I would start by copy paste the entire logic from scratch and then learn the OO logic and read the docs. what you are asking is in the docs and how airbyte works. It is quite difficult to explain where exactly put what since it really depends on the entire structure of the code