I would like to see what kind of requests Airbyte ...
# help-connector-development
d
I would like to see what kind of requests Airbyte is sending. Usually I can see it in the testing module but sometimes I can’t. Is there an alternative way to see the requests? I am getting
400 Client Error: Bad Request for url
messages and I would like to investigate what is going wrong.
k
A message from kapa.ai
New message text here
d
I corrected my message. Could you have another look in the knowledge base? @kapa.ai
k
A message from kapa.ai
Hmm, I don't know enough to give you a confident answer yet. Please provide more information about your issue or question related to Airbyte, and I'll do my best to help you based on the information extracts I have. In the meantime, you can also check the [Airbyte documentation](https://docs.airbyte.com) for more information.
d
@kapa.ai I using oauth for authentication in the connector builder ui but I can’t seem to set it up correctly. I would like to investigate what is going wrong that is why I am interested in looking at the requests airbyte is ending out. Can see these requests somewherE?
k
A message from kapa.ai
Yes, you can view the outgoing HTTP requests made by the connector in the Connector Builder UI's testing panel. Here's how to access it: 1. Open the Connector Builder UI. 2. In the testing panel, you'll find several tabs. 3. Click on the Request tab. This tab displays the outgoing HTTP request made by the connector to retrieve the selected page of data, which can be useful for debugging. For more information on how to use the Connector Builder UI and its testing panel, please refer to the [Connector Builder UI documentation](https://docs.airbyte.com/connector-development/config-based/connector-builder-ui#connector-builder-testing-panel).
d
@kapa.ai Is there an alternative way to see these requests? Since I am not seeing this request tab in every occasion
k
A message from kapa.ai
I apologize for the confusion earlier. Based on the information extracts I have, you can view the outgoing requests and their associated responses by running the
read
operation with the
--debug
flag. Here's an example command:
Copy code
bash python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --debug
This command will print out debug information, including the outgoing request and its associated response, which can help you investigate the issue with your OAuth authentication setup. For more details, refer to the [reading data tutorial](https://docs.airbyte.com/connector-development/config-based/tutorial/reading-data#part-1).
d
@kapa.ai in which directory should I execute this command?
k
A message from kapa.ai
You should execute the command in the directory where your connector's
main.py
file is located. This is typically the root directory of your connector's source code. For example, if you're working on a custom connector, the directory would be something like
airbyte-integrations/connectors/source-your-connector-name
. Here's the command again for reference:
Copy code
bash python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --debug
Make sure to replace
source-your-connector-name
with the actual name of your connector.
d
@kapa.ai How can I find the correct directory for the connector I am developing. I can’t seem to find this directory
k
A message from kapa.ai
To find the correct directory for the connector you are developing, follow these steps: 1. First, locate the root directory of the Airbyte project on your local machine. This is the directory where you cloned the Airbyte repository. 2. Next, navigate to the
airbyte-integrations
directory within the Airbyte project root:
Copy code
cd airbyte-integrations
3. Inside the
airbyte-integrations
directory, you'll find the
connectors
directory. Navigate to it:
Copy code
cd connectors
4. In the
connectors
directory, you should see multiple subdirectories with names like
source-connector-name
or
destination-connector-name
. Find the one that corresponds to the connector you are developing. For example, if you're working on a custom source connector named
my-custom-source
, the directory would be
source-my-custom-source
. Navigate to that directory:
Copy code
cd source-my-custom-source
Now you are in the correct directory for the connector you are developing. You can execute the command mentioned earlier in this directory:
Copy code
bash python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --debug
Make sure to replace the
config.json
and
configured_catalog.json
file paths with the correct paths for your connector's configuration and catalog files.
m
@Dieter De Beuckeleer are you using the connector builder UI or the code version?
d
the connector builder ui
m
@Joe Reuter (Airbyte) any ideas here? 😄
d
just fyi, I just updated to the latest version. Before I updated it, it worked. I am not sure about my exact configuration that I had.
j
@Dieter De Beuckeleer can you share a screenshot of the testing panel? What is it showing? In general we had an issue for some errors being swallowed along the way but it should be fixed in the latest version. Alternatively you could check the log messages of the connector builder server docker container
d
@Joe Reuter here you go!
j
Ah, got it, the supplementary requests happening to do auth etc. are not shown in the testing panel - I created an issue for this a while back here: https://github.com/airbytehq/airbyte/issues/21014
I will make sure this issue is considered for the next development iteration
Could you show your authentication config?
I can try to help getting it working
d
Much appreciated! And I guess I can’t find this information in the logs of the docker container? I will send you the config
j
No, I think it's also not logged in the docker container
hmm, actually reading through https://developers.zoom.us/docs/api/rest/using-zoom-apis/ (I suspect this is what you are working with) it seems like zoom is only supporting the client credentials flow which is not supported in the low code cdk today: https://github.com/airbytehq/airbyte/issues/23450
d
This morning I got it to work 🙂 But after the update my configuration got lost since it did not work with the new connector builder. This is my current config
j
Not sure whether it will help, but could you try adding an additional request param
refresh_token
{{ None }}
?
d
like this?
j
yeah
ah, wait, I just checked the code and this won't work
So for now you either switch to the yaml view where you can set the
grant_type
on the authenticator, roll back to the previous version of Airbyte, or use another authenticaton type. Sorry for the hassle. I will raise this with the team today so we can fix it quickly
d
Okay, I will give that a go, but thank you for the help!