<@U04SE163WC8> is it possible to add custom loggin...
# replication-ask-ai
s
@kapa.ai is it possible to add custom logging in to airbyte connectors deployed locally in order to help with debugging?
k
Yes, it is possible to add custom logging to Airbyte connectors for debugging purposes when they are deployed locally. You can enable
LOG_CONNECTOR_MESSAGES=true
to log the messages the Airbyte platform receives from the source and destination. This can be done from your local
airbyte-platform
repository. Here is an example of how to use it:
Copy code
LOG_CONNECTOR_MESSAGES=true VERSION=dev docker compose up
This command will start the Airbyte platform with logging enabled for connector messages. For more details, you can refer to the [Airbyte documentation](https://docs.airbyte.com/contributing-to-airbyte/resources/developing-locally).
s
@kapa.ai Another question, is there a way to configure a stream to use a post call instead of get in the python cdk?
k
A message from kapa.ai
Yes, you can configure a stream to use a POST call instead of GET in the Python CDK by overriding the
request_method
property in your stream class. By default, it uses the GET method, but you can change it to POST or any other HTTP method as per your requirements. Here is a basic example of how you can do it:
Copy code
python class MyStream(HttpStream):     ...     @property     def request_method(self) -> str:         """         Override this method if your HTTP requests use a method other than GET.         """         return "POST"     ...
In the
MyStream
class, the
request_method
property is overridden to return "POST", which means that this stream will use the POST method for HTTP requests. For more details, you can refer to the [Airbyte documentation](https://docs.airbyte.com/connector-development/cdk-python/http-streams).
s
@kapa.ai, the endpoint I'm using in my stream is returning a 400 error when I try to perform a sync. Is there any way to see the exact payload being sent to my source so I can see what is wrong
k
A message from kapa.ai
Yes, you can enable detailed logging to see the exact payload being sent to your source. This can be done by setting the
LOG_LEVEL
environment variable to
DEBUG
when starting Airbyte. This will log all HTTP requests and responses, including their bodies, which can help you identify what's causing the 400 error. Here is an example of how to use it:
Copy code
bash LOG_LEVEL=DEBUG VERSION=dev docker compose up
This command will start Airbyte with debug-level logging enabled. Please note that enabling debug logging will significantly increase the amount of log output, so it's recommended to use it only for debugging purposes. For more details, you can refer to the [Airbyte documentation](https://docs.airbyte.com/contributing-to-airbyte/resources/developing-locally).