@kapa.ai We are getting a huge spike in logging events that we are trying to reduce. We have set the logging level to error but there are still spikes in the range of thousands of records flooding our elastic logs which is causing issues there. An example error message starts: Caused by: io.airbyte.api.client.invoker.generated.ApiException: getWorkspaceByConnectionId call failed with: 404. The UI and ingestions are working fine

Based on the information provided in the knowledge sources, I can offer some insights and suggestions regarding your logging issue with Airbyte: 1. Debug Logs: It's recommended to turn off DEBUG logs for any non-testing use of Self-Managed Airbyte. Failing to do so while running at-scale syncs may result in the

server

pod being overloaded, preventing most of the deployment from operating normally. [Scaling Airbyte After Installation](https://docs.airbyte.com/enterprise-setup/scaling-airbyte#debug-logs) 2. JDBC Source Logging: There's an ongoing issue with SQL integrations (MySQL, Postgres, BigQuery, etc.) where many log entries are generated at the INFO level, which can result in thousands of lines for large schemas. A proposal has been made to change these log levels from INFO to DEBUG to reduce log volume. [Github Issue: Sources JDBC: Change logging from INFO to DEBUG](https://github.com/airbytehq/airbyte/issues/24983) 3. API Exceptions: The error message you're seeing (ApiException: getWorkspaceByConnectionId call failed with: 404) suggests that there might be an issue with the API calls. This could be related to a known issue where Airbyte doesn't respond via UI or API, possibly due to rate limiting or resource exhaustion. [Github Issue: Airbyte not responding via UI or API](https://github.com/airbytehq/airbyte/issues/46554) To address your specific situation: 1. Ensure that DEBUG logs are turned off in your production environment. 2. Check if you're using any JDBC sources, as these might be contributing to the log volume. 3. Monitor your system resources and API call rates to ensure you're not hitting any limits. If the issue persists after implementing these changes, you may need to investigate further or consider reaching out to Airbyte support for more specific assistance. The fact that your UI and ingestions are working fine suggests that this might be primarily a logging issue rather than a functional problem with Airbyte itself.

@kapa.ai, what about the message Could not find configuration for STANDARD_SYNC

A message from kapa.ai

Based on the knowledge sources provided, there isn't specific information available about the error message "Could not find configuration for STANDARD_SYNC". However, in general, this error might occur if there's an issue with the configuration of your data sync process in Airbyte. It's possible that the configuration for the standard sync process is missing or not correctly set up. Here are some general steps you can take to troubleshoot: 1. Check your source and destination configurations in Airbyte. Make sure they are correctly set up and saved. 2. If you recently upgraded your Airbyte or connector versions, there might be changes in the configuration parameters. For example, in both [MySQL](https://docs.airbyte.com/integrations/sources/mysql/mysql-troubleshooting#upgrading-from-068-and-older-versions-to-069-and-later-versions) and [MSSQL](https://docs.airbyte.com/integrations/sources/mssql#upgrading-from-0417-and-older-versions-to-0418-and-newer-versions) source connectors, the

replication_method

configuration parameter was changed from

string

to

object

in certain versions. You might need to update your configurations accordingly. 3. If the issue persists, consider reaching out to the Airbyte community or support for more specific assistance. You can post your question on the [Airbyte Discussion Forum](https://discuss.airbyte.io/) or check the [Airbyte Documentation](https://docs.airbyte.com/) for more information. Remember, it's always important to keep your system and connectors up-to-date and to follow the recommended practices for configuring and managing your data sync processes in Airbyte.