control purging job history logs in a docker deployment

In Airbyte, you can control the purging of job history logs by setting the

TEMPORAL_HISTORY_RETENTION_IN_DAYS

environment variable. This variable determines the number of days for which job history logs are retained. After this period, the logs are automatically purged. This feature was reimplemented in Airbyte version 0.45.5. If you're using a Docker deployment, you can set this variable in your Docker configuration. Additionally, there's a component called

airbyte-cron

that is responsible for cleaning the logs. It cleans workspace logs (only relevant in Docker deployments), Temporal workflow logs, and in the cloud version, it retrieves the connectors catalog definition every 30 seconds. The logs are stored inside the

/tmp/workspace

folder. Please note that there was an issue where the jobs database grew too large and caused disk space issues because the automatic cleanup was temporarily removed. This has been planned to be migrated back. In the meantime, a workaround was suggested to manually clean up using a script. Sources: - [Forum Post: Unsure how to set workspace retention variables correctly in docker based deployment](https://discuss.airbyte.io/t/2200) - [Understanding the core components:

airbyte-cron

](https://github.com/airbytehq/airbyte/discussions/33810) - [Github Issue (closed): Provide a way to clean up old data (logs)](https://github.com/airbytehq/airbyte/issues/15567)