Ryan Schwartz
07/15/2024, 10:03 PMkapa.ai
07/15/2024, 10:04 PMTEMPORAL_HISTORY_RETENTION_IN_DAYS
environment variable in Airbyte is used to define the retention period of the job history in Temporal, and it defaults to 30 days. This value is also applied to the log retention when running in Docker [[source](https://docs.airbyte.com/operator-guides/configuring-airbyte#data-retention)]. The airbyte-cron
service is responsible for cleaning up old logs in the workspace folder, and it uses the TEMPORAL_HISTORY_RETENTION_IN_DAYS
variable to determine how long to retain logs [[source](https://github.com/airbytehq/airbyte/discussions/33810)]. If you're finding that the TEMPORAL_HISTORY_RETENTION_IN_DAYS
variable is not triggering the airbyte-cron
to clean up logs as expected, there could be a few reasons for this: 1. There might be an issue with how the variable is being set. Ensure that you're setting the variable correctly in your deployment. For example, in a Kubernetes environment, you might set the variable using a helm command like this: helm install -n foresight-infra airbyte airbyte/airbyte --set global.env_vars.TEMPORAL_HISTORY_RETENTION_IN_DAYS="7"
[[source](https://github.com/airbytehq/airbyte/issues/39358)]. 2. There could be a bug in Airbyte. There have been reported issues where users have set the TEMPORAL_HISTORY_RETENTION_IN_DAYS
variable but found that logs were not being cleaned up as expected [[source](https://github.com/airbytehq/airbyte/issues/39358), [source](https://github.com/airbytehq/airbyte/issues/28235)]. 3. The feature to change the default value of TEMPORAL_HISTORY_RETENTION_IN_DAYS
was reimplemented in Airbyte version 0.45.5 [[source](https://discuss.airbyte.io/t/2417), [source](https://discuss.airbyte.io/t/2200)]. If you're using an older version of Airbyte, you might need to upgrade to a newer version to use this feature. If you're still having issues, I would recommend reaching out to the Airbyte community or opening an issue on the [Airbyte GitHub page](https://github.com/airbytehq/airbyte/issues) for further assistance.