Hey everyone, I have a question related to tempora...
# replication-troubleshooting
t
Hey everyone, I have a question related to temporal storage in airbyte. When syncs are run, the data is read from source and written on the destination, i*n transitioning of data, where does it store the data temporarily?* If it stores in the cache, how to control the data persistence time, (like after 123 mins, clear the cache)?? And at which directory/path the data is stored.? I'd be highly grateful if anyone can help me understand this, as data caching time is very important for us to manage great workload. cc: @Grace Stern (Airbyte)
u
Most connectors with cache in the Source Containers Docker memory the records. You don't have access to them for security reason. Some destination have the options to store staging data in S3 or Google Buckets to apply more efficient load methods, all connectors with this options you'll see the optio in the connector configuration.
👍 1