https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
v

Vika Petrenko

06/06/2021, 11:45 PM
Is it possible to set partition field when using full refresh append? This way it would be possible to set partition expiration and automatically get rid of old data.
s

s

06/06/2021, 11:55 PM
@Vika Petrenko do you mean stream slices?
if so then yes they are also supported in the full refresh sync mode and will be passed into
read_records
and other methods
v

Vika Petrenko

06/07/2021, 12:04 AM
it mostly about BigQuery tables partition, when create a table its possible to specify field for partition and expiration time. As they wrote in doc,
When a partition expires, BigQuery deletes the data in that partition.
It could be useful for large data
s

s

06/07/2021, 12:18 AM
I believe the bigquery connector doesn’t currently support writing partitioned tables but it is on the roadmap. See here: https://github.com/airbytehq/airbyte/issues/2579
👍 1
v

Vika Petrenko

06/07/2021, 12:22 AM
cool, thanks!
6 Views