Regarding FILES() partition pruning:
table has da...
# questions-and-troubleshootingv
Victor Dozal
11/05/2026, 12:12 AMRegarding FILES() partition pruning:
table has daily partitions since 2021
select count(*)
from files(
“path” = “s3://bucket/data/table/partition_date=*/*.parquet”
, “case_sensitive” = “false”
, “format” = “parquet”
, “aws.s3.access_key” = “”
, “aws.s3.secret_key” = “”
, “aws.s3.region” = “us-east-1"
, “aws.s3.enable_path_style_access” = “true”
, “columns_from_path” = “partition_date”
, “files_enable_insert_push_down_schema ” = “true”
, “auto_detect_sample_files” = “1"
, “auto_detect_sample_rows” = “1"
)
where partition_date = ‘2026-10-01’
order by 1
this query takes 2.5mins
select count(*)
from files(
“path” = “s3://bucket/data/table/partition_date=2026-10-01/*.parquet” --noqa
, “case_sensitive” = “false”
, “format” = “parquet”
, “aws.s3.access_key” = “”
, “aws.s3.secret_key” = “”
, “aws.s3.region” = “us-east-1"
, “aws.s3.enable_path_style_access” = “true”
, “columns_from_path” = “partition_date”
, “files_enable_insert_push_down_schema ” = “true”
, “auto_detect_sample_files” = “1"
, “auto_detect_sample_rows” = “1"
)
where partition_date = ‘2026-10-01’
order by 1
this query takes 1 second
Victor Dozal
11/05/2026, 6:39 PMHi @Sida Shen, this is the use case related to FILES() partition pruning I mentioned during Next Query Q&A
if I try the equivalent query in DuckDB:
select count(*)
from *read_parquet*(‘s3://bucket/data/table/partition_date=*/*.parquet’, hive_partitioning = true)
where partition_date = ‘2026-10-01’
it returns in < 1 sec
👀 1