Raghunadh Nittala
04/18/2023, 1:14 PMINSERT INTO sink_table_s3
SELECT event_id, event_type, event_name, DATE_FORMAT(TUMBLE_END(proc_time, INTERVAL '1' HOUR), 'yyyy-MM-dd') AS record_date, COUNT(*) results_count
FROM source_table
GROUP BY event_id, event_type, event_name, TUMBLE(proc_time, INTERVAL '1' HOUR);
I am partitioning the table on event_id, event_type and date columns. I observed the parquet files are getting saved for an event_id, event_type, but the date is not changing. The data being processed today is being saved to 2023-04-14 folder. As I am using proc_time to derive the date, I expect data to be saved to 2023-04-18 folder.
PS. The deployment has been rolled back due to some errors in between, but running fine since a day. I’m curious what would lead to this behavior?