Hello All, I’m using the below SQL query to sink ...
# troubleshooting
r
Hello All, I’m using the below SQL query to sink data to S3 in parquet format:
INSERT INTO sink_table_s3
SELECT event_id, event_type, event_name, DATE_FORMAT(TUMBLE_END(proc_time, INTERVAL '1' HOUR), 'yyyy-MM-dd') AS record_date, COUNT(*) results_count
FROM source_table
GROUP BY event_id, event_type, event_name, TUMBLE(proc_time, INTERVAL '1' HOUR);
I am partitioning the table on event_id, event_type and date columns. I observed the parquet files are getting saved for an
event_id
,
event_type
, but the date is not changing. The data being processed today is being saved to 2023-04-14 folder. As I am using
proc_time
to derive the date, I expect data to be saved to 2023-04-18 folder. PS. The deployment has been rolled back due to some errors in between, but running fine since a day. I’m curious what would lead to this behavior?