Hello All, I’m trying to sink data to S3 in parquet format, creating the table using below DDL:

CREATE TABLE sink_table_s3 (

event_id STRING NOT NULL,

event_type STRING NOT NULL,

event_name STRING NOT NULL,

``date` STRING,`

results_count BIGINT

`) PARTITIONED BY (event_id, event_type,

date

) WITH (`

'connector' = 'filesystem',

'path' = '<path>',

'format' = 'parquet'

);

I’m using the below SQL query to sink data to S3 in parquet format:

INSERT INTO sink_table_s3

SELECT event_id, event_type, event_name, DATE_FORMAT(TUMBLE_END(proc_time, INTERVAL '1' HOUR), 'yyyy-MM-dd') AS record_date, COUNT(*) results_count

FROM source_table

GROUP BY event_id, event_type, event_name, TUMBLE(proc_time, INTERVAL '1' HOUR);

I am partitioning the table on event_id, event_type and date columns. I observed the parquet files are getting saved for an

event_id

,

event_type

, but the date is being wrong. The data being processed today is being saved to 2023-04-26 folder. As I am using

proc_time

to derive the date, I expect data to be saved to 2023-04-27 folder. Can someone help me in identifying the issue?

I would recommend filing a Jira (since I see this being cross-posted from Stackoverflow to Slack etc)

What is timezone configured for flink application?

For time being, I fixed this by not creating windows in the query. As the table already has partitionedBy on date, I can save the data date wise.