It took a bit of trial-and-error to figure out the...
# troubleshooting
a
It took a bit of trial-and-error to figure out the correct syntax, only to find out that this is not supported yet.
Copy code
save_sample_data = f"""
CREATE TABLE save_sample_data
PARTITIONED BY (EVENT_DAY, EVENT_HOUR)
WITH (
  'connector' = 'filesystem',
  'path' = '{EGRESS_S3_PATH_FOR_SAMPLES}',
  'format' = 'parquet'
)
AS SELECT * FROM input_data
WHERE MINUTE(ts) = 1 AND HOUR(ts) = 10;
"""
table_env.execute_sql(save_sample_data)
I tried to put the PARTITIONED.. at every location. πŸ˜„ I also would have wanted to sample a percentage of the input data.
m
You mean that PARTITIONED BY in a CTAS is not supported?
a
Yes. It's in the docs, which I found afterwards.
m
πŸ˜… I'm happy at least it's documented πŸ˜„