Mikhail Spirin
01/26/2023, 7:39 AMMikhail Spirin
01/26/2023, 7:40 AMdef main():
table_env.execute_sql(f"""
CREATE TABLE logs_source (
raw_row STRING
) WITH (
'connector' = 'filesystem',
'path' = '{logs_path}',
'source.monitor-interval' = '10',
'format' = 'raw'
)
""")
table_env.execute_sql("""
CREATE TABLE print_sink (
ip_number BIGINT NOT NULL
) WITH (
'connector' = 'print'
)
""")
table_env.execute_sql(f"""
INSERT INTO print_sink
SELECT
COUNT(raw_row)
FROM logs_source
""").wait()
I searched for any options or parameters to enable parsing for gzipped files specifically - but i’ve failed...
Flink version 1.16.0, im using pyflink, Python 3.9
Where should i search for an answer? Thanks for any ideas!Martijn Visser
01/26/2023, 8:24 AMMikhail Spirin
01/26/2023, 9:19 AM