Team,
I am using HOP window in Table API, I have t...
# troubleshootingk
Keyur Makwana
09/14/2023, 11:30 AMTeam,
I am using HOP window in Table API, I have tried to set slide as 10 minutes and window as 15 minutes but it’s not working for me. Can anyone help?
m
Martijn Visser
09/14/2023, 1:29 PMMy recommendation would be to check your watermark configuration
Martijn Visser
09/14/2023, 1:30 PMAnd make sure that you either push in data that allows the windows to close, or check things like configuring idleness etc
k
Keyur Makwana
09/14/2023, 1:37 PMI am getting below error while trying with 15 minute window size and 10 minutes slide
Exception in thread "main" org.example.exceptions.SQLParseException: Error in executing SQL query : HOP table function based aggregate requires size must be an integral multiple of slide, but got size 15000 ms and slide 10000 ms
Keyur Makwana
09/14/2023, 1:38 PMBasically my idea is to achieve latesness in thw window
m
Martijn Visser
09/14/2023, 1:39 PMLate events will still be dropped; that’s prior to the windowing even happening
k
Keyur Makwana
09/14/2023, 1:41 PMNo it will not be let’s say my original window is of 10 minute and I want to allow 5 minute lateness
Now to achieve this I will have total 15 minute window with 10 minute slide
Keyur Makwana
09/14/2023, 1:42 PMand I can some check from 10 to 15 minute for timestamp
m
Martijn Visser
09/14/2023, 1:42 PMYou’re making the assumption that late data will reach the window
Martijn Visser
09/14/2023, 1:42 PMBut it will already be dropped at the source
k
Keyur Makwana
09/14/2023, 1:42 PMI am working on processing time FYI
m
Martijn Visser
09/14/2023, 1:43 PMWith processing time, how will you have late data?
k
Keyur Makwana
09/14/2023, 1:43 PMAnyhow by previous processing in another env I have some delay in data arrival
Keyur Makwana
09/14/2023, 1:45 PMDo you think of any other way to handle late event in TUMBLE window?
Keyur Makwana
09/14/2023, 1:46 PMwith SQL
Keyur Makwana
09/15/2023, 12:07 PMHey @Martijn Visser Any suggestion do you think?
m
Martijn Visser
09/15/2023, 12:48 PMI don't understand how you are thinking about having late data with processing time
k
Keyur Makwana
09/15/2023, 12:58 PMIssue is because my data is ingesting let's say 10 minutes now user has written 5 minutes winodw. Does event time create 2 winodw of 5 minutes?
3 Views