Hi there, I’m trying to get to grips with temporal...
# troubleshootings
Simon Lawrence
04/13/2023, 2:11 PMHi there, I’m trying to get to grips with temporal table joins in the table API, and I cannot get a simple example to work.
The code compiles and runs successfully. I would expect that the attached code would produce a table of currency converted “amounts” and that this would print to the console, but instead nothing appears to be happening. Any help with this would be appreciated.
Simon Lawrence
04/13/2023, 2:14 PMExample kafka records:
flink-table-orders
Copy code
{"currency":"USD","order_id":1338,"order_value":93}
{"currency":"USD","order_id":1339,"order_value":7}
{"currency":"USD","order_id":1340,"order_value":72}
{"currency":"USD","order_id":1341,"order_value":66}
{"currency":"USD","order_id":1342,"order_value":60}
{"currency":"USD","order_id":1343,"order_value":95}
{"currency":"USD","order_id":1344,"order_value":93}
{"currency":"USD","order_id":1345,"order_value":94}
{"currency":"USD","order_id":1346,"order_value":50}
{"currency":"USD","order_id":1347,"order_value":44}
{"currency":"USD","order_id":1348,"order_value":6}
{"currency":"USD","order_id":1349,"order_value":49}
{"currency":"USD","order_id":1350,"order_value":82} Copy code
{"rate":1.3489401,"currency":"USD"}
{"rate":1.6078507,"currency":"EUR"}
{"rate":1.8044899,"currency":"NOK"}
{"rate":1.5689677,"currency":"DKK"}
{"rate":1.2515974,"currency":"USD"}
{"rate":1.8256688,"currency":"EUR"}
{"rate":1.6015472,"currency":"NOK"}
{"rate":1.5926261,"currency":"DKK"}
{"rate":1.5863882,"currency":"USD"}
{"rate":1.5896747,"currency":"EUR"}
{"rate":1.9921018,"currency":"NOK"}
{"rate":1.4763255,"currency":"DKK"}
{"rate":1.5786169,"currency":"USD"}j
Jirawech Siwawut
04/14/2023, 2:11 AMCan you try this? https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/config/#table-exec-source-idle-timeout
It works in my case. I guess that if the table on right side, rates, does not progress as time goes as order. It will not proceed streaming further AFAIK
s
Simon Lawrence
04/14/2023, 7:44 AMHmm, I will try it, but the set up that I have is with two kafka streams that are constantly producing new values. I will also try with a static rates table, to see if that works.
👀 1
Simon Lawrence
04/14/2023, 8:04 AMIt did work! The correct currency rates were used for the correct time in the order stream. I don’t understand why though. Does the temporal table function require a static table to evaluate?
j
Jirawech Siwawut
04/14/2023, 8:54 AMI do not think so. I think it should work on your case normally. I think my guess is not 100% correct then. I dont think it is required static table tho. Let me try on my side.
s
Simon Lawrence
04/14/2023, 12:15 PMThank you very much for the help 🙂
j
Jirawech Siwawut
04/15/2023, 4:13 AM@Simon Lawrence I tested with mine and found the same thing.
If i set idle source timeout to 0
• The fact table is active
• I see that the number of
It works fine if i set
I am not sure as well. why is that.
I tested and it seems to work as i suspected. Both streams need to be active if you set
Record recievedidle source timeoutidle source timeout = 0Jirawech Siwawut
04/15/2023, 6:42 AMPlease make sure that your watermark is enabled correctly.
If you get something like this, it should not produce record
2 Views