Hi all could anyone point me to an example using t...
# troubleshooting
e
Hi all could anyone point me to an example using the
ProcessFunctionTestHarnesses
to unit test a
ProcessWindowFunction
?
d
I dont know of that specific one. https://github.com/search?q=ProcessFunctionTestHarnesses++language%3AJava+&type=code Has a few examples but I dont think specifically for ProcessWindowFunction. Not sure ProcessFunctionTestHarness is suitable for testing a window function which maybe why there do not appear to be examples.
e
Do you know a way to unit test a Process Window Function?
d
Do you know if this is a keyed situation?
if your Process Window Function is part of a larger keyed topology then it might affect testing options
e
Yes the data stream is keyed, then applied a tumbling window and the applied the operator I want to test
Copy code
DataStream.keyBy { event -> CityLineKey(event.city, event.line) }
            .window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
            .process(BoxInductedCountFunc()).uid("BoxInductedCountStream").name("BoxInductedCountFunc")
the BoxInductedCountFunc extends the ProcessWindowFunction, that’s the operator I want to test
d
Ok, I see
So I think the test harness is probably not ideal for this
I think you can approach this as an integrated test scenario using a local flink minicluster to handle the keyed and window processing.
Copy code
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.environment.TestStreamEnvironment;
import org.apache.flink.streaming.api.functions.source.SourceFunction;
import org.apache.flink.streaming.api.watermark.Watermark;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.test.util.MiniClusterWithClientResource;
import org.junit.ClassRule;
import org.junit.Test;

import java.time.Duration;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.TimeUnit;
These are some classes that you can use to build it.
e
Ok thanks I will look into the mini cluster example, thanks a lot for your help
d
sure let me know if you have any further questions about it.