Hi 🙂 I'm trying to run the Nexmark (https://github.com/nexmark/nexmark) benchmark with Flink (locally so far). I currently see that the CpuMetricReceiver is not started and thus my CpuMetricSenders are failing with

java.net.ConnectException: Connection refused

. That makes sense. I digged through the code and it seems that the script metric-server.sh is actually never called anywhere and I also don't see it mentioned in the README. So I guess it's not a surprise that it fails. Questions: • Am I missing something? • Is this a bug in the Nexmark benchmark? How was it ever supposed to run? • If it is a bug, it seems that the Nexmark benchmark is actually not really used, is it? Are there better ways how to test Flink from a performance perspective?

Flink runs its own performance tests, see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=115511847

Thanks a lot for the response. I'll look at the Flink's benchmarks as well, thanks for the pointers. I thought that Nexmark is kind of industry standard benchmark, more reflecting the production scenarios. The benchmarks that you point are all standalone (IIUC), so it's hardly representative of real-life scenario. Would you agree? We're always trying to evaluate the performance optimizations against as close to production as possible setup.

There's definitely TPC-DS tests being run, but they aren't open sourced

Oh, nice! That's very helpful. So let me flip it around - if a user wanted to evaluate Flink on different environments (different hardware, or different JVM, wink wink), would the TPC-DS test be the way to go?