Dear Team. If I have • some popular source (like File) • popular destination (like Postgres) • standard normalization • users with small data, but realtime or near realtime requirement • K8S run mode The best I can do now for them (after tuning)

KubePodProcess.STATUS_CHECK_INTERVAL_MS

is 20 second, amng them POD preparation takes 14 sec. Can I have mentioned 3 pods prepared beforehand and then reused? (Like pre-heat worker pool with queue maybe)?

Hi, this is not possible today. You would have to develop a very different worker model from the one we use. We intentionally did not go with this model because it's tricky to efficiently manage the idle/hot worker ratio. Feel free to open an issue if this is important for your use case. The team will discuss it 🙂

If there’s a specific source that you want to use, and you have relatively few connections, you could create a custom source that just always polls for new data. Then the source/destination pods would be always-on. Would that work for your use case? If not, I’d be curious to hear more about the application and the constraints around the latency for data to appear in the destination.