Hi all, needed some suggestions, what are some too...
# troubleshooting
s
Hi all, needed some suggestions, what are some tools that we can use to submit a Flink job from a UI and (or) REST API using SQL or SQL like languages, the use-case is, we have a set of ML models in dbt, which we would like to convert into Flink jobs for data transformations, we managed to parse dbt queries into Flink SQL, however, for testing and building the models, I am trying to find a better way, these are some options that I found • Zeppelin: Notebook style UI to create a Flink job, but might not support latest version of Flink, also, a bit tricky to make it work with existing Flink instances (at-least, what I have experienced). Docker image is really large in size. • AthenaX: It was developed by Uber, but it seems like, no longer actively maintained. • dbt-flink adapter: Its a nice tool, but uses SQL gateway for launching the Flink jobs, could not find a way to make with k8 operators. • Flink SQL Client: Great tool for debugging, but might be hard to integrate with dbt. I was wondering, what are some other options out there