Hello all. We're using Apache Zeppelin notebooks ...
# troubleshooting
e
Hello all. We're using Apache Zeppelin notebooks to debug FlinkSQL and it's a really slow process. Wondering if there's another flow folks use for debugging/developing that they can share?
r
locally, or cloud-based?
and which part of it are you finding slow?
e
Cloud-based in AWS. Waiting for results to first show takes many seconds/ minutes . The starting stream position is set to latest.
r
are you using MSF? I've only tried zeppelin through that (I work at Decodable), and it was kinda slow. FWIW I do flink SQL development locally using the CLI and that works fine, nice and responsive.
πŸ‘ 1
e
Yuop same
m
Hi @Eric Chea. I was wondering if you found a better solution than Apache Zeppelin? We are currently considering using Apache Zeppelin as a front-end for a user-facing self-service Flink SQL debugging/developing environment. Using CLI locally is fine for developers and I use that myself however for our non-developers the barriers of entry has proven to high as many of these users doesn't even have Docker Desktop installed - not sure if there is easier ways to get the CLI in the hands of these people?
e
Hey @Mark Rud Christensen I wouldn't use Apache Zeppelin if you're going to test with some large amount of data. We spent alot of time thinking that there was a bug in our code, but there were systems limits that prevented us from getting refreshed computations. In retrospect it makes sense, but wasn't clear to us and we went back and forth with AWS for weeks to realize the limits of Zeppelin notebooks.
m
Thanks - appreciate you sharing your experiences! We ended up setting up a Flink cluster on a virtual machine dedicated for this purpose. Users can ssh into the VM and execute Flink SQL Client. It actually works quite well.
πŸ‘ 1