Does flink still support scala? I tried going over...
# troubleshooting
v
Does flink still support scala? I tried going over the main website for latest docs - https://flink.apache.org/ and couldnt find any scala examples or Scala API. So going forward, is it possible to use Flink with scala? I also came across this blog - https://flink.apache.org/2022/02/22/scala-free-in-one-fifteen/
Copy code
Users can now leverage the Java API from any Scala version
Im finding it a little hard to understand the above message, please let me know if its possible process using Flink through scala? And i would highly appreciate it if the community could also share some Scala-Flink tutorials
m
You can, but the Flink Scala APIs are deprecated. It’s recommended to use the Flink Java APIs, even if you would build your job in Scala
https://github.com/sjwiesman/flink-scala-3 contains an example of using the Java APIs in a Scala application
v
Thanks @Martijn Visser!! Im a little new to this but what exactly do you mean by Java API in scala application? I will be deploying my job in AWS-EMR-6.9.0 cluster which currently supports scala-2 & Flink-1.16 Can i still use the above mentioned examples? By Java API do you mean that - I will have to pass class & variables similar to those defined in JAVA examples but using Scala Code?
For example - the given Java code from docs - https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/learn-flink/datastream_api/#a-complete-example
Copy code
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.api.common.functions.FilterFunction;

public class Example {

    public static void main(String[] args) throws Exception {
        final StreamExecutionEnvironment env =
                StreamExecutionEnvironment.getExecutionEnvironment();

        DataStream<Person> flintstones = env.fromElements(
                new Person("Fred", 35),
                new Person("Wilma", 35),
                new Person("Pebbles", 2));

        DataStream<Person> adults = flintstones.filter(new FilterFunction<Person>() {
            @Override
            public boolean filter(Person person) throws Exception {
                return person.age >= 18;
            }
        });

        adults.print();

        env.execute();
    }

    public static class Person {
        public String name;
        public Integer age;
        public Person() {}

        public Person(String name, Integer age) {
            this.name = name;
            this.age = age;
        }

        public String toString() {
            return this.name.toString() + ": age " + this.age.toString();
        }
    }
}
Here is the scala conversion using ChatGPT but still maintains class API style -
Copy code
import org.apache.flink.streaming.api.scala._
import org.apache.flink.api.common.functions.FilterFunction

object Example {
  def main(args: Array[String]) {

    val env = StreamExecutionEnvironment.getExecutionEnvironment

    val flintstones = env.fromElements(
      new Person("Fred", 35),
      new Person("Wilma", 35),
      new Person("Pebbles", 2))

    val adults = flintstones.filter(new FilterFunction[Person] {
      override def filter(person: Person): Boolean = {
        person.age >= 18
      }
    })

    adults.print()

    env.execute("Example")
  }

  class Person(var name: String, var age: Int) {
    def this() {
      this("", 0)
    }

    override def toString: String = {
      name + ": age " + age.toString
    }
  }
}
Is the second Scala code equivalent to using a Java API through Scala code?
s
Java API and Scala API are different APIs provided by Flink. The Scala API is getting deprecated, but your Scala code can still leverage the Java API just like any other Java dependency.
Your example still uses the Scala API, notice the first import.
v
Thanks @sap1ens Is the
FilterFunction
method correctly used in the above scala code? As per the Java API docs - https://nightlies.apache.org/flink/flink-docs-release-1.17/api/java/
FilterFunction
is a interface with
filter
method