Hi im trying to use datastream with python this is...
# troubleshooting
n
Hi im trying to use datastream with python this is my code but i got this error: TypeError: Could not found the Java class 'org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer'. The Java dependencies could be specified via command line argument '--jarfile' or the config option 'pipeline.jars' from pyflink.common.serialization import Encoder from pyflink.common.typeinfo import Types from pyflink.datastream import StreamExecutionEnvironment from pyflink.datastream.connectors import StreamingFileSink from pyflink.datastream.connectors import FlinkKafkaProducer from pyflink.common.serialization import SimpleStringSchema def tutorial(): env = StreamExecutionEnvironment.get_execution_environment() env.set_parallelism(1) ds = env.from_collection( collection=[(1, 'aaa'), (2, 'bbb')], type_info=Types.ROW([Types.INT(), Types.STRING()])) producer_properties = {"bootstrap.servers": "localhost:9092"} ds.add_sink(FlinkKafkaProducer("my-topic", SimpleStringSchema(), producer_properties)) env.execute("tutorial_job") if name == '__main__': tutorial()
d
@Narges The JAR packages of the connectors are not part of the distribution of PyFlink. You need to specify them manually. • Kafka Connector JAR: https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka/1.17.0/flink-sql-connector-kafka-1.17.0.jar • Use JAR packages in PyFlink: https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/python/dependency_management/#jar-dependencies
👍 1