We are using Apache Spark for batch processing and are now planning to extend the same for Stream Processing. I was checking the stream-native pulsar connector and was wondering does it supports Continuous Trigger or RealTime trigger like Databricks? The idea is to have sub-milliseconds end-to-end latency. Is this at all possible with Pulsar <> Spark?

This connector allows Spark applications to read data from Pulsar topics and write data back to Pulsar topics, supporting batch processing with Spark SQL and real-time stream processing with Spark Structured Streaming.

@David K, I know about the connector, and I was asking about the same. I guess it does not support Continuous streaming. Apache Spark provides. I was checking the repo and found out Continuous Streaming support was removed under PR https://github.com/streamnative/pulsar-spark/pull/124