This message was deleted.

Sounds really cool. Perhaps you should also post this on the #C030CMF6B70 channel.

Sounds awesome. Once you put the Java code together and have the supervisor API going, I can help guide you in adding support for Pravega in the web-console

Nice. you should start looking at classes in druid-kafka-indexing-service. There are two pieces in the streaming ingestion • supervisor - this is like a controller. if you are coming from yarn, this is like an application manager. if you are coming from spark, this is like a long-running driver. You would have to extend

SeekableStreamSupervisor

class for your own extension and override the methods depending on how pravaga works. • index task - This is the task that does the heavylifting of processing messages and publishing segments. The class that you need to extend is SeekableStreamIndexTask. you should look at these abstract classes first. These classes has certain concepts such as offsets, partitions, task assignment. If pravaga is a system similar to kafka from an API perspective, then it would be easier to build this extension.

@Abhishek Balaji Radhakrishnan Thank you for the tips! We will be sure to look at those classes and try to understand them. So far, we our work in PravegaEventSupplier.java, this was originally named KafkaRecordSupplier.java. We started our work here because there is a method named poll() which grabbed records from Kafka. We modified it to instead grab events from Pravega. The events are placed into a

List<OrderedPartitionableRecord<Integer, Long, KafkaRecordEntity>>

We modified this list by making it support ByteEntity objects instead because Pravega events can be represented in bytes. We've been following this helpful page highlighting the pravega api if you're interested! https://cncf.pravega.io/docs/latest/javadoc/clients/index.html