https://www.linkedin.com/feed/update/urn:li:activity:7468563396825780225/

In this work, we introduce Nexus, a transparent data management layer that sits between event streaming systems (e.g., Kafka, Pulsar, Pravega) and external storage (e.g., object stores). Nexus intercepts tiered storage operations and runs in-line programmable functions on chunks of stream data across the edge-cloud. This enables things like high data compression without impacting the hot path, semantic routing or labeling of stream data using AI, or caching/buffering stream data close to the edge. The policy-based,  programmable nature of the system allows for exploring many other potential use cases.

any one head about this ?

Hi Ali, thanks for sharing. I’m one of the authors of the paper. If you want to know more, I’m happy to share more details about it :)

Can you give a link to the full paper or the draft ?