This message was deleted.

Hi @Dragos Cirtorosan, you would definitely need to build your own images, and a custom fetch provider is probably needed but i am not sure you need to change anything else. If you set up the broadcaster to use the Kafka backbone channel, you would be able to simply send messages to OPAL through Kafka, you will then send normal DataUpdate messages and you only need a custom fetch provider for custom logic.

So I could change the broadcasting mechanism from the default Postgres listen/notify to Kafka and publish events like this which will be processed by the fetcher, right?

{
  "id": "1",
  "entries": [
    {
      "data": {
        "<mailto:test@test.com|test@test.com>": {}
      },
      "config": {
        "fetcher": "UsersFetchProvider"
      },
      "topics": [
        "policy_data"
      ],
      "dst_path": "users/test@test.com",
      "save_method": "PUT"
    }
  ],
  "reason": "test",
  "callback": {
    "callbacks": []
  }
}

You can follow this guide for the format of messages to send through kafka: https://docs.opal.ac/tutorials/run_opal_with_kafka

oh, awesome! I didn’t see the example at the bottom of the page 😄 thanks a lot for the input!

that guide is just for the server it seems - not the clients? AKA clients still expect to be able to connect direct to the server and do websocket messaging?

Yes, that is correct - you can pass the messages from kafka to the server, and the server (based on topic) will forward the message to clients via websockets