This message was deleted.

HI @Robert Philp at a glance it seems the current broadcaster Kafka backend doesn’t support SASL. https://github.com/permitio/broadcaster/blob/master/broadcaster/_backends/kafka.py Unless there’s a way to pass this configuration via URL to the underlying Kafka consumer. It can be supported without a ton of work (as AIOKafka does support it) , but currently the SASL params are not exposed. I’d suggest one of the following: 1. You can upgrade the broadcaster code (to expose these variables - either as part of the URL , or as Env-vars (If you do please consider contributing this back to the project 2. Use a Kafka proxy to add the needed params - e.g. https://github.com/grepplabs/kafka-proxy 3. Open an issue on our GitHub requesting we do item 1 for you; and we’ll do our best to prioritize it. Sorry it’s not a quick fix.

Thanks for the quick response @Or Weis. I'll have a look into the options.

Hi @Robert Philp, sorry for the late reply (timezone difference). First I really wouldn't recommend forking the broadcaster , you should be able to pass secrets via env vars. To the specific point: You can't add commands to tthe requirements.txt but only module names. If you want to add this as a command you can it to the docker file instead.

Ah, it seems I'm missing something here @Or Weis. Okay, so my current understanding: • AIOKafka allows you to define the SASL parameters via the producer and consumer classes, • Broadcaster (and therefore fastapi_websocket_pubsub and opal-server) does not pass in these optional arguments and therefore they are not exposed in OPAL-server I was assuming I would need to expose these in broadcaster (passing them into AIOKafka) first to be able to pass them in to OPAL as environment variables? At least I thought that was what you were getting at in your point 1, but it doesn't sound like it? What approach did you have in mind? Perhaps I'm missing something simple here?

Oh I'm sorry, I misread the parts about the secrets; Yes I agree forking the broadcaster is the easiest option here.

Hi @Or Weis - I've put a PR in for the updates you suggested to Broadcaster here https://github.com/permitio/broadcaster/pull/2 It also includes an update to allow it to pass through a connection to multiple kafka brokers. I've run this with OPAL against and MSK configuration with SASL in our dev environment and it connected fine. Let me know your thoughts. Happy to update if needed.

Hi @Robert Philp thanks for your contribution! @Ro'e Katz please open a ticket to review this PR carefully. @Robert Philp, it might take us a few days to review, we'll do our best to get there quickly.

Hey, any update on this? we're looking into using opal with kafka + SASL aswell

Hi @Samuel Carlier, i will make sure someone reviews this today, sorry for the delay.

Hi @Samuel Carlier & @Robert Philp Approved and Merged it Thanks for your contribution 🙏

cool, is it possible to make a tagged release of it? so it is available on https://pypi.org/project/permit-broadcaster/? then we only need to

make docker-build-server

to have the functionality before a new OPAL release.

Hi @Samuel Carlier, @Asaf Cohen will do it tomorrow

Side note @Robert Philp - thank you for your contribution; we’d love to thank you with some awesome Permit / OPAL swag - please DM @Filip your address so we can send you some 😉

Hey any update on this? could this be the 0.5.1-rc release i see on dockerhub?

Hi @Samuel Carlier we’ll add this to the next minor release of OPAL. 0.5.1-rc does not include this change