Hello Everyone, is it possible to ingest data from a kafka topic into multiple datasources by a supervisor?
a
Amatya Avadhanula
04/04/2024, 7:32 AM
Not with a single supervisor, no. Multiple supervisors can read from the same topic
s
Sanyog Singh
04/04/2024, 7:41 AM
ok. Since with 8 core CPU, ideally 7 supervisors can be created and my requirement is to have limited supervisors to ingest data (from 1 kafka topic) to multiple datasources. Any alternative to achieve this?
a
Amatya Avadhanula
04/04/2024, 7:49 AM
I don't think there is a mapping of 1 core <-> 1 supervisor on the Overlord.
What you said is generally true for tasks in a given middlemanager, I think