Hello there, Nick!
This is an interesting case. Indeed smaller jobs are more reliable and easy to maintain.
When I had to do this, I created a monorepo with many entrypoint classes. The monorepo would generate a single jar (for the sake of simplicity) and when deploying, I'd specify the class to be the entrypoint for the job.
All platform bits were shared among them and each one had its own consumergroup to kafka.