Hello all. We are trying to spin a minimal version of datahub on ECS to let our BA team try the tool. We intent to use the aws managed services for MYSQL and Elastic search, and run the gms and the frontend on ECS containers. So in total, we would have 4 services running on this ‘minimal’ setup. Given that we are looking to ingest the data via http (At least at this stage), can we skip KAFKA entirely? Also, what would be the advantages of using neo4j over elasticsearch? Is there someone in here with a similar deployment architecture? Thank you! 🙂

okay nice glad you found the answer about kafka! and then to get to your second piece - you can either use neo4j or elasticsearch for your graph index, and it's totally up to you which you want to use! elasticsearch is already being used for search queries, so you need that no matter what. neo4j has difficulties running locally with M1 chip Macs, so that's also something to consider.

Thanks!