Can the Multilingual /English model be prewarmed? ...
# agents
h
Can the Multilingual /English model be prewarmed? We’re facing high memory usage and if we run multiple agents on a single server. We found that Multilingual model and Silero VAD were initialized and loaded separately for each agent. This is not scalable. We’ve moved SIlero VAD model to be common across all agents on a server but Multilingual doesn’t work. Has anyone tried/achieved this feat? Any example? What are the typical memory usage agents face?
h
I'm curious about this, I thought Multilingual was shared across agents, do you mean you move Silero and Multilingual model to a different server to run inference there?
h
No i meant all of this within a server. All agent processes within a server are not sharing the loaded models like multilingual, silero vad and noise cancellation. We were able to move silero and noise to shared memory. but can’t move multilingual. Each instance of our agent are taking ~ 1GB of memory for running as multilungual and silero are 100s of MBs in size.
for a 2GB machine crashes if 3 agents run. 4GB crashes if 5 agents are running. With optimization we’ve improved but still feels we’re wasting memory by having a separate copy of multilingual model for each instance.
h
Hmm yes, my understanding is that it only loads once. Will check and report here