@elegant-coat-62120 @creamy-monitor-7376 @Gangda Deng and i were working to figure out localization problem in coding agents using OpenHands 🤔 Finally be able to share the pre-print: • We build a graph-based repository indexing (that only requires static analyzer) ◦ Allow LLMs to perform complex multi-hop navigation & reasoning ◦ only takes a few seconds per codebase (faster than RAG!) • LLM as agents equip with this indexing actually perform better than the existing tools we have in openhands (e.g., grep via bash) • And most interesting to me, we finally show that frontier model + agent can outperform plain embedding-based or RAG-based procedure (Agentless/Moatless) ◦ This probably tells us "a strong agent + graph-based indexing is all you need". There's no need to construct the embedding for the ENTIRE repo for retrieval -- it adds cost of embedding & a lot of engineering challenges: you need to update the embedding when the codebase is updated; you need to wait for the codebase to be indexed when you first open a new repo ◦ The only downside here is that running these strong agents are expensive • 👆 BUT! We were able to train a Qwen 2.5 32B model that almost perform on-par as Claude-3.5 -- and the fine-tuned 7B model is already competitive with OpenHands + sonnet 3.5 in function-level Acc@10. This should greatly help with the cost @elegant-coat-62120 will help integrating this LocAgent into OpenHands in the comming weeks so stay tuned 🙂 https://arxiv.org/abs/2503.09089