Gemini ACP • software-agent-sdk https://github.com/OpenHands/software-agent-sdk/pull/2619 • evaluation https://github.com/OpenHands/evaluation/pull/379 • benchmarks https://github.com/OpenHands/benchmarks/pull/594 Version (N-1) of the code has been validated by successful benchmark runs. The last version requires rebuilding all base images for validation, which takes a few hours. One upstream problem: Gemini-acp (as well as codex) does not transmit cost data. This is why I am trying this new way of retrieving costs: one virtual litellm key per instance + ask litellm for overall costs instead of using built-in telemetry.

This is awesome, thank you! ❤️ So now we have four SOTA-level agents via ACP to play and compare with

4!

My counting/SKILL.md is a bit outdated 🫠

Thanks for this, this is great! It’d be really nice if we could get upstream fixed too, I feel like they would like to report costs?