quick little summary on the new activity umpire (v2), based on the traffic through the weekend: • 4.4x faster than before • 10x cheaper than v1 (…granted that the ump is deeefinitely not our most expensive agent) • 7.5x less requests to do the umping note that we’re using openai’s (i-can’t-believe-its-not-jev)-decisions api - not typesafe’s jev one small issue is that we’re seeing a very small percentage 0.5% of ‘refusals’, which seem to be around sensitive issues like in one case the ‘Office of Women’s Health’ from the HHS website and not from anything actually adult or spywarey or gambling. kinda odd as openai’s services are usually a bit more… libertarian than most so the refusals feel a little weird. i’m digging in a bit more to see if there’s some wording stuff that might fix it, or maybe some settings somewhere that’ll fix it. but otherwise there’s a parallel version built with typesafe’s api that might do it claude.ai/artifact/FUduihAuRgzvNBNxnfHWxo?sk=…