@fresh-machine-37269 examine all of the recent discussion on this slack regarding deadlocks, and also the recent perf-related improvements to OpenHands/software-agent-sdk. Once you have that context, open a ready-for-dev issue on software-agent-sdk regarding the development of pre-release load tests. The acceptance criteria should include failing on all of the branches before the recent perf improvements, and passing on the current main (unless main has obvious perf issues, in which case they should be fixed in the process of deploying these tests). These tests should be added to the release-please workflow. Seperately check and comment on whether the posthog telemetry that we calculate on Openhands/OpenHands may be able to be used to identify p50/p95 on particular operations such as the ones cited above.

✅ Done! Done. I opened a ready-for-dev issue on software-agent-sdk: github.com/OpenHands/software-agent-sdk/issues/4588 — "Add pre-release load tests as a blocking release gate" It's labelled

enhancement

+

performance

(the repo's

issue-readiness-check

workflow will auto-apply

ready-for-dev

since the body has

### Desired Behavior

with a checklist of

### Acceptance Criteria

). The acceptance criteria cover the key points you asked for: the tests must fail on the pre-fix branches (pre-#4570 lifecycle-lock, pre-#4481 bashevent glob/eval-loop, pre-#4417 ConversationInfo GC wedge, pre-#4473 LLM global-config serialization) and pass on current

main

, with any obvious

main

perf defects fixed in the course of deploying them. One accurate nuance I baked in: software-agent-sdk does not currently use

release-please

itself — its release path is

prepare-release.yml

→

create-release.yml

→

pypi-release.yml

. (

release-please

lives in OpenHands/OpenHands and downstream repos like

automation

.) So the issue asks to wire the gate into `prepare-release`/`create-release`, "or the equivalent release-please path if the SDK later adopts it." Relevant context I found and referenced: existing stress suite

tests/agent_server/stress/

(opt-in,

-m stress

, with

budgets.py

and a

agent-server-stress-tests

CI job), existing complementary issue #4477 ("Add cheap load tests for concurrent conversations"), and the deadlock/perf issues #4514/#4513/#4548/#4569. On the PostHog p50/p95 question: short answer — not with what's captured today. There are two layers, and neither currently emits per-operation latency: • Agent-server telemetry (in software-agent-sdk,

openhands-agent-server/openhands/agent_server/telemetry/

): emits

agent_server.conversation_created/finished/failed

with a

duration_bucket

, but durations are deliberately bucketized to coarse bins (`DURATION_BOUNDS = 1,5,15,60,300,1800`s) and raw counts/durations are never reported (re-identification/privacy policy in `models.py`/`sanitizer.py`). That gives a coarse histogram of whole-conversation duration — not precise p50/p95, and it has no per-operation timing at all (no TTFT, tool-call latency, event-search latency, or db write/hydration path). • Canvas/frontend telemetry (OpenHands/OpenHands

src/services/telemetry.ts

): tracks funnel/product events (

conversation_start_requested

,

user_message_sent

,

automation_executed

,

backend_added

, etc.). These are action/funnel events with no

duration

-style numeric properties, so they can't produce p50/p95 either. So while PostHog can compute percentile aggregations over a numeric event property, that data doesn't exist in the current schemas. To get p50/p95 on the operations Jamie cited (switch-chat, db write/hydration, create chat, tool execution, TTFT), you'd need to add per-operation timing instrumentation — best done server/SDK-side (that's where those operations actually run), e.g. OpenTelemetry/OpenInference spans that also emit a numeric

duration_ms

property to PostHog. That's a natural companion to the load-test issue, and emitting a raw numeric duration is a deliberate departure from the current bucketed/privacy-constrained policy, so worth calling out explicitly rather than assuming it falls out.

I think this is a good start, but I wonder if coverage of the UI will also be necessary... I'm new to the architecture, but maybe somebody can consider whether a companion issue on the front-end/agent-canvas is necessary. For example, how will we know if a server span is 20ms but a client span is 5000ms? Or whether there is some websocket backpressure issues or some JS event-loop stalls, etc.

Also, I declare a

default_stuck_budget_ms

@ 20s. If anything takes longer than that it's declared jammed or locked. Maybe if another issue gets opened to plumb in the remaining timing instrumentation, some consideration can also be paid to exactly what the

budget_ms

should be on a per call site basis.