A more out there idea, using an agent to decide which subset of the e2e tests to run for a pr (to save time) (not likely to get this in anytime soon, but thought would be interesting idea to share) https://github.com/OpenHands/agent-canvas/pull/1267

interesting, but i feel there must be a deterministic way to do this too 🙂

The alternative typically is where we detect file changes, and exercise the e2e that directly cover them; I see two major downsides to this 1. For trivial changes (e.g a css change), it will trigger e2e tests for these paths which is overkill. We can probably come up with some heuristics maybe to avoid this 2. It's hard to reason about indirection. E.g a change introduced buggy logic for a util constructing a websocket URL for a conversation. We don't have e2e test explicitly covering websockets logic. But the e2e test for conversation creation would likely fail; (perhaps there is a heuristic here as well that can track these relationships) I'll come up with a deterministic approach too, I'll see whether these hunches hold true