still seeing pretty severe performance degradation...
# proj-agent-canvas
w
still seeing pretty severe performance degradation when running multiple conversations. Sometimes the UI doesn't load at all 😕
logging some things that feel "stuck" for posterity
lots of "fetching older messages"
semi-regular "disconnected" errors
not much going on in the logs ``````
Copy code
14:47:45 [agent-server] INFO     Settings accessed  settings_router.py:159
14:47:45 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:47:45 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:47:45 [agent-server] INFO     207.180.140.243:0 - "GET /api/conversations/c9bc27e5-280f-42c8-979f-d7fef3a6a72c HTTP/1.1" 200
14:47:55 [agent-server] INFO     Settings accessed  settings_router.py:159
14:47:55 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:47:55 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:47:56 [automation] {"message": "127.0.0.1:60548 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:48:00 [agent-server] INFO     207.180.140.243:0 - "GET /api/conversations?ids=c9bc27e5-280f-42c8-979f-d7fef3a6a72c HTTP/1.1" 200
14:48:05 [agent-server] INFO     Settings accessed  settings_router.py:159
14:48:05 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:48:05 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:48:11 [automation] {"message": "127.0.0.1:43818 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:48:15 [agent-server] INFO     Settings accessed  settings_router.py:159
14:48:15 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:48:15 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:48:16 [agent-server] INFO     207.180.140.243:0 - "GET /api/conversations/c9bc27e5-280f-42c8-979f-d7fef3a6a72c HTTP/1.1" 200
14:48:25 [agent-server] INFO     Settings accessed  settings_router.py:159
14:48:25 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:48:25 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:48:26 [automation] {"message": "127.0.0.1:44980 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:48:31 [agent-server] INFO     207.180.140.243:0 - "GET /api/conversations?ids=c9bc27e5-280f-42c8-979f-d7fef3a6a72c HTTP/1.1" 200
14:48:35 [agent-server] INFO     Settings accessed  settings_router.py:159
14:48:35 [agent-server] INFO     207.180.140.243:0 - "GET /api/settings HTTP/1.1" 200
14:48:35 [agent-server] INFO     207.180.140.243:0 - "GET /server_info HTTP/1.1" 200
14:48:41 [automation] {"message": "127.0.0.1:42056 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:48:57 [automation] {"message": "127.0.0.1:54706 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:49:12 [automation] {"message": "127.0.0.1:44436 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:49:27 [automation] {"message": "127.0.0.1:56290 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:49:42 [automation] {"message": "127.0.0.1:36342 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:49:58 [automation] {"message": "127.0.0.1:39334 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:50:13 [automation] {"message": "127.0.0.1:39224 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:50:28 [automation] {"message": "127.0.0.1:51016 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:50:43 [automation] {"message": "127.0.0.1:38470 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:50:58 [agent-server] INFO     [bubus] 🗣️  EventBus_fbea4140🟢(⏳ 1 | ▶️  0 | ✅ 32 ➡️  31 👂).dispatch(NavigateToUrlEvent) ➡️  NavigateToUrlEvent#4bb7 (#1 pending)
14:50:58 [automation] {"message": "127.0.0.1:48522 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:51:03 [agent-server] INFO     Event websocket disconnected  sockets.py:361
14:51:14 [automation] {"message": "127.0.0.1:56030 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
14:51:29 [automation] {"message": "127.0.0.1:42598 - \"GET /api/automation/v1/c6ec5fb4-3c72-4eb1-a82e-33157aa7b95e/runs?limit=30 HTTP/1.1\" 200", "severity": "INFO"}
currently stuck on this page
now here
it's possible that the agent is running commands that are killing my server...
I wonder if we can surface CPU/memory metrics in a way that makes this transparent
I've got 3 active convos but 11 chrome instances running 🤔
Copy code
❯ ps -aux | grep chrome | wc -l
11
(this is on a VM, so it's not my regular browser)
I don't think my agent is really using chrome but I'll check when I get back in
OK idea: we should have the agent's commands run with lower priority than the uvicorn servers. That should help the server from locking up when the agent tries to run a bunch of heavy tasks
👍 2