hi, i want to run tests in order to evaluate the q...
# ask-ai
b
hi, i want to run tests in order to evaluate the quality of the response. I already set the tests to run sinchronously, is there a way to run them in parallel? I want to run multiple conversations, so i'd need to run multiple AgentSession (i think), but i get RuntimeError: cannot start agent: an activity is already running
t
Thanks for reaching out, Antonio. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I see you're trying to run multiple conversations in parallel and hitting
RuntimeError: cannot start agent: an activity is already running
. I’ll help you get parallel test runs working. Short answer • That error happens because an
AgentSession
(and related runtime resources) isn’t designed to have multiple concurrent activities in the same process/context. To run tests in parallel, isolate each
AgentSession
so no two run in the same process or share the same runtime objects. What you can do (practical options) • Run each session in its own process ◦ Use Python’s
multiprocessing
or run tests across separate processes. This ensures each
AgentSession
has its own runtime and avoids the “activity already running” conflict. • Use a test runner that distributes work across processes ◦ Use
pytest-xdist
to run test files or test functions in separate worker processes. Each worker will create its own
AgentSession
, avoiding shared activity conflicts. • Create process-isolated workers manually ◦ Start a small pool of worker processes where each worker creates and owns its
AgentSession
. Send test scenarios to workers and collect results. • If you must stay in a single process, ensure strict sequential usage ◦ Only start one
AgentSession
activity at a time (not parallel). Use sequential
async with AgentSession(...)
blocks or an execution queue to avoid overlap. • Ensure you’re not sharing server/job resources ◦ Don’t share
JobProcess
,
AgentServer
, or any runtime objects between tests. Each parallel test should initialize its own resources if those objects are not concurrency-safe. Async-specific notes • If using asyncio test frameworks (e.g.,
pytest-asyncio
), ensure each concurrent test uses separate event loop contexts or run them in separate processes. Running multiple sessions on the same event loop can lead to conflicts if the agent runtime expects exclusive control. Example approaches 1) Quick multiprocessing sketch
Copy code
from multiprocessing import Process

def run_session(test_input):
    async def _run():
        async with AgentSession(...) as session:
            return await session.run(user_input=test_input)
    asyncio.run(_run())

procs = []
for inp in inputs:
    p = Process(target=run_session, args=(inp,))
    p.start()
    procs.append(p)
for p in procs:
    p.join()
2) Use pytest-xdist for test suites • Install
pytest-xdist
and run tests with
pytest -n auto
(or a fixed number). Each pytest worker runs in its own process, so each
AgentSession
is isolated. Questions to narrow it down • Are you running tests with pytest (and
pytest-asyncio
) or a custom runner? • Do you need true parallelism (multiple processes) or just concurrency on a single machine? • Are you currently sharing any objects like an
AgentServer
or
JobProcess
across tests? If you share which runner you’re using and a short snippet of how you create/start
AgentSession
, I can suggest the most direct code changes to run your sessions in parallel. Sources: Testing and evaluation | LiveKit Documentation | Job lifecycle | LiveKit Documentation