https://discord.gg/botpress logo
Powered by
Autonomous Node LLM Execution taking roughly 60sec...
# 🤝help
i
We can't seem to get botpress to respond to anything quickly with an autonomous node, it just takes so long with simple queries like "can you find me a red jacket" before it decides to then do an algolia search. Timeline: 0.05s incoming rewrite hook 2.05s KnowledgeAgent check 2.75s Single Choice capture matched the message to “Ask a product question” 0.19s Standard6 pass-through message 0.22s pre-LLM hook / learning injection 58.5s Products node LLM execution 52.8s first model pass 0.78s Algolia search tool call 4.87s second model pass 0.03s final message tool call 0.64s iteration tracking / wrap-up What can we do to speed up that LLM execution step??
l
Hey what's your bot ID? I'll take a look
The rule of thumb would be choosing a faster LLM have you tested a few different ones?
i
3f115c98-069d-4e68-b8e8-e35582f513e8 is this the id from the url?
l
Correct
But before I do that let's check this
i
We also have this bot: 9b763256-16a6-4d04-964c-b00c712452e5 which botpress made for us in the build program
which had the exact same issue, which was why we tried to make smaller autonomous nodes for specific tasks
We haven't, we've just stuck with haiku 4.5
l
Yes so before anything test with faster LLMs
i
Do you have a recommendation?
l
Try gpt 4.1 mini for example
i
Okay let me try that. Give me a few
Okay that was quicker yes, it didn't actually return the results to the user though; Aborted(Assertion failed: list_empty(&rt->gc_obj_list)...)
l
You can test with a few different LLMs to find one that works. Was the results being retuned before?
i
I'll continue testing.
Thanks
@loud-crowd-60573 i've raised this with support last week but it still isn't unblocked; my account has 3/3 bots but i only acttually have two bots, one has been deleted for over a week but won't free up the slot so i can't duplicate and test-things out.
Are you able to help?
l
Yes I am @important-musician-39432
what is your workspace ID
i'll look into it right now
i
wkspace_01JWTFNQR2VMPQ7NMH6SZ2Y7KJ
l
🫡
i
Thank you!
l
anything for the community
i
I need all the help i can get! Struggling to get the kind of smooth response we're looking for from our bot at the moment.
l
i'll look into that next
madison or test agent? @important-musician-39432
i
I'm currently looking at madison again as this is the one botpress built but struggling to chain tools together in one question
l
Can you quickly create a new bot now @important-musician-39432
i
Done
l
😄
i
I'm beginning to understand some of the problems. I think i just need to review all of our workflows. Currently it all seems quite restrictive. As an example, we have a workflow called "Fetch Recent Orders" which talks to an api to get the data. However the actual excute code within that flow always makes a card carousel of orders. Which makes questions like this (attached) makes it feel very derpy. https://cdn.discordapp.com/attachments/1483486430264295617/1483523236036612136/2Q.png?ex=69bae62e&is=69b994ae&hm=40be74616211e17ee0e81ef552b065fbdae0704b96ec02142062bdeb909e26d0&
Ideally the goal is to have a tool where i can ask chained intent questions such as; "I've recently ordered {item}, its faulty and i want to return it." and the chain would be; - Look up recent orders - Look up items within each order to find it - Start a return with details from found order
In my own attempt, using all the same workflows in "test agent" i've tried to use a router to work out intent and create seperate autonomous nodes so i can give each one more specific instructions however this falls down even worse with multi-intent prompts like the above as the orders node, doesn't have access to the return tools.
l
I was going to suggest using an AN with sub workflows but it seems you already do that
the key here would probably be making sure each step works individually then linking them all together
i
Ill continue tinkering, I think the concept is each subworkflow needs to be more flexible than it currently seems to be. I guess get orders should really return data then the AN is the one to decide how to display or use that data to power further tools
l
Amazing, do update me.
2 Views
PreviousNext
Powered by