Here is short demo (ignore the brutal crash from the UI, sometimes doesn't crash it is some socket error; I think I saw PR with fix of this)
quick-cricket-82634
05/06/2024, 9:23 PM
I just tried it twice and it is much better. I will actually remove the history part to see if this improves much. I think llama3 is this strange case of supper capable model with this ridiculous context window