I am at chapter 4 now and have been ambushed only once in chapter 2 when my Oathsworn wanted to rescue the kid and that definitely turned out to be the wrong strategy. There the ambush made sense story wise