Hey Kilian!
Agh this is a tough one - sounds like you have the
dismiss stale pull request approvals BPR enabled for this repo? Unfortunately, stack merge isn't possible if this rule is enabled and we haven't found a great workaround. The only suggestion we have here is to speak with the repo maintainer to see if it's possible to disable this rule