hi graphite, we recently have added one required check to our merge queue and it has led to some serious issues with getting prs merged, we will have some get caught in the queue until they time out, sometimes the draft pr that is opened in the queue gets closed immediately which causes the main pr to close without ever being merged. what are we doing wrong here? more than happy to share code samples among anything else that might help us fix this it just has been very difficult to figure out what is going wrong.

(If applicable) To help us repro, please send the following if you haven't already: - PR links, screenshots, and/or debug output if applicable - Repro steps - Does this happen consistently? Just to you or others on your team also? - Is this a regression? Is the problem new? - If you're having an issue with the CLI, please send us debug logs using

gt feedback --with-debug-context <any message>

1. https://app.graphite.com/github/pr/sandstone-team/sandstone/5588?mode=tour (this entire stack is one example of things not working correctly, but i can share more) 2. approve a pr, add to queue, sometimes it works flawlessly, stacks tend to get held up, the second pr (even if solo) in the queue usually fails after the one in the lead gets merged and we have to re-add it in order to get it to work 3. happens to everyone on the team, quite inconsistently 4. new since we added a required check to our ci

hey Liam, do you know when was the

Type Check & Unit Tests

check set to be required? Is it possible it wasn't required at the time the PRs initially ran their CI and so they were technically mergeable? Then when it re-ran during the merge process it became required and so its failure correctly causes merge failure

it was around a week ago at this point that it was made required so it isnt a new addition and it just causes intermittent errors, like here is one happeing right now: https://app.graphite.com/github/pr/sandstone-team/sandstone/5938?mode=tour there are three in the queue, all passing from what i can tell but then the graphite draft pr was already closed that defintiely does seem possible and that it is a misconfig with the ordering of when checks are running and that is mostly what i have been experimenting with to fix

I see, it's strange that the draft PR has closed already. In our logs, the merge attempt is still running but getting held up by

Type Check & Unit Tests

still being detected as

IN_PROGRESS

. Do you see that check running in the CI environment it's hosted in?

ya individual ones seem to be working ok..? ya i do see the check running (and already succeeded)

so this seems to only be an issue when you're merging multiple PRs at the same time? When that occurs, your MQ settings have CI running on the top-most PR. I'm wondering if this is causing some issue with that

Type Check & Unit Tests

should i switch it so it runs on every pr instead of topmost? i feel like i tried this too already but am more than happy to try again. should i provide the file for this pr-check? would that help?

yeah worth trying that switch and please share a PR's URL that's attempted to merge and will get fresh logs for it

ok i JUST made the switch 10s ago, heere is a solo one that just worked: https://app.graphite.com/github/pr/sandstone-team/sandstone/5949 here is another stack: https://app.graphite.com/github/pr/sandstone-team/sandstone/5591?mode=tour

the stack one again is showing the

Type Check & Unit Tests

as being

IN_PROGRESS

ya not sure what happened there, sitll has been inconsistent, single prs still are working fine

yeah, it looks like

Type Check & Unit Tests

is sometimes taking over 20 minutes to complete, so the queue is timing out as per the settings. We recommend setting the queue timeout to double your CI's average time, so you'll need to tweak that and see what's consistent

but is than issue with the action? that normally takes under 3 minutes to complete so i have assumed it has been a triggering issue to begin with. like this is a stack that has been in the queue for 10 minutes or so and all the checks are passing and it still isnt being merged: https://app.graphite.com/github/pr/sandstone-team/sandstone/6069

I see, could you double-check in its run logs in the PR in GitHub to be certain it has completed running? I'm wondering if our UI state is incorrect here, because in our backend it is still showing as

IN_PROGRESS

any new ideas today? what else can we try? now we are getting individual pr's popped from the merge queue as well

when enqueuing a PR and it seems to be hanging in the queue, could you double-check the draft PR's run logs in GitHub to be certain it has completed running? I'm wondering if our UI state is incorrect here, because in our backend the CI is still showing as

IN_PROGRESS

and that is blocking the merge and eventually the PR times out in the queue.

i am wondering if we have other steps we shoudl specifically. turn off for draft branches..?

when using the merge queue with CI settings enabled, Graphite groups stacks in the merge queue for running CI in the draft PRs and it creates a branch with a predictable prefix:

gtmq_

. You can use this for customizing CI runs or other behavior for enqueued PRs.

so i did do a bit of an overhaul today and remove a lot of ci so that id specifically does NOT run on branches that start with gtmq_ and that seems to have helped some. it does seem like ti still gets tripped up when there are two prs in queue (not in a stack) and the first one merges fine, and then the second one fails to merge because the temp draft pr tries to merge in to the previous one and then has issues

For PR #6578 that's merging via draft PR #6589 I can see the CI still

IN_PROGRESS

for the

PR Status

check

Hey @Gareth Brickman im sorry to ping you again on this but we are still having a ton of inconsistencies here when merging things via the queue, is there a better way we can resolve this than this thread? i feel like we are just making a simple mistake somewhere in our config and it has been causing the team to stop using the merge queue and stacking effectively and i really want to bring back

hey Liam, could you please share the link to a current PR that's having issues being merged?

we have basically stopped using the queue because it was every pr and stack that was having issues. is there any way i can share code snippets or something?

we'd need to check the logs in our system for when an enqueued merge is in progress. Do you know maybe the last PR to be merged via the queue? This definitely seems something idiosyncratic with your setup because we have no other similar reports from other users

forgot to close the loop here, what fixed it was turning off running queues in parallel, once i did that it seemed to wokr

Thanks for closing the loop, Liam. Glad to hear turning off running queues in parallel fixed it.