https://linen.dev logo
Join Slack
Powered by
I have had THE WORST time getting a stack of PRs t...
# report-bugs
k
I have had THE WORST time getting a stack of PRs through the queue today. I'm so far behind because of it that I can't offer many details, but the TLDR is that EVERY SINGLE PR that merges causes either a (not real) merge conflict with the next one or fails to trigger github actions for some reason and hangs, forcing me to push an empty commit or something to kick it back into gear. And one of the times I did that the PR had already been in the queue for long enough that it then timed out while running CI and got ejected. It's literally been all day that I've been fighting with this 😭 . Top of stack is here.
p
(If applicable) To help us repro, please send the following if you haven't already: - PR links, screenshots, and/or debug output if applicable - Repro steps - Does this happen consistently? Just to you or others on your team also? - Is this a regression? Is the problem new? - If you're having an issue with the CLI, please send us debug logs using 
gt feedback --with-debug-context <any message>
k
What I mean by "not real" merge conflict is that to fix it I simply sync the stack locally and re-submit and it's resolved without any further interaction from me.
There are two issues I could think of that you could possibly fix. The "easy" one I think is to do something about the merge conflicts. They're not real conflicts because I never have to actually modify any code myself to resolve them, so it seems there's a bug in there somewhere. The second issue is harder. I have no idea why github sometimes doesn't kick off actions, but I think maybe it would be good to implement some logic where Graphite will time something out while waiting for a GH action to report status. In this case, it was "waiting for status" for like an hour before I finally noticed and gave it a kick.
p
Hey Kael, apologies for the friction here, on initial scan i do see that 
3071
and 
3072
both timed out causing a rough time for 
3073
which then did see the merge conflict issue youre talking about. looking at the diff it doesnt appear to have any changes since the approval (ie versions 20-25 are rebases). I have not seen this behavior myself so need to do a bit more digging into why a rebase would fail with merge conflicts that seem non existent as you mention. Will circle back soon as i have any more info / direction. re: your github action issue, yes this may be harder to diagnose but let me see what enhancement requests exist around this and if nothing i can definitely write something up!
k
lovely, thanks for the attention! Yall are doing great ❤️
Sorry to beat on this, but it's really killing my productivity now, and it feels super shitty that we just 4x-ed our spend on graphite and got a bucket of bugs in return. This screenshot shows the absolutely needless churn I'm experiencing on a single PR, culminating (I pray) in the last entry, which clearly shows that for no reason at all I had to add the PR to the merge queue twice.
(this is the PR)
I also had a mind-boggling conflict experience today that I can't even describe. I ran 
gt sync
and one random branch in the stack (this one) tripped four times trying to restack. It was the most perplexing thing - I had a file in which, over the course of the stack, I defined 5 functions, one in each PR. The first "conflict" showed the current change with all five functions and the incoming change as empty. The second "conflict" showed 4 of the five functions and the incoming change as empty. The thrid showed 3, etc..... I had to simply "accept current change" four times before it finally went through.
(note that none of these were actual conflicts. Somehow graphite had come to believe that the incoming PR was for some reason trying to delete all of the code I had written, which it clearly wasn't)
Ok.... I'm gonna need either a fix or a discount now. Graphite merged commits in this stack to production yesterday that didn't pass our CI. I cannot understand how it got so bad so quickly, and especially immediately after raising prices.
There's something seriously wrong going on and I need answers from someone (cc @Brett Bedevian?). I'm still struggling through trying to merge this stack I've been working through for days (i'ts been approved and ready for that long) and just now I put the bottom three PRs into the merge queue. The bottom one merged and the other two were spontaneously ejected for no reason. When I went to the top of the remaining two and hit "merge", they immediately merged without passing through CI. Someone please give me an answer here. This makes graphite so dangerous and frustrating for us that we may have to drop it.
This is the PR that triggered the spontaneous merge.
p
Hey Kael, really sorry youre running into all this friction, and apologies for the delayed response here as i unpack your reports and run some questions by colleagues.
the bottom one merged and the other two were spontaneously ejected for no reason
Looking at the activity tab I can see that this was caused by CI failure. Specifically in my logs i see it was the 
api-checks
test that failed. We need to be better about indicating which test failed here. > When I went to the top of the remaining two and hit "merge", they immediately merged without passing through CI.
I do see that for 3277 there was a number of CI marked skipped in the UI as if part of the optimization but when I tested last night how 
CI optimizations
and 
fast forward merge queue
worked I was not able to reproduce the skipped checks. Im going to test again this morning on a partial stack merge to see if that behaves any differently.
k
Thanks for the info. I'm looking at it myself and am focusing on the event at 6:35AM CDT. I see that PR #3247 was evicted, but I don't see where it indicates there was a CI failure. I'm looking at github CI runs and all I see is two successes at 6:20am CDT (screenshot below). The activity feed in graphite also seems to reflect this inconsistency. It shows that the PR was added to the queue at 6:35AM, then added again at 6:37AM with no interceding ejection.... (Although I'm also not sure why it's not showing as being added earlier, since gh is showing the commit as having entered the 
main
branch at 6:20AM 🤔 ). Anyway, this is making my head spin, and if I have another stack like this one I might explode 🤯 . Please help 🥺
Sorry, meant to show a little more context on the above but now the page has turned over. Here are the CI runs immediately after that 6:20AM run:
👍 1
p
Some additional findings here Kael: • Our logs for #3277 show no outstanding required tests though a number have been marked 
completed - skipped
. After looking further at the CI in both the UI and our logs, i am seeing skipped that is different from when skipped for optimization purposes (which shows "optimized to run later" screenshot below). Is there any other mechanism on these actions that could have skipped these? Im also wondering if you could share what the branch protection rule looks like for main/master, specifically Require status checks to pass before merging. • Ive reached out to the merge team regarding the failure at 7:35AM that evicted PR #3247 and #3277. I do not see anything in the logs for why this would have failed.
k
Sorry, didn't see your reply here. Here are the settings:
Definintely weird on that GH actions run. It says "canceling since a higher priority waiting request for ci-refs/pull/3277/merge exists". Never seen that before and don't know what it means. Does that make sense to you guys?
Open in Slack
PreviousNext
Powered by