Hi, we've never seen the graphite review feature ever work for us. It either says "found no issues" when coderabbit finds plenty of major bugs, or it says "something went wrong" and never triggers, like for this PR https://app.graphite.com/github/pr/ST0x-Technology/st0x.liquidity/768

Hey Juani, More than happy to take a deep dive on this, can you navigate to that PR page and hit cmd+k then send search for feedback and send in a bug report. This will send me the run event and I am able to recreate exactly what you see. Thanks!

@Chris Bowie sent

OK so I think I have a better picture now, what I think is happening is the AI Reviewer isn't set to a specific repo, so its basically doing nothing. Please navigate here and then select the repo you want it to start working reviews on. Now this is obviously a bit non intuitive as far as the error reporting goes and something we can certainly improve. So let me know if this ultimately gets it working and I can bring this to the team.

oh wow ok thank you! I'll configure that and give it a try

100% agree on that, and I will get that to the team.

Hi @Chris Bowie I'm still finding that graphite never finds any issues, look at this stack: https://app.graphite.com/github/pr/ST0x-Technology/st0x.liquidity/744/refactor-extract-wrapper-contract-handling-into-its-own-crate?mode=tour

Ok, this is unexpected. Let me spend a bit of time on this today and I will double back once I have something to share. Thanks!

@Chris Bowie any updates?

Sorry for the delay! I do actually show its running since the 8th, and it has found 6 total issues over 37 prs https://app.graphite.com/ai-reviews?org=ST0x-Technology, do you have a PR since the 8th that you are seeing a mismatch? Thanks!

Don't you find it odd @Chris Bowie that it only finds 6 issues across 37 PRs?

Ok, let me do a bit of research on the difference between the two checks and try and give you a better read of the differences. Was there a PR with something coderabbit found that we ignored I can compare? Thanks!

@Chris Bowie You can basically look at any of the PRs we have open and you see coderabbit finding things (major and critica) where graphite says "no issues found). Take a look at this for example: https://app.graphite.com/github/pr/ST0x-Technology/st0x.liquidity/877/fix-gate-dependent-on-chain-writes-on-RPC-node-reaching-prerequisite-tx-block Look how many important things coderabbit found. How is it possible Graphite reviewer found nothing?

This is correct, it is intended to be high signal and will not necessarily comment on all PRs. This is why you are seeing some variance with coderabbit. Thanks!

I understand, but it still doesn't explain why it doesn't trigger in most PRs when there are a lot of critical/major findings from coderabbit. Looks more like a bug to me @Chris Bowie

I do not have that answer for you right now, let me spend a few days trying to understand what the difference is and get back to you. Most likely early next week so we can stop talking in high level and I can dig in to some specifics.

thanks!

Hey Juan! Spent a bit of time on this today, I compared CodeRabbit’s output on PR 877 with how Graphite AI Reviews is built. CodeRabbit’s prompt is a broad verify-and-fix pass (inline comments, outside-diff notes, test/doc/style). Graphite is tuned for high-signal issues on changed lines that would likely break in production (mostly logic, security, concurrency with clear evidence). From what I compared it’s normal to see fewer comments than a fuller scanner. On that PR specifically, several CodeRabbit items are test hardening, comment formatting, or doc nits we intentionally don’t flag; one concurrency note was outside the diff entirely. TLDR Coderabbit is checking a lot more areas than us, and some of the items that appear to be a miss are really just outside of what we are looking for. I can only really speak to this one specific PR, but on this it really comes down to what each surface is looking for as to why you are seeing varied results. Thanks!

Thanks for the deep dive @Chris Bowie