Diamond has really started throwing a lot of false...
# report-bugs
b
Diamond has really started throwing a lot of false positives lately. Here’s a particularly egregious one: we have a rule that our UI text standard is sentence case, not title case. One PR added the text “Reveal Logs,” and so Diamond said:
According to the UI text style guidelines, menu labels should follow sentence case when not at the beginning of a sentence. The label
Reveal Logs…
should be changed to
Reveal logs…
to maintain consistency with this standard.
Great, okay, valuable! So then we fixed that, push again, and Diamond commented on the SAME text:
The menu label
'Reveal logs…'
should use sentence case formatting. Change
'logs'
to
'Logs'
to comply with the UI Text should always be sentence case rule. The label should read
'Reveal Logs…'
.
Which is both obviously wrong, and also frustrating.
p
(If applicable) To help us repro, please send the following if you haven't already: - PR links, screenshots, and/or debug output if applicable - Repro steps - Does this happen consistently? Just to you or others on your team also? - Is this a regression? Is the problem new? - If you're having an issue with the CLI, please send us debug logs using
gt feedback --with-debug-context <any message>
b
The applicable rule simply says “UI Text should always be sentence case”
And then on the next push after THAT, it flagged another string that had been there the whole time. My bad for not actually checking for other instances like that, but also, that’s pretty inconsistent and frustrating.
p
hey Brendan, please downvote/thumbs down any comments you come across that are false positive or inconsistent and that will flag them for the team!
One suggestion that came to mind is to make the rule explicit and verbose about its intention as the LLM seems not able to infer the meaning of "sentence case" correctly
b
We’ve been downvoting them for sure. But the issue isn’t just limited to this rule. For example, we have another rule that’s much more explicit, about PR title/desc, and that’s been more aggressive and incorrect lately as well.