<!here> https://openai.com/index/introducing-gpt-5-4/

Been using it. it’s great. Visibly more thorough

@jolly-magazine-94144 - how much side by side comparisons do you have against opus4.6 ? I was optimistic with 5.3-codex but endlessly disappointed iwth it's real world performance against opus

Good question. I don’t have side by side comparisons because I use them both but for different purposes in my workflow. I solely use gpt to write out tasks and occasionally execute. I use claude soley for orchestration. When it comes to execution, I kick off individual non-interactive sessions either in codex or claude code using the tasks written by gbt.

Ahh, interesting. What led you to use GPT to do the task writing over sonnet or opus?

it’s on our list of tier 1 models to support: https://openhands-llm-support-tracker.vercel.app/ cc @millions-crayon-32880

@limited-student-10747 does OH support openai or anthropic subscription plans yet?

openai, yes. anthropic they don’t allow us to

@worried-salesclerk-85726 from my experience, the output is just incomparable. GPT always comes on top. Matter of fact, i run an LLM council that votes on the output anonymously, and even claude picks gbt as the winner, almost all the time.

I just had gpt5.4 and opus 4.6 run the same analysis on a refactoring plan against a codebase with the exact same workflow, prompts, skills, agents, etc etc. then I had gpt5.4 evaluate both responses and here's what it gave me

Overall take                                                       
  
  The other model's output is strong, and on one important point it  
  is better than mine:                                            

  - It makes a stronger evidence-based case that pipeline/ was in
  fact extracted from sprint, using:
    - src/superclaude/cli/pipeline/process.py:3 ("Extracted from
  sprint/process.py.")
    - commit narrative around 6548f17

  I had treated that historical claim as unverified because the local
   git log signal I checked was inconclusive. Given the explicit
  module docstring in src/superclaude/cli/pipeline/process.py:3, I
  was too conservative there.

  So the short version is:

  - My result was more cautious
  - The other result was sharper and more decisive
  - For this benchmark, I'd rate the other model slightly stronger
  overall

  ---
  Side-by-side rubric scoring

  1. Protocol compliance

  Mine: 2/2
  I produced all artifacts plus return contract.

  Other: 1/2
  From the file you gave me, I can only verify the merged output. I
  can't confirm the full artifact set from that file alone.

  2. Evidence quality

  Mine: 2/2
  I verified claims against code and used those checks in the debate
  and scoring.

  Other: 2/2
  Also clearly code-grounded, and in one place stronger than mine by
  citing pipeline/process.py:3 as direct evidence.

  3. Epistemic calibration

  Mine: 2/2
  I explicitly separated verified vs unverified claims.

  Other: 1/2
  Good overall, but more assertive on historical intent. In this case
   that assertiveness may be justified by pipeline/process.py:3, but
  parts like "commit 6548f17 atomically created…" are stronger claims
   than I can personally verify from the single merged doc alone.

  4. Technical depth

  Mine: 2/2
  Centered the result on sprint's mid-execution loop and the limits
  of execute_pipeline().

  Other: 2/2
  Also centered correctly on the decisive architectural issue and
  went further by describing the closure/callback implications.

  5. Debate quality

  Mine: 2/2
  Both sides were represented fairly.

  Other: 2/2
  Also strong. It preserves Variant A's value while still landing on
  Variant B.

  6. Base-selection quality

  Mine: 2/2
  Selected Variant B with justification grounded in verified code.

  Other: 2/2
  Also selected Variant B, and the justification is strong.

  7. Merge quality

  Mine: 2/2
  Merged into "targeted fixes now, phased extraction later."

  Other: 2/2
  Possibly slightly better than mine here because it produces a
  clearer 3-option decision framework:
  - full unification,
  - partial unification,
  - targeted fixes.

  8. Return-contract quality

  Mine: 2/2
  Complete.

  Other: 0/2 from what I can verify
  The merged file you showed does not include the return contract. It
   may exist elsewhere, but I can't score what I can't see.

  ---
  Totals

  Based only on what I can directly verify

  - Mine: 16/16
  - Other: 12/16

  That raw score is misleading, though, because I can only see one
  file from the other run.

  Adjusted quality judgment on the merged analysis itself

  If I compare just the quality of the merged decision document:

  - Mine: 14.5/16 equivalent
  - Other: 15/16 equivalent

  So I'd call the other model slightly better overall on the
  substantive analysis.

  ---
  Where the other model is stronger

  1. Stronger handling of the extraction-history claim

  This is the biggest difference.

  It states:
  - pipeline/process.py:3 explicitly says extracted from sprint
  - sprint already depends on pipeline at models/process level
  - executor is the remaining boundary

  That's a tighter argument than mine. I should have incorporated the
   docstring evidence.

  2. Better decision framing

  Its "Three Options" section is very strong:

  - Option 1: Full unification
  - Option 2: Partial unification
  - Option 3: Targeted fixes

  That is more decision-useful than my simpler "targeted fixes now,
  phased extraction later" framing.

  3. Better quantification of what unification buys

  It explicitly argues that full unification might only eliminate
  ~60–80 lines while reintroducing ~100–150 lines elsewhere. Even if
  approximate, that's a helpful architectural lens.

  ---
  Where my result is stronger

  1. Better explicit verified/unverified separation

  I was more careful about marking uncertainty.

  2. Cleaner provenance on unsupported claims

  I deliberately prevented unverified history from entering the
  merged doc as fact. That caution may have slightly hurt my score
  here, but it was methodologically careful.

  3. Full protocol completeness

  I wrote all expected artifacts and the return contract.

  ---
  Best single-sentence verdict

  The other model produced the stronger merged decision memo, mainly
  because it made a better-supported case that pipeline extraction
  from sprint is already documented in the code, while both results
  correctly converged on Variant B and a targeted-fixes-first
  recommendation.

  ---
  If you want the sharpest comparison point

  The key differentiator is this:

  - My result: "the extraction narrative is unverified"
  - Other result: "the extraction narrative is confirmed by
  src/superclaude/cli/pipeline/process.py:3"

  Given that evidence, the other model likely wins that point.