Oh lil' DeepSeek Flash on Digital Ocean is fun! It didn't have any secrets in env, but it has its own install, so it... just figured out the DeepSeek key, and it was off to the races -> it's testing OpenHands with live LLM And, it's so fast! Like incredibly fast

Deepseek inference is fast? Good to know!

I mean from direct deepseek 🤔

What I found: it's fast with most expensive providers, but slow and unreliable for the cheapest ones

┌──────────────────────┬────────┬─────────┬────────────────┬──────────────────┐
  │       Provider       │ $/M in │ $/M out │ $/M cache read │ Production tok/s │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ Together             │ 0.30   │ 1.20    │ 0.006          │ 157–197          │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ InferenceNet         │ 0.103  │ 0.715   │ 0.008          │ 168–265          │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ StreamLake           │ 0.15   │ 0.60    │ 0.003          │ 190              │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ AtlasCloud           │ 0.135  │ 0.54    │ 0.013          │ 180–191          │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ CoreWeave            │ 0.15   │ 0.65    │ 0.002          │ –                │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ Parasail             │ 0.30   │ 1.20    │ 0.006          │ –                │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ Fireworks            │ 0.30   │ 1.20    │ 0.006          │ –                │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ DeepInfra            │ 0.14   │ 0.42    │ 0.004          │ 36–87            │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ Morph                │ 0.056  │ 0.60    │ 0.007          │ 28–29            │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤
  │ Wafer                │ 0.10   │ 0.65    │ 0.006          │ 29–57            │
  ├──────────────────────┼────────┼─────────┼────────────────┼──────────────────┤

Can't we just use deepseek? Maybe keep other providers as fallback in potential outages