After accounting for premiums + expected medical costs, how much more per month is it usually worth paying for an HSA? Is something like $20/month extra a reasonable tradeoff to be able to max out an HSA, or does that generally negate the advantage?

Assuming you are making out your other retirement vehicles, the tax benefits of a single year of HSA contribution, assuming you take it out in 40 years is $3750, so yes it's worth it.

Another benefit is that you don't lose money at the end of each year. With an FSA, you have to guess what you'll spend and you may not have enough or lose if you put too much. You can roll over $600, but it's not much. With an HSA, you can roll over whatever is left. You can also use future payments to pay old bills as long as that HSA account was open when the service happened.

Can HSA money be used for premiums? I thought not

No, you cannot use HSA money to pay for regular health insurance premiums. There is one exception: after you turn 65, you can use your HSA to pay for certain types of Medicare premiums (like Part A, B, D, and Medicare Advantage), but not for Medigap premiums. For most other insurance premiums, HSA money cannot be used.