That's the route I ended up going as well. Seemed to work fine upgrading a test HA pair we typically start upgrades with, then I upgraded a standalone VPX in a secondary site with the "in place" upgrade model, then also did a production HA pair in our primary site like you did, starting with secondary, failing over, then updating former primary. The only "problem" I ran into was the custom certificate applied to the management IP/services had to be re-bound, and we had to re-activate with LAS afterwards, even though it'd already been done.