I think I found my issue, I wasn't setting the fit axis in the operator