Has anyone here used cognito with resource servers along with HTTP APIs? Any ideas how to configure my clients and authorizers to make them honour custom scopes in the resource server. Any help would be highly appreciated because I am just not able to make this work.

I am not sure this qualifies as help, but I would advise managing this on your own instead. Cognito does not seem to be well-supported by AWS, and I have found the greatest stability using it for Authentication only. For the price, it is still a steal even if you do nothing with authorization.