Is it possible to set resource limits for

airbyte-connector-builder-server

?

No, that is currently not possible unfortunately!

I wonder if infra is part of your codebase what is open for community pull requests?

You're welcome to look through the codebase and make an issue/PR! https://github.com/airbytehq/airbyte