Okay, I've found my aurthelia issue(s).
I am using authelia via dex and made 3 mistakes
1. I not set the scopes.
2. I did set the scopes at the wrong place in the config
3. I did not set "insecureEnableGroups: true"
Everything works fine now, thank you for this great tool to manage users and groups