This message was deleted.

Hi @Pranyt T great question, I will check this out with my team and get back to you

Every time a new OPAL pod is brought up, it should fetch authorization data from an external source.. that means the external source of authorization data should expose an API to return all authorization data right ?

that is correct 🙂 But you can also fetch data from other sources (for example a database) by writing a custom OPAL fetcher.

If the authorization data is huge, how do we return it in plain text format ? if the data is returned as a zip file, will OPAL client be able to unzip and push that data to OPA policy agent ?

OPAL can deal with huge data set, but your real limitation is OPA. OPA loads all data into memory and typically starts struggling at 2-3GB of data. The good news are: • typically your authorization data is smaller than you think (you only need ids and relationships typically, not the entire db object) • you can use OPAL to shard the data into many OPA agents, each one managed by a different OPAL client and subscribed to a different data topic

Makes sense @Asaf Cohen . When OPAL has to fetch authorization data from external sources, it needs to have a secure communication strategy because it is sensitive data. Do you have any prescription around how to do this ?

Simply load the credentials inside the data sources within OPAL_DATA_CONFIG_SOURCES and run OPAL in secure mode (with encryption keys). The OPAL server <-> OPAL client communication is then encrypted.