Hi all. Sorry if this is really stupid question a...
# opal
c
Hi all. Sorry if this is really stupid question and I should "RTFM", but I'm under pressure and pulling my hair out between ChatGPT, online docs etc. The scenario is really simple. I need to make a policy decision based on updated user data in Entra ID, we're talking about 10s maybe 100s of thousands of users. I want to dynamically pull data using a fetcher and make sure that it is avaible in the data store before the OPA executes. ChatGPT led me to believe this was possible (see diagram), but online opal docs dont seem to support this. Am I completely on the wrong track? It is fundamental that the data recovered from Entra ID relates to the user represented by the JWT, no for all the users.
a
Hi @Christopher McMahon, I'm an AI assistant for Permit.io. I'll analyze available sources and get back to you shortly.
Yes, you can dynamically pull user data from Entra ID using OPAL's data fetchers. Here's how it works: OPAL tracks data from various sources via webhooks and Fetch-Providers (extensible python modules that teach it to load data from sources) (1) . The OPAL server sends instructions to the OPAL-clients to fetch data, and the event via the config field indicates which type of data-fetch-provider the client should use (2) . For your Entra ID scenario, you have two main approaches: ​*Initial Data Load:*​ Initial data is indicated by the server based on
OPAL_DATA_CONFIG_SOURCES
(1) . You can configure this to fetch from your Entra ID endpoint on startup. ​*Dynamic Updates:*​ Subsequent data updates are triggered via the data update webhook . When user data changes in Entra ID, you can trigger OPAL to fetch the updated data by calling:
Copy code
curl -X POST <http://localhost:7002/data/config> \
 -H "Content-Type: application/json" \
 -d '{
 "entries": [{
 "url": "<http://example_external_data_provider:80/acl/tenant1>",
 "topics": ["tenant_data"],
 "dst_path": "/acl/tenant1"
 }],
"reason": "Add tenant1 data source"
 }'
(3) The key point is that OPAL-clients use data-fetchers both to load the initial baseline data and for subsequent updates as part of an update event (2) . The fetcher will retrieve the data from your Entra ID API and store it in OPA before policy evaluation occurs. You can write custom data-fetch-providers to handle the specific authentication and data transformation needed for Entra ID integration.
To learn more, see the sources I considered: [[1] OPAL TL;DR](https://docs.opal.ac/getting-started/tldr) [[2] Data updates](https://docs.opal.ac/faq/) [[3] Single-Topic Multi-Tenant Configuration](https://docs.opal.ac/tutorials/single-topic-multi-tenant)
o
AI is correct, let me know if any clarifications are needed
c
But it does fit my scenario, I don't want to pull the entire Entra ID data into OPAL, just for the sub-set of users that access this app.
Sorry "does not fit"
I want something similiar to what is shown in the diagram
The app receives a request with the user id in a token (jwt). I want to call OPA with that token and call Entra ID dynamically based on that user id.
I can't find documentation here about calling the OPAL Client API directly
o
> I don’t want to pull the entire Entra ID data into OPAL You can pull in selective data, no requirement to load everything You can use HTTP.send in OPA’s Rego code to fetch data as part of an AuthZ query - highly not recommended for production use You can use OPAL’s data triggering to load new data as it arrives (e.g. new users) in an event driven fashion Please listen to attached recording for added important context
c
Thank you @Or Weis!!! Ok so summarizing. OPA http.send, bad idea. Best practice would be to sync Entra ID data with OPA via asynchronous events based on user lifecycle. But if I was willing to take a performance hit the first time a user accessed the app (taking advantage of the fetcher's caching), could I call the OPA Clients /fetch endpoint directly at execution time, or is this considered an anti-pattern?
o
Yes you can do that, and yes to some degree this is an anti pattern (coupling of different SLA services) Note that this is less about performance, and more about SLA and availability - you usually want to have a very high SLA for your authorization layer and it being non-responsive (timeouts, generally unavailable) is usually a deal breaker, which might happen by coupling to EntraID API which is highly scalable but not highly responsive (SLA is usually in seconds not milliseconds)
c
Ok, thanks again @Or Weis, that's been really helpful.
🤘 1