This message was deleted.
# opal
s
This message was deleted.
permit 1
o
Hi @Soumya Mishra , we’re excited to hear you love OPAL - helping fellow engineers is what we are all about 😇
How does OPAL guarantee that the policy data is actually in sync with the actual application state
Thanks to the lightweight nature of the OPAL pub/sub channel - clients learn of changes in the data state even before they have the data itself very quickly - this allows the OPAL-clients, their data-fetcher components, their managed OPA instances (OPAL-client stores this state in OPA), and other subscribing services to respond to data change and handle the interim state. The simplest form of here is - 1. Your Rego policy can use the health state value to make different interim decisions 2. The OPAL-client will continue to try and fetch data the data until it gets it (with exponential backoff) Read about the mechanism here: https://docs.opal.ac/tutorials/healthcheck_policy_and_update_callbacks
We are using the official OPAL postgres data fetcher (link). When the opal client is being spun up, it will fetch the data from the postgres instance it is connected to. We are concerned that during that spin up phase if any updates to the data source happens how is it handled.
Once connected the OPAL-client will start tracking data update events that you send and handle them. It does so from the start even before getting the first base data - and will process the events in order (So you don’t need to track them specifically). To your sub question -
Does the client fetches the update messages from opal server and re runs them after it builds the initial state?
the client doesn’t re-run, but more correctly spins tasks to update as events come, and processes them once possible. Usually loading the client is pretty fast, so we haven’t encountered issues here. In general this is not a pure OPAL problem but a general multiprocess read/write sync challenge If it is an issue for you , I can suggest a few options: • Create a clone source where clients get their base data set, instead of working directly with the rapidly changing data source (this can be another OPA+OPAL instance for example - kinda like a master node) • Temporarily lock access to the DB for writing until the client is ready (reader/writer lock pattern) CC: @Filip (add these to the new FAQ, please)
s
Thanks for the quick response @Or Weis. On an initial read through this makes sense. Let us get back after a proper read through and discussions.
🤟 1