This message was deleted.

@Or Weis I have used

opal-client

to update the users location

opal-client publish-data-update --src-url <https://api.country.is/23.54.6.78> -t policy_data --dst-path /users/bob/location

as per https://docs.opal.ac/getting-started/quickstart/opal-playground/publishing-data-update and I got the

Event Published Successfully

message but after querying it again instead of

{"result": false}

still getting true. May i ask what other configuration i need to do, please note that I am using kind cluster in my machine and used helm charts to bring up the postgres, client and server and port-forwarded before using

opal-client

Hi @raghanag, ☺️ You'd need to make sure the client is configured to accept the event (connects to server, has right topic

policy_data

, ...) What are you seeing in the client log? Does it accept the message? Does it try to fetch the data?

@Or Weis it keeps printing the below logs in opal-client

Got policy bundle with 2 rego files, 1 data files, commit hash: 'fa45fc65800a983c3159ca6cc4e7a21a05c8c7f2'
2023-05-04T05:58:19.203031+0000 | opal_client.policy_store.opa_client     | INFO  | processing store transaction: {'id': 'fa45fc65800a983c3159ca6cc4e7a21a05c8c7f2', 'actions': ['set_policies'], 'transaction_type': <TransactionType.policy: 'policy'>, 'success': True, 'error': '', 'creation_time': '2023-05-04T05:58:19.183647', 'end_time': '2023-05-04T05:58:19.202705', 'remotes_status': [{'remote_url': '<http://titanopal-server:7002/policy>', 'succeed': True, 'error': None}]}
2023-05-04T05:58:19.203642+0000 | opal_client.policy_store.opa_client     | INFO  | persisting health check policy: ready=true, healthy=true
2023-05-04T05:58:19.203859+0000 | opal_client.policy_store.opa_client     | INFO  | Policy and data statistics: policy: (successful 79, failed 0);	data: (successful 0, failed 0)
2023-05-04T05:58:22.036595+0000 | opal_client.opa.runner                  | INFO  | Running OPA inline: opa run --server --addr=:8181 --authentication=off --authorization=off --log-level=info
2023-05-04T05:58:22.207411+0000 | opal_client.opa.runner                  | INFO  | OPA exited with return code: 1
2023-05-04T05:58:23.060130+0000 | opal_client.opa.runner                  | INFO  | Running OPA rehydration callbacks
2023-05-04T05:58:23.060607+0000 | opal_client.policy.updater              | INFO  | full update was forced (ignoring stored hash if exists)
2023-05-04T05:58:23.060691+0000 | opal_client.policy.updater              | INFO  | Refetching policy code (full bundle)
2023-05-04T05:58:23.061824+0000 | opal_client.policy.fetcher              | INFO  | Fetching policy bundle from <http://titanopal-server:7002/policy>
2023-05-04T05:58:23.084940+0000 | opal_client.policy.fetcher              | INFO  | Fetched valid bundle, id: fa45fc65800a983c3159ca6cc4e7a21a05c8c7f2

It looks like it's not getting the update events. You'd need to check it's configuration, and also the configuration of your broadcater if you're using more than one opal-server worker, or multiple opal-servers

no i am using only one opal-server and i just used the same charts and configuration as https://github.com/permitio/opal-helm-chart and have 2 more properties in values.yaml for client

OPAL_OPA_HEALTH_CHECK_POLICY_ENABLED: True
    OPAL_DATA_UPDATER_ENABLED: False

Are you also using only one worker for the opal server?

yes one pod for opal-server, one for opal-client and one for postgres

Hi @raghanag, This is either: 1. a broadcaster configuration issue Please check that your server configuration includes this var:

- UVICORN_NUM_WORKERS=1

2. an issue with the OPA agent exiting - are you passing special config to the agent? i can see

OPA exited with return code: 1

in the logs

Nothing I am just using opal helm charts AS IS

please set uvicorn workers to 1

@raghanag I believe the issue is

OPAL_DATA_UPDATER_ENABLED: False

, which disables getting data updates in the client…

@Ro'e Katz the latest chart values also does not have any config https://github.com/permitio/opal-helm-chart/blob/master/values.yaml so do i still need to make the above change that you said if i want to do data publish

Yes. You still should do as @Ro'e Katz suggests- i.e. sending empty data, instead of disabling it completely. And as Asaf pointed out you need to set uvicornWorkers to 1

may i ask why is the default at 4 for uvicornWorkers https://github.com/permitio/opal-helm-chart/blob/master/values.yaml#L31

Most people do setup a broadcaster as per the instructions, and then you can have a multi-worker server that can use multiple cores providing better performance

so if i use only one server and one client, and also may i ask what is a uvicornworker, is it like a container base image that opal server and client uses

If you use only one worker then you don't need the broadcaster. Uvicorn is the underlying http server framework OPAL uses

and why is the data publish event fails when we have more than one worker, it should not stop data publishes right? i am using broadcaster too as per the charts, which is postgres in this case. broadcaster will be used only when we have more than one server and one client right? correct me if i am wrong

The broadcaster channel shares messages between OPAL server instances and workers. Let's say we have worker-1 and worker-2, and client-A. And client-A is connected to worker-2. If you send a mesage to client A, and that mesaage gets to worker-2 all ia good, but if it gets to worker-1 , without a broadcaster it won't get to worker-2 and then not to client A ...

and also may i ask why the data is being fetched by client instead of server, what i meant is if client can fetch the data, we can use the same to fetch policies too from git right, thus eliminating one more component(server).

It's part of the zero trust architecture, if the server is exposed to all the data it creates a very vulnerable spot.

broadcaster setup correctly -- I am just using the opal charts at https://github.com/permitio/opal-helm-chart no change has been done, do you think there is misconfiguration in the charts i cloned thus causing the data publish error

If you're using the helm charts as is, it should work. You do need to set the server.broadcastUri to match a relevant service (postgres listen notify, Kafka, redis, ...)

i haven't changed anything in helm charts except followed this comment since the opal-client is not starting up

I can't really say what is the issue you are experiencing. But if you disabled updates, yes of course updates won't work.

but if i set the empty data, then how do i do the publish data event step 4 https://docs.opal.ac/getting-started/quickstart/opal-playground/publishing-data-update

There's a difference between the baseline data set with config sources , and follow on updates

i am assuming the helm charts will use the data and policies at https://github.com/permitio/opal-helm-chart/tree/master/test for the simple testing and if we want to configure the server to read policies at a different location and client to read data from different location then we have to configure at values.yaml or i might be wrong altogether, since the server.policyRepoUrl at https://github.com/permitio/opal-helm-chart/blob/master/values.yaml#L9 is already pointing to an example repo it just uses the data and policies available at https://github.com/permitio/opal-example-policy-repo and disregards the data publish update step #4

You should set values.yaml. I'm not sure if it's configured with example defaults. If you are having a hard time with the helm-chart I'd suggest using the docker compose examples and guide instead, or at least at first to get familiar with the system.

@Ro'e Katz @Or Weis does the below statement meant we can avoid fetching data with opal-client and subscribe to opal-server topics to get latest data and policy

• Subscribes to Pub/Sub updates by topics for data and policy

The clients always fetch the data directly (unless it's already included in the event itself). Topics are used to control which clients receives which events

• Fetches data from multiple sources (DBs, APIs, 3rd party services)

but what about the above statement?

That refers to the OPAL-client data fetchers: https://docs.opal.ac/tutorials/write_your_own_fetch_provider

does that meant we can just ask opal-client to call a http endpoint to get data too right? that meant it can avoid getting updates from opal-server?

Yes. Exactly

event telling it to fetch data from OPAL server

does that meant data and policies always comes to opal-client via opal-server

The knowledge that there is a policy or data to fetch comes from the server. The data does not

but in architecture diagram there is no connection between data updates and server right?

Correct. Doesn't have to be. Though you can send webhooks from there to the server for example to let it know there's new data (and it will tell all the clients per-topic)

got it, so can i maintain the data and policies in the same repo and same folder just like the example repo and if there is a change in .rego files then it will emit a policy update event to client and if there is a .json change the it emits a data change event right? correct me if i am wrong and based on the event type the client will either fetch directly from github repo or gets the policy itself if its a policy update right

You almost got it. The data.json change will trigger an update butbit will be considered a policy update. Data.json is meant for rather static policy data. Nonetheless the client will get that data and update it into the policy agent

Saw the video above, but the example in the demo has policies and demo in the same repo but used different ways to update policy and data, I think if we keep both in the same repo, can we just update the git repo and get the latest on policy and data updates to opal-client which updates the OPA

You can definitely use data.json via git as your way of loading data, it's your choice.