This message was deleted.

Hi @Maxime Broussart 🙂 Glad to hear OPAL is working for you, Currently OPAL doesn’t detected changes in data sources automatically (though that is a feature we are working on (… 😉 @Shaul Kremer) So you need to trigger a data-update trigger yourself, here are a few options for Postgres: • (Generic) Have the code writing to DB send the update event • (Generic) Use a library like https://github.com/inqueryio/inquery • (AWS RDS) Use a Lambda as part of queries to trigger the event : https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL-Lambda.html • (Generic Postgres) Use Postgres triggers + functions https://www.postgresql.org/docs/current/sql-createtrigger.html , https://stackoverflow.com/questions/3325292/how-can-i-send-some-http-request-from-postgresql-function-or-trigger Also, if your system is already event-driven - you might consider sending the event directly from the message-queue you’re using There are probably other options, but If you can if I were in your shoes I’d go with the first option

Great thank you for your very clear answer!

My pleasure

Hi @Or Weis ! I succeed to trigger a data update following the option 2 with Server REST API --> when there is a change on my postgres db, I have an SQL triggered function that fetches OPAL Server API and the OPA data gets well updated. But, I am not really satisfied with this approach because I have to pass all the postgres credentials as plaintext in the body of the request. Do you think there is a more secure way? In the meantime, I am trying option 1 with CLI using token but I am stuck in getting a datasource identity token. I added an env variable on my docker compose for the OPAL server :

OPAL_AUTH_MASTER_TOKEN=JwMXIIyrFR

, then with the cli :

opal-client obtain-token JwMXIIyrFR --type datasource --server-url <http://localhost:7002>

gives me this error :

File "/home/maxime/.local/lib/python3.11/site-packages/opal_common/cli/commands.py", line 78, in fetch
    return data["token"]
           ~~~~^^^^^^^^^
KeyError: 'token'
# I only pasted the last Tracedback error.

Any idea for this issue? Thank you in advance for your help 😉

That’s great @Maxime Broussart

But, I am not really satisfied with this approach because I have to pass all the postgres credentials as plaintext in the body of the request. Do you think there is a more secure way?

You can include user/password as part of the URL if you prefer https://github.com/permitio/opal-fetcher-postgres#-possible-user-issues And you can fork the data-provider, and create one that take credentials from elsewhere (e.g. an env var, a file) Re:

In the meantime, I am trying option 1 with CLI using token but I am stuck in getting a datasource identity token. I added an env variable on my docker compose for the OPAL server :

OPAL_AUTH_MASTER_TOKEN=JwMXIIyrFR

, then with the cli :

opal-client obtain-token JwMXIIyrFR --type datasource --server-url <http://localhost:7002>

gives me this error :

I’m not sure what’s wrong, but it seems like the reply from the server side didn’t contain the token; you can probably debug it and see an error message from the server

Thank you @Or Weis! So with

--no-just-the-token

, I get : {'detail': 'opal server was not configured with security, cannot generate tokens!'} and at the same time, the OPAL Server logs

uvicorn.protocols.http.httptools_impl   | INFO  | 172.22.0.1:54224 - "POST /token HTTP/1.1" 503

However, on the docker-compose, I well added for the OPAL Server config : OPAL_AUTH_PRIVATE_KEY, OPAL_AUTH_PUBLIC_KEY, OPAL_AUTH_MASTER_TOKEN Any ideas? 🤔

indeed “🤔” Maybe the values you gave for the keys weren’t parsed correctly? @Asaf Cohen, @Ro'e Katz , @Shaul Kremer - any input ?

You should see in the opal server logs that it's indeed running in secure mode

Thank you for your advices! 😉 I may keep the Rest API option to refresh data