This message was deleted.
# opals
Slackbot
11/29/2022, 11:18 AMThis message was deleted.
a
Asaf Cohen
11/29/2022, 11:22 AMHi @Bart And and welcome 🙂
Check out this example repo, and specifically the configuration in docker compose:
https://github.com/permitio/opal-fetcher-postgres
Copy code
OPAL_DATA_CONFIG_SOURCES={"config":{"entries":[{"url":"<postgresql://postgres@example_db:5432/postgres>","config":{"fetcher":"PostgresFetchProvider","query":"SELECT * from city;","connection_params":{"password":"postgres"}},"topics":["policy_data"],"dst_path":"cities"}]}}Asaf Cohen
11/29/2022, 11:22 AMyou do not manipulate the save method at all (this determines how the data is saved to OPA)
instead you use the fetcher custom config to pass what query you need to run against the db
Asaf Cohen
11/29/2022, 11:22 AM Copy code
"query":"SELECT * from city;"Asaf Cohen
11/29/2022, 11:23 AMhope it helps 🙂
b
Bart And
11/29/2022, 12:52 PMHi, thanks a lot for quick response. I went trough all the documentation, but I still do not catch one thing. If we change the record in db, we still have ex. "select ... where id=xxx". When we add new record, have the same select.
Bart And
11/29/2022, 12:54 PMand question how we can adapt/transform selected data to specific data structure on OPA side?
a
Asaf Cohen
11/29/2022, 1:13 PMre:2 - you transform the data returned from DB to json in the process method of the fetcher
Asaf Cohen
11/29/2022, 1:13 PMsee the code here: https://github.com/permitio/opal-fetcher-postgres
Asaf Cohen
11/29/2022, 1:15 PMre:1 i'm not sure i am following.
sending the update will fetch the select query in that moment and update the data in OPA.
you will have to re-send the update through OPAL every time there is a change. OPAL will not know to track the database.
b
Bart And
11/29/2022, 1:46 PMThanks. ad1, may be add/update was wrong example as OPA API rest call looks the same. So just one more question, how can I notify OPA that record was deleted (in example data structure I remove specific city from cities table and want to update this record in OPA)? doing delta update, not reloading all cities.
a
Asaf Cohen
11/29/2022, 1:49 PMCurrently we do not have a delete instruction. There is an outstanding PR but there is still some work to do in order to merge it in.
The the meanwhile structure your data as dictionaries / hash maps and not as lists.
You can override a specific city record key by doing PUT on the city unique key and putting {} as value
b
Bart And
11/29/2022, 2:02 PMOK, thanks a lot! Now it looks clear.
2 Views