This message was deleted.

use case is very specific, my OPAL client and the service whose url is configured in data sources configuration run in the same pod so OPAL client tries to fetch data but my service isn’t yet available, and hence data fetch fails if there is a way OPAL client could retry it would solve this issue

Hi @Thilak Reddy we actually don't have configurable timeouts for getting the data source configuration:

async def get_policy_data_config(self, url: str = None) -> DataSourceConfig:
        try:
            async with ClientSession(headers=self._extra_headers) as session:
                ...
                response = await session.get(url, **self._ssl_context_kwargs)
                ...
        except:
            logger.exception(f"Failed to load data sources config")
            raise

That's not a hard fix, is that something you are willing to contribute as a PR? Otherwise i can put it in @Ro'e Katz’s queue.

seems like a small add, happy to contribute. i will take a look when i get some time if you think this is a good feature in general question around it • should it be an exponential retry? and should the exponent be configurable? ◦ asking to get your knowledge on how retry configurations in general look like

You can take a look how retries are done for the fetching engine. I think exponential backoff is ok and maybe also max retries. Best if it’s configurable.

Hi @Asaf Cohen have created a PR for adding retries to data source configurations, i see that tests in pipeline fail with

Cannot connect to host localhost:9321 ssl:default [Connect call failed ('127.0.0.1', 9321)]

not sure why that is happening, could someone help me with that? https://github.com/permitio/opal/pull/502

Hi @Ext Tchung thanks for the contribution! I'm currently in meetings, will take a look later today 🙂

you mentioned someone else xD, i see that you are clearly in a meeting 😆

haha @Thilak Reddy sorry about that, multitasking 😅

haha, i figured out why tests were failing, fixed it

oh cool

yes

👍