https://linen.dev logo
Join Slack
Powered by
Hello. I'm running a quick proof of concept with O...
# opal
j
Hello. I'm running a quick proof of concept with OPA + OPAL on a local workstation where I have OPAL configured to fetch data with the 
HttpFetchProvider
from an API. Is there a working example of using the 
HttpFetchProvider
for a 
POST
request that includes a request body? I've tried a number of things, but every time I try to include a request body, it appears the 
HttpFetcherProvider
doesn't even attempt to make a call out to the configured endpoint. For example, I have this set in 
OPAL_DATA_CONFIG_SOURCES
in the 
permitio/opal-server:latest
container I have running locally:
Copy code
{
  "config": {
    "entries": [
      {
        "url": "<http://myapi:8080/v1/path>",
        "config": {
          "fetcher": "HttpFetchProvider",
          "method": "post",
          "data": {
            "blah": "blah"
          }
        },
        "topics": [
          "blah"
        ],
        "dst_path": "blah"
      }
    ]
  }
}
This results in a 
aiohttp.client_exceptions.ClientResponseError: 400, message='Bad Request', url='<http://myapi:8080/v1/path>
in the 
permitio/opal-client-standalone:latest
container I have running locally, and with a 
ERROR  | Timeout while fetching url: <http://openfga:8080/stores/01K7MTJYKRN6V5TQ1WT8D83QDQ/list-user>
error. The logs from the app running at 
<http://myapi:8080>
don't seem to indicate that the request made it to my app. However, if I remove 
config.data
from the configuration, the logs from my app do indicate the request was received. My app responds with a 
400
in this case because the request body is required, but it at least proves the request has been received. I will admit it is confusing that the OPAL client indicates a 
400
was received from the API when 
config.data
is included but, my app logs don't indicate the request was received.
a
Hi @JT Wheeler, I'm an AI assistant for Permit.io. I'll analyze available sources and get back to you shortly.
👀 1
Based on the OPAL documentation, the 
HttpFetchProvider
configuration structure for POST requests with data is documented through the 
OPALHttpFetcherConfig
interface (1) . The configuration includes:
Copy code
export interface OPALHttpFetcherConfig {
  fetcher?: string;
  headers?: { [key: string]: string };
  is_json?: boolean;
  process_data?: boolean;
  method?: HttpMethods;
  data?: any;
}
Your configuration structure appears correct. The issue you're experiencing might be related to missing configuration parameters. Try adding the 
is_json
flag to your configuration:
Copy code
{
  "config": {
    "entries": [
      {
        "url": "<http://myapi:8080/v1/path>",
        "config": {
          "fetcher": "HttpFetchProvider",
          "method": "post",
          "is_json": true,
          "data": {
            "blah": "blah"
          }
        },
        "topics": [
          "blah"
        ],
        "dst_path": "blah"
      }
    ]
  }
}
The 
is_json
parameter tells the HttpFetchProvider how to handle the data . You might also want to explicitly set content-type headers:
Copy code
"config": {
  "fetcher": "HttpFetchProvider",
  "method": "post",
  "is_json": true,
  "headers": {
    "Content-Type": "application/json"
  },
  "data": {
    "blah": "blah"
  }
}
The timeout error you're seeing suggests the request might not be formatted correctly for your API. The 
process_data
flag is another option that affects how the fetcher handles the response data . Since you're experiencing this issue with your PDP/policy checks, could you please share: 1. Which Workspace / Project / Env you are using 2. What calls you are performing (as SDK code or CURL) 3. Which PDP version you are running 4. Relevant logs from your PDP This will help our engineers investigate the specific behavior you're encountering.
a
Hey @JT Wheeler, thanks for reaching out, we will circle back to you soon
👍 1
j
I'm trying the suggestion, but 
is_json
appears to be set to 
True
by default.
It's the same behavior if I include 
is_json
and 
headers
in the 
OPAL_DATA_CONFIG_SOURCES
config
o
Hi, welcome 🤗 The built-in HTTP data fetcher doesn't support passing a request body. But you can: 1. Write your own custom fetcher, that's the purpose of this entire mechanism- to be extensible 2. Contribute an option to do so in the built in fetcher via a PR
j
I see. I misinterpreted the data field of the 
HttpFetcherConfig
class.
I'll try my own custom fetcher as a starting place. Thanks for clarifying!
💜 1
Open in Slack
PreviousNext
Powered by