How to include paremeters into the request body? I have an API that the start date, end date, dimensions paremeters must be sent in the body

Injecting Parameters into the Request Body In Airbyte's low-code connector framework, you can use

request_body_json

in the

HttpRequester

to send parameters (like

startDate

,

endDate

, and

dimensions

) in the POST request body. Using request_body_json in YAML You can directly define the JSON body in your requester configuration using Jinja expressions with

stream_interval

variables for dynamic date ranges:

requester:
  type: HttpRequester
  url_base: "https://api.example.com"
  path: "/your/endpoint"
  http_method: POST
  request_headers:
    Content-Type: "application/json"
  request_body_json:
    startDate: "{{ stream_interval.get('start_time') }}"
    endDate: "{{ stream_interval.get('end_time') }}"
    dimensions:
            - date
            - country

The

stream_interval.start_time

and

stream_interval.end_time

variables are automatically populated by the incremental sync cursor. [YAML Reference] Real-world Example (Google Search Console) Here's how the Google Search Console connector does it:

requester:
  type: HttpRequester
  url_base: https://www.googleapis.com/webmasters/v3
  path: "/sites/{{ site_url }}/searchAnalytics/query"
  http_method: POST
  request_headers:
    Content-Type: "application/json"
  request_body_json:
    startDate: "{{ stream_interval.get('start_time') }}"
    endDate: "{{ stream_interval.get('end_time') }}"
    dimensions: ["date", "country"]
    type: "web"
    aggregationType: auto

[Google Search Console manifest] Using the Connector Builder UI If you're using the Connector Builder UI instead of writing YAML directly: 1. In the Incremental Sync form, disable the "Inject start time / end time into outgoing HTTP request" option. 2. Go to the generic parameter section at the bottom of the stream configuration form. 3. Add your body fields there using Jinja expressions: ◦ Field Name:

startDate

→ Value:

{{ stream_interval.start_time }}

◦ Field Name:

endDate

→ Value:

{{ stream_interval.end_time }}

◦ Field Name:

dimensions

→ Value: your dimensions list [Custom Parameter Injection] Nested Body Parameters If your API requires nested JSON (e.g.,

{ "filter": { "fromDateTime": "...", "toDateTime": "..." } }

), you can nest objects directly in `request_body_json`:

request_body_json:
  filter:
    fromDateTime: "{{ stream_interval.start_time }}"
    toDateTime: "{{ stream_interval.end_time }}"

[Forum discussion] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai how the

incremental_sync

must be defined? I have the following now:

cursor_datetime_formats:
          - "%Y-%m-%d"
        datetime_format: "%Y-%m-%d"
        start_datetime:
          type: MinMaxDatetime
          datetime: "{{ config['start_date'] }}"
          datetime_format: "%Y-%m-%dT%H:%M:%SZ"
        start_time_option:
          type: RequestOption
          field_name: start_time
          inject_into: request_parameter
        end_time_option:
          type: RequestOption
          field_name: end_time
          inject_into: request_parameter
        end_datetime:
          type: MinMaxDatetime
          datetime: "{{ config['end_date'] if config.get('end_date') else day_delta(-1, format='%Y-%m-%dT%H:%M:%SZ') }}"
          datetime_format: "%Y-%m-%dT%H:%M:%SZ"

Defining incremental_sync for Request Body Injection Since your API requires parameters in the request body (not query parameters), you should not use

start_time_option

/

end_time_option

with

inject_into: request_parameter

. Instead, remove those options and handle the date injection directly in your

request_body_json

. Here's how your

incremental_sync

should look:

incremental_sync:
  type: DatetimeBasedCursor
  cursor_field: date
  cursor_datetime_formats:
    - "%Y-%m-%d"
  datetime_format: "%Y-%m-%d"
  step: P1D
  cursor_granularity: PT1S
  start_datetime:
    type: MinMaxDatetime
    datetime: "{{ config['start_date'] }}"
    datetime_format: "%Y-%m-%dT%H:%M:%SZ"
  end_datetime:
    type: MinMaxDatetime
    datetime: "{{ config['end_date'] if config.get('end_date') else day_delta(-1, format='%Y-%m-%dT%H:%M:%SZ') }}"
    datetime_format: "%Y-%m-%dT%H:%M:%SZ"

Key changes: • Remove

start_time_option

and

end_time_option

— these inject into request parameters, not the body. • Add

step

and

cursor_granularity

— these are required fields for

DatetimeBasedCursor

. [DatetimeBasedCursor docs] Then, in your requester, inject the dates via

request_body_json

using `stream_interval`:

requester:
  type: HttpRequester
  http_method: POST
  request_body_json:
    startDate: "{{ stream_interval.start_time }}"
    endDate: "{{ stream_interval.end_time }}"
    dimensions:
      - your_dimension

The

stream_interval.start_time

and

stream_interval.end_time

are automatically populated by the cursor for each time window partition. [DatetimeBasedCursor docs] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai This is my configuration:

retriever:
        type: SimpleRetriever
        requester:
          $ref: "#/definitions/base_requester"
          path: reports
          http_method: POST
          # Dimensions requested under DATA-3554: <https://kolibrigames.atlassian.net/jira/servicedesk/projects/DATA/queues/custom/105/DATA-3554>
          request_body_json:
            date_range:
              start: "{{ stream_interval.get('start_time') }}"
              end: "{{ stream_interval.get('end_time') }}"
            dimensions:
              - DATE
              - APP_OR_SITE
              - CAMPAIGN
              - AD_GROUP
              - CREATIVE_GROUP
              - CREATIVE
              - SUB_PUBLISHER
              - TRAFFIC
              - EXCHANGE
            ad_account_id: '{{ config[''ad_account_id''] }}'

The connector is failing because the request body is not being defined. I have a custom requester

The knowledge sources do not contain enough information about custom requesters and how they interact with

request_body_json

to give you a fully confident answer. However, based on what the knowledge sources do show, here are a few things to check: 1. Custom Requester May Not Inherit request_body_json Since you're using a

CustomRequester

(with a

class_name

), the

request_body_json

defined in the YAML may not be automatically picked up depending on how your custom class is implemented. The LinkedIn Ads connector, for example, uses a

CustomRequester

but relies on

request_parameters

instead:

requester:
  type: CustomRequester
  class_name: source_declarative_manifest.components.SafeEncodeHttpRequester
  url_base: https://api.linkedin.com/rest/
  request_parameters:
    dateRange: "..."

[LinkedIn Ads manifest] 2. Check How Your $ref: "#/definitions/base_requester" Is Defined If your

base_requester

already defines a

request_body_json

, the child definition may be overriding it rather than merging. YAML

$ref

with overrides replaces the entire field, not merges it. 3. Try Using request_body_json at the base_requester Level Move the

request_body_json

into the

base_requester

definition directly, or ensure your custom requester class explicitly handles the body fields passed from the YAML. I'd recommend checking: • Your custom requester's Python implementation to confirm it reads

request_body_json

from the YAML config. • Whether the

$ref

+ override pattern is causing the body to be dropped. Could you share your

base_requester

definition and custom requester class? That would help narrow down the issue. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.