https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> I need to extract high-resolution (...
# replication-ask-ai
d
@kapa.ai I need to extract high-resolution (1024x1024) image URLs from the Facebook Marketing API, similar to how Windsor.ai does. Currently, the standard connector returns many 
NULL
values for 
thumbnail_url
because the data is deeply nested in 
object_story_spec
or 
asset_feed_spec
. Also, when it does return a URL, it's a low-res cropped version. How can I modify the 
DPathExtractor
in the Connector Builder to reach these nested hashes and URLs? 1. Is there a way to inject query parameters like 
thumbnail_width=1024
and 
thumbnail_height=1024
into the stream request to force Meta to generate the high-res 
scontent
signed URL? 2. Should I use a "Custom Transformation" after the extraction to handle the different JSON structures (Single Image vs DCO)?
k
The knowledge sources you provided cover: • How 
DpathExtractor
works and how to configure its 
field_path
to reach nested fields. [DpathExtractor] • How to inject custom query parameters (including complex or conditional ones) into requests in Connector Builder, both for incremental sync and partitioning. [Custom parameter injection; Partitioning injection] They do not include a full Facebook Marketing connector YAML or explicit examples for 
object_story_spec
/ 
asset_feed_spec
, nor do they mention 
thumbnail_width
/ 
thumbnail_height
specifically. So I’ll describe what is supported in Airbyte generically, and where I have to stay high‑level. ------------------------- 1. Reaching nested hashes/URLs with 
DpathExtractor
 
DpathExtractor
lets you walk nested JSON using an array path, with 
*
to iterate arrays. [DpathExtractor] Examples from the docs:
Copy code
field_path:
  - data
Copy code
field_path:
  - data
  - records
Copy code
field_path:
  - data
  - "*"
  - record
To reach deeply nested image hashes/URLs in 
object_story_spec
or 
asset_feed_spec
, you would: 1. Set the stream’s record selector to the top‑level list (e.g. 
["data"]
if the API returns 
{ "data": [ ...ad_creatives... ] }
). 2. Then, in the schema / field definitions, map nested properties using JSON paths relative to each record (Connector Builder UI lets you pick nested fields visually; in YAML you’d use the appropriate path). If the images are in an array, you’d use 
*
in the 
field_path
to iterate over them, e.g.:
Copy code
field_path:
  - data
  - "*"
  - object_story_spec
  - "*"
  - image_hash
The docs don’t show a Facebook‑specific example, but this is the supported pattern for nested extraction. [DpathExtractor] ------------------------- 2. Injecting 
thumbnail_width
/ 
thumbnail_height
query parameters The docs explicitly support custom parameter injection for query params, headers, and body fields using Jinja and available variables. [Custom parameter injection] Key points: • You can disable the built‑in “Inject start time / end time” or “Inject Parameter / Parent Key Value” options. • Then use the generic parameter section at the bottom of the stream configuration to add arbitrary query parameters. • Values can be static or Jinja expressions using variables like 
config
, 
stream_interval
, 
stream_partition
, etc. [Custom parameter injection; Partitioning injection] So, for your case, you can configure the stream like: • Add query parameter: ◦ key: 
thumbnail_width
◦ value: 
1024
• Add query parameter: ◦ key: 
thumbnail_height
◦ value: 
1024
This is fully supported by the generic parameter mechanism; the docs just don’t mention these specific Facebook parameters. Whether Meta actually returns a higher‑res 
scontent
URL for those values is API‑specific and not covered in the Airbyte docs, so I can’t confirm the effect, only that Airbyte can send those params. ------------------------- 3. Using “Custom Transformation” for different JSON structures The knowledge sources you shared do not describe the “Custom Transformation” feature or give examples of handling different shapes (e.g. Single Image vs DCO) post‑extraction, so I can’t give a documented, step‑by‑step recipe for that. What is documented is: • Complex logic and composition of values can be done at the request parameter level using Jinja (e.g. combining multiple values into one query parameter, adding prefixes/suffixes, conditional logic). [Custom parameter injection] • Similar custom injection is available for partitioning, using variables like 
stream_partition
. [Partitioning injection] The docs do not explicitly say whether you should normalize Single Image vs DCO via a “Custom Transformation” vs via schema/field mapping, so I can’t state a recommended pattern from the sources. ------------------------- Summary 1. DPathExtractor: You can reach nested image hashes/URLs in 
object_story_spec
/ 
asset_feed_spec
by configuring 
field_path
with nested keys and 
*
for arrays (e.g. 
["data", "*", "object_story_spec", "*", "image_hash"]
), as per the generic 
DpathExtractor
behavior. [DpathExtractor] 2. High‑res thumbnails: You can inject 
thumbnail_width=1024
and 
thumbnail_height=1024
as query parameters using the generic parameter section in Connector Builder; this is supported, but the docs do not confirm how Facebook’s API will respond to those values. [Custom parameter injection] 3. Custom Transformation: The provided docs don’t cover using “Custom Transformation” to unify Single Image vs DCO structures, so I can’t give a documented best practice there. You may still use it, but that’s outside what’s described in these sources. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
Persistent HTTP 400 & StreamThreadException in Custom Meta Ads Connector (v25.0) @kapa.ai I am experiencing a critical failure in a custom connector built via Connector Builder. While the "Test" feature works for single partitions, the production sync crashes repeatedly. Environment details: • Airbyte Type: Self-hosted on VPS (12GB RAM). • CDK Component: Declarative Manifest (YAML). • API: Meta Graph API v25.0 (
/adcreatives
). • Meta App Tier: Development Access. • Architecture: 
ListPartitionRouter
iterating over 11 account IDs with 
concurrency_level: 1
. The Error Trace: Plaintext
Copy code
airbyte_cdk.sources.concurrent_source.stream_thread_exception.StreamThreadException: 
Exception while syncing stream ad_creatives_hd: Bad request. Please check your request parameters.
Context & Observations: The error occurs when requesting high-resolution fields: 
thumbnail_width: "1024"
and 
thumbnail_height: "1024"
. In Meta's Development Access tier, I suspect they return a 400 Bad Request (Error code 80004) instead of a 429 when hitting rate limits for heavy media fields. 1. I have implemented a 
DefaultErrorHandler
with 
ConstantBackoffStrategy
and 
response_filters
to catch HTTP 400 and trigger a RETRY. 2. The problem: Even with the retry logic defined in YAML, the 
StreamThreadException
seems to bypass the error handler and kills the entire sync process immediately. Specific Questions: Why is the 
StreamThreadException
crashing the sync instead of allowing the 
DefaultErrorHandler
to perform the retry on the 400 error? Is there a known issue in the Concurrent Source implementation where thread-level exceptions ignore declarative error handlers? 1. Given the Development Tier restrictions, what is the recommended way to "throttle" a declarative connector to wait between partitions/accounts without triggering a thread timeout? Relevant YAML Snippet: YAML
Copy code
error_handler:
  type: DefaultErrorHandler
  backoff_strategies:
    - type: ConstantBackoffStrategy
      backoff_time_in_seconds: 20
  response_filters:
    - http_codes: [400, 429, 500]
      action: RETRY
k
I’m sorry, but the provided knowledge sources do not contain information specific to: • 
StreamThreadException
behavior in 
concurrent_source
• How declarative 
error_handler
interacts with thread-level exceptions • Throttling/waiting between partitions/accounts in a concurrent/declarative source They only document: • Declarative error handling (
DefaultErrorHandler
, 
CompositeErrorHandler
, response filters, backoff strategies) [Config error handling] • Connector Builder error handling and backoff strategies in general [CB error handling; Backoff strategies] • Some practical notes from a forum thread about Builder tests vs real syncs, and how default behavior on 429/5xx can be overridden only by editing YAML directly [Forum 9985] Given that limitation, I can only answer at a high level and cannot confirm internal 
concurrent_source
behavior or known bugs. ------------------------- 1. Why is 
StreamThreadException
crashing the sync instead of retrying on 400? From the docs, declarative error handling works like this: • By default, only HTTP 429 and 5XX are retried up to 5 times with exponential backoff. [Config error handling] • You can override this with a 
DefaultErrorHandler
on the 
requester.error_handler
, specifying: ◦ 
response_filters
with 
http_codes
and 
action
(
RETRY
, 
IGNORE
, 
FAIL
, 
SUCCESS
) ◦ 
backoff_strategies
(constant, exponential, wait-from-header, etc.) [Config error handling; Backoff strategies] Example from docs to retry 404:
Copy code
requester:
  ...
  error_handler:
    response_filters:
      - http_codes: [404]
        action: RETRY
And to ignore 404 and retry 429:
Copy code
requester:
  ...
  error_handler:
    response_filters:
      - http_codes: [404]
        action: IGNORE
      - http_codes: [429]
        action: RETRY
[Config error handling] However, the docs do not describe: • How 
StreamThreadException
is raised or propagated in 
concurrent_source
• Whether all HTTP 4xx/5xx responses are guaranteed to pass through the declarative 
error_handler
before being wrapped in 
StreamThreadException
So I cannot say from the documentation why, in your case, the 400 is not being handled by the 
DefaultErrorHandler
and instead surfaces as a 
StreamThreadException
that stops the sync. This could be: • A limitation/bug in how concurrent streams wrap exceptions, or • A mismatch between where the 
error_handler
is attached (e.g., not on the exact 
requester
that is making the failing call), or • A non-HTTP error being raised before the declarative layer sees the response But none of that is explicitly covered in the docs, so I can’t confirm. One relevant forum insight (different issue, but about behavior differences) is: • Builder “Test” does not actually perform backoff/retries; only real syncs do. [Forum 9985] • Editing YAML directly can change how error handling overrides defaults, whereas the UI-generated 
CompositeErrorHandler
may not behave as expected in some cases. [Forum 9985] This suggests that behavior can differ between UI tests and production syncs, but it does not address 
StreamThreadException
specifically. ------------------------- 2. Is there a known issue where concurrent threads ignore declarative error handlers? The knowledge sources do not mention: • Any known issues/bugs with 
concurrent_source
• Any statement that thread-level exceptions bypass declarative 
error_handler
So I cannot confirm a known issue from the official docs or the provided forum thread. ------------------------- 3. Throttling / waiting between partitions/accounts The docs describe backoff strategies for retrying failed requests: • Constant backoff: fixed wait time between retries. [Backoff strategies] • Exponential backoff: increasing wait times (factor * 2^attempt). [Backoff strategies] • Wait time from header / wait until time from header. [Backoff strategies] These are configured on the 
error_handler
and apply per failed request, not between partitions/accounts in general. The docs do not describe: • A declarative way to “sleep” or throttle between partitions/accounts • A configuration to globally slow down a 
ListPartitionRouter
or 
concurrent_source
beyond per-request backoff So I cannot point to a documented way to “wait between partitions” other than: • Using backoff strategies triggered by specific HTTP codes (e.g., 429, 5xx) via 
response_filters
. [Config error handling; Backoff strategies] Your YAML snippet:
Copy code
error_handler:
  type: DefaultErrorHandler
  backoff_strategies:
    - type: ConstantBackoffStrategy
      backoff_time_in_seconds: 20
  response_filters:
    - http_codes: [400, 429, 500]
      action: RETRY
is consistent with the documented schema for 
DefaultErrorHandler
and 
HttpResponseFilter
. [Config error handling] But the docs don’t say how to add an additional delay between partitions when there is no error. ------------------------- What you can reliably rely on from the docs Within the limits of the documentation: • Your use of 
DefaultErrorHandler
with 
response_filters
and 
ConstantBackoffStrategy
is aligned with the declarative error handling model. [Config error handling; Backoff strategies] • Only 429 and 5xx are retried by default; adding 400 with 
action: RETRY
is the correct way (per docs) to attempt retries on 400. [Config error handling] • Builder “Test” will not actually perform backoff/retries; you must inspect logs from a real sync to see retry behavior. [Forum 9985] For the specific 
StreamThreadException
behavior, concurrent source internals, and partition-level throttling, the knowledge sources do not provide enough detail to give a definitive explanation or a documented workaround. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai I am building a custom connector for Facebook Ads. I need to get the Ad Creative image in 1024x1024 resolution. Currently, the 
image_url
field in the 
/ads
endpoint returns a 
64x64
thumbnail or a dynamic asset hash. How can I implement a parent-child stream (Substream) in YAML to fetch the 
url
from the 
/adimages
endpoint using the 
image_hash
found in the 
/ads
creative spec, ensuring I get the high-resolution source URL?
k
The knowledge sources don’t mention Facebook Ads specifically, but they do document exactly how to build a parent–child (substream) relationship in a declarative YAML connector using 
ParentStreamConfig
, and how to extract nested fields with 
DpathExtractor
. I’ll show how to wire those pieces together; you’ll still need to adapt field names to your exact Facebook responses. ------------------------- 1. Concept: 
/ads
as parent, 
/adimages
as child You want: • Parent stream: 
/ads
(or 
/adcreatives
) ◦ Extract 
image_hash
from the creative spec. • Child stream: 
/adimages
◦ Use 
image_hash
as a partition key to request the high‑res image and read its 
url
. This is exactly what 
ParentStreamConfig
is for: it lets you build partitions (slices) for a child stream from records of a parent stream, passing a 
parent_key
into the child’s requests. [ParentStreamConfig] ------------------------- 2. Parent stream: extract 
image_hash
with 
DpathExtractor
 Assume the 
/ads
(or 
/adcreatives
) response looks roughly like:
Copy code
{
  "data": [
    {
      "id": "123",
      "creative": {
        "object_story_spec": {
          "link_data": {
            "image_hash": "abc123"
          }
        }
      }
    }
  ]
}
You configure the parent stream to select the list of ads/creatives with `DpathExtractor`:
Copy code
streams:
  - type: DeclarativeStream
    name: ads
    retriever:
      type: SimpleRetriever
      requester:
        # your base_url, path, auth, etc.
        path: "/v25.0/act_{account_id}/ads"
      record_selector:
        extractor:
          type: DpathExtractor
          field_path: ["data"]  # root array of ads
DpathExtractor
uses 
field_path
as an array of keys, with 
*
to iterate arrays. [Record selector; DpathExtractor] You then ensure the parent stream’s schema exposes 
image_hash
(e.g., via a property path like 
creative.object_story_spec.link_data.image_hash
in the Builder UI or equivalent JSON schema), so that 
image_hash
is available as a field in each parent record. ------------------------- 3. Child stream: 
/adimages
as a substream using 
ParentStreamConfig
 Now define the 
/adimages
stream as a substream of 
ads
, using `ParentStreamConfig`:
Copy code
- type: DeclarativeStream
    name: adimages
    retriever:
      type: SimpleRetriever
      requester:
        # base_url, auth, etc.
        path: "/v25.0/act_{account_id}/adimages"
        request_parameters:
          # Inject the parent image_hash into the child request
          hash: "{{ stream_slice.parent_id }}"   # see partition_field below
      partition_router:
        type: ListPartitionRouter
        parent_stream_configs:
          - type: ParentStreamConfig
            stream: "#/streams/0"          # reference to the 'ads' stream above
            parent_key: "image_hash"       # field from parent records
            partition_field: "parent_id"   # name used in stream_slice
            request_option:
              type: RequestOption
              inject_into: request_parameter
              field_name: "hash"          # query param name expected by /adimages
Key pieces from `ParentStreamConfig`: [ParentStreamConfig] • `stream`: reference to the parent stream (
ads
). • `parent_key`: the field in parent records used to build partitions (here, 
image_hash
). • `partition_field`: the name under which that value is exposed in 
stream_slice
(e.g., 
parent_id
). • `request_option`: describes where and under what name to inject the parent key into the child request (here, as query parameter 
hash
). During sync: 1. Airbyte reads 
ads
records. 2. For each record, it takes 
image_hash
and creates a slice with 
stream_slice.parent_id =
image_hash. 3. For each slice, the child stream calls `/adimages?hash=`image_hash and reads the image record(s), including the high‑res 
url
. ------------------------- 4. Extracting the high‑res 
url
from 
/adimages
 Assume 
/adimages
returns:
Copy code
{
  "data": [
    {
      "hash": "abc123",
      "url": "https://...1024x1024..."
    }
  ]
}
Configure the child stream’s record selector:
Copy code
retriever:
      type: SimpleRetriever
      requester:
        path: "/v25.0/act_{account_id}/adimages"
        request_parameters:
          hash: "{{ stream_slice.parent_id }}"
      record_selector:
        extractor:
          type: DpathExtractor
          field_path: ["data"]   # list of image records
DpathExtractor
with 
["data"]
will yield each image record; your schema then exposes the 
url
field as the high‑resolution image URL. [Record selector; DpathExtractor] ------------------------- 5. Notes / limitations from docs • The docs show 
ParentStreamConfig
and 
DpathExtractor
generically; they don’t include Facebook‑specific examples or confirm that 
/adimages
always returns 1024×1024, so you’ll need to verify the exact Graph API behavior. [ParentStreamConfig; DpathExtractor] • If your 
image_hash
is nested differently (e.g., arrays, DCO), adjust the parent stream’s schema/paths accordingly, using 
*
in 
field_path
where needed. [Record selector] But structurally, the pattern above—`ads` as parent, 
adimages
as substream with 
ParentStreamConfig
and `request_option`—is the documented way to implement this in YAML. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai I am building a custom Declarative Connector for Facebook Ads using the Connector Builder. My goal is to extract the High-Resolution media URLs (1024x1024 or original) for ad creatives.
Currently, querying 
fields=id,name,creative{image_url,thumbnail_url}
returns highly compressed 64x64 pixel thumbnails (the CDN URL contains the parameter 
p64x64
). Competitor tools like Windsor return the exact same CDN URL but with 
p1024x1024
.
My creatives are a mix of Images, Videos, and Carousels. I previously tried a Parent-Child stream (Parent: 
/ads
, Child: 
/adimages
using 
creative.image_hash
), but it failed because video ads don't return an 
image_hash
, causing the child stream request to crash.
What is the best and most reliable way to implement this in the Declarative YAML? Options I'm considering (but need the YAML syntax for):
Is there a way to extract the HD URL directly from the 
object_story_spec
using 
DpathExtractor
with fallbacks?
Can I use a Transformation in Airbyte to do a string replacement on the URL (replace 
p64x64
with 
p1024x1024
)?
How can I safely configure a Parent-Child stream that handles missing 
image_hash
values without failing the entire sync?
Please provide the recommended YAML implementation.
k
The knowledge sources you provided cover the generic declarative building blocks you need (
DpathExtractor
, 
ParentStreamConfig
, record filtering/transformations), but they do not contain Facebook‑specific examples, nor do they document any guarantee about 
p64x64
vs 
p1024x1024
behavior on Meta’s CDN. So I can show you how to wire this in YAML, but I can’t confirm which exact Graph fields/URLs will be HD in all creative types. Below are patterns that are supported by the declarative framework. 1. Extracting nested URLs with DpathExtractor (with “fallbacks”) 
DpathExtractor
lets you walk nested JSON using a list of keys, with 
*
for arrays. [DpathExtractor] Examples from the docs:
Copy code
field_path: ["data"]
field_path: ["data", "records"]
field_path: ["data", "*", "record"]
In your case, you can: 1. Use 
DpathExtractor
to select the top‑level list of creatives/ads (e.g. 
["data"]
). 2. Then, in the schema / field definitions, map multiple possible nested paths for the URL (image, video, carousel) and let them be nullable. The declarative spec itself doesn’t support “if A is null then use B” inside 
DpathExtractor
, but you can expose multiple candidate fields and then combine them in a transformation (see section 2). For example, parent stream for 
/ads
or `/adcreatives`:
Copy code
streams:
    - type: DeclarativeStream
    name: ad_creatives
    retriever:
      type: SimpleRetriever
      requester:
        path: "/v25.0/act_{account_id}/adcreatives"
        # add your fields=id,name,object_story_spec,... here
      record_selector:
        extractor:
          type: DpathExtractor
          field_path: ["data"]
Then in the schema (or via transformations) you can expose: • 
object_story_spec.link_data.image_url
• 
object_story_spec.video_data.image_url
• 
object_story_spec.carousel_data.child_attachments[*].image_url
using 
*
where needed in 
field_path
if you want to explode arrays. [DpathExtractor] Because the docs don’t show a built‑in “fallback” operator, the usual pattern is: • Extract multiple nullable fields. • Use a transformation to coalesce them into a single “best URL” field. 2. String replacement transformation (e.g. p64x64 → p1024x1024) The knowledge sources don’t document a dedicated “replace” transformation type, but they do show that you can use Jinja expressions in transformations (e.g. 
AddField
) and that 
AddField
will overwrite an existing key. [Replace transformation forum] From the forum example:
Copy code
{{ record['iri'] | replace(config['base_url'], '') | replace('/task/', '') }}
The answer notes: > 
AddField
will replace if the key already exists, it might be useful here. [Replace transformation forum] So you can: 1. Extract the original URL into a field, e.g. 
image_url_raw
. 2. Use an 
AddField
transformation with a Jinja expression that: ◦ Picks the first non‑null candidate URL (image / video / carousel). ◦ Replaces 
p64x64
with 
p1024x1024
. Example transformation (conceptual YAML, exact placement depends on your manifest version):
Copy code
transformations:
    - type: AddField
    field_name: hd_image_url
    value: >-
      {{
        (
          record.get('image_url_raw')
          or record.get('video_image_url_raw')
          or record.get('carousel_image_url_raw')
        )
        | replace('p64x64', 'p1024x1024')
      }}
This pattern (Jinja + 
replace
+ 
AddField
overwriting/adding a field) is explicitly shown in the forum thread and is supported. [Replace transformation forum] Important limitation: The docs do not confirm that simply changing 
p64x64
to 
p1024x1024
is always valid for Meta’s CDN URLs, so this is a heuristic, not a guaranteed API contract. 3. Safe Parent–Child stream using ParentStreamConfig (handling missing image_hash) 
ParentStreamConfig
is the documented way to build a substream whose partitions come from a parent stream’s records. [ParentStreamConfig] Key fields: • `stream`: reference to the parent stream. • `parent_key`: field from parent records used to build partitions. • `partition_field`: name under which that value is exposed in 
stream_slice
. • `request_option`: where/how to inject the parent key into the child request. Example pattern (parent: 
ad_creatives
, child: 
adimages
):
Copy code
streams:
    - type: DeclarativeStream
    name: ad_creatives
    retriever:
      type: SimpleRetriever
      requester:
        path: "/v25.0/act_{account_id}/adcreatives"
      record_selector:
        extractor:
          type: DpathExtractor
          field_path: ["data"]

    - type: DeclarativeStream
    name: adimages
    retriever:
      type: SimpleRetriever
      requester:
        path: "/v25.0/act_{account_id}/adimages"
        request_parameters:
          hash: "{{ stream_slice.image_hash }}"
      partition_router:
        type: ListPartitionRouter
        parent_stream_configs:
                    - type: ParentStreamConfig
            stream: "#/streams/0"          # reference to ad_creatives
            parent_key: "image_hash"       # field in parent records
            partition_field: "image_hash"  # name in stream_slice
            request_option:
              type: RequestOption
              inject_into: request_parameter
              field_name: "hash"
[ParentStreamConfig] Handling missing `image_hash`: The docs do not explicitly describe skipping partitions when 
parent_key
is null, nor do they show a built‑in filter on parent records inside 
ParentStreamConfig
. However, a forum thread explains that you can use a record filter on the parent stream to avoid creating slices for records that don’t have the required field. [Parent filtering forum] From that thread: • Adding 
record_filter
to the parent stream YAML lets you pre‑filter parent records (e.g., only those that have the field needed by the child). • This is not yet exposed in the Builder UI, but can be added in YAML mode. [Parent filtering forum] So, to avoid crashing on video ads without 
image_hash
, you can: 1. Add a 
record_filter
to the parent stream to keep only records where 
image_hash
is present. 2. Then 
ParentStreamConfig
will only see valid parents. Conceptual example (exact syntax may vary by manifest version; the docs link to record selector filtering): [Record selector filtering; Parent filtering forum]
Copy code
- type: DeclarativeStream
    name: ad_creatives
    retriever:
      type: SimpleRetriever
      requester:
        path: "/v25.0/act_{account_id}/adcreatives"
      record_selector:
        extractor:
          type: DpathExtractor
          field_path: ["data"]
        record_filter:
          condition: "{{ record.get('image_hash') is not none }}"
This way: • Only creatives with 
image_hash
produce child partitions. • Video‑only creatives (no 
image_hash
) are filtered out and won’t cause 
/adimages
calls with empty hashes. The forum post confirms this pattern conceptually (filtering parent records to avoid invalid child requests), though it doesn’t show your exact field names. [Parent filtering forum] 4. Putting it together: recommended approach Given the docs and your constraints: 1. Primary approach ◦ Use a single parent stream (
/adcreatives
or 
/ads
with 
creative{...}
) with 
DpathExtractor
to get the list of creatives. [DpathExtractor] ◦ Expose multiple candidate URL fields from 
object_story_spec
(image, video, carousel). ◦ Use an 
AddField
transformation with Jinja to: ▪︎ Coalesce the candidates into one field. ▪︎ Optionally replace 
p64x64
with 
p1024x1024
. [Replace transformation forum] 1. Optional enhancement for images only ◦ Add a child 
/adimages
stream using 
ParentStreamConfig
with 
parent_key: image_hash
. [ParentStreamConfig] ◦ Add a 
record_filter
on the parent to keep only records with 
image_hash
, so missing hashes (e.g. pure video creatives) don’t create invalid child requests. [Parent filtering forum] 1. Limitations to be aware of ◦ The docs do not guarantee that changing 
p64x64
to 
p1024x1024
is always valid; that’s inferred from observed CDN behavior, not from Airbyte docs. ◦ There is no documented built‑in “fallback” operator inside `DpathExtractor`; fallbacks are implemented via transformations (Jinja) over multiple nullable fields. If you share a sample JSON payload for your 
object_story_spec
(image, video, carousel), I can help you sketch the exact 
field_path
arrays and Jinja expressions within these documented patterns. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
3 Views
Open in Slack
PreviousNext
Powered by