Hello all, I'm confused about how to implement thi...
# help-connector-development
j
Hello all, I'm confused about how to implement this feature. I want to iterate over some data coming back from an API and pass it to another API. Think of it like a list of product descriptions and then calling another API to get more details about each product. However, the API in question just has a list of string GUIDs to pass to the individual API to get more info. Here's an example response:
Copy code
{
  "StartDate": "2025-12-15T04:00:21.433-05:00",
  "EndDate": "2025-12-16T04:00:23.51-05:00",
  "Orders": [
    "75571f2a-d568-4334-a184-9115a84b6e0a",
    "ee11b78f-3810-460e-b582-e6209745b9fb",
    "55fa53b5-3873-46e1-aa4c-b8adf2226748",
    "f6890b8d-f9da-4a13-92ec-fe6a27c463ea",
    "f8f4b9ed-1505-42e9-a5ac-a918f32ac6eb",
    "f9ecedea-cca7-46af-a202-31780169972c",
    "25a39645-7d09-4de0-ae37-f9a22b534fb9",
    "8ae84f2a-7963-41f3-8c5c-c30e3b787f0c",
    "b8229ce9-09b9-48b3-8d02-76600a0e5ff6",
    "346f16a0-4238-4d9f-ac79-b3f24d3f0361",
    "01baab54-a9a0-484e-9ac7-36022dfa93b7",
    "9429584f-8fb0-4258-8cee-691ba58b5261",
    "3175128e-abe6-40f4-aae3-073392a15d3f",
    "bbbf0d91-9367-49d6-b1c4-3ce602218721",
    "ffc55808-49b8-43db-bbaa-f49e656aa948",
    "6966de64-ae6d-491e-9465-56263c32decf",
    "6e0727c9-bf33-4fdb-818f-79afb6fdf674",
    "f6346cbf-c9a1-4669-92fc-da1c05b59ea1",
    "d6bba87e-3bb5-45a8-8c31-67555be7d14c",
    "588df1b9-3914-4ccf-97c4-97b2f7148f4b",
    "4ac390c3-0adf-4c54-a4d9-edb1621982ce",
    "735e8e33-e958-4c26-9e77-a1cf51830f99",
    "73773186-8d6c-46b8-be82-64c7ceadaa1c",
    "d95e8085-d3b4-4567-ae3c-01d47addf1a2",
    "92a8135d-3c2a-4fe3-8a36-629052d19a9b",
    "abc830b9-e362-46cc-99a6-838da53ab3b4",
    "83cd43d8-8edd-4cdb-ba65-0e24bc2fc97e",
    "4876a5a6-9057-4663-bd7f-ea9b435b1848",
    "7e853697-ca8b-46df-a57d-0712addc03b1",
    "c095c377-19ee-4fee-8feb-d9f2d75a9c8e",
    "94df99a2-401e-4d60-bcd4-18b9f66ab74b",
    "eeaa634e-f1b7-41a2-bc67-3ab6ddb3e7ad",
    "089ad64e-6b76-4401-808a-de1da39ee9e8",
    "0ba50fd3-9ed0-406f-be57-99e779c4eef5",
    "f1030b96-3365-4162-90cb-c2c3b89adc31",
    "24e2e019-6bf2-4659-8cae-3c2337e7d177",
    "42cc3075-59ac-445f-abb3-0a0dd4af66bd",
    "2a50cb28-865f-404d-b5c0-e6ed97f2b449",
    "32444194-3705-463e-bb12-0d96b6fc33ef",
    "12e1344a-d183-4a38-b1ec-2e8837cc7892",
    "ccc84fe8-ffac-4942-9f96-8791e4b53049",
    "55e02e9f-2560-407a-8c43-b25a2984a726",
    "9b272b6f-e21c-4566-b0d1-6d6ab6a80649",
    "0b45f736-0bd5-431e-9168-88c505ae6aa7",
    "f1c3214f-477f-428c-8fd4-b5629a7143e4",
    "a201cd7d-1a8e-40eb-82a9-15a0ff3225b0",
    "60f28ed3-3ae1-4015-94c5-0575d4ac87ea",
    "f22592e6-c792-4bab-a357-5a8f0ec13489",
    "e2eaeb98-b0ed-414c-ac34-5cbbb236cc42",
    "d45429cd-465e-479e-ab66-71073b4a4352",
    "c1afe225-efa7-4431-9c17-266a5cd4b4c9",
    "633c1b7d-45f9-48d1-8039-827987a91013",
    "b064bb28-955a-4ac2-8cd7-e608ef27383e",
    "7df85ce8-6b3c-4d0f-a1be-6b336ce5bd70",
    "de3cdddb-ea2c-48c4-a60b-8e36e37ea4b2",
    "237ffeb2-af25-4dc1-ba08-9f4c9505ebe7",
    "c9c0bee0-02d2-4fb7-b317-313fdf504fc6",
    "e31e58f2-4e3a-4006-b598-d0fde84e5935",
    "7dc4725f-6120-40f4-80c4-e5e39ec89d54",
    "e950343d-da28-4d23-b37b-e5732f6408d3",
    "388457ee-098c-4586-a20b-774a2a318486",
    "49be8c99-24e1-437a-b60c-d3791e708a29",
    "af46ec3d-a449-456a-bef3-c597bf6939ad",
    "32191c98-d5ea-4832-a75c-d5c918615472",
    "f6de0301-79d9-40fc-8d2f-f79e9d2fb41a",
    "64940ff1-5af6-4937-9421-c219dad21a92",
    "554f0423-834b-4a46-96e9-b8ca21b74a01",
    "83214a17-a64c-43a5-b590-2941509e1838",
    "0a748d68-ff36-4999-b870-fed5804ccf1b",
    "dc68ec70-3a09-4155-a252-af9cedb45e15",
    "28e7b6c4-69f9-4b63-85e7-4164aa367d5a",
    "e4f96caf-2f00-44f5-aeda-87d361847d98",
    "e561942b-c8b6-41f9-9486-3b56a08153c0",
    "e5bd8fc8-d35e-4003-90f2-bef588cb7ce4",
    "a5ae6a4e-01ce-4e77-ac15-be86f344d94b",
    "a3a9ff4e-b2b5-4fa0-a2ab-e43046c0946d",
    "82bb80bb-99e0-43ed-bef6-29966ab23a5d",
    "db510b2b-94ac-4de3-b27b-8191642df9d7",
    "52321798-e9dc-4a09-8352-a5beede4c4f8",
    "c17675fb-b4dd-4bb4-bd92-287c5e575588",
    "52b0e504-69f4-4e2c-8ed9-783f5c3be225",
    "19e9241c-c687-4a7f-a9a5-ba8aa4daa408",
    "1e81a7ca-a68c-42df-a34d-2e4e786386e6",
    "66dd56ae-c8fe-48ad-a029-dc5b947d8dde",
    "8f3bcacb-1ae2-43df-b6ab-f305a8992188",
    "1f4322e8-c7d9-4667-b558-fb0317e01c88",
    "ca86b41f-f34b-41ce-8d20-7c6d9ab56f45",
    "14ec45eb-7fc4-44cf-ba1a-58b66058f756",
    "9be14534-46c3-44de-a38a-0f2eb9723d91",
    "135e7cf5-2c5f-4f10-80bc-3ba1b7256329",
    "70a677f9-fc01-45fc-88e7-7107afabcaab",
    "e94cbdf6-e6f5-4604-a93e-23df3cf2c9bf",
    "e7ca1dd2-e076-449c-aa5f-51a8c41ab927",
    "4ffeb60c-1588-403d-b09b-b7bd31730bcf",
    "061fdf1e-5922-46a8-8fe3-5500c9203ba9",
    "320aa362-8055-43f7-a6c4-15dbe296c78d",
    "b1d5afa0-c830-4361-9296-59a364934969",
    "695a5e1a-afd3-474d-a787-fc16212620b1",
    "a77ccc6a-59a5-4785-8318-adacd4798721",
    "1be74378-9614-47c1-af9b-514d7c97ef23"
  ],
  "Success": true
}
I'd like to split this list of strings into individual records, but whenever I try to get the Orders field I get this error: Any idea what I can do to split this array of strings into records?
j
You need to use Record Selector to unpack the array of orders (to get individual UUIDs/GUIDs), then you can use this as a Parent Stream for the default view
j
That was my thought, but I get the error above when I use the Dpath record selector to select Orders.
Screenshot 2025-12-16 at 1.47.29 PM.png
I've queried several LLMs and each one has said that Airbyte's partition router requires wrapped JSON objects. I tried to wrap it with a python function, but I can't figure out where and how to use custom components. I'm writing python code in the UI, btw.
Screenshot 2025-12-16 at 1.50.50 PM.png
j
Try
Orders *
as the field path in dpath
j
Same thing.
Is there a way I can use custom extractor with a class defined in the components.py file in the yaml editor?
j
ah, missed that this was just a normal array of strings, not an array of objects. To use a custom extractor, it would look something like:
Copy code
record_selector:
          type: RecordSelector
          extractor:
            type: CustomRecordExtractor
            class_name: source_declarative_manifest.components.YourCustomExtractorName
j
That was the missing piece thank you so much!
Except now I'm running into an error with being unable to load the components file in the child. Is there a way I can declare the requirement to load the python file in the child stream?
j
The child stream doesn't need to be aware of the extractor . . . you alias the name of the column from the parent stream, then use it in the child stream's request (so it isn't interacting with that data structure at all, and I'm assuming returns a more normal object listing) If that's not the case, provide some examples of the data structure and you should be good to go. (It's also worth noting that those components aren't stream-specific; so there isn't really a "loading into the child stream" at all)
b
hey @Jonathan Ling, I've built a tool that can solve this problem for you. Check out Forge