Hey guys. I am currently attempting to create a custom source for CircleCI using a low-code YAML-based feature. I have encountered a specific use case that requires the following steps: 1. First, I make a request to the project stats endpoint to receive some data. In the response, along with other fields, there is a list of available workflows, which I refer to as "all_workflows". This list is structured in JSON format as follows:

json
{
       ......
	"all_workflows": ["workflow_1", "workflow_2", ..., "workflow_N"]
}

2. Next, I need to fetch some workflow stats from a different endpoint, which is structured as follows: "https://circleci.com/api/v2/insights/time-series/{project-slug}/workflows/{workflow-name}/jobs". In this URL, the "project-slug" is defined as a variable, while the "workflow-name" should be taken from the previous step's response. To accomplish this, I created a separate stream with partitioning settings defined (refer to pic1.png). In these partitioning settings, I refer to the stream/field from the previous step and define some alias on that field. 3. When I test this stream, I do not receive any response, which is expected because the workflows for the request are concatenated with commas. Unfortunately, the endpoint does not support multiple workflow names in a single request, so I need to send N requests to fetch the required stats. My questions are as follows: 1. Is it possible to iterate over fields from another stream to make multiple requests? 2. If not, what is a workaround that I can use with a low-code approach to implement this use case? 3. (optional) What is the correct Airbyte way to implement such an integration? Thank you in advance

A message from kapa.ai

pic1.png