Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi, everyone.

Can I send “parent_key” from the parent stream in manifest.yaml to the child stream in schemas/file_name.json?

I miss the id of product_attributes_stream for the product_attribute_terms_stream in source-woocommerce. In the final table, after normalization, I can’t understand which line belongs to which attribute.

The schema has no parent id because they don’t have this parameter in API:
<https://github.com/airbytehq/airbyte/blob/ba040c9462c285c50a68d41b9210ac5c14acf1cc/airbyte-integrations/connectors/source-woocommerce/source_woocommerce/schemas/product_attribute_terms.json.|github-airbyte>
<https://woocommerce.github.io/woocommerce-rest-api-docs/#product-attribute-terms|woocommerce-documentation>

However, the manifest.yaml has a link to its path: “/products/attributes/{{ stream_slice.id }}/terms” (<https://github.com/airbytehq/airbyte/blob/ba040c9462c285c50a68d41b9210ac5c14acf1cc/airbyte-integrations/connectors/source-woocommerce/source_woocommerce/manifest.yaml#L141|github-airbyte>)

Can this parameter be added to the schema? Or do I need to use another way to add this parameter to the final table in the database?

Yes, you can use the `parent_key` from the parent stream in the child stream by configuring a SubstreamPartitionRouter in the manifest.yaml file. This allows you to fetch records from the child stream based on the records of the parent stream.  To do this, you need to configure the following fields in the SubstreamPartitionRouter:  - Parent stream: The stream whose records should be used as partitions. - Parent key: The property on the parent stream record that should become the partition value (usually an ID). - Current partition value identifier: A freely chosen identifier for the variable holding the current partition value. You can use this in the path of the stream using the `{{ stream_partition.&lt;identifier&gt; }}` interpolation placeholder.  Here's an example of how to configure the SubstreamPartitionRouter in the manifest.yaml file:  ```yaml # manifest.yaml stream_config:   partition_router:     type: SubstreamPartitionRouter     parent_stream_configs:       - parent_key: "id"         stream: "#/definitions/ParentStream"         partition_field: "parent_id"         request_option:           type: Path           field: "parent_id" ```  In this example, the `parent_key` is set to `id`, and the `partition_field` is set to `parent_id`. The `request_option` is configured to use the `Path` type and the `field` is set to `parent_id`. This configuration will allow you to use the parent key value in the child stream's path URL.  For more information on configuring SubstreamPartitionRouter, refer to the [Airbyte documentation on partitioning](<https://docs.airbyte.com/connector-development/connector-builder-ui/partitioning>).  However, if the API does not provide the parent ID in the child stream's response, you might need to modify the connector's code to include the parent ID in the child stream's records before they are written to the destination. This would require custom development and is not directly supported by the low-code connector configuration.

How to convert connector from low code to custom development