Hi team! I’m using the low code CDK and one the stream has is defined as follow:

questions_stream:
    $ref: "*ref(definitions.base_stream)"
    $options:
      name: "questions"
      primary_key: "id"
      path: "/question/{{ stream_slice.parent_id }}"
    retriever:
      $ref: "*ref(definitions.retriever)"
      record_selector:
        extractor:
          field_pointer: []
      stream_slicer:
        type: SubstreamSlicer
        parent_stream_configs:
          - stream: "*ref(definitions.reviews_stream)"
            parent_key: "question_id"
            stream_slice_field: "parent_id"

The problem is that in the parent stream, most the the reviews have the same question_id. So in the question stream I’m getting many many duplicates and the extract takes a lot of time. What would you recommend to fetch the unique question_id in this stream?

what's the relationship between questions and reviews? It sounds like one question can have multiple reviews, in which case I would recommend setting the questions stream as the parent and the reviews as the substream

Hi @Alexandre Girard (Airbyte) The same question can be used in multiple review, but the endpoint for the question is the following: https://api.latticehq.com/v1/question/{id} And I can only get the question_id from the review stream