https://linen.dev logo
Join Slack
Powered by
I have the following class which has a next_page_t...
# help-connector-development
a
I have the following class which has a next_page_token and parse_response function which uses a data_field property. How do i change my GetMemberInfo class which does not have a data_field in the api response?
Copy code
class MailChimpStream(HttpStream, ABC):
    primary_key = "id"
    page_size = 1000

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.current_offset = 0
        self.data_center = kwargs["authenticator"].data_center

    @property
    def url_base(self) -> str:
        return f"https://{self.data_center}.<http://api.mailchimp.com/3.0/|api.mailchimp.com/3.0/>"

    @property
    def availability_strategy(self) -> Optional["AvailabilityStrategy"]:
        return None

    def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
        decoded_response = response.json()
        api_data = decoded_response[self.data_field]
        if len(api_data) < self.page_size:
            self.current_offset = 0
            return None
        else:
            self.current_offset += self.page_size
            return {"offset": self.current_offset}

    def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapping]:
        response_json = response.json()
        yield from response_json[self.data_field]


class GetMemberInfo(MailChimpStream):
    data_field = ""
k
A message from kapa.ai
To modify the 
GetMemberInfo
class to handle API responses without a 
data_field
, you can override the 
next_page_token
and 
parse_response
methods in the 
GetMemberInfo
class. Here's an example of how you can do this:
Copy code
python class GetMemberInfo(MailChimpStream):     data_field = ""      def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:         decoded_response = response.json()         if len(decoded_response) < self.page_size:             self.current_offset = 0             return None         else:             self.current_offset += self.page_size             return {"offset": self.current_offset}      def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapping]:         response_json = response.json()         yield from response_json
In this example, the 
next_page_token
and 
parse_response
methods are modified to work with the API response directly, without using the 
data_field
property. The 
next_page_token
method checks the length of the decoded response, and the 
parse_response
method yields items from the response JSON directly.
a
@kapa.ai the code you provided gives me the following error
Copy code
lambda req, res, state, _slice: self.parse_response(res, stream_slice=_slice, stream_state=state), stream_slice, stream_state\nTypeError: GetMemberInfo.parse_response() got an unexpected keyword argument 'stream_slice'\n", "failure_type": "system_error"}}}
k
A message from kapa.ai
Hmm, I don't know enough to give you a confident answer yet. However, I recommend reviewing the Airbyte documentation and forum discussions to find a solution to your issue: - [Airbyte Documentation](https://docs.airbyte.com/) - [Airbyte Forum](https://discuss.airbyte.io/) Make sure to carefully review the code and compare it with the examples provided in the documentation and forum discussions.
a
@kapa.ai DO you know why I get this error 
ValueError: Unexpected type for data_or_message: <class 'str'>: id\n", "failure_type": "system_error"
k
A message from kapa.ai
Sorry @Aazam Thakur, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?
a
@kapa.ai DO you know why I get this error ValueError: Unexpected type for data_or_message: <class 'str'>: id\n", "failure_type": "system_error"
k
A message from kapa.ai
Sorry @Aazam Thakur, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?
Open in Slack
PreviousNext
Powered by