Hi, I am developing a custom connector (generic Py...
# help-connector-development
e
Hi, I am developing a custom connector (generic Python, extending SOURCE class ), I am also emitting a state like this:
yield AirbyteMessage(
type=Type.STATE,
state=AirbyteStateMessage(data={stream.name: {"date": max_date}})
)
it works well, but as I have multiple streams, I need to emit a state per each of them. For some reasons, it looks like just the latest gets saved.
k
A message from kapa.ai
It seems like you're trying to implement per-stream state for your custom connector. In Airbyte, per-stream state is supported by emitting
AirbyteStateMessage
for each stream. However, you need to make sure that your custom connector's
source
class has the
per_stream_state_enabled
property set to
True
. Here's an example of how to do that:
Copy code
python class MyCustomSource(AbstractSource):     @property     def per_stream_state_enabled(self) -> bool:         return True
By setting this property to
True
, your custom connector should be able to save the state for each stream separately. For more information on incremental streams and state management, you can refer to the [Incremental Streams documentation](https://docs.airbyte.com/connector-development/cdk-python/incremental-stream).