In a Source implementation, is there an upper size...
# help-connector-development
c
In a Source implementation, is there an upper size limit on the contents of the
state
parameter passed to the
read
method (and what is emitted in state messages during read)?
k
A message from kapa.ai
Based on the provided extracts, there is no explicit mention of an upper size limit on the contents of the
state
parameter passed to the
read
method in a Source implementation, or what is emitted in state messages during read. The
state
is treated as a black box by the Airbyte protocol and is modeled as a JSON blob. It is used to allow a Source to pick up where it left off and to checkpoint data replication. The Source can track state however makes most sense to it. The state for an actor is emitted as a complete black box and is wrapped in the
AirbyteStateMessage
. The contents of the
data
field is what is passed to the Source on start up. However, it's important to note that while there's no specified size limit in the extracts, practical limitations may be imposed by the specific data store or system you're working with. For more details, you can refer to the [Airbyte Protocol](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol) documentation.
c
@kapa.ai In the Python source connector cdk, I notice that the
state
parameter is provided as a defaultdict. Under the hood is the serialization of the state blob implemented using
pickle
?
k
A message from kapa.ai
Hmm, I don't know enough to give you a confident answer yet. The information extracts provided do not mention the use of
pickle
for serialization of the
state
blob in the Python source connector CDK. However, they do mention that the
state
attribute helps the CDK figure out the current state of sync at any moment and that the setter typically deserializes state saved by CDK and initializes the internal state of the stream. The getter should serialize the internal state of the stream. Here is an example of how the
state
getter and setter are implemented:
Copy code
python @property   def state(self) -> Mapping[str, Any]:    return {self.cursor_field: str(self._cursor_value)}      @state.setter   def state(self, value: Mapping[str, Any]):    self._cursor_value = value[self.cursor_field]
For more details, you might want to check the [Airbyte documentation](https://docs.airbyte.com/connector-development/cdk-python/incremental-stream).