Join Slack
Powered by
If I add a new DAFT custom data source, and I read...
# general
n
Navneeth Krishnan
11/17/2025, 4:30 AM
If I add a new DAFT custom data source, and I read from it, is it streaming in nature by default?
c
Colin Ho
11/17/2025, 5:42 AM
Yes! The DataSourceTask has a
Copy code
def get_micro_partitions(self) -> Iterator[MicroPartition]:
method that allows you to define an iterator, which daft will pull from in a streaming fashion
❤️ 1
6
Views
Open in Slack
Previous
Next