Hi team! I was thinking if we should add tests to 6606 and in general to Write stats/metrics. Any ideas on how to do that?

Hey Ruthvik! Great idea. We don't have any test coverage for write metrics today. For approach, Python integration tests via

df.metrics

would give us the most coverage for the least effort. Two cases worth testing: 1. All data buffered (the #6606 scenario): write a small DataFrame with a large parquet_target_row_group_size so everything flushes in close(). Assert

rows.written == <http://rows.in|rows.in>

and

bytes.written > 0

. 2. Multi-batch with leftovers: write enough data with a small target row group size to trigger mid-stream flushes plus a leftover batch. Assert

rows.written == <http://rows.in|rows.in>

It would be create to write tests for Parquet/csv/json writes to make sure metrics work across formats. Happy to help if you run into issues!

Hey Desmond! Let me work on the tests