Hello Greptime team. Saw the "Pandas DataFrame SQ...
# general
n
Hello Greptime team. Saw the "Pandas DataFrame SQL query, zero-copy" item in the 2026 roadmap . We're trying to figure out if it fits what we're building. Our setup: • Pull from ClickHouse and a few other sources, compute in Polars (financial metrics, LazyFrame pipelines). • Write results back or pass them downstream. The annoying part is when we need the computed output to JOIN back against data still in the DB. Currently that's a full round-trip. Two questions: 1. Is the intended direction DataFrame → DB (register a local df as a virtual table), DB → DataFrame (results returned as Arrow), or both? 2. The roadmap mentions Pandas specifically — is Polars in scope? It uses Arrow natively so the zero-copy path should be the same. (We switched our near-real-time computation to Polars, and both its expressiveness and speed are satisfactory.)
r
Is the intended direction DataFrame → DB (register a local df as a virtual table), DB → DataFrame (results returned as Arrow), or both?
Both, we plan to provide dataframe on API level to interact with DB, so both direction should work
The roadmap mentions Pandas specifically — is Polars in scope? It uses Arrow natively so the zero-copy path should be the same.
Pandas is as an widely used example when play with dataframe. It should not be very hard to extend the support to other systems with similar interfaces. And Arrow is one of the data transmission protocol internal of greptimedb, which should also be easy to integrate with other arrow native systems
🎉 1