Hi folks, looking to use GreptimeDB to store some ...
# general
h
Hi folks, looking to use GreptimeDB to store some high volume LLM conversation data (instead of Clickhouse). I see that there is a SQL ingestion interface supported, but it doesn't seem to support the dynamic data mode, does it? Or does it support it for additional columns, just not the initial table creation? If not, I'm leaning towards the gRPC interface instead, but can't find a JS/TS API for that, Am I'm I overlooking something?
n
Hey @Heinrich Maximilian Wendel, great to hear. The SQL interface doesn't support auto schema, we may need gRPC or OpenTelemetry for that. We don't have js/ts wrapper for gRPC but we can build one soon. Some quick questions: 1. is the llm conversation data like json based trace data? 2. are using sending data from nodejs server? 3. is there a time frame for this project? we are building a new json data type that may work for this scenario exactly
h
Hi @Ning, thank you for the super quick reply. The project is on-going and we are doing new releases on a week-by-week basis: https://hebo.ai/, happy to test out new things along the way. Specifically right now, we are looking to store OpenAI-compatible conversation data for our /conversation endpoint: https://developers.openai.com/api/reference/resources/conversations/methods/create This one allows a flexible metadata KV object. My plan is to store each metadata item as it's own column (tag) so that queries are optimized. I could store it as JSON, but I understand queries wouldn't be as fast?
(We're running on Bun)