hi all, how do you declaratively manage GreptimeDB...
# general
k
hi all, how do you declaratively manage GreptimeDB? as best practices, i suppose we shouldn't run ad-hoc DDL queries against prod GreptimeDB? whats the recommended workflow to do
CREATE OR REPLACE FLOW
, CREATE or ALTER DATABASE etc.?
l
Hi Kevin, GreptimeDB does not support declarative DDL per se, but since it supports MySQL/PostgreSQL wire protocol, most modern database-as-code tools like atlas/bytebase will work perfectly for GreptimeDB.
k
thanks for the reply. i am aware of atlas/bytebase, but they cant do
CREATE OR REPLACE FLOW
or CREATE or ALTER DATABASE and many other things. also i am afraid there are many GreptimeDB specific syntax that are not compatible. additionally, the wire protocol for Postgres still needs more work, e.g.
information_schema.tables
only has MySQL-style metadata columns (table_collation, auto_increment, row_format…).
i am curious how GreptimeDB team recommends to manage it in prod? i was thinking about historical reviews, state management, rollback, automation etc. these perspective.
l
Taking the monitoring standalone instance in GreptimeDB enterprise cluster for example: monitoring standalone instance serves as the metrics/logs store for a running GerptimeDB enterprise edition that runs in our client’s env. It has serveral built-in tables like cluster CPU usage, logs from different components. Instead of manageing the schemas of these tables manually, GreptimeDB supports a schema-less manner: the table automatically adds a column when it ingests a row containing some column that currently does not exist. But as you mentioned, creating database/flow is not automatic, but in most cases
CREATE DATABASE/FLOW IF NOT EXIST
would be enough. If you still find it useful to manage GreptimeDB cluster in a declarative manner, I think the best approach would be add some post-installation task in the greptimedb-cluster in the helmchart. You can create a feature rqeuest in that project.
k
thanks for your answer, i think operationalizing tasks in heavily underrated nowadays, declarative, reproducible, automated GreptimeDB management should be the default, no manual ad-hoc commands.
i think for this, you need a data plane operator. for creating a data plane operator, GreptimeDB needs to expose APIs. but all changes are done using SQL, now it becomes a whole different problem, similar to what Atlas tries to solve. even if we want to implement ourselves, it will be very complicated and fragile without deep domain knowledge, testing and verification.