Hello, I’m new to StarRocks and recently learned ...
# questions-and-troubleshooting
k
Hello, I’m new to StarRocks and recently learned about it through a friend. After reading more about it online, I found it quite interesting. We have a use case where we receive a stream of Kafka messages containing table metadata changes, such as column additions/deletions and primary key information. After consuming these messages, we need to synchronize data from the source system to the destination system (currently, we are using ClickHouse as the destination). Would StarRocks be a suitable fit for this use case?
k
you mean starrocks consumes ddl change from kafka message and apply the change accordingly?
k
Hi @Kevin Cai I ment i want to replace my existing clickhouse DB with StarRocks. so just wanted to check if this is suitable for my use case as i explained in the my previous msg. will the StarRocks support DDL and DML operation at run time with streaming along with data synchronisation ? i.e every seconds either a rows will get inserted or updated.
j
Yes, StarRocks is a strong fit for this use case, with one distinction worth clarifying. Streaming DML (inserts and updates every second): Fully supported. Use a Primary Key table with Routine Load from Kafka. The Primary Key model handles real-time upserts natively — row-level updates and inserts at high frequency are exactly what it's designed for. This is actually where StarRocks beats ClickHouse significantly, since ClickHouse's mutation model for updates is asynchronous and heavyweight. DDL changes at runtime (column additions/deletions from Kafka): StarRocks supports online DDL — you can add or drop columns on a live table without downtime. However, StarRocks won't automatically parse a Kafka message that says "add column X" and apply it. You'd need an application layer consumer that reads those DDL events from Kafka and executes the corresponding
ALTER TABLE
statements against StarRocks. That's not a limitation unique to StarRocks — no database natively consumes schema change instructions from a message queue without some middleware. Recommended architecture: CDC pipeline (Flink CDC or Kafka Connect) → StarRocks Primary Key table via Routine Load or Flink StarRocks connector • Separate consumer for DDL events → executes
ALTER TABLE
on StarRocks This pattern is commonly used and well-documented. If you're already using Kafka, the migration from ClickHouse to StarRocks for this use case is straightforward.
k
Thanks @John Bergamini for detailed clarification. it really help to proceed further.
hello @John Bergamini @Kevin Cai Could you please point me to a document or guide for setting up a StarRocks cluster on Kubernetes (including both FE and BE nodes)? I’m looking for a simple, step-by-step approach that can help deploy a 4-node cluster within about 30 minutes. I’ve already referred to the official StarRocks documentation, but I wasn’t able to successfully deploy it in cluster mode. Any guidance or recommended resources would be really helpful.