@Rocky can you help me find resources to onboard my team of engineers onto Starrocks?

Onboarding your engineering team to StarRocks involves understanding its architecture, data modeling strategies, and performance tuning tools. Here is a structured roadmap to get them started. 1. Architecture Fundamentals StarRocks is a Linux-based, distributed MPP (Massively Parallel Processing) database. * Components: * FE (Frontend): Manages metadata, handles client connections, and generates execution plans. * BE (Backend): Stores data and executes query plans. * CN (Compute Node): Stateless nodes for computation, typically used in shared-data (cloud-native) architectures to query data from object storage. * Compatibility: It is MySQL-protocol compatible. Engineers can use standard MySQL clients, BI tools (like Grafana or Tableau), and drivers (JDBC/ODBC) to connect. 2. Core Data Modeling (The "Key" to Performance) Engineers must choose the correct table type based on the workload: * Primary Key Table: Best for real-time updates and deletes (e.g., CDC from MySQL/Postgres). * Duplicate Key Table: Best for raw logs or event data where no aggregation is needed. * Aggregate Table: Pre-aggregates data during ingestion (e.g., summing metrics with the same key). * Unique Key Table: Replaces existing rows with the same key (Primary Key tables are generally preferred for performance now). 3. Data Distribution Strategy StarRocks uses a two-tier distribution strategy that is critical for performance: * Partitioning (Level 1): Usually by time (e.g.,

PARTITION BY RANGE(event_date)

). This enables partition pruning and easier TTL management. * Bucketing (Level 2): Inside each partition, data is hashed into buckets (e.g.,

DISTRIBUTED BY HASH(user_id)

). Choose high-cardinality columns frequently used in

JOIN

or

WHERE

clauses as bucket keys. 4. Data Ingestion & Integration * Stream Load: An HTTP-based synchronous interface for loading local files (CSV/JSON). * Routine Load: For continuous streaming from Apache Kafka. * Broker Load: For high-volume batch loading from cloud storage (S3, GCS, Azure) or HDFS. * External Catalogs: Use these to query data directly from Hive, Iceberg, Hudi, or Delta Lake without migrating data into StarRocks. 5. Advanced Performance Tools * Materialized Views (MV): * Synchronous: Simple rollups on a single table. * Asynchronous: Supports multi-table joins and can be used for transparent query rewrite and pre-computing complex dashboards. * Query Profiling: Use

EXPLAIN

to see the plan and

EXPLAIN ANALYZE <query>

to see real-time execution metrics. * Index Tuning: Understand Sort Keys (the order of columns in the table definition) and Bitmap/Bloom Filter indexes for specific filtering needs. Recommended Next Steps for the Team 1. Hands-on: Run the Quick Start with Docker to launch a cluster in minutes. 2. Training: Review the Best Practices guide for schema design. 3. Community: Join the StarRocks Slack for technical support and architectural discussions. References • best_practices/query_tuning/query_planning.md • best_practices/query_tuning/query_profile_text_based_analysis.md • best_practices/query_tuning/query_plan_intro.md

• @Rocky Cluster architecture and best practices • Deployment and ongoing maintenance • High availability (HA) • Data lifecycle management • Cluster sizing and configuration • Monitoring and observability • Upgrades and operational best practices • Troubleshooting slow queries, compaction issues, and FE/BE failures • Incident management and day-to-day operations Has anyone worked with: • Does StarRocks has official training or workshops? • AWS partners that provide StarRocks training? • Third-party consultants or companies you'd recommend? • Any particularly good courses, documentation, or hands-on labs?

Onboarding a team for the operational and architectural side of StarRocks requires a mix of official documentation and deep-dive technical content. While there is no centralized "StarRocks University" certification yet, there are several highly structured resources and partner programs available. 1. Training, Workshops, and Consultants * Official Training: CelerData (the commercial company founded by the creators of StarRocks) provides formal, hands-on training. They offer tailored programs ranging from half-day to 5-day workshops specifically for engineering teams. * Third-Party Consultants: * *Deep.BI:* They offer specialized StarRocks consulting, including cluster optimization, migration delivery, and hands-on training from senior engineers with 24/7 SLA coverage. * AWS Marketplace: You can find CelerData Cloud (the enterprise managed version) on the AWS Marketplace, which often includes architectural support as part of the onboarding. * AWS Partners: StarRocks has an official AWS Partner Solution (formerly Quick Start) that provides an automated reference deployment following AWS best practices for HA and networking. 2. Operational & Deep-Dive Resources For the specific operational topics you mentioned, these are the best "source of truth" links: * Cluster Architecture & Sizing: * Plan a StarRocks Cluster: Covers node types (FE/BE/CN), CPU/Memory planning, and storage estimation. * Capacity Planning Guide: A specialized guide on calculating CPU cores based on scan volume and QPS. * High Availability (HA) & Deployment: * FE Metadata Management: Explains the Raft protocol and how to set up 3 Follower FEs to avoid a Single Point of Failure. * Operational FAQ: Practical answers for day-to-day cluster management. * Monitoring & Observability: * Monitoring and Alerting: The standard setup guide for Prometheus and Grafana with pre-built dashboards. * Manage Alerts: Lists specific PromSQL queries for alerting on compaction failures and backend exceptions. * Troubleshooting & Performance: * Query Profile Analysis: Essential for diagnosing slow queries. * Compaction Troubleshooting: How to diagnose "Compaction Score" issues and version count alerts. * FE Memory Troubleshooting: Deep dive into JVM heap issues and "Full GC" troubleshooting. 3. Recommended Hands-on Labs & Community * StarRocks Summit Replays: The StarRocks Summit 2025 YouTube playlist includes engineering keynotes from Intuit, Coinbase, and Pinterest specifically focusing on incident management and at-scale operations. * 5-Part Best Practice Series: A community-contributed series on Medium (by a StarRocks DBA) is highly recommended for team reading. It covers: 1. Deployment & Planning 2. Data Modeling 3. Ingestion Best Practices 4. Query Optimization 5. Monitoring & Incident Response * The Slack Community: For issues that go beyond documentation (e.g., specific incident management scenarios), the

#support

channel in the StarRocks Slack is the primary place where core maintainers respond to operational failures. References