@Rocky can I measure starrocks shared cluster -> object store latency and its metrics?

Yes, you can measure the latency and performance between a StarRocks shared-data cluster and your object storage (e.g., S3, OSS, GCS). StarRocks provides a comprehensive set of metrics specifically for this purpose, primarily categorized under the Starlet and Shared-data monitoring dashboards. Key Metrics for Object Store Latency These metrics are exported via Prometheus and can be viewed in Grafana: 1. Read Latency (S3/Object Store) *

fslib_read_io_latency

: Provides both quantile (P75, P95, P99) and average latency for reads from the object store. *

fslib_read_iops

: The number of read I/O operations per second. *

fslib_read_throughput

: The I/O throughput (bytes per second) for reads. 2. Write/Upload Latency *

fslib_s3_single_upload_latency

: Quantile latency for S3

PutObject

operations. *

fslib_s3_multi_upload_latency

: Quantile latency for S3 Multipart Upload operations. *

fslib_s3_complete_multi_upload_latency

: Latency for the final "complete" step of a multipart upload. *

fslib_write_io_latency

: Measures application-level write latency. Note that this may appear lower than actual S3 latency because it often reflects writes to an internal buffer. 3. Metadata Latency Since shared-data clusters store metadata on the object store, monitoring these is crucial: *

Get/Put Tablet Metadata Latency

: Latency for fetching or updating tablet metadata. *

Get/Put Txn Log Latency

: Latency for transaction log operations on the object store. Monitoring Tools * Prometheus/Grafana: StarRocks exposes these metrics at the

/metrics

endpoint of the FE and BE (or CN) nodes. * Dashboard Templates: StarRocks provides official Grafana dashboard templates specifically for shared-data clusters: * Shared-data Dashboard for VM * Shared-data Dashboard for K8s Data Cache Impact If you have the Data Cache enabled, some "reads" will be served from local disk/memory instead of the object store. To distinguish between them, monitor: *

fslib_cache_hit_ratio

: A high hit ratio means most reads are staying local, reducing observed query latency regardless of object store performance. *

fslib_cache_hits/misses

: The frequency of local vs. remote access-to-object-store operations. References • administration/management/monitoring/metrics-shared-data.md • using_starrocks/caching/block_cache.md • administration/management/monitoring/metrics.md • administration/management/monitoring/Monitor_and_Alert.md