<@U0A71G31CDV> is it normal that I have 3x logs fo...
# questions-and-troubleshooting
z
@Rocky is it normal that I have 3x logs for followers than leader? for instance 11g log files for leader fe and 34 for followers?
r
It is quite common to see differences in log volume between a Leader FE and Follower FEs, but whether "3x" is normal depends on which logs you are looking at and how many followers you have. 1. Check if you are comparing a single node to the "Total" If your cluster has 3 Followers (plus 1 Leader, making 4 FEs total, though 3 or 5 is standard), then having 3x more logs for "the followers" collectively (e.g., 34GB total for all followers vs 11GB for the leader) is perfectly normal. Each node is likely generating roughly the same amount of data. 2. Check the Log Type Identify which directory or file is consuming the space: *
meta/bdb/
(Metadata Journals):
StarRocks uses BDBJE to replicate metadata. All FEs (Leader and Followers) should have nearly identical metadata journals (
.jdb
files). * Normal: Small differences in file counts (e.g., 11 files vs 15 files) can occur because the Leader triggers log cleaning, and followers may lag slightly in deleting old files. * Abnormal: If one node has 3x more
.jdb
files than the others, it often indicates the Follower is not performing checkpoints correctly or has a metadata synchronization issue. Check
fe.log
for
InsufficientLogException
or BDB-related errors. *
log/fe.log
and
log/fe.warn.log
(System Logs):
* Normal: Followers often log more "noise" (e.g., heartbeat checks, replication status, or "transfer: follower" messages). * Abnormal: If
fe.log
or
fe.warn.log
is massive on a follower, it is likely being spammed with connection errors or sync failures. Grep for
WARN
or
ERROR
in the follower logs to see if it's struggling to talk to the Leader or the BEs. *
log/fe.audit.log
(Audit Logs):
Audit logs record every query. Each FE records the queries it directly receives from clients. * If your load balancer or application is sending most queries to the Followers (to offload the Leader), the Followers will naturally have much larger audit logs. 3. Verification Steps To determine if this is a problem, check the following: 1. Check FE Status: Run
SHOW PROC '/frontends';
in MySQL. Ensure all followers have
Alive: true
and their
ReplayedJournalId
is close to the Leader's. 2. Compare Configuration: Ensure
sys_log_roll_num
and
sys_log_delete_age
are the same across all FEs. If a follower is configured to keep more historical logs, it will naturally have a larger folder. 3. Inspect Log Content: If the followers have significantly more files, check the last few entries of
fe.log
. Persistent warnings about "failed to get master info" or "heartbeat timeout" can quickly bloat logs. Summary: If "34" refers to the total across all followers, it is normal. If it refers to each individual follower having 3x more logs than the leader, you should check for error spam in
fe.warn.log
or a checkpointing delay in the BDBJE metadata. Referencesadministration/management/logs.mdadministration/management/FE_configuration.mdadministration/Meta_recovery.md