This message was deleted.

With no fsync, the journal entries are written in the OS page cache but not flushed to disk. In case of power failure, the affected bookie might lose the unflushed data. You let OS control when to flush page cache. You may need to check your OS kernel settings.

If it is controlled at OS level, then how does the metric

bookie_journal_JOURNAL_SYNC_count

figure the value out?

The total number of journal fsync operations happening at the bookie. The

success

label is used to distinguish between successes and failures.

yes, but if its kernel triggered, how does bookkeeper process figure it out?

fsync is a system call. if journal sync is disabled (os pagecache on), sync system call should be used.

so, does it mean that irrespective of the fsync happening by kernel itself due to page cache dirty write sysctl values, bookies also trigger fsync system call every second?

I am not sure your question.

the mtric value of journal_sync_count is always 1/sec - irrespective of the throughput each bookie is observing. I am assuming if bookies write to page cache and fsync is handled by kernel, it should vary based on throughput.. i.e. can't be constant.. thus, i was wondering what that 1/sec represent.

👍

It's still unclear what the 1/second represent though 🙂

according to the issue, I think there is no more sync syscall. You either use journalsyncdata=true (fsync journal log before ack) or journalsyncdata=false (fsync every 1 second).

Yea, I was checking the system defaults. the default sync interval is 30 seconds, or some memory dirty ratio (which we don't breach). I checked the ForceWriteThread logic, which does the fsync, but couldn't find how its limiting to 1 second rate.. https://github.com/apache/bookkeeper/blob/branch-4.14/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java#L495

@Hang Chen I wonder if you can validate this? Very interesting thread.

You can use

journalPageCacheFlushIntervalMSec

to control the OS PageCache flush interval when disabled the fsync