Running [SERVICE]
Flush 1
Log_Level debug
storage.path /tmp/fluent-bit-storage
storage.sync normal
storage.pause_on_chunks_overlimit On
storage.max_chunks_up 10
# storage.total_limit_size 3M
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_PORT 2020
[INPUT]
Name tail
Tag test.tail
Path /var/log/test.log
Refresh_Interval 1
Skip_Long_Lines On
storage.type filesystem
[OUTPUT]
Name forward
Match *
Host 127.0.0.1
Port 9999
Retry_Limit False
storage.total_limit_size 30M . Then populate /var/log/test.log with -> while true; do echo "$(date) hello world" >> /var/log/test.log; sleep 0.001; done. My input never gets paused. The chunks will be created until reaching storage.total_limit_size 30M. Monitoring shows a fluctuation of chunks in memory from 1 to 8 watch -n 1 'curl -s
http://localhost:2020/api/v1/storage | jq ".storage_layer.chunks.fs_chunks_up"'. This combination does not allow me to pause the tail input: storage.pause_on_chunks_overlimit + storage.max_chunks_up + storage.type filesystem. Simply UP chunks never reach the threshold value. Maybe I do miss some other extra setting required here? Docker compose file -> services:
fluentbit:
image: fluent/fluent-bit:4.0.1-debug
container_name: fluentbit_lab
volumes:
- ./fluent-bit.conf
/fluent bit/etc/fluent bit.confro
environment:
- TZ=UTC
#command: ["fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.conf"]
command: ["/fluent-bit/bin/fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.conf"]