Hi Team, I am recently installed flunet-bit 4.0.3 ...
# fluent-bit
j
Hi Team, I am recently installed flunet-bit 4.0.3 on my windows server where its frequently crashed. can someone advise.
p
Can you provide more details on the specific Windows server version, etc. ? Did you use the official packages? What config are you using?
j
we are using windows server 2016, yes we are using the official package
p
Windows Server 2016 has been EOL for over 3.5 years now...
j
Copy code
[SERVICE]
  # Flush
  # =====
  # set an interval of seconds before to flush records to a destination
  flush 30
  
  # Daemon
  # ======
  # instruct Fluent Bit to run in foreground or background mode.
  daemon Off
  
  # Log_Level
  # =========
  log_level debug
  log_file fluentbit.log
  
  # Parsers File
  # ============
  # specify an optional 'Parsers' configuration file
  parsers_file parsers.conf
  
  # Plugins File
  # ============
  # specify an optional 'Plugins' configuration file to load external plugins.
  plugins_file plugins.conf
  
  # HTTP Server
  # ===========
  # Enable/Disable the built-in HTTP Server for metrics
  http_server Off
  http_listen 0.0.0.0
  http_port 2020
  storage.metrics on

[INPUT]
  Name winlog
  Tag  winlog
  Channels Application,System,Security,Microsoft-Windows-TaskScheduler/Operational,Microsoft-Windows-SMBServer/Audit,Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
  Interval_Sec 10
  DB winlog.sqlite

[FILTER]
  Name aws
  Match winlog
  imds_version v2
  ec2_instance_id true
  hostname true

[OUTPUT]
  Name s3
  Match winlog
  bucket ops-bucket-test
  upload_timeout 1m
  s3_key_format /fluent-bit-logs/$TAG/%Y/${HOSTNAME}/%m/%d/%H/%M/%S
  store_dir s3-buffer
  total_file_size 1M

[INPUT]
  name         tail
  Tag          xx_assist_app
  path         C:\Program Files\xxxxx\yyyyyyy\Logs\Admin\xxxxxx\*
  Path_Key     filename
  DB           tail.sqlite
  read_from_head   true
  multiline.parser multiline-regex

[INPUT]
  name         tail
  Tag          xx_assist_app
  path         C:\Program Files\xxxxx\yyyyyyy\Logs\Admin\xxxxxx\*
  Path_Key     filename
  DB           tail.sqlite
  read_from_head   true
  multiline.parser   multiline-anchor

[INPUT]
  name         tail
  Tag          xx_assist_app
  path        C:\Program Files\xxxxx\yyyyyyy\Logs\Admin\xxxxxx\*
  Path_Key     filename
  DB           tail.sqlite
  read_from_head   true
  multiline.parser   multiline-anchor

[FILTER]
  Name      lua
  Match     xx_assist_app
  Script    extract_filename.lua
  Call      extract_filename

[FILTER]
  Name record_modifier
  Match xx_assist_app
  Record hostname ${HOSTNAME}

[OUTPUT]
  name   loki
  match  xx_assist_app
  Host   loki-xx-dev-in.internal
  port   443
  tls    on
  tls.verify  off
  labels job=Classix-App, instance=${HOSTNAME}, filename=$filename
Is fluent-bit latest version doesn't have support for windows 2016
parsers used
Copy code
[MULTILINE_PARSER]
    name          multiline-regex
    type          regex
    flush_timeout 1000
    #
    # Regex rules for multiline parsing
    # ---------------------------------
    #
    # configuration hints:
    #
    #  - first state always has the name: start_state
    #  - every field in the rule must be inside double quotes
    #
    # rules |   state name  | regex pattern                  | next state
    # ------|---------------|--------------------------------------------
    rule      "start_state"   "/(^[A-Za-z]{3}-\d{2}-\d{4}\s\d+\:\d+\:\d+)(.*)/"  "cont"
    rule         "cont"       "/^(?![A-Za-z]{3}-\d{2}-\d{4}\s\d+\:\d+\:\d+).+/"  "cont"

[MULTILINE_PARSER]
    name          multiline-anchor
    type          regex
    flush_timeout 1000
    rule      "start_state"   "/(^AetherPal\.)(\w+)(.*)/"  "cont"
    rule         "cont"       "/^(?!AetherPal\.)(\w+)(.*)/"  "cont"
lau parser
Copy code
function extract_filename(tag, ts, record)
    local path = record["filename"]
    if path ~= nil then
        local normalized = path:gsub("\\\\", "/"):gsub("\\", "/")
        local base = normalized:match("^.+/(.+)$") or normalized
        record["filename"] = base  -- overwrite the original field
    end
    return 1, ts, record
end
Is there anything wrong on my config
p
Even Microsoft does not support Windows 2016 😞 We primarily supported 2019 and 2022 - if you need older versions your best bet is to talk to an enterprise provider, e.g. https://www.fluent.do/ or another here: https://fluentbit.io/enterprise/ We are OSS so rely on whatever is supported by the Github CI: https://github.com/fluent/fluent-bit/blob/master/.github/workflows/call-build-windows.yaml At the moment you can see the targets we build there and the runner is based on Windows Server 2022: https://github.com/actions/runner-images
j
okay. This is the OS version running on my production env as well. So i can't update the OS. let me try with some lower version. currently we have 3.1.8
Is there anything wrong on my config
p
I imagine some dependencies have been updated since that was built which no longer work now
j
okay
p
I'm afraid supporting an EOL platform is not really something OSS can do, you can always build directly for the target yourself or ask an enterprise provider to do so
j
okay
Is OSS have diff URL to download
p
No, that is the official one
j
So OSS also the same URL to download. Am i right?
p
That is the official package yes
j
okay