Hi team, I need help for logrotation using appendb...
# fluent-bit
v
Hi team, I need help for logrotation using appendblob for azure_blob plugin, is it possible?
p
It's a bit vague, did you check the docs? https://docs.fluentbit.io/manual/data-pipeline/outputs/azure_blob appendblob is in there
v
Yes i did, but couldnt get it
p
Couldn't get what? What do you need to do?
v
i need to rotate logs at specific size lets say 10 mb
p
I don't think the MS API lets you do that so you'll probably have to control it by changing the blob path (e.g. with a time or something to use a new path after a while)
Maybe it's something you can configure on the Azure side to automatically do it for you? That would be my expectation but I've not used it
v
Hey I have another approach, would it be possible, I have 6 files, namely file1.log,file2.log, I want each file to be sent as a new appendblob
p
I thought each log file would get its own blob already
the screenshot in the docs suggests it does: https://docs.fluentbit.io/manual/data-pipeline/outputs/azure_blob
v
For some strange reason, I have files named as file-1,file-2 etc there
Copy code
[INPUT]
    Name                 tail
    Tag                  tmptag
    Path                 /home/tpmDir/file-*.log
    Path_Key             file
    Buffer_Chunk_Size    32KB
    Buffer_Max_Size      256KB
    Skip_Long_Lines      On
    Refresh_Interval     10
    Rotate_Wait          5
    Read_From_Head       On
    storage.type         filesystem
    storage.pause_on_chunks_overlimit on



[OUTPUT]
    Name                   azure_blob
    Match                  tmptag
    account_name           ${AZURE_ACCOUNT_NAME}
    auth_type              key
    shared_key             ${AZURE_SHARED_KEY}
    container_name         ${AZURE_CONTAINER}
    tls                    on
    blob_type              appendblob
    path                   /${LOG_PATH_PREFIX}/logs/${UNIQUE_ID}/test3/
but the output, i am getting in a single append blob
p
your tag is fixed
try with a wildcard in it
v
is there a way to embed file name in the tag?
p
e.g.
Copy code
[INPUT]
  name tail
  tag file.*
 ...
again, please check the docs - it's all there 🙂 https://docs.fluentbit.io/manual/data-pipeline/inputs/tail
Copy code
Tag expansion is supported: if the tag includes an asterisk (*), that asterisk will be replaced with the absolute path of the monitored file, with slashes replaced by dots.
v
Lemme try that and get back
p
and please check docs too if you have questions - if stuff is missing we can add
v
Thanks it worked