Hi, folks! How are y’all shipping your file-based ...
# dockerl
Leon Miller-Out
10/04/2023, 8:40 PMHi, folks! How are y’all shipping your file-based logs (application.log, exception.log, etc) from your containers to your log aggregator service?
Leon Miller-Out
10/04/2023, 8:41 PMI’ve got STDOUT logging covered, but not the file-based logs.
Leon Miller-Out
10/04/2023, 9:02 PMSpecifically, I’m using docker-commandbox and deploying to a Docker Swarm.
Leon Miller-Out
10/04/2023, 9:03 PMMy log aggregator has an agent that runs as a system service to ship logs from files, but that doesn’t really make sense in Docker-land.
q
quetwo
10/04/2023, 9:15 PMI pipe ALL my logs to either stdout or stderr. Since the log names show up in the console, the log service can split them up
quetwo
10/04/2023, 9:21 PMRUN ln -sf /dev/stdout /opt/lucee/web/logs/application.log && ln -sf /dev/stderr /opt/lucee/web/logs/exception.logRUN ln -sf /dev/stdout /opt/lucee/server/lucee-server/context/logs/application.log && ln -sf /dev/stdout /opt/lucee/server/lucee-server/context/logs/out.logl
Leon Miller-Out
10/05/2023, 3:09 PMThanks, @quetwo!
Leon Miller-Out
10/05/2023, 3:10 PMCan I ask how the log names show up in the console? My app (quite rationally) doesn’t print the log file name on each log line.
q
quetwo
10/06/2023, 12:53 AMI guess looking at it further, they don't actually say the old log file names.. But the format of cataliana.log and applicaiton.log are different. And by putting Exception to the /dev/stderr, it would be very different as well.
l
Leon Miller-Out
10/06/2023, 11:54 AMThat makes sense. Thanks for letting me know! I’m using your /dev/stdout trick.
j
jamiejackson
10/10/2023, 4:19 PMi use filebeat and logstash. i have separate handling for the tomcat stuff and the standardized lucee logging. (i parse all the regular lucee logs as CSV.) they all get shipped to elasticsearch
jamiejackson
10/10/2023, 4:22 PMi never thought to send to /dev/stdout (or stderr). i didn't even know that was a thing you could do.
i think the problem in doing so may be that the different log types have different parsing needs, etc., (as @quetwo alluded to) which is why my approach seems to work fine.
q
quetwo
10/10/2023, 6:38 PMJamie -- thing to keep in mind is as you evolve in your docker deployments, most of the tools /expect/ to read logs from stdout. On AWS (cloudwatch), Azure, etc. all do their log parsing from there. There are ways to get the logs coming from log4j to have filenames in the prefix to make that work better.
j
jamiejackson
10/10/2023, 8:31 PMyeah i like your idea if i can get the wrinkles ironed out. however, i haven't had any luck influencing the output of lucee logs with log4j configuration.
there were some details on my (failed) attempt at doing so, here: https://dev.lucee.org/t/logging-in-json-format/3753/8
jamiejackson
10/10/2023, 8:35 PMspeaking of logs, i really want this: https://luceeserver.atlassian.net/browse/LDEV-101
it's not easy to correlate logs without it
l
Leon Miller-Out
10/11/2023, 2:03 PMWe wrote our own logger.cfc and we just use that everywhere so we have full control over the output. It uses FileWrite instead of WriteLog
Leon Miller-Out
10/11/2023, 2:03 PMWe’ve been including a request ID for a while now. It’s very helpful!
j
jamiejackson
10/11/2023, 4:06 PMhmm, not a bad idea. sounds easy to implement but can you share the source?
j
jclausen
10/13/2023, 7:00 PMFor file-based, logs, Filebeat is a great option. You can point it to a directory and specify the logging patterns separately for different directories or file names. You just need an elasticsearch server. https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html
jclausen
10/13/2023, 7:00 PMThen you can configure index lifecycle policies for retention, etc.
jclausen
10/13/2023, 7:01 PMThe nice thing is that they become easily searchable in Kibana ( or StacheBox )
2 Views