Carlos Aguni

Highly motivated self-taught IT analyst. Always learning and ready to explore new skills. An eternal apprentice.


Metrics from Logs Study

16 Aug 2022 »

logrecyler

https://github.com/grosser/logrecycler

Re-process logs from applications you cannot modify to:

  • convert plaintext or glog logs from stdin to json on stdout
  • remove noise
  • add log levels / timestamp / details / captured values
  • emit prometheus metric
  • emit statsd metric

Example

stdin: I0530 10:13:00.740596      33 foo.go:132] error connecting to remote host foobar.com:12345
stdout: {"ts":"2020-05-30 10:13:00","level":"error","message":"error connecting to remote host","host":"foobar.com","port":"1234","pattern":"connection-error"}
/metrics: log_total{level="error",host="foobar.com",port="1234",pattern="connection-error"} 1

fluentbit