Carlos Aguni

Highly motivated self-taught IT analyst. Always learning and ready to explore new skills. An eternal apprentice.

Log AIOPS Study

09 Mar 2022 »
  • Logging in Action - Phil Wilkins

GrafanaConline 2022 Day 3 - Medallia

initial path to logging

fragmentation of loggin pipelines

kubernets brings new abilities

logging costs and running at scale

grafana and loki for the win

future of logging

final thoughts/questions

Store applications persist logs (human-readable) to disk or S3 Storage. #766

https://github.com/grafana/loki/issues/766

Currently logs are written in Chunks which can only be readable by Grafana. There is no way if I want to download it to human readable file logs. This is only partly correct. Loki indeed has an HTTP API-endpoint which is used by logcli and grafana to query the logs.

Nevertheless I see the use-case of archiving the logs to a human readable long term storage. However, I would not implement this directly in Loki but rather use the tailing feature to create a standalone agent (loki-archive) that does exactly that:

  • accept a set of logql queries
  • persist each query into a separate file on disk, s3, whatever