从kubernetes转发日志到splunk [英] Forwarding logs from kubernetes to splunk

问题描述

我是Kubernetes的新手，没有实践经验.

I'm pretty much new to Kubernetes and don't have hands-on experience on it.

我的团队面临有关到 splunk 的问题.

My team is facing issue regarding the log format pushed by to splunk.

{"logname" : "app-log", "level" : "INFO"}

Splunk最终获得此格式(使用了splunkforwarder)

{
  "log" : "{\"logname\": \"app-log\", \"level\": \"INFO \"}",
  "stream" : "stdout",
  "time" : "2018-06-01T23:33:26.556356926Z" 
 }

这种格式使Splunk中的属性难以查询.

This format kind of make things harder in Splunk to query based on properties.

Kubernetes中是否有任何选项可以从应用程序转发raw logs，而不是分组为另一个json?

Is there any options in Kubernetes to forward raw logs from app rather than grouping into another json ?

我在Splunk中遇到了此帖子，但是配置在Splunk端完成

I came across this post in Splunk, but the configuration is done on Splunk side

请让我知道我们是否可以从Kubernetes一方发送来自应用程序的原始日志

Please let me know if we have any option from Kubernetes side to send raw logs from application

推荐答案

Kubernetes体系结构提供了三种收集日志的方式:

Kubernetes architecture provides three ways to gather logs:

1.使用在每个节点上运行的节点级日志记录代理.

您可以通过在每个节点上包括一个节点级日志记录代理来实现群集级日志记录.日志记录代理是专用工具，可公开日志或将日志推送到后端.通常，日志记录代理是一个容器，可以访问该节点上所有应用程序容器中包含日志文件的目录.

You can implement cluster-level logging by including a node-level logging agent on each node. The logging agent is a dedicated tool that exposes logs or pushes logs to a backend. Commonly, the logging agent is a container that has access to a directory with log files from all of the application containers on that node.

日志格式取决于Docker设置.您需要在每个节点的/etc/docker/daemon.json中设置log-driver参数.

The logs format depends on Docker settings. You need to set up log-driver parameter in /etc/docker/daemon.json on every node.

例如，

{
  "log-driver": "syslog"
}

或

{
  "log-driver": "json-file"
}

• 无-容器无可用日志，而docker日志无可用 返回任何输出.
• json-file-日志格式为JSON.这 Docker的默认日志记录驱动程序.
• syslog-将日志消息写入 syslog工具.
• none - no logs are available for the container and docker logs does not return any output.
• json-file - the logs are formatted as JSON. The default logging driver for Docker.
• syslog - writes logging messages to the syslog facility.

有关更多选项，请检查链接

For more options, check the link

2.包括一个专用的sidecar容器，用于登录应用程序窗格.

您可以通过以下方式之一使用边车集装箱:

You can use a sidecar container in one of the following ways:

• sidecar容器将应用程序流记录到其自己的stdout.
• sidecar容器运行一个日志记录代理，该代理配置为从应用程序容器中获取日志.

通过将sidecar容器流传输到它们自己的stdout和stderr流，您可以利用已经在每个节点上运行的kubelet和日志记录代理.边车容器从文件，套接字或日志记录中读取日志.每个单独的sidecar容器都将日志打印到其自己的stdout或stderr流中.

By having your sidecar containers stream to their own stdout and stderr streams, you can take advantage of the kubelet and the logging agent that already run on each node. The sidecar containers read logs from a file, a socket, or the journald. Each individual sidecar container prints log to its own stdout or stderr stream.

3.从应用程序内部将日志直接推送到后端.

您可以通过直接从每个应用程序公开或推送日志来实现群集级日志记录.

You can implement cluster-level logging by exposing or pushing logs directly from every application.

有关更多信息，您可以检查官方Kubernetes的文档

For more information, you can check official documentation of Kubernetes

这篇关于从kubernetes转发日志到splunk的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！