如何将kubernetes集群中的流利数据发送到集群外的远程独立服务器中的elasticsearch? [英] How can I send the data from fluentd in kubernetes cluster to the elasticsearch in remote standalone server outside cluster?
问题描述
我在GCP中设置了三个kubernetes集群环境. 我已经在所有这些环境中将Fluentd作为daemonset安装了，以便从所有pod收集日志. 我还在集群外部的单独服务器中安装了elasticsearch和kibana. 我需要将流利的日志输入远程服务器中的elasticsearch，从而运行集中式日志记录平台.
I have three kubernetes cluster environments set up in GCP. I have installed Fluentd as daemonset in all these environments to collect the logs from all the pods. I have also installed elasticsearch and kibana in a separate server outside the cluster. I need to feed the logs in fluentd to the elasticsearch in remote server and thereby run a centralised logging platform.
如何将数据从流利的发送到远程服务器中的elasticsearch?
How can I send the data from fluentd to the elasticsearch in remote server?
收到的错误是:
error_class = Fluent :: Plugin :: ElasticsearchOutput :: ConnectionFailure error =无法到达Elasticsearch集群
error_class=Fluent::Plugin::ElasticsearchOutput::ConnectionFailure error="Can not reach Elasticsearch cluster
推荐答案
文档以便从Pod内部访问外部资源:
There are two common ways mentioned in documentation to access external resources from inside the Pod:
-
创建服务和端点对象.在Endpoint的规范中设置外部IP地址:
Create a Service and Endpoint objects. Set external IP address in Endpoint's specification:
kind: Service
apiVersion: v1
metadata:
name: ext-elastic
namespace: default
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9200
---
kind: Endpoints
apiVersion: v1
metadata:
name: ext-elastic
namespace: default
subsets:
- addresses:
- ip: 1.2.3.4
ports:
- port: 9200
注意::终结点IP可能不是环回(
127.0.0.0/8)，本地链接 (169.254.0.0/16)或本地链接多播(224.0.0.0/24).他们不能 成为其他Kubernetes服务的群集IP，因为 kube-proxy组件尚不支持将虚拟IP作为目标.
NOTE: The endpoint IPs may not be loopback (
127.0.0.0/8), link-local (169.254.0.0/16), or link-local multicast (224.0.0.0/24). They cannot be the cluster IPs of other Kubernetes services either because the kube-proxy component doesn’t support virtual IPs as destination yet.
您可以通过在相同名称空间内使用http://ext-elastic或从不同名称空间使用http://ext-elastic.default.svc.cluster.local来访问此服务.
You can access this service by using http://ext-elastic inside the same namespace or by using http://ext-elastic.default.svc.cluster.local from a different namespace.
- 创建ExternalName服务并在规范中指定外部资源的名称:
ExternalName服务是服务的特殊情况，它不 有选择器.它没有定义任何端口或端点.而是 用作将别名返回到驻留的外部服务的方法 在集群之外.
An ExternalName service is a special case of service that does not have selectors. It does not define any ports or Endpoints. Rather, it serves as a way to return an alias to an external service residing outside the cluster.
kind: Service
apiVersion: v1
metadata:
name: ext-elastic
namespace: default
spec:
type: ExternalName
externalName: my.external.elasticsearch.com
ports:
- port: 80
查找主机my-service.prod.svc.CLUSTER时，群集DNS 服务将返回带有值的CNAME记录 my.database.example.com.访问此类服务的工作原理相同 和其他方式一样，唯一的不同是重定向发生 在DNS级别，并且没有发生代理或转发.你应该 稍后决定将数据库移至您的集群中，您可以开始 其广告连播，添加适当的选择器或端点并更改 服务类型.
When looking up the host my-service.prod.svc.CLUSTER, the cluster DNS service will return a CNAME record with the value my.database.example.com. Accessing such a service works in the same way as others, with the only difference that the redirection happens at the DNS level and no proxying or forwarding occurs. Should you later decide to move your database into your cluster, you can start its pods, add appropriate selectors or endpoints and change the service type.
查看其他文章，以查看更多示例.
Check out another article to see some more examples.
这篇关于如何将kubernetes集群中的流利数据发送到集群外的远程独立服务器中的elasticsearch?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
