使用 prometheus-POD 指标进行 K8s 监控 [英] K8s monitoring with prometheus-POD metrics

查看:222
本文介绍了使用 prometheus-POD 指标进行 K8s 监控的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

是否有任何工具/方法可以获取 POD 的 CPU、MEM、NET 指标.除了下面的链接,还有没有可用的工具

Is there any tools/way to get CPU, MEM, NET metrics of PODs. Other than below links, is there any tools available

  • kube-state-metrics - Able to deploy but no useful POD metrics. You can see the POD metrics here.
  • kubernetes-monitoring-with-prometheus-in-15-minutes- Installed kube-prometheus with "helm" tool, No POD metrics. Metrics List here
  • prometheus-kubernetes - But it strucks at registering custom service forever. check here
  • Monitoring K8s with Prometheus - In blog they mentioned container_cpu metrics, but I dont see any metrics like that

更新1

尝试使用 yaml 文件 启动 POD,正如他们在 博客.安装 go langGOPATH &GOROOT

Tried launch POD with yaml file as they mentioned in blog. Installed go lang with GOPATH & GOROOT

ubuntu@ip-172-:~$ kubectl create -f prometheus.yaml 
panic: interface conversion: interface {} is []interface {}, not map[string]interface {}

goroutine 1 [running]:
k8s.io/kubernetes/pkg/kubectl/cmd/util/openapi/validation.getObjectKind(0x14dcb20, 0xc420c56480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xffffffffffffff01, 0xc420f6bca0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/openapi/validation/validation.go:111 +0x539
k8s.io/kubernetes/pkg/kubectl/cmd/util/openapi/validation.(*SchemaValidation).ValidateBytes(0xc4207b01d0, 0xc420b3ca80, 0x16c, 0x180, 0xc420b51628, 0x4ed384)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/openapi/validation/validation.go:49 +0x8f
k8s.io/kubernetes/pkg/kubectl/validation.ConjunctiveSchema.ValidateBytes(0xc42073cba0, 0x2, 0x2, 0xc420b3ca80, 0x16c, 0x180, 0x4ed029, 0xc420b3ca80)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/validation/schema.go:130 +0x9a
k8s.io/kubernetes/pkg/kubectl/validation.(*ConjunctiveSchema).ValidateBytes(0xc42073cbc0, 0xc420b3ca80, 0x16c, 0x180, 0xc420b51700, 0x443693)
    <autogenerated>:3 +0x7d
k8s.io/kubernetes/pkg/kubectl/resource.ValidateSchema(0xc420b3ca80, 0x16c, 0x180, 0x2183f80, 0xc42073cbc0, 0x20, 0xc420b51700)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:222 +0x68
k8s.io/kubernetes/pkg/kubectl/resource.(*StreamVisitor).Visit(0xc420c2eb00, 0xc420c3d440, 0x218a000, 0xc420c3d4a0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:543 +0x269
k8s.io/kubernetes/pkg/kubectl/resource.(*FileVisitor).Visit(0xc420c3d2c0, 0xc420c3d440, 0x0, 0x0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:502 +0x181
k8s.io/kubernetes/pkg/kubectl/resource.EagerVisitorList.Visit(0xc420f6bc30, 0x1, 0x1, 0xc420903c50, 0x1, 0xc420903c50)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:211 +0x100
k8s.io/kubernetes/pkg/kubectl/resource.(*EagerVisitorList).Visit(0xc420c3d360, 0xc420903c50, 0x7ff854222000, 0x0)
    <autogenerated>:115 +0x69
k8s.io/kubernetes/pkg/kubectl/resource.FlattenListVisitor.Visit(0x2183d00, 0xc420c3d360, 0xc420c2eac0, 0xc420c2eb40, 0xc420c3d401, 0xc420c2eb40)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:417 +0xa3
k8s.io/kubernetes/pkg/kubectl/resource.(*FlattenListVisitor).Visit(0xc420c3d380, 0xc420c2eb40, 0x18, 0x18)
    <autogenerated>:130 +0x69
k8s.io/kubernetes/pkg/kubectl/resource.DecoratedVisitor.Visit(0x2183d80, 0xc420c3d380, 0xc420c3d3c0, 0x3, 0x4, 0xc420c3d400, 0xc420386901, 0xc420c3d400)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:325 +0xd8
k8s.io/kubernetes/pkg/kubectl/resource.(*DecoratedVisitor).Visit(0xc420903c20, 0xc420c3d400, 0x151b920, 0xc420f6bc60)
    <autogenerated>:153 +0x73
k8s.io/kubernetes/pkg/kubectl/resource.ContinueOnErrorVisitor.Visit(0x2183c80, 0xc420903c20, 0xc420c370e0, 0x7ff854222000, 0x0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/visitor.go:352 +0xf1
k8s.io/kubernetes/pkg/kubectl/resource.(*ContinueOnErrorVisitor).Visit(0xc420f6bc50, 0xc420c370e0, 0x40f3f8, 0x60)
    <autogenerated>:144 +0x60
k8s.io/kubernetes/pkg/kubectl/resource.(*Result).Visit(0xc4202c23f0, 0xc420c370e0, 0x6, 0x0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/resource/result.go:95 +0x62
k8s.io/kubernetes/pkg/kubectl/cmd.RunCreate(0x21acd60, 0xc420320e40, 0xc42029d440, 0x2182e40, 0xc42000c018, 0x2182e40, 0xc42000c020, 0xc420173000, 0x176f608, 0x4)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/create.go:187 +0x4a8
k8s.io/kubernetes/pkg/kubectl/cmd.NewCmdCreate.func1(0xc42029d440, 0xc4202aa580, 0x0, 0x2)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/create.go:73 +0x17f
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc42029d440, 0xc4202aa080, 0x2, 0x2, 0xc42029d440, 0xc4202aa080)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:603 +0x22b
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420235b00, 0x8000102, 0x0, 0xffffffffffffffff)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:689 +0x339
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420235b00, 0xc420320e40, 0x2182e00)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:648 +0x2b
k8s.io/kubernetes/cmd/kubectl/app.Run(0x0, 0x0)
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/app/kubectl.go:39 +0xd5
main.main()
    /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:26 +0x22

prometheus.yaml

# This scrape config scrapes kubelets
- job_name: 'kubernetes-nodes'
  kubernetes_sd_configs:
    - role: node

  # couldn't get prometheus to validate the kublet cert for scraping, so don't bother for now
  tls_config:
    insecure_skip_verify: true
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

  relabel_configs:
  - target_label: __scheme__
    replacement: https
  - source_labels: [__meta_kubernetes_node_label_kubernetes_io_hostname]
    target_label: instance

推荐答案

您正在寻找 cadvisor.

这篇关于使用 prometheus-POD 指标进行 K8s 监控的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆