Kubernetes HPA使用来自另一个部署的指标 [英] Kubernetes HPA using metrics from another deployment

问题描述

我目前正在尝试使用Prometheus和Prometheus适配器运行自动缩放演示，而我想知道是否有一种方法可以根据Prometheus从另一个部署中删除的指标来自动缩放我的一个部署.

我现在有2种不同的部署，kafka-consumer-application(我要扩展)和kafka-exporter(公开了我将用于扩展的kafka指标).我知道如果在同一部署中将它们都作为容器都可以自动缩放，但是问题是kafka-exporter也可以自动缩放并且不理想，所以我想将它们分开.我尝试使用以下HPA，但无法正常工作:

 种类:Horizo​​ntalPodAutoscalerapiVersion:自动缩放/v2beta1元数据:名称:consumer-hpa规格:scaleTargetRef:apiVersion:apps/v1种类:部署名称:kafka-consumer-applicationminReplicas:1maxReplicas:10指标:-类型:物件目的:目标:kafka出口商metricName:"kafka_consumergroup_lag"targetValue:5 

我不确定我是不是做错了什么，或者这不是我可以做的，所以任何建议都值得赞赏.

谢谢！

注意:我正在使用此配置运行适配器:

 规则:默认值:false资源:{}风俗:-seriesQuery:"kafka_consumergroup_lag"资源:覆盖:kubernetes_namespace:{资源:名称空间"}kubernetes_pod_name:{资源:"pod"}名称:匹配:"kafka_consumergroup_lag"作为:"kafka_consumergroup_lag"metricsQuery:"avg_over_time(kafka_consumergroup_lag {topic ="my-topic"，consumergroup ="we-consume"} [1m])"`` 

解决方案

在

如果您有多个kafka出口商，则可以使用选择器对其进行过滤(此Stack问题.

Im currently trying to run an autoscaling demo using prometheus and the prometheus adapter, and i was wondering if there is a way to autoscale one of my deployments based on metrics that prometheus scrapes from another deployment.

What i have right now are 2 different deployments, kafka-consumer-application (which i want to scale) and kafka-exporter (which exposes the kafka metrics that I'll be using for scaling). I know that if I have both of them as containers in the same deployment the autoscaling works, but the issue is that the kafka-exporter also gets autoscaled and its not ideal, so i want to separate them. I tried with the following HPA but i could not get it to work:

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
  name: consumer-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kafka-consumer-application
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: object
    object:
      target: kafka-exporter
      metricName: "kafka_consumergroup_lag"
      targetValue: 5

Im not sure if im doing something wrong or if this is just not something i can do, so any advice is appreciated.

Thanks!

Note: im running the adapter with this config:

rules:
  default: false
  resource: {}
  custom:
    - seriesQuery: 'kafka_consumergroup_lag'
      resources:
        overrides:
          kubernetes_namespace: {resource: "namespace"}
          kubernetes_pod_name: {resource: "pod"}
      name:
       matches: "kafka_consumergroup_lag"
       as: "kafka_consumergroup_lag"
      metricsQuery: 'avg_over_time(kafka_consumergroup_lag{topic="my-topic",consumergroup="we-consume"}[1m])'
``

解决方案

In kubernetes documentation you can read:

Autoscaling on metrics not related to Kubernetes objects Applications running on Kubernetes may need to autoscale based on metrics that don’t have an obvious relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case with external metrics

So using external metrics, your HPA yaml could look like following:

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta2
metadata:
  name: consumer-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kafka-consumer-application
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: kafka_consumergroup_lag
        #selector:
        #  matchLabels:
        #    topic: "my-topic"
      target:
        type: AverageValue
        averageValue: 5

If you have more than one kafka-exporter you can use selector to filter it (source):

selector is the string-encoded form of a standard kubernetes label selector for the given metric When set, it is passed as an additional parameter to the metrics server for more specific metrics scoping. When unset, just the metricName will be used to gather metrics

Also have a look at this Stack question.

这篇关于Kubernetes HPA使用来自另一个部署的指标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！