没有自定义指标的水平 pod 自动缩放 [英] Horizontal pod Autoscaling without custom metrics

查看:29
本文介绍了没有自定义指标的水平 pod 自动缩放的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我们希望根据 Kafka 主题中的消息量水平扩展 Pod.标准的解决方案是将指标发布到 Kubernetes 的自定义指标 API.但是,由于公司指南,我们不允许使用 Kubernetes 的自定义指标 API.我们只被允许使用非管理员功能.是否有具有 kubernetes-nativ 功能的解决方案,或者我们是否需要实施定制的解决方案?

We want to scale our pods horizontally based on the amount of messages in our Kafka Topic. The standard solution is to publish the metrics to the custom metrics API of Kubernetes. However, due to company guidelines we are not allowed to use the custom metrics API of Kubernetes. We are only allowed to use non-admin functionality. Is there a solution for this with kubernetes-nativ features or do we need to implement a customized solution?

推荐答案

我不确定这是否适合您的需求,但您可以使用 自动缩放与 Kubernetes 对象无关的指标.

I'm not exactly sure if this would fit your needs but you could use Autoscaling on metrics not related to Kubernetes objects.

在 Kubernetes 上运行的应用程序可能需要根据与 Kubernetes 集群中的任何对象没有明显关系的指标进行自动缩放,例如描述与 Kubernetes 命名空间没有直接关联的托管服务的指标.在 Kubernetes 1.10 及更高版本中,您可以使用外部指标来解决此用例.

Applications running on Kubernetes may need to autoscale based on metrics that don’t have an obvious relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case with external metrics.

使用外部指标需要了解您的监控系统;该设置类似于使用自定义指标时所需的设置.外部指标允许您根据监控系统中可用的任何指标自动扩展集群.只需提供一个带有 nameselectormetric 块,如上所述,并使用 External 度量类型而不是<代码>对象.如果 metricSelector 匹配多个时间序列,则 Horizo​​ntalPodAutoscaler 使用它们值的总和.外部指标同时支持 ValueAverageValue 目标类型,其功能与使用 Object 类型时完全相同.

Using external metrics requires knowledge of your monitoring system; the setup is similar to that required when using custom metrics. External metrics allow you to autoscale your cluster based on any metric available in your monitoring system. Just provide a metric block with a name and selector, as above, and use the External metric type instead of Object. If multiple time series are matched by the metricSelector, the sum of their values is used by the HorizontalPodAutoscaler. External metrics support both the Value and AverageValue target types, which function exactly the same as when you use the Object type.

例如,如果您的应用程序处理来自托管队列服务的任务,您可以将以下部分添加到您的 Horizo​​ntalPodAutoscaler 清单中,以指定每 30 个未完成的任务需要一个工作程序.

For example if your application processes tasks from a hosted queue service, you could add the following section to your HorizontalPodAutoscaler manifest to specify that you need one worker per 30 outstanding tasks.

- type: External
 external:
   metric:
     name: queue_messages_ready
     selector: "queue=worker_tasks"
   target:
     type: AverageValue
     averageValue: 30

如果可能,最好使用自定义指标目标类型而不是外部指标,因为集群管理员可以更轻松地保护自定义指标 API.外部指标 API 可能允许访问任何指标,因此集群管理员在公开它时应该小心.

When possible, it’s preferable to use the custom metric target types instead of external metrics, since it’s easier for cluster administrators to secure the custom metrics API. The external metrics API potentially allows access to any metric, so cluster administrators should take care when exposing it.

你也可以看看zalando-incubator/kube-metrics-adapter 并使用 Prometheus 收集器 外部指标.

You may also have a look at zalando-incubator/kube-metrics-adapter and use Prometheus collector external metrics.

这是配置为基于 Prometheus 查询获取指标的 HPA 的示例.查询在注释 metric-config.external.prometheus-query.prometheus/processed-events-per-second 中定义,其中 processed-events-per-second 是将与查询结果相关联的查询名称.必须在指标定义的 matchLabels 中定义匹配的 query-name 标签.这允许将多个普罗米修斯查询与单个 HPA 关联.

This is an example of an HPA configured to get metrics based on a Prometheus query. The query is defined in the annotation metric-config.external.prometheus-query.prometheus/processed-events-per-second where processed-events-per-second is the query name which will be associated with the result of the query. A matching query-name label must be defined in the matchLabels of the metric definition. This allows having multiple prometheus queries associated with a single HPA.

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
 name: myapp-hpa
 annotations:
   # This annotation is optional.
   # If specified, then this prometheus server is used,
   # instead of the prometheus server specified as the CLI argument `--prometheus-server`.
   metric-config.external.prometheus-query.prometheus/prometheus-server: http://prometheus.my->namespace.svc
   # metric-config.<metricType>.<metricName>.<collectorName>/<configKey>
   # <configKey> == query-name
   metric-config.external.prometheus-query.prometheus/processed-events-per-second: |
     scalar(sum(rate(event-service_events_count{application="event-service",processed="true"}[1m])))
spec:
 scaleTargetRef:
   apiVersion: apps/v1
   kind: Deployment
   name: custom-metrics-consumer
 minReplicas: 1
 maxReplicas: 10
 metrics:
 - type: External
   external:
     metric:
       name: prometheus-query
       selector:
         matchLabels:
           query-name: processed-events-per-second
     target:
       type: AverageValue
       averageValue: "10"

这篇关于没有自定义指标的水平 pod 自动缩放的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆