Flink 1.7.0 Dashboard 不显示任务统计 [英] Flink 1.7.0 Dashboard not show Task Statistics

问题描述

我使用 Flink 1.7 仪表板并选择一个流式作业.这应该会显示一些指标，但它仍然需要加载.

I use Flink 1.7 dashboard and select a streaming job. This should show me some metrics, but it remains to load.

我在 Flink 1.5 集群中部署了相同的作业，我可以查看指标.Flink 在 docker swarm 中运行，但如果我在 docker-compose 中运行 Flink 1.7(不在 swarm 中)，它可以工作

I deployed the same job in a Flink 1.5 cluster, and I can watch the metrics. Flink is running in docker swarm, but if I run Flink 1.7 in docker-compose (not in the swarm), it works

我可以做到，删除 docker-compose.yaml 文件中的主机名

I can do it work, deleting the hostname in docker-compose.yaml file

version: "3"
services:
  jobmanager17:
    image: flink:1.7.0-hadoop27-scala_2.11
    hostname: "{{.Node.Hostname}}"
    ports:
      - "8081:8081"
      - "9254:9249"
    command: jobmanager
....

我删除了主机名:

version: "3"
services:
  jobmanager17:
    image: flink:1.7.0-hadoop27-scala_2.11
    ports:
      - "8081:8081"
      - "9254:9249"
    command: jobmanager
....

现在指标有效，但没有主机名...

and now the metrics works, but without the hostname...

可以同时拥有吗?

PD:我读过一些关于分离模式"的内容......但我不使用它

PD: I read something about 'detached mode'... but I don't use it

推荐答案

我猜你是在 Kubernetes 或 docker swarm 上运行你的集群.使用 Kubernetes 上的 Flink 1.7，您需要确保任务管理器是使用他们的 IP 地址而不是他们的 IP 地址注册到作业管理器主机名.如果您查看 jobmanagers 日志，您会发现许多无法访问 Taskmanager 的警告.

I guess you are running your cluster on Kubernetes or docker swarm. With Flink 1.7 on Kubernetes you need to make sure the task managers are registering to the job manager with their IP addresses and not the hostnames. If you look at the jobmanagers log you'll find a lot of warnings that the Taskmanager can't be reached.

您可以通过传递定义taskmanager.host 参数来实现.一个示例部署可能如下所示:

You can do that by passing defining the taskmanager.host parameter. An example depoyment might look like this:

apiVersion: extensions/v1beta1
kind: Deployment
....
spec:
  template:
    spec:
      containers:
      - name: "<%= name %>"
        args: ["taskmanager", "-Dtaskmanager.host=$(K8S_POD_IP)"]
        env:
          - name: K8S_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP

如果您不是在 K8 上运行，则可能值得尝试手动传递此参数(通过提供可从作业管理器作为 taskmanager.host 访问的 IP 地址)

If you are not running on K8 it might be worth a try to pass this parameter manually (by providing an IP adress which is reachable from the jobmanager as the taskmanager.host)

希望有所帮助.

更新:Flink 1.8 解决了这个问题.属性 taskmanager.network.bind-policy 默认设置为ip"，这与上述解决方法的作用大致相同(https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#taskmanager)

Update: Flink 1.8 solves the problem. The property taskmanager.network.bind-policy is by default set to "ip" which does more or less the same what the above described workaround does (https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#taskmanager)

这篇关于Flink 1.7.0 Dashboard 不显示任务统计的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！