Spark执行者无法访问kubernetes集群内部的点火节点 [英] Spark executors not able to access ignite nodes inside kubernetes cluster

问题描述

我正在将我的火花作业与现有的点火集群连接起来.我为此使用了一个服务帐户名称 spark .我的驱动程序可以访问点火容器，但是我的执行者不能访问该容器.

I am connecting my spark job with an existing ignite cluster. I use a service account name spark for it. My driver is able to access the ignite pods, but my executors are not able to access that.

这是执行程序日志的样子

This is what executor log looks like

Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://35.192.214.68/api/v1/namespaces/default/endpoints/ignite

我想这是由于某些特权.有没有办法为执行人明确指定服务帐户?

I guess it's due to some privileges. Is there a way to explicitly specify service account for executors as well?

谢谢.

推荐答案

讨论了类似的问题您很可能需要向用于运行Ignite的服务帐户授予更多权限.

Most likely you need to grant more permissions to a service account which is used for running Ignite.

通过这种方式，您可以创建另一个角色并将其绑定到服务帐户:

This way you are able to create and bind one more role to the service account:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ignite
  namespace: default
rules:
- apiGroups:
  - ""
  resources: # Here is resources you can access
  - pods
  - endpoints
  verbs: # That is what you can do with them
  - get
  - list
  - watch

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ignite
roleRef:
  kind: ClusterRole
  name: ignite
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: <service account name>
  namespace: default

此外，如果您的命名空间不是default，则需要在yaml文件中更新该命名空间，并在 TcpDiscoveryKubernetesIpFinder配置.

Also, if your namespace is not default you need to update that one in yaml-files and specify it in TcpDiscoveryKubernetesIpFinder configuration.

这篇关于Spark执行者无法访问kubernetes集群内部的点火节点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！