没有 Google Cloud Storage 的 Kubeflow [英] Kubeflow without Google Cloud Storage

问题描述

是否可以使用替代的本地解决方案替换 Google Cloud Storage 存储桶的使用，以便可以运行例如Kubeflow Pipelines 完全独立于 Google Cloud Platform?

Is it possible to replace the usage of Google Cloud Storage buckets with an alternative on-premises solution so that it is possible to run e.g. Kubeflow Pipelines completely independent from the Google Cloud Platform?

推荐答案

是的，这是可能的.您可以使用 minio，它类似于 s3/gs，但它在本地存储的持久卷上运行.

Yes it is possible. You can use minio, it's like s3/gs but it runs on a persistent volume of your on-premises storage.

以下是有关如何将其用作 kfserving 推理存储的说明:

Here are the instructions on how to use it as a kfserving inference storage:

验证 minio 是否在您的 kubeflow 安装中运行:

Validate that minio is running in your kubeflow installation:

$ kubectl get svc -n kubeflow |grep minio
minio-service                                  ClusterIP   10.101.143.255   <none>        9000/TCP            81d

为您的 minio 启用隧道:

Enable a tunnel for your minio:

$ kubectl port-forward svc/minio-service -n kubeflow 9000:9000
Forwarding from 127.0.0.1:9000 -> 9000
Forwarding from [::1]:9000 -> 9000

浏览 http://localhost:9000 以进入 minio UI 并创建存储桶/上传您的模型.凭证 minio/minio123.或者，您可以使用 mc 命令从终端执行此操作:

Browse http://localhost:9000 to get to the minio UI and create a bucket/upload your model. Credentials minio/minio123. Alternatively you can use the mc command to do it from your terminal:

$ mc ls minio/models/flowers/0001/
[2020-03-26 13:16:57 CET]  1.7MiB saved_model.pb
[2020-04-25 13:37:09 CEST]      0B variables/

为minio访问创建secret&ser​​viceaccount，注意s3-endpoint定义了minio的路径，keyid&acceskey是base64编码的凭证:

Create a secret&serviceaccount for the minio access, note that the s3-endpoint defines the path to the minio, keyid&acceskey are the credentials encoded in base64:

$ kubectl get secret mysecret -n homelab -o yaml
apiVersion: v1
data:
  awsAccessKeyID: bWluaW8=
  awsSecretAccessKey: bWluaW8xMjM=
kind: Secret
metadata:
  annotations:
    serving.kubeflow.org/s3-endpoint: minio-service.kubeflow:9000
    serving.kubeflow.org/s3-usehttps: "0"
  name: mysecret
  namespace: homelab

$ kubectl get serviceAccount -n homelab sa -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa
  namespace: homelab
secrets:
- name: mysecret

最后，创建您的 inferenceservice 如下:

Finally, create your inferenceservice as follows:

$ kubectl get inferenceservice tensorflow-flowers -n homelab -o yaml
apiVersion: serving.kubeflow.org/v1alpha2
kind: InferenceService
metadata:
  name: tensorflow-flowers
  namespace: homelab
spec:
  default:
    predictor:
      serviceAccountName: sa
      tensorflow:
        storageUri: s3://models/flowers

这篇关于没有 Google Cloud Storage 的 Kubeflow的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！