我可以在本地文件系统中使用flinkrocksDB状态后端吗? [英] can i use flink rocksDB state backend with local file system?

问题描述

我正在探索使用FlinkrocksDb状态后端，该文档似乎暗示我可以使用常规文件系统，例如: file:///data/flink/checkpoints ，但是该代码仅适用于javadoc在这里提到hdfs或s3选项.

I am exploring using Flink rocksDb state backend, the documentation seems to imply i can use a regular file system such as: file:///data/flink/checkpoints, but the code javadoc only mentions hdfs or s3 option here.

我想知道是否可以在flink rocksdb后端使用本地文件系统，谢谢！

I am wondering if it's possible to use local file system with flink rocksdb backend, thanks!

Flink文档:https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/state_backends.html#the-rocksdbstatebackend

Flink代码: https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L175

推荐答案

不，您不应该这样做！

使用此路径可以配置Flink将检查点写入其中的目录.检查点是您的应用程序状态的副本，用于在发生故障(例如机器故障)时恢复应用程序状态.该路径必须指向持久性和远程存储，以便在进程被杀死或机器死亡时能够读取检查点.如果将检查点写入失败的计算机的本地文件系统，则将无法恢复作业并恢复状态.

With this path you configure the directory into which Flink writes checkpoints. A checkpoint is a copy of your application state that is used to restore the application state in case of a failure such as a machine failure. The path must point to a persistent and remote storage to be able to read the checkpoint in case that a process was killed ore a machine died. If a checkpoint was written to the local filesystem of a machine that failed, you would not be able to recover the job and restore the state.

但是，如果这是可以从其他计算机上挂载的NFS(或任何其他远程存储)的挂载点，则可以将检查点写入本地路径.

However, you can write the checkpoint to a local path if this is a mount point of an NFS (or any other remote storage) that can be mounted from other machines as well.

这篇关于我可以在本地文件系统中使用flinkrocksDB状态后端吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！