重用缓存的Spark RDD [英] Reuse a cached Spark RDD

问题描述

Spark是否有可能在另一个应用程序(或同一应用程序的另一个运行)中重用缓存的RDD?

Is there a possibility in Spark to re-use a cached RDD in another application (or in another run of the same application)?

JavaRDD<ExampleClass> toCache = ... // transformations on the RDD
toCache.cache();                    // can this be reused somehow in another application or further runs?

推荐答案

否，Spark RDD不能在其他应用程序或其他运行中使用.

No, Spark RDD cannot be used in other application or in another run.

您可以将Spark与例如 Hazelcast 或 Apache Ignite 将RDD保存在内存中.其他应用程序将有可能读取保存在第一个应用程序中的数据

You can connect Spark with for example Hazelcast or Apache Ignite to save RDDs in memory. Other application will have possibility to read data saved in first application

这篇关于重用缓存的Spark RDD的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！