如何在Zeppelin笔记本电脑和pyspark中导入Delta Lake模块? [英] How to import Delta Lake module in Zeppelin notebook and pyspark?

问题描述

我正在尝试在带有pyspark的Zeppelin笔记本中使用Delta Lake，似乎无法成功导入该模块.例如

I am trying to use Delta Lake in a Zeppelin notebook with pyspark and seems it cannot import the module successfully. e.g.

%pyspark
from delta.tables import *

它失败并出现以下错误:

It fails with the following error:

ModuleNotFoundError:没有名为"delta"的模块

ModuleNotFoundError: No module named 'delta'

但是，使用delta格式保存/读取数据帧没有问题.如果使用scala spark %spark

However, there is no problem to save/read the data frame using delta format. And the module can be loaded successfully if using scala spark %spark

在齐柏林飞艇和pyspark有什么方法可以使用三角洲湖吗?

Is there any way to use Delta Lake in Zeppelin and pyspark?

推荐答案

最终设法将其加载到齐柏林飞艇pyspark上.必须明确包含jar文件

Finally managed to load it on zeppelin pyspark. Have to explicitly include the jar file

%pyspark
sc.addPyFile("**LOCATION_OF_DELTA_LAKE_JAR_FILE**")
from delta.tables import *

这篇关于如何在Zeppelin笔记本电脑和pyspark中导入Delta Lake模块?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！