使用火花CSV阅读飞艇的CSV文件 [英] Reading csv files in zeppelin using spark-csv

问题描述

我想读齐柏林CSV文件，并想用databricks
火花CSV包： https://github.com/databricks/spark-csv

I wanna read csv files in Zeppelin and would like to use databricks' spark-csv package: https://github.com/databricks/spark-csv

在火花壳，我可以使用火花CSV以

In the spark-shell, I can use spark-csv with

spark-shell --packages com.databricks:spark-csv_2.11:1.2.0

但我怎么告诉飞艇使用该包？

But how do I tell Zeppelin to use that package?

在此先感谢！

推荐答案

您需要星火包存储库添加到齐柏林，才能在火花包使用DEP％。

You need to add the Spark Packages repository to Zeppelin before you can use %dep on spark packages.

%dep
z.reset()
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.load("com.databricks:spark-csv_2.10:1.2.0")

另外，如果这是你想要在你所有的笔记本电脑可用的东西，你可以在--packages选项添加到在齐柏林间preters配置的火花提交命令设置，然后重新启动跨preTER。这应该开始已经加载按火花壳方法包的上下文。

Alternatively, if this is something you want available in all your notebooks, you can add the --packages option to the spark-submit command setting in the interpreters config in Zeppelin, and then restart the interpreter. This should start a context with the package already loaded as per the spark-shell method.

这篇关于使用火花CSV阅读飞艇的CSV文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！