如何在HDP的zeppelin-spark2中将库安装到python [英] How to install libraries to python in zeppelin-spark2 in HDP

问题描述

我正在使用HDP版本:2.6.4

I am using HDP Version: 2.6.4

您能否提供有关如何将库安装到spark2下的以下python目录的分步说明?

Can you provide a step by step instructions on how to install libraries to the following python directory under spark2 ?

sc.version(火花版本)返回

The sc.version (spark version) returns

res0: String = 2.2.0.2.6.4.0-91

spark2解释器的名称和值如下

The spark2 interpreter name and value is as following

zeppelin.pyspark.python:    /usr/local/Python-3.4.8/bin/python3.4

python版本和当前库是

The python version and current libraries are

%spark2.pyspark

import pip
import sys

sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])

print("--")     
print (sys.version)
print("--")
print(installed_packages_list)

--
3.4.8 (default, May 30 2018, 11:05:04) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)]
--
['pip==9.0.1', 'setuptools==28.8.0']

更新1:using pip install [package name]实际上导致两个问题

1)HDP指向python2.6而不是python3.4.8

Update 1: using pip install [package name] actually leads to two problems

1) The HDP is pointing at python2.6 rather than python3.4.8

2)pip3由于某种原因不存在

2) pip3 is not there for some reason

因此，我正在考虑安装miniconda并将Zeppelin指向那里，并在conda中安装所有软件包，以防止python 2.6和3.4.8之间发生冲突.

Therefore, I am thinking of installing miniconda and pointing Zeppelin there and installing all the packages in conda to prevent conflict between python 2.6 and 3.4.8

推荐答案

您需要打开终端并输入pip并按TAB键.您的沙盒上可用的点子版本应列出.使用 pip3 安装所需的软件包.这样做的方法保持不变.这将使您希望在Zeppelin中使用的Python3安装中可以使用该软件包.

You need to open your terminal and type pip and press the TAB key. The pip versions available on your sandbox shall be listed. Use pip3 to install the packages you require. The way to do so remains the same pip3 install "packageName". This would make the package available with the Python3 installation you wish to use in Zeppelin.

这篇关于如何在HDP的zeppelin-spark2中将库安装到python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！