通过Python命令和dbutils在Databricks上安装Maven库 [英] Installing Maven library on Databricks via Python commands and dbutils

问题描述

在Databricks上，如果尚未安装，我想通过Python Notebook中的命令安装Maven库.

如果它是Python PyPI库，我将执行以下操作:

 #获取所有可用库的列表library_name_list = dbutils.library.list()#假设感兴趣的库是"scikit-learn"，而如果"scikit-learn"不在library_name_list中:#安装库dbutils.library.installPyPI("scikit-learn") 

如何对Maven库执行相同的操作"com.microsoft.azure.kusto:spark-kusto-connector:2.0.0&"；即检查它是否已经安装，如果尚未安装，则安装吗?

我可以使用UI，通过转到集群"(Custers)来安装Maven库.->图书馆"->安装新"->"Maven".但是，我想通过脚本以编程方式进行操作.

解决方案

注意:库实用程序( dbutils.library.installPyPI(''))允许您安装 Python库并创建一个环境仅限于笔记本会话.

以下是通过编程方式从Maven存储库安装库的步骤:

您可以使用Databricks CLI在Azure Databricks中安装Maven库.

第1步::您可以从Maven坐标中转到Maven存储库，然后选择要查找的版本，并记下相关性(groupId，artifactId和Version).

步骤2 ，使用databricks CLI获取群集ID.

要获取群集ID: databricks群集列表

第3步::使用下面的Databricks CLI命令在Databricks中安装" com.microsoft.azure.kusto:spark-kusto-connector:2.0.0 "./p>

语法: databricks库安装--cluster-id集群ID"--maven-coordinates"GroupId:ArtifactId:Version"(即org.jsoup:jsoup:1.7.2)

要使用databricks CLI安装Maven库，请执行以下操作: databricks库安装--cluster-id"1013-095611-mazes551"；--maven-coordinates" com.microsoft.azure.kusto:spark-kusto-connector:2.0.0"

要检查群集上的安装库，请执行以下操作: databricks库列表--cluster-id"1013-095611-mazes551"

有关在Azure Databricks中安装软件包的不同方法，请参考: 解决方案

Note: Library utilities (dbutils.library.installPyPI("")) allow you to install Python libraries and create an environment scoped to a notebook session.

Here are the steps to programmatically way to install libraries from the maven repository:

You can use Databricks CLI to install maven libraries in Azure Databricks.

Step1: From maven coordinates, you can go to Maven Repository and pick the version which you are looking for and note the dependency (groupId, artifactId, and Version).

Step2 Get the cluster-ID using databricks CLI.

To get the cluster-ID: databricks clusters list

Step3: Use the below Databricks CLI command to install 'com.microsoft.azure.kusto:spark-kusto-connector:2.0.0' in Databricks.

Syntax: databricks libraries install --cluster-id "Cluster ID" --maven-coordinates "GroupId:ArtifactId:Version" (i.e.org.jsoup:jsoup:1.7.2)

To install the maven library using databricks CLI: databricks libraries install --cluster-id "1013-095611-mazes551" --maven-coordinates "com.microsoft.azure.kusto:spark-kusto-connector:2.0.0"

To check the install libraries on the cluster: databricks libraries list --cluster-id "1013-095611-mazes551"

For different methods to install packages in Azure Databricks, refer: How to install a library on a databricks cluster using some command in the notebook?

这篇关于通过Python命令和dbutils在Databricks上安装Maven库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！