DATABRICKS connect 6.4无法再与服务器通信 [英] DATABRICKS connect 6.4 not able to communicate with server anymore

问题描述

我在MacBook上运行Pycharm.

I am running Pycharm on my MacBook.

客户端设置:Python解释器->Python 3.7(dtabricks-connect-6.4)

Client settings: Python Interpreter -> Python 3.7 (dtabricks-connect-6.4)

集群设置:Databricks运行时版本->6.4(包括Apache Spark 2.4.5，Scala 2.11)

Cluster settings: Databricks Runtime Version -> 6.4 (includes Apache Spark 2.4.5, Scala 2.11)

几个月以来它一直运行良好，但是突然之间，没有进行任何更新，我无法再将Pycharm的python脚本针对databricks集群运行.错误是...

It worked well for months but suddenly, without any updates made, I cant run my python script from Pycharm against databricks cluster anymore. The Error is ...

Caused by: `java.lang.IllegalArgumentException: The cluster is running server version `dbr-6.4` but this client only supports Set(dbr-5.5)....`

我重新启动了Pycharm，回头了，为解释器，我重新启动了集群，甚至重新启动了MacBook，但它没有帮助.错误消息只是错误的，因为群集和客户端都使用SAME版本.我可以看到，当我执行python脚本时，集群正在启动，但最终失败.

I restarted Pycharm, I witched back and for the interpreter, I restarted the cluster and I even restarted my MacBook but it didnt helped. The Error message is simply false, because both, cluster and client are using the SAME version. I can see that when I execute my python script that the cluster is being started but fails at the end.

pyenv activate databricks-connect-6-4

pip freeze

Cython==0.29.21
databricks-connect==6.4.0
numpy==1.19.2
pandas==1.0.1
py4j==0.10.7
pyarrow==0.13.0
pycountry==20.7.3
python-dateutil==2.8.1
pytz==2020.1
six==1.15.0

推荐答案

它看起来像是服务器端的一些内部更改所致，这阻止了databricks-connect的正常工作.您始终可以通过以下方式禁用此检查:在执行 databricks-之前，在控制台中将环境变量 DEBUG_IGNORE_VERSION_MISMATCH 设置为值 1 ( export DEBUG_IGNORE_VERSION_MISMATCH = 1 ).连接测试)-您还可以在PyCharm中设置此环境变量.

It looks like that it was caused by some internal changes on the server side, and this prevents databricks-connect from working. You can always disable this check by setting environment variable DEBUG_IGNORE_VERSION_MISMATCH to value 1 (export DEBUG_IGNORE_VERSION_MISMATCH=1 in the console before executing databricks-connect test) - you can also set this environment variable in the PyCharm.

更新:应通过 Databricks Connect 6.4.2 进行修复.刚刚被释放.

Update: it should be fixed by Databricks Connect 6.4.2 that was just released.

这篇关于DATABRICKS connect 6.4无法再与服务器通信的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！