火花连接到Phoenix NoSuchMethod异常 [英] spark connecting to Phoenix NoSuchMethod Exception

问题描述

我正在尝试通过Spark/Scala连接到Phoenix以读取和写入数据作为DataFrame.我在

我搜索了该警告，但没有任何参考，也找不到任何建议方法的示例.我找不到任何其他好的方法来指导如何连接Phoenix.谢谢您的宝贵时间.

解决方案

请使用.read而不是如下所示的负载

  val df = sparkSession.sqlContext.read.format("org.apache.phoenix.spark").option("zkUrl"，"localhost:2181").option("table"，"TABLE1").load() 

I am trying to connect to Phoenix through Spark/Scala to read and write data as a DataFrame. I am following the example on GitHub however when I try the very first example Load as a DataFrame using the Data Source API I get the below exception.

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Put.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Put;

There are couple of things that are driving me crazy from those examples:

1)The import statement import org.apache.phoenix.spark._ gives me below exception in my code:

cannot resolve symbol phoenix

I have included below jars in my sbt

"org.apache.phoenix" % "phoenix-spark" % "4.4.0.2.4.3.0-227" % Provided,
"org.apache.phoenix" % "phoenix-core" % "4.4.0.2.4.3.0-227" % Provided,

2) I get the deprecated warning for symbol load.

I googled about that warnign but didn't got any reference and I was not able to find any example of the suggested method. I am not able to find any other good resource which guides on how to connect to Phoenix. Thanks for your time.

解决方案

please use .read instead of load as shown below

val df = sparkSession.sqlContext.read
        .format("org.apache.phoenix.spark")
        .option("zkUrl", "localhost:2181")
        .option("table", "TABLE1").load()

这篇关于火花连接到Phoenix NoSuchMethod异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！