HiveUDF + saxon 9.1.0.8 + Java8 =创建XPathFactory失败 [英] HiveUDF + saxon 9.1.0.8 + Java8 = failed to create an XPathFactory

问题描述

除非在代码中未定义UDF，否则我的带有HiveContext和Saxon的Spark作业可以正常工作.如果使用UDF，则HiveContext初始化失败，并显示错误.我听说saxon 9.5.1.5中解决了saxon \ java8不兼容的问题，该文件尚未在中央maven存储库中作为免费版本发布:

My Spark job with HiveContext and Saxon working fine unless no UDFs defined in code. In case of UDF implementation - HiveContext initialization failed with error. I heard there are saxon\java8 incompability solved in saxon 9.5.1.5, which is not released yet as free version in central maven repository:

由以下原因引起:java.lang.RuntimeException:XPathFactory#newInstance() 无法为默认对象模型创建XPathFactory: http://java.sun.com/jaxp/xpath/dom 与 XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: java.util.ServiceConfigurationError:javax.xml.xpath.XPathFactory: jar:文件:/JBOD_D19/hadoop/cdh/yarn/nm/usercache/u23120d1/appcache/application_1477998759081_5017/container_e45_1477998759081_5017_01_000001/saxon-xpath-9.1.0.8.jar！/META-INF/services/javax.xml.xpath.XPathFactory 2: 位于的配置文件语法非法 javax.xml.xpath.XPathFactory.newInstance(XPathFactory.java:102)在 org.apache.hadoop.hive.ql.udf.xml.UDFXPathUtil.(UDFXPathUtil.java:41) 在 org.apache.hadoop.hive.ql.udf.xml.GenericUDFXPath.(GenericUDFXPath.java:53)

Caused by: java.lang.RuntimeException: XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: java.util.ServiceConfigurationError: javax.xml.xpath.XPathFactory: jar:file:/JBOD_D19/hadoop/cdh/yarn/nm/usercache/u23120d1/appcache/application_1477998759081_5017/container_e45_1477998759081_5017_01_000001/saxon-xpath-9.1.0.8.jar!/META-INF/services/javax.xml.xpath.XPathFactory:2: Illegal configuration-file syntax at javax.xml.xpath.XPathFactory.newInstance(XPathFactory.java:102) at org.apache.hadoop.hive.ql.udf.xml.UDFXPathUtil.(UDFXPathUtil.java:41) at org.apache.hadoop.hive.ql.udf.xml.GenericUDFXPath.(GenericUDFXPath.java:53)

推荐答案

正确，在类路径上使用Java8中的XPathFactory.newInstance()方法和较旧版本的Saxon会导致此失败.因此，请使用较新版本的Saxon.当前版本是9.7.0.11.

Correct, using the XPathFactory.newInstance() method from Java8 with an older release of Saxon on the classpath will cause this failure. So use a newer version of Saxon. The current release is 9.7.0.11.

Saxon的最新版本可以在Maven中找到.我们拒绝将Saxon放入Maven已有很多年，因为从Maven下载不满足我们使用的某些第三方软件组件所要求的条件，也就是说，您必须在不分发法律条款和条件的情况下分发JAR文件.尽管由于通过Maven进行分发仍然违反了这种条件，但最终由于需求压倒，我们最终松懈了.关心这些事情的组织不应使用Maven.

Recent releases of Saxon can be found in Maven. We resisted putting Saxon in Maven for many years because downloading from Maven does not satisfy the condition required some of the third-party software components we use, that say you must not distribute the JAR files without also distributing the legal terms and conditions. We eventually relented because of overwhelming demand, despite the fact that distributing via Maven still violates this condition. Organisations that care about these things should not use Maven.

这篇关于HiveUDF + saxon 9.1.0.8 + Java8 =创建XPathFactory失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！