在Spark SQL中更新查询 [英] update query in Spark SQL

问题描述

我想知道我是否可以像下面这样在sparksql中使用更新查询:

I wonder can I use the update query in sparksql just like:

sqlContext.sql("update users set name = '*' where name is null")

我得到了错误:

org.apache.spark.sql.AnalysisException: 
Unsupported language features in query:update users set name = '*' where name is null

如果sparksql不支持更新查询，或者我写的代码不正确?

If the sparksql does not support the update query or am i writing the code incorrectly?

推荐答案

Spark SQL尚不支持UPDATE语句.

Spark SQL doesn't support UPDATE statements yet.

Hive自从Hive 0.14版本以来就开始支持UPDATE.但是即使使用Hive，它也仅在支持事务的那些表上支持更新/删除，这在

Hive has started supporting UPDATE since hive version 0.14. But even with Hive, it supports updates/deletes only on those tables that support transactions, it is mentioned in the hive documentation.

在databricks论坛中查看答案，以确认 UPDATE/DELETES在Spark SQL中不支持，因为它

See the answers in databricks forums confirming that UPDATES/DELETES are not supported in Spark SQL as it doesn't support transactions. If we think, supporting random updates is very complex with most of the storage formats in big data. It requires scanning huge files, updating specific records and rewriting potentially TBs of data. It is not normal SQL.

这篇关于在Spark SQL中更新查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！