如何写入Microsoft SQL Server表使用PySpark即使表中 [英] How to write into Microsoft SQL Server table even if table exist using PySpark

问题描述

我有一个PySpark code其中写入这样的SQL Server数据库

I have a PySpark Code which writes into SQL Server database like this

 df.write.jdbc(url=url, table="AdventureWorks2012.dbo.people", properties=properties)

但问题是，我要继续写在表格中的人即使表存在，我在星火文件，有可能的选择错误，附加，而是覆盖，而忽略模式和他们都抛出错误，请参阅如果数据库表中已经存在的对象已经存在。

However problem is that I want to keep writing in the table people even if the table exist and I see in the Spark Document that there are possible options error, append, overwrite and ignore for mode and all of them throws error, the object already exist if the table already exist in the database.

星火抛出下面的错误
    py4j.protocol.Py4JJavaError：同时呼吁o43.jdbc发生错误。
    com.microsoft.sqlserver.jdbc.SQLServerException：有已在数据库中名为'人'的对象。

Spark throw following error py4j.protocol.Py4JJavaError: An error occurred while calling o43.jdbc. com.microsoft.sqlserver.jdbc.SQLServerException: There is already an object named 'people' in the database

有没有办法将数据写入即使表已经存在的表？
请让我知道你需要更多的解释

Is there way to write data into the table even if the table already exits ? Please let me know you need more explanation

推荐答案

有关我的问题是与星火1.5.2。它检查的方式，如果表中存在（<一个href=\"https://github.com/apache/spark/blob/v1.5.2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala#L48\"相对=nofollow>这里）是通过运行 SELECT 1 FROM $表LIMIT 1 。如果查询失败，则表不存在。即使当表在那里，查询失败。

For me the issue was with Spark 1.5.2. The way it checks if the table exists (here) is by running SELECT 1 FROM $table LIMIT 1. If the query fails, the tables doesn't exist. That query failed even when the table was there.

这已更改为 SELECT * FROM $表，其中1 = 0 在1.6.0（<一个href=\"https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala#L99\"相对=nofollow>这里）。

This was changed to SELECT * FROM $table WHERE 1=0 in 1.6.0 (here).

这篇关于如何写入Microsoft SQL Server表使用PySpark即使表中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！