使用python多处理程序进行并行表更新(PostGIS/PostgreSQL) [英] Parallel table update using python multiprocessing (PostGIS/PostgreSQL)
本文介绍了使用python多处理程序进行并行表更新(PostGIS/PostgreSQL)的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
Hi I have been using the multiprocessing approach for updating a huge postgis table from here and it works quite well, but my table doesn't seem to get updated unless I add a commit;
statement at the end of
procQuery = 'UPDATE city SET gid_fkey = gid FROM country WHERE ST_within((SELECT the_geom FROM city WHERE city_id = %s), country.the_geom) AND city_id = %s' % (self.a, self.a)
喜欢
procQuery = 'UPDATE city SET gid_fkey = gid FROM country WHERE ST_within((SELECT the_geom FROM city WHERE city_id = %s), country.the_geom) AND city_id = %s;commit;' % (self.a, self.a)
这里可能是什么问题?
推荐答案
好的,我通过将连接设置为autommit = True
ok, I solved by setting the connection to autommit = True
class Consumer(multiprocessing.Process):
def __init__(self, task_queue, result_queue):
multiprocessing.Process.__init__(self)
self.task_queue = task_queue
self.result_queue = result_queue
self.pyConn = psycopg2.connect(**connstring)
self.pyConn.autocommit = True
def run(self):
proc_name = self.name
while True:
next_task = self.task_queue.get()
if next_task is None:
print 'Tasks Complete'
self.task_queue.task_done()
break
answer = next_task(connection=self.pyConn)
self.task_queue.task_done()
self.result_queue.put(answer)
return
这篇关于使用python多处理程序进行并行表更新(PostGIS/PostgreSQL)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文