如何使用python并行查询数据库 [英] How to use python to query database in parallel

问题描述

我有两个用于查询数据库的函数.假设有两个单独的查询，如何并行运行这些查询以查询同一个数据库，并在继续执行其余代码之前等待两个结果返回?

I have two functions which I use to query database. Assuming two separate queries, how to run these in parallel to query same database, and also wait for both results to return before continuing the execution of the rest of the code?

def query1(param1, param2):
    result = None
    logging.info("Connecting to database...")
    try:
        conn = connect(host=host, port=port, database=db)
        curs = conn.cursor()
        curs.execute(query)
        result = curs
        curs.close()
        conn.close()
    except Exception as e:
        logging.error("Unable to access database %s" % str(e))
    return result


def query2(param1, param2):
    result = None 
    logging.info("Connecting to database...")
    try:
        conn = connect(host=host, port=port, database=db)
        curs = conn.cursor()
        curs.execute(query)
        result = curs
        curs.close()
        conn.close()  
    except Exception as e:
        logging.error("Unable to access database %s" % str(e))    
    return result

推荐答案

这是一个多线程代码，可以完成您想要完成的任务:

Here is a multi-threaded code that does what you're trying to accomplish:

from threading import Thread, Lock

class DatabaseWorker(Thread):
    __lock = Lock()

    def __init__(self, db, query, result_queue):
        Thread.__init__(self)
        self.db = db
        self.query = query
        self.result_queue = result_queue

    def run(self):
        result = None
        logging.info("Connecting to database...")
        try:
            conn = connect(host=host, port=port, database=self.db)
            curs = conn.cursor()
            curs.execute(self.query)
            result = curs
            curs.close()
            conn.close()
        except Exception as e:
            logging.error("Unable to access database %s" % str(e))
        self.result_queue.append(result)

delay = 1
result_queue = []
worker1 = DatabaseWorker("db1", "select something from sometable",
        result_queue)
worker2 = DatabaseWorker("db1", "select something from othertable",
        result_queue)
worker1.start()
worker2.start()

# Wait for the job to be done
while len(result_queue) < 2:
    sleep(delay)
job_done = True
worker1.join()
worker2.join()

这篇关于如何使用python并行查询数据库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！