异步数据库插入-python + mysql [英] Asynchronous database inserts - python + mysql

问题描述

我正在制作一个实用程序，需要在其中提取用户的推文并将其存储在数据库中. Twitter API在一个调用中仅发送20条推文，并且要检索更多推文，您需要一次翻阅20条推文.因此，我不想从Twitter API同步读取推文并等待插入到数据库中，而是想异步启动数据库插入以优化流程.
如何使用Python和Mysql实现此目标?
伪代码可以写成(不检查语法有效性):-

I am making a utility in which I need to extract tweets for a user and store them in database. Twitter API only sends 20 tweets in one call and to retrieve more tweets, you need to page through the tweets 20 at a time. So instead of synchronously reading tweets from Twitter API and waiting to insert into DB, I want to start the database inserts asynchronously to optimize the process.
How can I achieve this with Python and Mysql?
Pseudo-code for this can be written as (not checked for syntax validity):-

    def readTweets():
       x=0
       while true: 
          tweets= twitterAPI.getusertimeline(id='twitterUser',count=20,page=x)
          #Need to know how to call the below function asynchronously
          callDBSaveAsynchronously(tweets)
          if len(tweets) < 20: break
          x=x+1

    def callDBSaveAsynchronously(tweets):
        for tweet in tweets:
            mysqldb.insertTweet(tweet)

 

提前谢谢！

推荐答案

有一个名为 ultramysql 可以利用 gevent 进行异步插入.您可以那样避免线程.在最近的项目中，这对我来说非常有用.

There's a third-party MySQL driver called ultramysql that can take advantage of gevent for async inserts. You could avoid threads that way. It worked great for me on a recent project.

这篇关于异步数据库插入-python + mysql的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！