使用Tweepy提取推文时出错 [英] Error while fetching Tweets with Tweepy

问题描述

我有一个获取推文的Python脚本.在脚本中，我使用了库 Tweepy .我使用有效的身份验证参数.运行此脚本后，某些推文存储在我的MongoDB中，而某些则被if语句拒绝.但仍然出现错误

I have a Python script that fetch tweets. In the script i use the libary Tweepy . I use a valid authentication parameters. After running this script some tweets are stored in my MongoDB and some are refused by the if statement. But still i get the error

requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read, 2457 more expected)'

我的问题是我可以改进脚本的哪一部分，所以我没有得到上面的错误.

My question is which part of the script can i improve, so i do not get the error above.

这是我的剧本

    from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import time
import json
from pymongo import MongoClient

#Mongo Settings
client = MongoClient()
db = client.Sentiment
Tweets = db.Tweet

#Twitter Credentials
ckey ='myckey'
csecret ='mycsecret'
atoken = 'myatoken'
asecret = 'myasecret'

class listener(StreamListener):

    def on_data(self, data):
        try:  

            tweet = json.loads(data)

            if tweet["lang"] == "nl":
                print tweet["id"]
                Tweets.insert(tweet)



            return True
        except BaseException, e:
            print 'failed on_date,', str(e)
            time.sleep(5)

    def on_error(self, status):
        print status

auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter( track=["geld lenen"
                            ,"lening"
                            ,"Defam"
                            ,"DEFAM"
                            ,"Credivance"
                            ,"CREDIVANCE"
                            ,"Alpha Credit"
                            ,"ALPHA CREDIT"
                            ,"Advanced Finance"
                            ,"krediet"
                            ,"KREDIET"
                            ,"private lease"
                            ,"ing"
                            ,"Rabobank"
                            ,"Interbank"
                            ,"Nationale Nerderlanden"
                            ,"Geldshop"
                            ,"Geldlenen"
                            ,"ABN AMBRO"
                            ,"Independer"
                            ,"DGA adviseur"
                            ,"VDZ"
                            ,"vdz"
                            ,"Financieel Attent"
                            ,"Anderslenen"
                            ,"De Nederlandse Kredietmaatschappij"
                            ,"Moneycare"
                            ,"De Financiele Makelaar Kredieten"
                            ,"Finanplaza"
                            ,"Krediet"
                            ,"CFSN Kredietendesk"
                            ,"De Graaf Assurantien en Financieel Adviseurs"
                            ,"AMBTENARENLENING"
                            ,"VDZ Geldzaken"
                            ,"Financium Primae"
                            ,"SNS"
                            ,"AlfamConsumerCredit"
                            ,"GreenLoans"
                            ], languages="nl" 
                     )

希望您能帮助我...

I hope you can help me...

推荐答案

我遇到了同样的问题，当我从过滤器功能中删除languages时解决了该问题

I had this same problem, solved when I remved languages from the filter function

由于尚未运行，尽管Twitter表示已运行

since it's not yet functional, although Twitter says it is

相反，我会像您在on_data(..)

我也按如下方式使用on_status(..)而不是on_data(..):

Also I use the on_status(..) instead of on_data(..) as follows:

def on_status(self, status):
    ...
    tweet = json.dumps(status)
    if tweet["lang"] == "nl":
        print tweet["id"]
        Tweets.insert(tweet)
    ...

其他人报告说使用了twitterStream.filter(track=['word'], languages=['nl'])，但是我不赞成.

Other people reported that using twitterStream.filter(track=['word'], languages=['nl']), but it didn't with me.

这篇关于使用Tweepy提取推文时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！