Twitter 流 API 从 None 给出 JSONDecodeError("Expecting value", s, err.value) [英] Twitter stream API gives JSONDecodeError("Expecting value", s, err.value) from None

问题描述

我正在使用 Twitter 的流 API(通过 tweepy)来收集符合特定条件的推文，但是当我使用 json.loads() 解析创建的 jsonl 文件时，我收到以下错误:

I am using the stream API of Twitter (through tweepy) to collect tweets matching certain criteria, but when I use json.loads() to parse the created jsonl file I get this following error:

File "twitter_time_series.py", line 19, in <module> 
    tweet = json.loads(line)

File "C:\Program Files\Anaconda3\lib\json\__init__.py", line 319, in loads 
    return _default_decoder.decode(s)

File "C:\Program Files\Anaconda3\lib\json\decoder.py", line 339, in decode
   obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "C:\Program Files\Anaconda3\lib\json\decoder.py", line 357, in raw_decode 
    raise JSONDecodeError("Expecting value", s, err.value) from None

json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)

我无法确定为什么与其他类型的 jsonl 文件(例如获取 twitter 时间线等)一样，不会发生这种情况.任何人都可以帮助我?谢谢！

I cannot determine why, as with other types of jsonl files (like when getting twitter timelines etc), this does not occur. Anyone that can help me? thanks!

我正在使用基本的流推文:

I am using the basic for stream tweets:

auth = get_twitter_auth() 

twitter_stream = Stream(auth, CustomListener(query_fname)) 

twitter_stream.filter(track=['curiosity'], async=True)

用于加载 json:

fname = sys.argv[1] 

with open(fname, "r") as f: 

    for line in f:
      tweet = json.loads(line)

我使用的是 python 3.5 和 tweepy 3.3.0 版，这是 json 文件的一行:

I am using python 3.5 and tweepy version 3.3.0 and this is one line of the json file:

{"created_at":"Mon Dec 26 16:03:06 +0000 2016",

"id":813414846033170432,

"id_str":"813414846033170432",

"text":"Battle proper and at greater risk https:\/\/t.co\/W6U9Irgst9","source":"\u003ca href=\"http:\/\/twitter.com\" rel=\"nofollow\"\u003eTwitter Web Client\u003c\/a\u003e",

"truncated":false,

"in_reply_to_status_id":null,

"in_reply_to_status_id_str":null,

"in_reply_to_user_id":null,

"in_reply_to_user_id_str":null,

"in_reply_to_screen_name":null,

"user":{"id":544570972,"id_str":"544570972","name":"Fay Moore","screen_name":"MooreFay","location":"Maryland","url":"http:\/\/faymoore.wordpress.com","description":"Author, writer for hire, crazy woman in a crazy world","protected":false,"verified":false,"followers_count":482,"friends_count":322,"listed_count":22,"favourites_count":92,
"statuses_count":17326,"created_at":"Tue Apr 03 20:30:46 +0000 2012","utc_offset":-18000,"time_zone":"Eastern Time (US & Canada)","geo_enabled":false,"lang":"en","contributors_enabled":false,"is_translator":false,"profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_tile":false,"profile_link_color":"1DA1F2","profile_sidebar_border_color":"C0DEED","profile_sidebar_fill_color":"DDEEF6","profile_text_color":"333333","profile_use_background_image":true,"profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/3252060596\/e06263f9a226ca0e3f83915812c331e6_normal.jpeg","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/3252060596\/e06263f9a226ca0e3f83915812c331e6_normal.jpeg","profile_banner_url":"https:\/\/pbs.twimg.com\/profile_banners\/544570972\/1360842914","default_profile":true,"default_profile_image":false,"following":null,"follow_request_sent":null,"notifications":null},

"geo":null,

"coordinates":null,

"place":null,

"contributors":null,

"is_quote_status":false,

"retweet_count":0,

"favorite_count":0,

"entities":{"hashtags":[],"urls":[{"url":"https:\/\/t.co\/W6U9Irgst9","expanded_url":"http:\/\/a.msn.com\/01\/en-us\/BBxzSWF?ocid=st","display_url":"a.msn.com\/01\/en-us\/BBxzS\u2026","indices":[81,104]}],"user_mentions":[],"symbols"[]},

"favorited":false,

"retweeted":false,

"possibly_sensitive":false,

"filter_level":"low",

"lang":"en",

"timestamp_ms":"1482768186468"}

推荐答案

写入文件时，在 Listener 类中，可以尝试:

When writing to file, in the Listener class, you can try:

def on_data(self, data):
    with open('filename.json', 'a', newline='\n') as f:
        f.write(data)

我在 Windows 机器上也遇到过这个问题.这是因为 Windows 使用 CR LF 行尾，而 Linux 使用 LF.

I also faced this problem on Windows machines. That's because Windows uses CR LF line ending while Linux uses LF.

如果您尝试阅读一条推文，我认为它不会出现错误，正如您在此处看到的:

If you try to read a single tweet, I don't think it would give an error as you can see here:

json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)  

这更可能是 json python 包中的错误，而不是 TwitterAPI.

This is more likely a bug in the json python package rather than the TwitterAPI.

参见:现代方式:使用 newline=''

关于行尾的更多信息:在维基百科上，一如既往.

More on line endings: On Wikipedia, as always.

这篇关于Twitter 流 API 从 None 给出 JSONDecodeError("Expecting value", s, err.value)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！