lxml错误"IOError:读取文件时出错"；在python scraper脚本中解析facebook mobile时 [英] lxml error "IOError: Error reading file" when parsing facebook mobile in a python scraper script
问题描述
我使用来自使用python登录到Facebook 的修改后的脚本帖子:
I use a modified script from Logging into facebook with python post :
#!/usr/bin/python2 -u
# -*- coding: utf8 -*-
facebook_email = "YOUR_MAIL@DOMAIN.TLD"
facebook_passwd = "YOUR_PASSWORD"
import cookielib, urllib2, urllib, time, sys
from lxml import etree
jar = cookielib.CookieJar()
cookie = urllib2.HTTPCookieProcessor(jar)
opener = urllib2.build_opener(cookie)
headers = {
"User-Agent" : "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7",
"Accept" : "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,text/png,*/*;q=0.5",
"Accept-Language" : "en-us,en;q=0.5",
"Accept-Charset" : "utf-8",
"Content-type": "application/x-www-form-urlencoded",
"Host": "m.facebook.com"
}
try:
params = urllib.urlencode({'email':facebook_email,'pass':facebook_passwd,'login':'Log+In'})
req = urllib2.Request('http://m.facebook.com/login.php?m=m&refsrc=m.facebook.com%2F', params, headers)
res = opener.open(req)
html = res.read()
except urllib2.HTTPError, e:
print e.msg
except urllib2.URLError, e:
print e.reason[1]
def fetch(url):
req = urllib2.Request(url,None,headers)
res = opener.open(req)
return res.read()
body = unicode(fetch("http://www.facebook.com/photo.php?fbid=404284859586659&set=a.355112834503862.104278.354259211255891&type=1"), errors='ignore')
tree = etree.parse(body)
r = tree.xpath('/see_prev')
print r.text
执行代码时，出现问题:
When I execute the code, problems appears :
$ ./facebook_fetch_coms.py
Traceback (most recent call last):
File "./facebook_fetch_coms_classic_test.py", line 42, in <module>
tree = etree.parse(body)
File "lxml.etree.pyx", line 2957, in lxml.etree.parse (src/lxml/lxml.etree.c:56230)
File "parser.pxi", line 1533, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:82313)
File "parser.pxi", line 1562, in lxml.etree._parseDocumentFromURL (src/lxml/lxml.etree.c:82606)
File "parser.pxi", line 1462, in lxml.etree._parseDocFromFile (src/lxml/lxml.etree.c:81645)
File "parser.pxi", line 1002, in lxml.etree._BaseParser._parseDocFromFile (src/lxml/lxml.etree.c:78554)
File "parser.pxi", line 569, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:74498)
File "parser.pxi", line 650, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:75389)
File "parser.pxi", line 588, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:74691)
IOError: Error reading file '<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.0//EN" "http://www.wapforum.org/DTD/xhtml-mobile10.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Facebook</title><meta name="description" content="Facebook helps you connect and share with the people in your life."
目标是首先获得id=see_prev与lxml的链接，然后使用while循环打开所有注释，最终获取文件中的所有消息.任何帮助将不胜感激！
The goal is first to get the link with id=see_prev with lxml, then using a while loop to open all comments, to finally fetch all messages in a file. Any help will be very appreciated !
修改: 我在archlinux x86_64和lxml 2.3.3上使用Python 2.7.2.
Edit: I use Python 2.7.2 on archlinux x86_64 and lxml 2.3.3.
推荐答案
这是您的问题:
tree = etree.parse(body)
文档说，"source是包含XML数据的文件名或文件对象."您提供了一个字符串，因此lxml将HTTP响应正文的文本作为您要打开的文件的 name .没有这样的文件，所以您得到一个IOError.
The documentation says that "source is a filename or file object containing XML data." You have provided a string, so lxml is taking the text of your HTTP response body as the name of the file you wish to open. No such file exists, so you get an IOError.
您收到的错误消息甚至说错误读取文件"，然后将您的XML字符串作为要读取的文件的名称提供，这是正在发生的事情的重要提示.
The error message you get even says "Error reading file" and then gives your XML string as the name of the file it's trying to read, which is a mighty big hint about what's going on.
您可能需要 etree.XML() ，从字符串输入.或者，您可以执行tree = etree.parse(res)直接从HTTP请求读取到lxml中(opener.open()的结果是一个类似文件的对象，并且etree.parse()应该非常乐意使用它).
You probably want etree.XML(), which takes input from a string. Or you could just do tree = etree.parse(res) to read directly from the HTTP request into lxml (the result of opener.open() is a file-like object, and etree.parse() should be perfectly happy to consume it).
这篇关于lxml错误"IOError:读取文件时出错"；在python scraper脚本中解析facebook mobile时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
