在R中使用XML包时出错 [英] Error using XML package in R

问题描述

我正在收集有关不同大学的数据，在执行以下代码后，我对追随错误有疑问.问题是使用htmlParse()

I am gathering data about different universities and I have a question about the follow error after executing the following code. The problem is when using htmlParse()

代码:

url1<-"http://nces.ed.gov/collegenavigator/?id=165015"

url1 <- "http://nces.ed.gov/collegenavigator/?id=165015"

webpage1<-getURL(url1)

webpage1<- getURL(url1)

doc1<-htmlParse(webpage1)

doc1 <- htmlParse(webpage1)

输出:

htmlParse(webpage1)中的错误:文件

Error in htmlParse(webpage1) : File

！DOCTYPE html PUBLIC-//W3C//DTD XHTML 1.0 Transitional//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"

!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"

html xmlns ="http://www.w3.org/1999/xhtml"头id ="ctl00_hd"元http-equiv =内容类型" content ="text/html;字符集= UTF-8" /标题

html xmlns="http://www.w3.org/1999/xhtml" head id="ctl00_hd"meta http-equiv="Content-type" content="text/html;charset=UTF-8" /title

    College Navigator - National Center for Education Statistics

/titlelink href ="css/md0.css" type ="text/css" rel ="stylesheet" meta name ="keywords" content =学院导航员，学院搜索，中学后教育，中学后统计，NCES，IPEDS ，college locator"/meta meta name =" description"content =" College Navigator是一个免费的消费者信息工具，旨在帮助学生，父母，高中辅导员和其他人获取有关美国7,000多家高等教育机构的信息-提供的计划，保留率和毕业率，价格，可用的援助，授予的学位，校园安全和认证."meta>元名称="机器人"content ="索引，nofollow"/metalink

/titlelink href="css/md0.css" type="text/css" rel="stylesheet" meta name="keywords" content="college navigator,college search,postsecondary education,postsecondary statistics,NCES,IPEDS,college locator"/meta meta name="description" content="College Navigator is a free consumer information tool designed to help students, parents, high school counselors, and others get information about over 7,000 postsecondary institutions in the United States - such as programs offered, retention and graduation rates, prices, aid available, degrees awarded, campus safety, and accreditation."meta>meta name="robots" content="index,nofollow"/metalink

在使用此程序包之前，我的网站已抓取网页，但我从未遇到过问题.名称="robots"与它有关系吗?任何帮助将不胜感激.

I have webs scraped pages before using this package and I never had an issue. Does the name="robots" have anything to do with it? Any help would be greatly appreciate.

推荐答案

http://validator.w3.org/check?verbose=1&uri=http%3A%2F%2Fnces.ed.gov%2Fcollegenavigator%2F%3Fid%3D165015 indicates the webpage is badly formed. Your browser can compensate for this but your R package is having problems.

如果您使用的是Windows，则可以通过以下方式获取IE浏览器对其进行修复:

if you are using windows you can get the IE browser to fix it for you as follows:

library(rcom)
library(XML)
ie = comCreateObject('InternetExplorer.Application')
ie[["visible"]]=T # true for debugging
comInvoke(ie,"Navigate2","http://nces.ed.gov/collegenavigator/?id=165015")
while(comGetProperty(ie,"busy")||comGetProperty(ie,"ReadyState")<4){
 Sys.sleep(1)
 print(comGetProperty(ie,"ReadyState"))
}
myDoc<-comGetProperty(ie,"Document")
webpage1<-myDoc$getElementsByTagName('html')[[0]][['innerHTML']]
ie$Quit()
doc1 <- htmlParse(webpage1)

这篇关于在R中使用XML包时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！