rvest 中的 html 与 XML 中的 htmlParse [英] html in rvest verses htmlParse in XML

问题描述

如下代码所示，rvest包中的html使用了XML包中的htmlParse..

As the following code shows, html in rvest package uses htmlParse from XML package. .

html
function (x, ..., encoding = NULL) 
{
    parse(x, XML::htmlParse, ..., encoding = encoding)
}
<environment: namespace:rvest>

htmlParse
function (file, ignoreBlanks = TRUE, handlers = NULL, replaceEntities = FALSE, 
    asText = FALSE, trim = TRUE, validate = FALSE, getDTD = TRUE, 
    isURL = FALSE, asTree = FALSE, addAttributeNamespaces = FALSE, 
    useInternalNodes = TRUE, isSchema = FALSE, fullNamespaceInfo = FALSE, 
    encoding = character(), useDotNames = length(grep("^\\.", 
        names(handlers))) > 0, xinclude = TRUE, addFinalizer = TRUE, 
    error = htmlErrorHandler, isHTML = TRUE, options = integer(), 
    parentFirst = FALSE) 
.....

因此，对于以下网址:

 myurl<-"http://www.nepalstock.com.np/"
parse_XML<-htmlParse(myurl) #runs without error
parse_rvest<-html(myurl) # throws out the Internal Sever error
Error in parse.response(r, parser, encoding = encoding) : 
  server error: (500) Internal Server Error

有什么想法吗?

推荐答案

从底层 httr::GET 请求重置默认用户代理，然后它工作:

Reset the default user-agent from the underlying httr::GET request, then it works:

library(httr)
library(rvest)
parse_rvest <- html(myurl, add_headers("User-Agent" = "myagent"))

或

parse_rvest <- html(myurl, user_agent("myagent"))

请注意，出于调试目的，您可以将 verbose() 添加到 html(...).

Note that for debugging purposes, you can add verbose() to html(...).

使用新的 rvest/xml2/curl 组合，它应该看起来像:

Using the new rvest / xml2 / curl combo, it should look like:

library(xml2)
library(rvest)
library(curl)
parse_rvest <- curl(myurl, handle = new_handle("useragent" = "myua")) %>%
               read_html()

这篇关于rvest 中的 html 与 XML 中的 htmlParse的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！