在使用 rvest 抓取时输入缺失值的 NA [英] Inputting NA where there are missing values when scraping with rvest
问题描述
我想使用 rvest
抓取一个页面,该页面包含最近一次会议上的标题和运行时间,然后将这些值组合成一个 tibble
I want to use rvest
to scrape a page which has titles and run times of talks at a recent conference and then combine the values into a tibble
library(tibble)
library(rvest)
url <- "https://channel9.msdn.com/Events/useR-international-R-User-conferences/useR-International-R-User-2017-Conference?sort=status&direction=desc&page=14"
title <- page %>%
html_nodes("h3 a") %>%
html_text()
length <- page %>%
html_nodes(".tile .caption") %>%
html_text()
df <- tibble(title,length)
如果您查看该页面,您会发现其中一个演讲没有任何价值 - 而在查看源代码中,此演讲没有 class="caption"
If you look at the page, you will see that for one of the talks there is no value - and in View source there is no class="caption"
for this talk
有什么办法可以替换 NA
来显示缺失值?
Is there any way I can substitute an NA
to show missing values?
推荐答案
最简单的方法是选择一个节点,每行都包含你想要的两个节点,然后遍历它们,拉出你想要的两个节点立刻.purrr::map_df
不仅可以方便地迭代,甚至可以将结果组合成一个漂亮的小标题:
The simplest way is to select a node that encloses both of the nodes you want for each row, then iterate over them, pulling out both of the nodes you want at once. purrr::map_df
is handy for not only iterating, but even combining the results into a nice tibble:
library(rvest)
library(purrr)
url <- "https://channel9.msdn.com/Events/useR-international-R-User-conferences/useR-International-R-User-2017-Conference?sort=status&direction=desc&page=14"
page <- read_html(url)
df <- page %>%
html_nodes('article') %>% # select enclosing nodes
# iterate over each, pulling out desired parts and coerce to data.frame
map_df(~list(title = html_nodes(.x, 'h3 a') %>%
html_text() %>%
{if(length(.) == 0) NA else .}, # replace length-0 elements with NA
length = html_nodes(.x, '.tile .caption') %>%
html_text() %>%
{if(length(.) == 0) NA else .}))
df
#> # A tibble: 12 x 2
#> title length
#> <chr> <chr>
#> 1 Introduction to Natural Language Processing with R II 01:15:00
#> 2 Introduction to Natural Language Processing with R 01:22:13
#> 3 Solving iteration problems with purrr II 01:22:49
#> 4 Solving iteration problems with purrr 01:32:23
#> 5 Markov-Switching GARCH Models in R: The MSGARCH Package 15:55
#> 6 Interactive bullwhip effect exploration using SCperf and Shiny 16:02
#> 7 Actuarial and statistical aspects of reinsurance in R 14:15
#> 8 Transformation Forests 16:19
#> 9 Room 2.02 Lightning Talks 50:35
#> 10 R and Haskell: Combining the best of two worlds 14:45
#> 11 *GNU R* on a Programmable Logic Controller (PLC) in an Embedded-Linux Environment <NA>
#> 12 Performance Benchmarking of the R Programming Environment on Knight's Landing 19:32
这篇关于在使用 rvest 抓取时输入缺失值的 NA的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!