rvest，如何在html_nodes中具有NA值以创建数据表 [英] rvest, How to have NA values in html_nodes for creating datatables

问题描述

因此，我正在尝试制作网站上某些信息的数据表。

So I'm trying to make a data table of some information on a website. This is what I've done so far.

library(rvest)
url <- 'https://uws-community.symplicity.com/index.php?s=student_group'
page <- html_session(url)

name_nodes <- html_nodes(page,".grpl-name a")
name_text <- html_text(name_nodes)

df <- data.frame(matrix(unlist(name_text)), stringsAsFactors = FALSE)

library(tidyverse)
df <- df %>% mutate(id = row_number())

desc_nodes <- html_nodes(page, ".grpl-purpose")
desc_text <- html_text(desc_nodes)

df <- left_join(df, data.frame(matrix(unlist(desc_text)), 
                               stringsAsFactors = FALSE) %>% 
                  mutate(id = row_number()))

email_nodes <- html_nodes(page, ".grpl-contact a")

email_text <- html_text(email_nodes)
df <- left_join(df, data.frame(matrix(unlist(email_text)), 
                               stringsAsFactors = FALSE) %>% 
                  mutate(id = row_number()))

我到了电子邮件部分。一些条目没有电子邮件。在数据框中，最后三行显示的是NA值，而不是适当的行显示了电子邮件的NA值。

This has been working until I got to the emails part. A few of the entries do not have emails. In the data frame, instead of the appropriate rows showing the NA value for the email, the last three rows show an NA value.

如何使它合适显示行是否具有NA值，而不仅仅是最后3行？

How do I make it so the appropriate rows show have the NA value instead of just the last 3 rows?

推荐答案

解决此问题的关键是找到20已知每个学生组都存在的父节点。使用此父节点列表，在每个父节点上使用 html_node 函数。 html_node 函数将返回一个结果或不适用，这取决于所需的标记是否存在。每当有可变数量的子节点时，我都会推荐这种技术。

The key for solving this problem is to find the 20 parent nodes which are known to exist for each student group. With this list of parent nodes, use the html_node function on each parent node. The html_node function will return one result or NA depending if the desired tag exists. I would recommend this technique, any time there is a variable number of sub nodes.

library(rvest)
library(dplyr)
url <- 'https://uws-community.symplicity.com/index.php?s=student_group'
page <- html_session(url)

#find group names
name_text <- html_nodes(page,".grpl-name a") %>% html_text()
df <- data.frame(name_text, stringsAsFactors = FALSE)
df <- df %>% mutate(id = row_number())

#find text description
desc_text <- html_nodes(page, ".grpl-purpose") %>% html_text()
df$desc_text <- trimws(desc_text)

#find emails
#  find the parent nodes with html_nodes
#  then find the contact information from each parent using html_node
email_nodes<-html_nodes(page, "div.grpl-grp") %>% html_node( ".grpl-contact a") %>% html_text()
df$emails<-email_nodes

我也借此机会简化了您的代码，因为列表都是20个元素，所以没有理由取消列表/基质/突变函数确实会将其他列添加到数据框中。

I also took the opportunity to simplify your code, since the lists are all 20 elements long, there is no reason for the unlist/ matrix/ mutate function do add the additional columns onto the data frame.

这篇关于rvest，如何在html_nodes中具有NA值以创建数据表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！