使用dplyr过滤postgreSQL数据库中的多个值 [英] Filter multiple values in postgreSQL database with dplyr
问题描述
我想过滤一个postgres数据库,所以我可以把一个数据的子集进入R进行分析。我可以通过单个条件(选择一个featureid)成功进行过滤,但不能通过值向量进行过滤。例如,如果我设置与数据库的连接,那么
I want to filter a postgres database so I can bring a subset of the data into R for analysis. I can successfully filter by a single condition (pick a single featureid) but not by a vector of values. For example, if I set up the connection to the database as such
library(dplyr)
db <- src_postgres(dbname = 'conte_dev', host = '155.0.0.x', port = '1234', user = '...', password = '...')
tbl_daymet <- tbl(db, 'daymet')
那么如果我过滤到一个值,那么它的工作
then it works if I filter to a single value
tbl_filtered <- tbl_daymet %>%
dplyr::filter(featureid == 739554)
tbl_filtered
Source: postgres 9.3.5 [conte@127.0.0.1:5432/conte_dev]
From: daymet [12,410 x 9]
Filter: featureid == 739554
featureid date tmax tmin prcp dayl srad vp swe
1 739554 1980-01-18 -1.9375 -12.2500 0.000 32140.8 199.6 240 100.5
2 739554 1980-01-19 1.1250 -3.4375 0.000 32140.8 100.4 480 99.0
3 739554 1980-01-20 0.0000 -7.5000 0.000 32486.4 160.4 360 99.0
4 739554 1980-01-21 -6.5000 -15.7500 0.000 32486.4 193.6 180 99.0
5 739554 1980-01-22 -11.8125 -18.7500 0.000 32486.4 156.8 140 99.0
6 739554 1980-01-23 -6.4375 -16.5000 3.000 32832.0 157.2 160 102.5
7 739554 1980-01-24 -6.8750 -19.0000 3.125 32832.0 178.0 120 105.0
8 739554 1980-01-25 -15.0000 -23.0625 0.000 32918.4 184.4 80 105.0
9 739554 1980-01-26 -9.9375 -20.7500 0.000 33177.6 229.2 120 105.0
10 739554 1980-01-27 -7.0625 -15.9375 0.000 33177.6 202.4 165 105.0
.. ... ... ... ... ... ... ... ... ...
但是,如果我尝试过滤到一组featureid
However if I try to filter to a group of values in featureid
catches <- c(739554, 739554)
tbl_derived_metrics <- tbl_daymet %>%
dplyr::filter(featureid %in% catches)
我收到错误
postgresqlExecStatement(conn,statement,...)中的错误:RS-DBI
驱动程序:(无法检索t他的结果是:错误:或
附近的语法错误739554LINE 3:WHEREfeatureidIN 739554
^)另外:警告消息:在postgresqlQuickSQL(conn,statement,...)中:无法创建
executeSELECT count(*)FROM(SELECTfeatureid,date,tmax,
tmin,prcp,dayl,srad,vp ,sweFROMdaymetWHERE
featureidIN 739554)ASmaster
Error in postgresqlExecStatement(conn, statement, ...) : RS-DBI driver: (could not Retrieve the result : ERROR: syntax error at or near "739554" LINE 3: WHERE "featureid" IN 739554 ^ ) In addition: Warning message: In postgresqlQuickSQL(conn, statement, ...) : Could not create executeSELECT count(*) FROM (SELECT "featureid", "date", "tmax", "tmin", "prcp", "dayl", "srad", "vp", "swe" FROM "daymet" WHERE "featureid" IN 739554) AS "master"
我相信这样如果它是R中的数据帧,而不是postgres中的链接表,则可以工作。但是,我需要先进行过滤,因为该表包含几十行。有没有特别的命令我可以使用相关的postgres?
I believe this would work if it were a dataframe in R rather than a linked table in postgres. However, I need to do the filtering first since the table contains a few billion rows. Is there a special command I can use related to postgres? The current code doesn't work whether I use characters or integers.
推荐答案
使用%in%
过滤器
函数中的/ code>函数不起作用,如果它们只是一个值,而不是具有多个值的向量。
Using %in%
in the filter
function doesn't work if their is only a single value rather than a vector with multiple values.
它作为一个函数,对于具有1个或更多值的案例,它具有 ifelse
语句的功能。
It works as a function with an ifelse
statement for cases with 1 or more values.
retreiveDaymet <- function(catchmentid, num.catch) {
catches <- catchmentid[1:num.catch]
if(num.catch == 1) {
tbl_derived_metrics <- tbl_daymet %>%
dplyr::filter(featureid == catches)
} else {
tbl_derived_metrics <- tbl_daymet %>%
dplyr::filter(featureid %in% catches)
}
derived_metrics <- collect(tbl_derived_metrics)
return(derived_metrics)
}
然后可以这样使用
catchment.numbers <- rep(c(1, 10, 50, 100, 200, 400, 800, 1000, 1500, 2000, 2500, 3000), each = 3)
daymet.times <- data.frame(matrix(NA, length(catchment.numbers), 4))
for(i in 1:length(catchment.numbers)) {
time1 <- system.time(foo <- retreiveDaymet(catchmentid = catchmentid, num.catch = catchment.numbers[i]))
daymet.times[i, ] <- c(catchment.numbers[i], time1[1:3])
rm(foo)
rm(time1)
gc(verbose = FALSE)
}
names(daymet.times) <- c("num.catchments", names(system.time(1+1))[1:3])
这个例子有点傻,因为 foo
每次都被丢弃这只是用于计时的目的。在将来,此代码可以添加一个函数,以便每次使用 foo
执行某些操作,并将其附加到数据框或列表。
This example is a bit silly because foo
is thrown away each time. This is just used for timing purposes. In the future this code could add a function to do something with foo
each time and append it to a dataframe or list.
这篇关于使用dplyr过滤postgreSQL数据库中的多个值的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!