如何使用 sparklyr 过滤部分匹配 [英] How to filter on partial match using sparklyr
问题描述
我是 sparklyr 的新手(但熟悉 spark 和 pyspark),我有一个非常基本的问题.我正在尝试根据部分匹配过滤列.在 dplyr 中,我会这样写我的操作:
I'm new to sparklyr (but familiar with spark and pyspark), and I've got a really basic question. I'm trying to filter a column based on a partial match. In dplyr, i'd write my operation as so:
businesses %>%
filter(grepl('test', biz_name)) %>%
head
在火花数据帧上运行该代码但是给了我:
Running that code on a spark dataframe however gives me:
错误:org.apache.spark.sql.AnalysisException:未定义的函数:'GREPL'.该函数既不是已注册的临时函数,也不是在数据库project_eftpos_failure"中注册的永久函数.第 5 行 pos 7
Error: org.apache.spark.sql.AnalysisException: Undefined function: 'GREPL'. This function is neither a registered temporary function nor a permanent function registered in the database 'project_eftpos_failure'.; line 5 pos 7
推荐答案
和标准 Spark 一样,你可以使用 rlike
(Java 正则表达式):
The same as in standard Spark, you can use either rlike
(Java regular expressions):
df <- copy_to(sc, iris)
df %>% filter(rlike(Species, "osa"))
# or anchored
df %>% filter(rlike(Species, "^.*osa.*$"))
或 like
(简单的 SQL 正则表达式):
or like
(simple SQL regular expressions):
df %>% filter(like(Species, "%osa%"))
这两种方法也可以使用后缀表示法
Both methods can be also used with suffix notation as
df %>% filter(Species %rlike% "^.*osa.*$")
和
df %>% filter(Species %like% "%osa%")
分别.
详情见vignette(sql-translation")
.
这篇关于如何使用 sparklyr 过滤部分匹配的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!