spark数据帧中过滤器的多种条件 [英] multiple conditions for filter in spark data frames

问题描述

我有一个包含四个字段的数据框.字段名称之一是状态，我试图在 .filter 中使用 OR 条件作为数据帧.我尝试了以下查询，但没有成功.

I have a data frame with four fields. one of the field name is Status and i am trying to use a OR condition in .filter for a dataframe . I tried below queries but no luck.

df2 = df1.filter(("Status=2") || ("Status =3"))

df2 = df1.filter("Status=2" || "Status =3")

有没有人用过这个.我在这里看到了一个关于堆栈溢出的类似问题.他们使用以下代码来使用 OR 条件.但该代码适用于 pyspark.

Has anyone used this before. I have seen a similar question on stack overflow here . They have used below code for using OR condition. But that code is for pyspark.

from pyspark.sql.functions import col 

numeric_filtered = df.where(
(col('LOW')    != 'null') | 
(col('NORMAL') != 'null') |
(col('HIGH')   != 'null'))
numeric_filtered.show()

推荐答案

代替:

df2 = df1.filter("Status=2" || "Status =3")

试试:

df2 = df1.filter($"Status" === 2 || $"Status" === 3)

这篇关于spark数据帧中过滤器的多种条件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！