PySpark:when 子句中有多个条件 [英] PySpark: multiple conditions in when clause
问题描述
我想修改当前为空白的数据框列 (Age) 的单元格值,并且只有在另一列 (Survived) 的相应行的值为 0 时才执行此操作,其中年龄为空白.如果它在 Survived 列中为 1 但在 Age 列中为空白,那么我将其保留为空.
I would like to modify the cell values of a dataframe column (Age) where currently it is blank and I would only do it if another column (Survived) has the value 0 for the corresponding row where it is blank for Age. If it is 1 in the Survived column but blank in Age column then I will keep it as null.
我尝试使用 &&
运算符,但没有用.这是我的代码:
I tried to use &&
operator but it didn't work. Here is my code:
tdata.withColumn("Age", when((tdata.Age == "" && tdata.Survived == "0"), mean_age_0).otherwise(tdata.Age)).show()
任何建议如何处理?谢谢.
Any suggestions how to handle that? Thanks.
错误信息:
SyntaxError: invalid syntax
File "<ipython-input-33-3e691784411c>", line 1
tdata.withColumn("Age", when((tdata.Age == "" && tdata.Survived == "0"), mean_age_0).otherwise(tdata.Age)).show()
^
推荐答案
你得到 SyntaxError
错误异常,因为 Python 没有 &&
操作符.它有 and
和 &
,后者是在 Column
(|
用于逻辑析取,~
用于逻辑否定).
You get SyntaxError
error exception because Python has no &&
operator. It has and
and &
where the latter one is the correct choice to create boolean expressions on Column
(|
for a logical disjunction and ~
for logical negation).
您创建的条件也无效,因为它不考虑运算符优先级.&
在 Python 中比 ==
有更高的优先级,所以表达式必须用括号括起来.
Condition you created is also invalid because it doesn't consider operator precedence. &
in Python has a higher precedence than ==
so expression has to be parenthesized.
(col("Age") == "") & (col("Survived") == "0")
## Column<b'((Age = ) AND (Survived = 0))'>
附注 when
函数等价于 case
表达式而不是 WHEN
子句.仍然适用相同的规则.连词:
On a side note when
function is equivalent to case
expression not WHEN
clause. Still the same rules apply. Conjunction:
df.where((col("foo") > 0) & (col("bar") < 0))
分离:
df.where((col("foo") > 0) | (col("bar") < 0))
您当然可以单独定义条件以避免括号:
You can of course define conditions separately to avoid brackets:
cond1 = col("Age") == ""
cond2 = col("Survived") == "0"
cond1 & cond2
这篇关于PySpark:when 子句中有多个条件的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!