PySpark 数据框的类似 NOT IN 子句的 SQL [英] SQL like NOT IN clause for PySpark data frames

问题描述

例如在SQL中，我们可以做select * from table where col1 not in ('A','B');

In SQL, we can for example, do select * from table where col1 not in ('A','B');

我想知道是否有对应的 PySpark.我能够找到像 IN 子句这样的 SQL 的 isin 函数，但对于 NOT IN 什么也没有.

I was wondering if there is a PySpark equivalent for this. I was able to find the isin function for SQL like IN clause, but nothing for NOT IN.

推荐答案

我刚刚遇到了同样的问题并找到了解决方案.如果要否定任何条件(在 pySpark 中表示为 Column 类)有否定运算符 ~，例如:

I just had the same issue and found solution. If you want to negate any condition (in pySpark represented as Column class) there is negation operator ~, for example:

df.where(~df.flag.isin(1, 2, 3)) # records with flag NOT IN (1, 2, 3)

这篇关于PySpark 数据框的类似 NOT IN 子句的 SQL的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！