Pyspark:如何向上或向下取整(四舍五入到最接近的整数) [英] Pyspark: how to round up or down (round to the nearest)

问题描述

我有一个看起来像这样的 df

I have a df that looks like this

TEST_schema = StructType([StructField("date", StringType(), True),\
                          StructField("col1", FloatType(), True),\
                          ])
TEST_data = [('2020-08-01',1.22),('2020-08-02',1.15),('2020-08-03',5.4),('2020-08-04',2.6),('2020-08-05',3.5),\
             ('2020-08-06',2.2),('2020-08-07',2.7),('2020-08-08',-1.6),('2020-08-09',1.3)]
rdd3 = sc.parallelize(TEST_data)
TEST_df = sqlContext.createDataFrame(TEST_data, TEST_schema)
TEST_df = TEST_df.withColumn("date",to_date("date", 'yyyy-MM-dd'))
TEST_df.show() 

+----------+-----+
|      date|col1 |
+----------+-----+
|2020-08-01| 1.22|
|2020-08-02| 1.15|
|2020-08-03| 5.4 |
|2020-08-04| 2.6 |
|2020-08-05| 3.5 |
|2020-08-06| 2.2 |
|2020-08-07| 2.7 |
|2020-08-08|-1.6 |
|2020-08-09| 1.3 |
+----------+-----+

逻辑:将 col1 舍入到最接近的值，然后返回整数，并返回 max(舍入值，0)

Logic : round col1 to the nearest and return as integer , and max( rounded value , 0)

生成的df如下:

+----------+----+----+
|      date|col1|want|
+----------+----+----+
|2020-08-01| 1.2|   1|
|2020-08-02| 1.1|   1|
|2020-08-03| 5.4|   5|
|2020-08-04| 2.6|   3|
|2020-08-05| 3.5|   4|
|2020-08-06| 2.2|   2|
|2020-08-07| 2.7|   3|
|2020-08-08|-1.6|   0|
|2020-08-09| 1.3|   1|
+----------+----+----+

推荐答案

首先，我在这里检查它是否小于零.在这里我们正在使用 pyspark函数中的when方法时，首先我们检查列中的值 小于零，如果小于零，则取零，否则我们将列中的实际值转换为整数 从pyspark.sql导入功能为F

First, here i am checking whether it's lessthan zero or not. Here we are using when method in pyspark functions, first we check whether the value in the column is lessthan zero, if it is will make it to zero, otherwise we take the actual value in the column then cast to int from pyspark.sql import functions as F

TEST_df.withColumn("want", F.bround(F.when(TEST_df["col1"] < 0, 0).otherwise(TEST_df["col1"])).cast("int")).show()
+----------+----+----+
|      date|col1|want|
+----------+----+----+
|2020-08-01|1.22|   1|
|2020-08-02|1.15|   1|
|2020-08-03| 5.4|   5|
|2020-08-04| 2.6|   3|
|2020-08-05| 3.5|   4|
|2020-08-06| 2.2|   2|
|2020-08-07| 2.7|   3|
|2020-08-08|-1.6|   0|
|2020-08-09| 1.3|   1|
+----------+----+----+

这篇关于Pyspark:如何向上或向下取整(四舍五入到最接近的整数)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！