如何将 Spark Row 的数据集转换为字符串? [英] How to convert the datasets of Spark Row into string?
问题描述
我已经编写了使用 SparkSQL 访问 Hive 表的代码.代码如下:
I have written the code to access the Hive table using SparkSQL. Here is the code:
SparkSession spark = SparkSession
.builder()
.appName("Java Spark Hive Example")
.master("local[*]")
.config("hive.metastore.uris", "thrift://localhost:9083")
.enableHiveSupport()
.getOrCreate();
Dataset<Row> df = spark.sql("select survey_response_value from health").toDF();
df.show();
我想知道如何将完整的输出转换为字符串或字符串数组?当我尝试使用另一个模块时,只有我可以传递字符串或字符串类型的数组值.
我尝试过其他方法,例如 .toString
或类型转换为 String 值.但对我不起作用.
请让我知道如何将数据集值转换为字符串?
I would like to know how I can convert the complete output to String or String array? As I am trying to work with another module where only I can pass String or String type Array values.
I have tried other methods like .toString
or typecast to String values. But did not worked for me.
Kindly let me know how I can convert the DataSet values to String?
推荐答案
这里是 Java 中的示例代码.
Here is the sample code in Java.
public class SparkSample {
public static void main(String[] args) {
SparkSession spark = SparkSession
.builder()
.appName("SparkSample")
.master("local[*]")
.getOrCreate();
//create df
List<String> myList = Arrays.asList("one", "two", "three", "four", "five");
Dataset<Row> df = spark.createDataset(myList, Encoders.STRING()).toDF();
df.show();
//using df.as
List<String> listOne = df.as(Encoders.STRING()).collectAsList();
System.out.println(listOne);
//using df.map
List<String> listTwo = df.map(row -> row.mkString(), Encoders.STRING()).collectAsList();
System.out.println(listTwo);
}
}
"row" 是 java 8 lambda 参数.请查看 developer.com/java/start-using-java-lambda-expressions.html
"row" is java 8 lambda parameter. Please check developer.com/java/start-using-java-lambda-expressions.html
这篇关于如何将 Spark Row 的数据集转换为字符串?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!