GUI或CLI创建镶木地板文件 [英] GUI or CLI to create parquet file

问题描述

我想提供一个与我一起工作的人的工具，该工具可以创建镶木地板文件，以用于读取和处理此类文件的模块的单元测试.

I want to provide the people I work with, a tool to create parquet files to be use for unit-tests of modules that read and process such files.

我使用ParquetViewer查看镶木地板文件的内容，但是我喜欢有一个制作(样本)镶木地板文件的工具.是否有这样的工具可以使用GUI或一些实用的CLI创建镶木地板文件?

I use ParquetViewer to view the content of parquet files, but I like to have a tool to make (sample) parquet files. Is there such a tool to create parquet file with a GUI or some practical CLI otherwise?

注意:我更喜欢跨平台的解决方案，但如果没有，我正在寻找Windows/mingw解决方案以便在工作中使用它-在这里我无法选择OS:\

Note: I would prefer a cross-platform solution, but if not I am looking for a windows/mingw solution in order to use it at work - where I cannot choose the OS :\

推荐答案

parquet-cli 可以从CSV转换为镶木地板.

parquet-cli written in Java can convert from CSV to parquet.

(这是Windows上的示例)

(This is a sample on Windows)

test.csv如下:

test.csv is below:

emp_id,dept_id,name,created_at,updated_at
1,1,"test1","2019-02-17 10:00:00","2019-02-17 12:00:00"
2,2,"test2","2019-02-17 10:00:00","2019-02-17 12:00:00"

在Windows上需要 winutils .下载并设置环境值.

It requires winutils on Windows. Download and set environment value.

$ set HADOOP_HOME=D:\development\hadoop

克隆 parquet-m ，构建所有文件并运行'convert-csv'命令-cli.

Clone parquet-mr, build all and run 'convert-csv' command of parquet-cli.

$ cd parquet-cli
$ java -cp target/classes;target/dependency/* org.apache.parquet.cli.Main convert-csv C:\Users\foo\Downloads\test.csv -o C:\Users\foo\Downloads\test-csv.parquet

'cat'命令显示该实木复合地板文件的内容.

'cat' command shows the content of that parquet file.

$ java -cp target/classes;target/dependency/* org.apache.parquet.cli.Main cat C:\Users\foo\Downloads\test-csv.parquet
{"emp_id": 1, "dept_id": 1, "name": "test1", "created_at": "2019-02-17 10:00:00", "updated_at": "2019-02-17 12:00:00"}
{"emp_id": 2, "dept_id": 2, "name": "test2", "created_at": "2019-02-17 10:00:00", "updated_at": "2019-02-17 12:00:00"}

这篇关于GUI或CLI创建镶木地板文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！