如何在 Java 中优雅地序列化和反序列化 OpenCV YAML 校准数据? [英] How to elegantly serialize and deserialize OpenCV YAML calibration data in Java?
问题描述
我正在尝试使用官方 OpenCV Java 绑定以 YAML 格式加载/保存 OpenCV 校准数据.我知道 OpenCV(至少是 c++ 版本)可以序列化为 XML 和 JSON，但我想支持旧的 YAML 校准文件.
I'm trying to load / save OpenCV calibration data in YAML format using the official OpenCV Java bindings. I am aware OpenCV (c++ version at least) can serialize to XML and JSON but I would like to support older YAML calibration files.
校准文件如下所示:
%YAML:1.0
cameraMatrix: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!opencv-matrix
rows: 5
cols: 1
dt: d
data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
-3.5244467228016116e-03, -7.0195032848241403e-04,
-2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01
我已经看过一些答案 这里和这里，但是我正在寻找一个优雅的解决方案，因为我不太了解如何将 Java 类最好地映射到 YAML 并返回.我尝试了一些库，例如 jyaml、yamlbeans(来自 SourceForge 的 1.0 和来自 Maven Central 的 1.13)和 SnakeYAML.
I had a look at a few answer already here and here, however I'm looking for an elegant solution as I haven't quite understood to best map java classes to YAML and back. I've tried a few libraries like jyaml, yamlbeans (both 1.0 from SourceForge and 1.13 via Maven Central) and SnakeYAML.
我目前尝试反序列化某种作品，但感觉很hacky:
My current attempt at deserialising sort of works but feels quite hacky:
CalibrationParseTest.java
CalibrationParseTest.java
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import org.opencv.core.Core;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.Constructor;
public class CalibrationParseTest {
public static void main(String[] args) {
// load OpenCV native
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
String yamlPath = "./data/calibration.yml";
try{
String yamlString = new String(Files.readAllBytes(Paths.get(yamlPath)), StandardCharsets.UTF_8);
// remove %YAML:1.0 to avoid scan directive error
yamlString = yamlString.replaceAll("%YAML:1.0", "");
// map custom class
yamlString = yamlString.replaceAll("opencv-matrix", "MatYAML");
System.out.println("<loaded>");
System.out.println(yamlString);
System.out.println("</loaded>");
Yaml yaml = new Yaml(new Constructor(CalibrationData.class));
CalibrationData data = yaml.load(yamlString);
// currently manually parsing data from the HashMap: can this be better ?
data.populateCV();
// double check data
System.out.println("<deserialized>");
System.out.println(data);
System.out.println("</deserialized>");
}catch (IOException e) {
e.printStackTrace();
}
}
}
CalibrationData.java
CalibrationData.java
import java.util.HashMap;
import org.opencv.core.Mat;
import org.opencv.core.Size;
public class CalibrationData extends HashMap{
public Mat cameraMatrix;
public Size imageSize;
public Size sensorSize;
public Mat distCoeffs;
public float reprojectionError;
public CalibrationData(){}
public void populateCV(){
cameraMatrix = ((MatYAML)get("cameraMatrix")).toMat();
imageSize = new Size((int)get("imageSize_width"),(int)get("imageSize_height"));
sensorSize = new Size((int)get("sensorSize_width"),(int)get("sensorSize_height"));
distCoeffs = ((MatYAML)get("distCoeffs")).toMat();
reprojectionError = (float)((double)get("reprojectionError"));
}
public String toString(){
if(cameraMatrix == null){
return String.format("[CalibrationData (not parsed to CV-> call populateCV()
data: %s
]",super.toString());
}
return String.format("[CalibrationData
" +
" calibrationMatrix: %s
" +
" imageSize: %s
" +
" sensorSize: %s
" +
" distCoeffs: %s
" +
" reprojectionError: %f
]", cameraMatrix.dump(), imageSize.toString(), sensorSize.toString(), distCoeffs.dump(), reprojectionError);
}
}
MatYAML.java
MatYAML.java
import java.util.List;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
public class MatYAML{
public int rows;
public int cols;
public String dt;
public List<Double> data;
Mat toMat(){
Mat out = new Mat(rows, cols, dt.equals("d") ? CvType.CV_64F : CvType.CV_32F);
int index = 0;
for (int row = 0; row < rows; row++) {
for (int col = 0; col < cols; col++) {
out.put(row, col, data.get(index++));
}
}
return out;
}
}
这会输出预期的结果:
<loaded>
cameraMatrix: !!MatYAML
rows: 3
cols: 3
dt: d
data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!MatYAML
rows: 5
cols: 1
dt: d
data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
-3.5244467228016116e-03, -7.0195032848241403e-04,
-2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01
</loaded>
<deserialized>
[CalibrationData
calibrationMatrix: [662.7859988712237, 0, 312.4425601600666;
0, 661.2927687519908, 227.4717976712425;
0, 0, 1]
imageSize: 640x480
sensorSize: 0x0
distCoeffs: [-0.1884833834146469; 1.072189041918385; -0.003524446722801612; -0.000701950328482414; -2.04128279990271]
reprojectionError: 0.217233
]
</deserialized>
有没有更优雅的方式在 Java OpenCV 类和 YAML 之间进行序列化/反序列化而不需要这些 hack?
Is there a more elegant way of serializing/deserializing between Java OpenCV classes and YAML without these hacks ?
我的意思是黑客:
- 手动删除 yaml 版本指令
- 用 MatYAML 字符串交换 opencv-matrix
- 手动转换 HashMap 值
- 可能避免手动填充 OpenCV
Mat数据?(如果可能的话?)
- manually removing yaml version directive
- swapping opencv-matrix with MatYAML string
- manually casting HashMap values
- potentially avoiding to manually populate OpenCV
Matdata ? (if possible ?)
更新 2amanin 的答案更简洁，可以避免随意替换!!opencv-matrix"，但它不会序列化/反序列化 Mat:
Update 2
amanin's answer is cleaner and makes it possible to avoid hackily replacing "!!opencv-matrix", however it doesn't serializing/deserializing Mat:
OpenCVConfig{imageSize_width=640, imageSize_height=480, sensorSize_width=0, sensorSize_height=0, camerMatrix=Matrix{rows=3, cols=3, dt=d, data=[662.7859988712237, 0.0, 312.4425601600666, 0.0, 661.2927687519908, 227.4717976712425, 0.0, 0.0, 1.0]}, distCoeffs=Matrix{rows=5, cols=1, dt=d, data=[-0.1884833834146469, 1.0721890419183855, -0.0035244467228016116, -7.01950328482414E-4, -2.04128279990271]}}
---
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
reprojectionError: 0.21723265945911407
cameraMatrix:
rows: 3
cols: 3
dt: "d"
data:
- 662.7859988712237
- 0.0
- 312.4425601600666
- 0.0
- 661.2927687519908
- 227.4717976712425
- 0.0
- 0.0
- 1.0
distCoeffs:
rows: 5
cols: 1
dt: "d"
data:
- -0.1884833834146469
- 1.0721890419183855
- -0.0035244467228016116
- -7.01950328482414E-4
- -2.04128279990271
请就解决方案与 org.opencv.core.Mat
推荐答案
你看过 Jackson 库吗?它允许将 JSON/Yaml 内容映射到 Java POJO.
Have you looked at Jackson library ? It allows mapping JSON/Yaml content to Java POJOs.
我做了一个小例子来解决你的两个问题:
I've made a little example which solves two of your problems:
- 用 MatYAML 字符串交换 opencv-matrix
- 手动转换 HashMap 值
但是，对于 yaml 版本指令，因为它看起来不是有效的 Yaml，我不知道如何处理它.在我的示例中，我事先手动将其删除.当然，可以找到更好的解决方案，但我不知道.
However, for yaml version directive, as it looks like it is not valid Yaml, I'm not sure how to handle it. In my example, I've removed it manually before-hand. Surely, a better solution can be found, but I don't know it.
EDIT2:对于矩阵对象，我制作了一个愚蠢的 POJO，Jackson 在内部使用它来读取原始的 YAML.然后，我添加了一个转换层(请参阅 @JsonSerialize 和 @JsonDeserialize 注释上的 OpenCVConfig 类)将这个简单的 POJO 转换为专门的 OpenCV矩阵.Jackson 提供了各种映射技术(流、自定义转换器/反序列化器、引导注释等)，因此您可以探索其功能以找到最适合您需要的解决方案.
EDIT2: For the matrix object, I've made a dumb POJO, used internally by Jackson to read brut YAML. Then, I've added a conversion layer (see @JsonSerialize and @JsonDeserialize anotations on OpenCVConfig class) to convert this simple POJO to specialized OpenCV matrix. Jackson offers various technics (streaming, custom converters/deserializers, guiding annotations, etc.) of mapping, so you can explore its capabilities to find the solution that fits best to you need.
因此，要使示例正常运行，您需要两个依赖项(此处以 maven 格式给出):
So, to make the example work, you'll need two dependencies (given here in maven format):
<!-- (De)serialization engine -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.0</version>
</dependency>
<!-- Yaml support -->
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<version>2.10.0</version>
</dependency>
这里是 Maven 存储库描述页面:
Here are maven repository description pages:
Jackson-databind:通过 maven 搜索 或 通过 mvnrepository
Jackson-dataformat-yaml:通过 maven 搜索 或 通过 mvnrepository
注意:我的示例包含很多样板，因为我手动(嗯，IDE)生成了 getter/setter.到目前为止，您应该通过以下任一方式减少代码量:
Note : My example contains lot of boilerplate, as I manually (well, IDE) generated getters/setters. You should by far reduce code amount by either:
- 使用 Lombok 库
- 使用 Kotlin 编码(数据类)
- 使用没有 getter/setter 的公共属性
- 使用 java 14 记录(目前不确定是否有效)
package fr.amanin.stackoverflow;
import java.util.Arrays;
import java.util.stream.Stream;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.type.TypeFactory;
import com.fasterxml.jackson.databind.util.Converter;
import com.fasterxml.jackson.dataformat.yaml.YAMLGenerator;
import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
public class YAMLOpenCV {
/**
* Engine in charge of YAML decoding.
*/
public static final ObjectMapper OPENCV_YAML_MAPPER = new YAMLMapper();
public static void main(String[] args) throws Exception {
nu.pattern.OpenCV.loadShared();
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
final String confStr =
"cameraMatrix: !!opencv-matrix
" +
" rows: 3
" +
" cols: 3
" +
" dt: d
" +
" data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
" +
" 6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
" +
"imageSize_width: 640
" +
"imageSize_height: 480
" +
"sensorSize_width: 0
" +
"sensorSize_height: 0
" +
"distCoeffs: !!opencv-matrix
" +
" rows: 5
" +
" cols: 1
" +
" dt: d
" +
" data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
" +
" -3.5244467228016116e-03, -7.0195032848241403e-04,
" +
" -2.0412827999027101e+00 ]
" +
"reprojectionError: 2.1723265945911407e-01";
OpenCVConfig conf = OPENCV_YAML_MAPPER.readValue(confStr, OpenCVConfig.class);
System.out.println(conf);
String serialized = OPENCV_YAML_MAPPER.writeValueAsString(conf);
System.out.println(serialized);
}
/**
* Java model mirroring YAML configuration. Jackson will fill it
* with values read from YAML configuration file, matching YAML
* fields with this class property names.
*/
public static class OpenCVConfig {
int imageSize_width;
int imageSize_height;
int sensorSize_width;
int sensorSize_height;
double reprojectionError;
/* Special case: Matrix objects are decoded in two passes:
* 1. Jackson will check below converters, and use their input
* type to decode YAML to `Matrix` object (intermediate step).
* 2. Jackson uses converter to delegate to user the mapping
* from this intermediate POJO to specialized target (`Mat` here)
*/
@JsonDeserialize(converter = ToMatConverter.class)
@JsonSerialize(converter = FromMatConverter.class)
Mat cameraMatrix;
@JsonDeserialize(converter = ToMatConverter.class)
@JsonSerialize(converter = FromMatConverter.class)
Mat distCoeffs;
public int getImageSize_width() {
return imageSize_width;
}
public OpenCVConfig setImageSize_width(int imageSize_width) {
this.imageSize_width = imageSize_width;
return this;
}
public int getImageSize_height() {
return imageSize_height;
}
public OpenCVConfig setImageSize_height(int imageSize_height) {
this.imageSize_height = imageSize_height;
return this;
}
public int getSensorSize_width() {
return sensorSize_width;
}
public OpenCVConfig setSensorSize_width(int sensorSize_width) {
this.sensorSize_width = sensorSize_width;
return this;
}
public int getSensorSize_height() {
return sensorSize_height;
}
public OpenCVConfig setSensorSize_height(int sensorSize_height) {
this.sensorSize_height = sensorSize_height;
return this;
}
public double getReprojectionError() {
return reprojectionError;
}
public OpenCVConfig setReprojectionError(double reprojectionError) {
this.reprojectionError = reprojectionError;
return this;
}
public Mat getCameraMatrix() {
return cameraMatrix;
}
public OpenCVConfig setCameraMatrix(Mat cameraMatrix) {
this.cameraMatrix = cameraMatrix;
return this;
}
public Mat getDistCoeffs() {
return distCoeffs;
}
public OpenCVConfig setDistCoeffs(Mat distCoeffs) {
this.distCoeffs = distCoeffs;
return this;
}
@Override
public String toString() {
return "OpenCVConfig{" +
"imageSize_width=" + imageSize_width +
", imageSize_height=" + imageSize_height +
", sensorSize_width=" + sensorSize_width +
", sensorSize_height=" + sensorSize_height +
", camerMatrix=" + cameraMatrix +
", distCoeffs=" + distCoeffs +
'}';
}
}
/**
* Converter used for serialization of Mat objects into YAML.
*/
private static class FromMatConverter implements Converter<Mat, Matrix> {
@Override
public Matrix convert(Mat value) {
final Matrix result = new Matrix();
result.cols = value.cols();
result.rows = value.rows();
final int type = value.type();
result.dt = Stream.of(MatrixDataType.values())
.filter(dt -> dt.mapping == type)
.findAny()
.orElseThrow(() -> new UnsupportedOperationException("No matching datatype found for "+type));
int idx = 0;
result.data = new double[result.rows * result.cols];
for (int r = 0 ; r < result.rows ; r++) {
for (int c = 0; c < result.cols; c++) {
final double[] v = value.get(r, c);
result.data[idx++] = v[0];
}
}
return result;
}
@Override
public JavaType getInputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Mat>() {});
}
@Override
public JavaType getOutputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Matrix>() {});
}
}
/**
* Converter used at read time, to map YAML object to OpenCV Mat.
*/
private static class ToMatConverter implements Converter<Matrix, Mat> {
@Override
public Mat convert(Matrix in) {
final Mat result = new Mat(in.rows, in.cols, in.dt.mapping);
int idx = 0;
for (int r = 0 ; r < in.rows ; r++) {
for (int c = 0; c < in.cols; c++) {
result.put(r, c, in.data[idx++]);
}
}
return result;
}
@Override
public JavaType getInputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Matrix>() {});
}
@Override
public JavaType getOutputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Mat>() {});
}
}
public static class Matrix {
int rows;
int cols;
MatrixDataType dt;
double[] data;
public int getRows() {
return rows;
}
public Matrix setRows(int rows) {
this.rows = rows;
return this;
}
public int getCols() {
return cols;
}
public Matrix setCols(int cols) {
this.cols = cols;
return this;
}
public MatrixDataType getDt() {
return dt;
}
public Matrix setDt(MatrixDataType dt) {
this.dt = dt;
return this;
}
public double[] getData() {
return data;
}
public Matrix setData(double[] data) {
this.data = data;
return this;
}
double at(int x, int y) {
if (x >= cols || y >= rows) throw new IllegalArgumentException("Bad coordinate");
return data[y*rows + x];
}
@Override
public String toString() {
return "Matrix{" +
"rows=" + rows +
", cols=" + cols +
", dt=" + dt +
", data=" + Arrays.toString(data) +
'}';
}
}
/*
public static class MatDeserializer extends StdDeserializer<Mat> {
protected MatDeserializer() {
super(Mat.class);
}
@Override
public Mat deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
final int rows, cols;
final MatrixDataType dtype;
final double[] data;
}
}
public static class MatSerializer extends StdSerializer<Mat> {
protected MatSerializer() {
super(Mat.class);
}
@Override
public void serialize(Mat value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeNumberField("rows", value.rows());
gen.writeNumberField("cols", value.cols());
gen.writeFieldName("data");
gen.writeStartArray();
gen.writeEndArray();
}
}
*/
public enum MatrixDataType {
d(CvType.CV_64F),
f(CvType.CV_32F);
public final int mapping;
MatrixDataType(int mapping) {
this.mapping = mapping;
}
}
}
希望对你有帮助，
很抱歉，我还没有找到可靠的方法:
I'm sorry, but I've not found reliable ways to:
- 选择数组样式.这似乎是 杰克逊 Github 上的一个未解决问题
- 关于 YAML 标签，我没有找到任何线索.如果您决定坚持下去，也许您将不得不填写有关杰克逊的问题.在寻找一种编码方式之后
编辑 2:我已经修改了上面的代码示例来序列化/反序列化 Mat 对象.但是，我使用循环来填充/获取矩阵值，因为我无法找到通过 ByteBuffers 传输值的方法(相信我，我试过了.但在这方面，Java API 和文档都不是很有帮助).
EDIT 2: I"ve modified above code example to serialize/deserialize Mat objects. However, I've used loops as you did to fill/get matrix values, as I was not able to find a way to transfer values through ByteBuffers (and believe me, I tried. But neither Java API nor documentation are very helpful in this regard).
您可以看到引入的 Matrix 对象仍然存在，因为它简化了转换工作恕我直言.如果你愿意的话，如果你使用 Jackson StdSerializer 对象而不是 Converter，你可以摆脱它.但是，我不确定它是否会更干净.
You can see that the introduced Matrix object is still around, becaus it eases conversion work IMHO. You can, if you want, get rid of it if you use Jackson StdSerializer object instead of Converter. However, I'm not sure that it would be cleaner.
遗言:
- 我找到了 其他 SO在搜索数据传输时发布有关 OpenCV 矩阵 的信息.也许对你有用.
- 请注意，我的代码只能正确管理 64 位浮点值.如果需要，您必须从这里开始添加其他 CvType 案例.
- I've found this other SO post about OpenCV matrix while searching for data transfer. Maybe it will be useful to you.
- Beware, my code only manage properly 64bit floating point values. You'll have to start from here to add other CvType cases if needed.
嗯，这一次可以肯定，我不能再帮你了.祝你好运;-)
Well, this time it is sure, I cannot help you any further. Good luck ;-)
这篇关于如何在 Java 中优雅地序列化和反序列化 OpenCV YAML 校准数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
