如何在Java中优雅地序列化和反序列化OpenCV YAML校准数据? [英] How to elegantly serialize and deserialize OpenCV YAML calibration data in Java?
问题描述
我正在尝试使用官方OpenCV Java绑定以YAML格式加载/保存OpenCV校准数据.我知道OpenCV(至少为c ++版本)可以序列化为XML和JSON，但我想支持较早的YAML校准文件.
I'm trying to load / save OpenCV calibration data in YAML format using the official OpenCV Java bindings. I am aware OpenCV (c++ version at least) can serialize to XML and JSON but I would like to support older YAML calibration files.
校准文件如下:
%YAML:1.0
cameraMatrix: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!opencv-matrix
rows: 5
cols: 1
dt: d
data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
-3.5244467228016116e-03, -7.0195032848241403e-04,
-2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01
我已经查看了一些答案 和此处，但是我正在寻找一个优雅的解决方案，因为我还不太了解如何将Java类最好地映射到YAML并返回. 我尝试了一些库，例如jyaml，yamlbeans(来自SourceForge的1.0和来自Maven Central的1.13)和SnakeYAML.
I had a look at a few answer already here and here, however I'm looking for an elegant solution as I haven't quite understood to best map java classes to YAML and back. I've tried a few libraries like jyaml, yamlbeans (both 1.0 from SourceForge and 1.13 via Maven Central) and SnakeYAML.
我目前尝试对一些作品进行反序列化，但是感觉很棘手:
My current attempt at deserialising sort of works but feels quite hacky:
CalibrationParseTest.java
CalibrationParseTest.java
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import org.opencv.core.Core;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.Constructor;
public class CalibrationParseTest {
public static void main(String[] args) {
// load OpenCV native
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
String yamlPath = "./data/calibration.yml";
try{
String yamlString = new String(Files.readAllBytes(Paths.get(yamlPath)), StandardCharsets.UTF_8);
// remove %YAML:1.0 to avoid scan directive error
yamlString = yamlString.replaceAll("%YAML:1.0", "");
// map custom class
yamlString = yamlString.replaceAll("opencv-matrix", "MatYAML");
System.out.println("<loaded>");
System.out.println(yamlString);
System.out.println("</loaded>");
Yaml yaml = new Yaml(new Constructor(CalibrationData.class));
CalibrationData data = yaml.load(yamlString);
// currently manually parsing data from the HashMap: can this be better ?
data.populateCV();
// double check data
System.out.println("<deserialized>");
System.out.println(data);
System.out.println("</deserialized>");
}catch (IOException e) {
e.printStackTrace();
}
}
}
CalibrationData.java
CalibrationData.java
import java.util.HashMap;
import org.opencv.core.Mat;
import org.opencv.core.Size;
public class CalibrationData extends HashMap{
public Mat cameraMatrix;
public Size imageSize;
public Size sensorSize;
public Mat distCoeffs;
public float reprojectionError;
public CalibrationData(){}
public void populateCV(){
cameraMatrix = ((MatYAML)get("cameraMatrix")).toMat();
imageSize = new Size((int)get("imageSize_width"),(int)get("imageSize_height"));
sensorSize = new Size((int)get("sensorSize_width"),(int)get("sensorSize_height"));
distCoeffs = ((MatYAML)get("distCoeffs")).toMat();
reprojectionError = (float)((double)get("reprojectionError"));
}
public String toString(){
if(cameraMatrix == null){
return String.format("[CalibrationData (not parsed to CV-> call populateCV()\n\tdata: %s\n]",super.toString());
}
return String.format("[CalibrationData\n" +
"\tcalibrationMatrix: %s\n" +
"\timageSize: %s\n" +
"\tsensorSize: %s\n" +
"\tdistCoeffs: %s\n" +
"\treprojectionError: %f\n]", cameraMatrix.dump(), imageSize.toString(), sensorSize.toString(), distCoeffs.dump(), reprojectionError);
}
}
MatYAML.java
MatYAML.java
import java.util.List;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
public class MatYAML{
public int rows;
public int cols;
public String dt;
public List<Double> data;
Mat toMat(){
Mat out = new Mat(rows, cols, dt.equals("d") ? CvType.CV_64F : CvType.CV_32F);
int index = 0;
for (int row = 0; row < rows; row++) {
for (int col = 0; col < cols; col++) {
out.put(row, col, data.get(index++));
}
}
return out;
}
}
这将输出预期结果:
<loaded>
cameraMatrix: !!MatYAML
rows: 3
cols: 3
dt: d
data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,
6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
distCoeffs: !!MatYAML
rows: 5
cols: 1
dt: d
data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,
-3.5244467228016116e-03, -7.0195032848241403e-04,
-2.0412827999027101e+00 ]
reprojectionError: 2.1723265945911407e-01
</loaded>
<deserialized>
[CalibrationData
calibrationMatrix: [662.7859988712237, 0, 312.4425601600666;
0, 661.2927687519908, 227.4717976712425;
0, 0, 1]
imageSize: 640x480
sensorSize: 0x0
distCoeffs: [-0.1884833834146469; 1.072189041918385; -0.003524446722801612; -0.000701950328482414; -2.04128279990271]
reprojectionError: 0.217233
]
</deserialized>
在没有这些hack的情况下，在Java OpenCV类和YAML之间是否有更优雅的序列化/反序列化方法?
Is there a more elegant way of serializing/deserializing between Java OpenCV classes and YAML without these hacks ?
通过骇客，我的意思是:
By hacks I mean:
- 手动删除yaml版本指令
- 用MatYAML字符串交换opencv-矩阵
- 手动转换HashMap值
- 是否避免手动填充OpenCV
Mat数据? (如果可能?)
- manually removing yaml version directive
- swapping opencv-matrix with MatYAML string
- manually casting HashMap values
- potentially avoiding to manually populate OpenCV
Matdata ? (if possible ?)
更新2
amanin的答案更简洁，可以避免被黑替换"!! opencv-matrix"，但是它不会序列化/反序列化Mat:
Update 2
amanin's answer is cleaner and makes it possible to avoid hackily replacing "!!opencv-matrix", however it doesn't serializing/deserializing Mat:
OpenCVConfig{imageSize_width=640, imageSize_height=480, sensorSize_width=0, sensorSize_height=0, camerMatrix=Matrix{rows=3, cols=3, dt=d, data=[662.7859988712237, 0.0, 312.4425601600666, 0.0, 661.2927687519908, 227.4717976712425, 0.0, 0.0, 1.0]}, distCoeffs=Matrix{rows=5, cols=1, dt=d, data=[-0.1884833834146469, 1.0721890419183855, -0.0035244467228016116, -7.01950328482414E-4, -2.04128279990271]}}
---
imageSize_width: 640
imageSize_height: 480
sensorSize_width: 0
sensorSize_height: 0
reprojectionError: 0.21723265945911407
cameraMatrix:
rows: 3
cols: 3
dt: "d"
data:
- 662.7859988712237
- 0.0
- 312.4425601600666
- 0.0
- 661.2927687519908
- 227.4717976712425
- 0.0
- 0.0
- 1.0
distCoeffs:
rows: 5
cols: 1
dt: "d"
data:
- -0.1884833834146469
- 1.0721890419183855
- -0.0035244467228016116
- -7.01950328482414E-4
- -2.04128279990271
请建议将解决方案与org.opencv.core.Mat
推荐答案
您看过Jackson图书馆吗?它允许将JSON/Yaml内容映射到Java POJO.
Have you looked at Jackson library ? It allows mapping JSON/Yaml content to Java POJOs.
我已经举了一个小例子来解决您的两个问题:
I've made a little example which solves two of your problems:
- 用MatYAML字符串交换opencv-矩阵
- 手动转换HashMap值
但是，对于yaml版本指令，由于它看起来不是有效的Yaml，因此我不确定如何处理它.在我的示例中，我已预先手动将其删除.当然，可以找到更好的解决方案，但我不知道.
However, for yaml version directive, as it looks like it is not valid Yaml, I'm not sure how to handle it. In my example, I've removed it manually before-hand. Surely, a better solution can be found, but I don't know it.
EDIT2 :对于矩阵对象，我制作了一个愚蠢的POJO，供Jackson用来内部读取Brut YAML.然后，我在 OpenCVConfig 类上添加了一个转换层(请参见 @JsonSerialize 和 @JsonDeserialize 注释)，以将该简单的POJO转换为专用的OpenCV矩阵. Jackson提供了多种映射技术(流，自定义转换器/解串器，指导注释等)，因此您可以探索其功能，以找到最适合您需要的解决方案.
EDIT2: For the matrix object, I've made a dumb POJO, used internally by Jackson to read brut YAML. Then, I've added a conversion layer (see @JsonSerialize and @JsonDeserialize anotations on OpenCVConfig class) to convert this simple POJO to specialized OpenCV matrix. Jackson offers various technics (streaming, custom converters/deserializers, guiding annotations, etc.) of mapping, so you can explore its capabilities to find the solution that fits best to you need.
因此，要使该示例正常工作，您将需要两个依赖项(这里以maven格式给出):
So, to make the example work, you'll need two dependencies (given here in maven format):
<!-- (De)serialization engine -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.0</version>
</dependency>
<!-- Yaml support -->
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<version>2.10.0</version>
</dependency>
以下是Maven存储库描述页面:
Here are maven repository description pages:
Jackson-databind :通过maven搜索或 Jackson-dataformat-yaml :通过mvnrepository
注意:我的示例包含许多样板，就像我手动(好吧，IDE)生成的getter/setter一样.到目前为止，您应该通过以下两种方法之一减少代码量:
Note : My example contains lot of boilerplate, as I manually (well, IDE) generated getters/setters. You should by far reduce code amount by either:
- 使用Lombok lib
- 使用Kotlin编码(数据类)
- 使用没有getter/setter的公共属性
- 使用Java 14记录(不确定当前是否可用)
package fr.amanin.stackoverflow;
import java.util.Arrays;
import java.util.stream.Stream;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.type.TypeFactory;
import com.fasterxml.jackson.databind.util.Converter;
import com.fasterxml.jackson.dataformat.yaml.YAMLGenerator;
import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
public class YAMLOpenCV {
/**
* Engine in charge of YAML decoding.
*/
public static final ObjectMapper OPENCV_YAML_MAPPER = new YAMLMapper();
public static void main(String[] args) throws Exception {
nu.pattern.OpenCV.loadShared();
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
final String confStr =
"cameraMatrix: !!opencv-matrix\n" +
" rows: 3\n" +
" cols: 3\n" +
" dt: d\n" +
" data: [ 6.6278599887122368e+02, 0., 3.1244256016006659e+02, 0.,\n" +
" 6.6129276875199082e+02, 2.2747179767124251e+02, 0., 0., 1. ]\n" +
"imageSize_width: 640\n" +
"imageSize_height: 480\n" +
"sensorSize_width: 0\n" +
"sensorSize_height: 0\n" +
"distCoeffs: !!opencv-matrix\n" +
" rows: 5\n" +
" cols: 1\n" +
" dt: d\n" +
" data: [ -1.8848338341464690e-01, 1.0721890419183855e+00,\n" +
" -3.5244467228016116e-03, -7.0195032848241403e-04,\n" +
" -2.0412827999027101e+00 ]\n" +
"reprojectionError: 2.1723265945911407e-01";
OpenCVConfig conf = OPENCV_YAML_MAPPER.readValue(confStr, OpenCVConfig.class);
System.out.println(conf);
String serialized = OPENCV_YAML_MAPPER.writeValueAsString(conf);
System.out.println(serialized);
}
/**
* Java model mirroring YAML configuration. Jackson will fill it
* with values read from YAML configuration file, matching YAML
* fields with this class property names.
*/
public static class OpenCVConfig {
int imageSize_width;
int imageSize_height;
int sensorSize_width;
int sensorSize_height;
double reprojectionError;
/* Special case: Matrix objects are decoded in two passes:
* 1. Jackson will check below converters, and use their input
* type to decode YAML to `Matrix` object (intermediate step).
* 2. Jackson uses converter to delegate to user the mapping
* from this intermediate POJO to specialized target (`Mat` here)
*/
@JsonDeserialize(converter = ToMatConverter.class)
@JsonSerialize(converter = FromMatConverter.class)
Mat cameraMatrix;
@JsonDeserialize(converter = ToMatConverter.class)
@JsonSerialize(converter = FromMatConverter.class)
Mat distCoeffs;
public int getImageSize_width() {
return imageSize_width;
}
public OpenCVConfig setImageSize_width(int imageSize_width) {
this.imageSize_width = imageSize_width;
return this;
}
public int getImageSize_height() {
return imageSize_height;
}
public OpenCVConfig setImageSize_height(int imageSize_height) {
this.imageSize_height = imageSize_height;
return this;
}
public int getSensorSize_width() {
return sensorSize_width;
}
public OpenCVConfig setSensorSize_width(int sensorSize_width) {
this.sensorSize_width = sensorSize_width;
return this;
}
public int getSensorSize_height() {
return sensorSize_height;
}
public OpenCVConfig setSensorSize_height(int sensorSize_height) {
this.sensorSize_height = sensorSize_height;
return this;
}
public double getReprojectionError() {
return reprojectionError;
}
public OpenCVConfig setReprojectionError(double reprojectionError) {
this.reprojectionError = reprojectionError;
return this;
}
public Mat getCameraMatrix() {
return cameraMatrix;
}
public OpenCVConfig setCameraMatrix(Mat cameraMatrix) {
this.cameraMatrix = cameraMatrix;
return this;
}
public Mat getDistCoeffs() {
return distCoeffs;
}
public OpenCVConfig setDistCoeffs(Mat distCoeffs) {
this.distCoeffs = distCoeffs;
return this;
}
@Override
public String toString() {
return "OpenCVConfig{" +
"imageSize_width=" + imageSize_width +
", imageSize_height=" + imageSize_height +
", sensorSize_width=" + sensorSize_width +
", sensorSize_height=" + sensorSize_height +
", camerMatrix=" + cameraMatrix +
", distCoeffs=" + distCoeffs +
'}';
}
}
/**
* Converter used for serialization of Mat objects into YAML.
*/
private static class FromMatConverter implements Converter<Mat, Matrix> {
@Override
public Matrix convert(Mat value) {
final Matrix result = new Matrix();
result.cols = value.cols();
result.rows = value.rows();
final int type = value.type();
result.dt = Stream.of(MatrixDataType.values())
.filter(dt -> dt.mapping == type)
.findAny()
.orElseThrow(() -> new UnsupportedOperationException("No matching datatype found for "+type));
int idx = 0;
result.data = new double[result.rows * result.cols];
for (int r = 0 ; r < result.rows ; r++) {
for (int c = 0; c < result.cols; c++) {
final double[] v = value.get(r, c);
result.data[idx++] = v[0];
}
}
return result;
}
@Override
public JavaType getInputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Mat>() {});
}
@Override
public JavaType getOutputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Matrix>() {});
}
}
/**
* Converter used at read time, to map YAML object to OpenCV Mat.
*/
private static class ToMatConverter implements Converter<Matrix, Mat> {
@Override
public Mat convert(Matrix in) {
final Mat result = new Mat(in.rows, in.cols, in.dt.mapping);
int idx = 0;
for (int r = 0 ; r < in.rows ; r++) {
for (int c = 0; c < in.cols; c++) {
result.put(r, c, in.data[idx++]);
}
}
return result;
}
@Override
public JavaType getInputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Matrix>() {});
}
@Override
public JavaType getOutputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<Mat>() {});
}
}
public static class Matrix {
int rows;
int cols;
MatrixDataType dt;
double[] data;
public int getRows() {
return rows;
}
public Matrix setRows(int rows) {
this.rows = rows;
return this;
}
public int getCols() {
return cols;
}
public Matrix setCols(int cols) {
this.cols = cols;
return this;
}
public MatrixDataType getDt() {
return dt;
}
public Matrix setDt(MatrixDataType dt) {
this.dt = dt;
return this;
}
public double[] getData() {
return data;
}
public Matrix setData(double[] data) {
this.data = data;
return this;
}
double at(int x, int y) {
if (x >= cols || y >= rows) throw new IllegalArgumentException("Bad coordinate");
return data[y*rows + x];
}
@Override
public String toString() {
return "Matrix{" +
"rows=" + rows +
", cols=" + cols +
", dt=" + dt +
", data=" + Arrays.toString(data) +
'}';
}
}
/*
public static class MatDeserializer extends StdDeserializer<Mat> {
protected MatDeserializer() {
super(Mat.class);
}
@Override
public Mat deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
final int rows, cols;
final MatrixDataType dtype;
final double[] data;
}
}
public static class MatSerializer extends StdSerializer<Mat> {
protected MatSerializer() {
super(Mat.class);
}
@Override
public void serialize(Mat value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeNumberField("rows", value.rows());
gen.writeNumberField("cols", value.cols());
gen.writeFieldName("data");
gen.writeStartArray();
gen.writeEndArray();
}
}
*/
public enum MatrixDataType {
d(CvType.CV_64F),
f(CvType.CV_32F);
public final int mapping;
MatrixDataType(int mapping) {
this.mapping = mapping;
}
}
}
希望有帮助，
很抱歉，但是我没有找到可靠的方法:
I'm sorry, but I've not found reliable ways to:
- 选择数组样式.它似乎是 Jackson Github上的一个未解决问题
- 至于YAML标签，我没有发现任何线索.如果您决定坚持下去，也许您需要在Jackson上写一个问题. 在寻找一种编码方式
- Choose array style. It appears to be an open issue on Jackson Github
- As for YAML labels, I've not found any clue about that. Maybe you'll have to fill an issue on the Jackson, if you decide to stick with it. After looking for a way to encode
我已经修改了上面的代码示例以对Mat对象进行序列化/反序列化.但是，由于您无法找到一种通过ByteBuffers传输值的方法，因此像填充/获取矩阵值一样使用循环.我尝试过.但是Java API和文档在这方面都没有帮助.
EDIT 2: I"ve modified above code example to serialize/deserialize Mat objects. However, I've used loops as you did to fill/get matrix values, as I was not able to find a way to transfer values through ByteBuffers (and believe me, I tried. But neither Java API nor documentation are very helpful in this regard).
您可以看到引入的 Matrix 对象仍然存在，因为它简化了IMHO的转换工作.如果愿意，可以使用Jackson StdSerializer对象而不是Converter来摆脱它.但是，我不确定它会更干净.
You can see that the introduced Matrix object is still around, becaus it eases conversion work IMHO. You can, if you want, get rid of it if you use Jackson StdSerializer object instead of Converter. However, I'm not sure that it would be cleaner.
最后一句话:
- 我找到了这另一种方法在搜索数据传输时发布有关OpenCV矩阵的信息.也许对您有用.
- 当心，我的代码仅正确管理64位浮点值.如果需要，您必须从这里开始添加其他CvType案例.
- I've found this other SO post about OpenCV matrix while searching for data transfer. Maybe it will be useful to you.
- Beware, my code only manage properly 64bit floating point values. You'll have to start from here to add other CvType cases if needed.
好吧，这一次可以肯定，我再也不能帮助您.祝你好运；-)
Well, this time it is sure, I cannot help you any further. Good luck ;-)
这篇关于如何在Java中优雅地序列化和反序列化OpenCV YAML校准数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
