在Golang中快速复制对象的更快方法 [英] Quicker way to deepcopy objects in golang
问题描述
我正在使用go 1.9.我想将对象的值复制到另一个对象中.我尝试用encoding/gob和encoding/json做到这一点.但是gob编码比json编码花费更多的时间.我看到了其他一些问题，例如这，并且它们建议gob编码应该更快.但是我看到了完全相反的行为.有人可以告诉我我做错了什么吗?还是有比这两个更好，更快的方法来进行深度复制?我对象的结构很复杂且嵌套.
测试代码:
package main
import (
"bytes"
"encoding/gob"
"encoding/json"
"log"
"time"
"strconv"
)
// Test ...
type Test struct {
Prop1 int
Prop2 string
}
// Clone deep-copies a to b
func Clone(a, b interface{}) {
buff := new(bytes.Buffer)
enc := gob.NewEncoder(buff)
dec := gob.NewDecoder(buff)
enc.Encode(a)
dec.Decode(b)
}
// DeepCopy deepcopies a to b using json marshaling
func DeepCopy(a, b interface{}) {
byt, _ := json.Marshal(a)
json.Unmarshal(byt, b)
}
func main() {
i := 0
tClone := time.Duration(0)
tCopy := time.Duration(0)
end := 3000
for {
if i == end {
break
}
r := Test{Prop1: i, Prop2: strconv.Itoa(i)}
var rNew Test
t0 := time.Now()
Clone(r, &rNew)
t2 := time.Now().Sub(t0)
tClone += t2
r2 := Test{Prop1: i, Prop2: strconv.Itoa(i)}
var rNew2 Test
t0 = time.Now()
DeepCopy(&r2, &rNew2)
t2 = time.Now().Sub(t0)
tCopy += t2
i++
}
log.Printf("Total items %+v, Clone avg. %+v, DeepCopy avg. %+v, Total Difference %+v\n", i, tClone/3000, tCopy/3000, (tClone - tCopy))
}
我得到以下输出:
Total items 3000, Clone avg. 30.883µs, DeepCopy avg. 6.747µs, Total Difference 72.409084ms
JSON与gob的区别
encoding/gob 包需要传递类型定义:
该实现会为流中的每种数据类型编译一个自定义编解码器，当使用单个Encoder传输值流时，效率最高，分摊了编译成本.
首先"序列化一个类型的值时，还必须包含/传输该类型的定义，以便解码器可以正确地解释和解码流:
一滴小滴是自我描述的.流中的每个数据项都以其类型的规范开头，以一小套预定义类型表示.
这在此处进行了详细说明: Effective Go序列化结构到磁盘
因此，在您的情况下，有必要每次都创建一个新的gob编码器和解码器，但这仍然是瓶颈"，这是使其变慢的部分.从JSON格式编码/从JSON解码，表示中不包含类型描述.
要证明这一点，请进行以下简单更改:
type Test struct {
Prop1 [1000]int
Prop2 [1000]string
}
我们在这里所做的是使字段数组的类型变为值"千倍，而类型信息实际上保持不变(数组中的所有元素都具有相同的类型).像这样创建它们的值:
r := Test{Prop1: [1000]int{}, Prop2: [1000]string{}}
现在运行您的测试程序，我的机器上的输出:
原文:
2017/10/17 14:55:53总项目3000，克隆平均. 33.63µs ，DeepCopy平均. 2.326µs ，总差93.910918ms
修改版本:
2017/10/17 14:56:38总项目3000，克隆平均值. 119.899µs ，DeepCopy平均. 462.608µs ，总差-1.02812648s
如您所见，在原始版本中，JSON更快，但是在修改版本中，gob变得更快，因为传输类型信息的成本被摊销.
测试/基准测试方法
现在继续您的测试方法.这种测量性能的方法不好，并且可能会产生非常不准确的结果.相反，您应该使用Go的内置测试和基准测试工具.有关详细信息，请阅读代码和性能的顺序. /p>
这些克隆的注意事项
这些方法与反射一起使用，因此只能克隆"可通过反射访问的字段，即:导出.同样，它们通常不管理指针相等.我的意思是，如果您在一个结构中有2个指针字段，都指向同一个对象(指针相等)，则在编组和解编组之后，您将获得2个指向2个不同值的不同指针.在某些情况下甚至可能会导致问题.
正确"的克隆方式
考虑到上述注意事项，通常正确的克隆方法需要内部"的帮助.也就是说，通常只有在特定类型(或该类型的包)提供此功能的情况下，才可以克隆该特定类型.
是的，提供手动"克隆功能并不方便，但另一方面，它的性能将优于上述方法(甚至可能高出几个数量级)，并且所需的工作"内存量最少.克隆过程.
I am using go 1.9. And I want to deepcopy value of object into another object. I try to do it with encoding/gob and encoding/json. But it takes more time for gob encoding than json encoding. I see some other questions like this and they suggest that gob encoding should be quicker. But I see exact opposite behaviour. Can someone tell me if I am doing something wrong? Or any better and quicker way to deepcopy than these two? My object's struct is complex and nested.
The test code:
package main
import (
"bytes"
"encoding/gob"
"encoding/json"
"log"
"time"
"strconv"
)
// Test ...
type Test struct {
Prop1 int
Prop2 string
}
// Clone deep-copies a to b
func Clone(a, b interface{}) {
buff := new(bytes.Buffer)
enc := gob.NewEncoder(buff)
dec := gob.NewDecoder(buff)
enc.Encode(a)
dec.Decode(b)
}
// DeepCopy deepcopies a to b using json marshaling
func DeepCopy(a, b interface{}) {
byt, _ := json.Marshal(a)
json.Unmarshal(byt, b)
}
func main() {
i := 0
tClone := time.Duration(0)
tCopy := time.Duration(0)
end := 3000
for {
if i == end {
break
}
r := Test{Prop1: i, Prop2: strconv.Itoa(i)}
var rNew Test
t0 := time.Now()
Clone(r, &rNew)
t2 := time.Now().Sub(t0)
tClone += t2
r2 := Test{Prop1: i, Prop2: strconv.Itoa(i)}
var rNew2 Test
t0 = time.Now()
DeepCopy(&r2, &rNew2)
t2 = time.Now().Sub(t0)
tCopy += t2
i++
}
log.Printf("Total items %+v, Clone avg. %+v, DeepCopy avg. %+v, Total Difference %+v\n", i, tClone/3000, tCopy/3000, (tClone - tCopy))
}
I get following output:
Total items 3000, Clone avg. 30.883µs, DeepCopy avg. 6.747µs, Total Difference 72.409084ms
JSON vs gob difference
The encoding/gob package needs to transmit type definitions:
The implementation compiles a custom codec for each data type in the stream and is most efficient when a single Encoder is used to transmit a stream of values, amortizing the cost of compilation.
When you "first" serialize a value of a type, the definition of the type also has to be included / transmitted, so the decoder can properly interpret and decode the stream:
A stream of gobs is self-describing. Each data item in the stream is preceded by a specification of its type, expressed in terms of a small set of predefined types.
This is explained in great details here: Efficient Go serialization of struct to disk
So while in your case it's necessary to create a new gob encoder and decoder each time, it is still the "bottleneck", the part that makes it slow. Encoding to / decoding from JSON format, type description is not included in the representation.
To prove it, make this simple change:
type Test struct {
Prop1 [1000]int
Prop2 [1000]string
}
What we did here is made the types of fields arrays, "multiplying" the values a thousand times, while the type information is effectively remained the same (all elements in the arrays have the same type). Creating values of them like this:
r := Test{Prop1: [1000]int{}, Prop2: [1000]string{}}
Now running your test program, the output on my machine:
Original:
2017/10/17 14:55:53 Total items 3000, Clone avg. 33.63µs, DeepCopy avg. 2.326µs, Total Difference 93.910918ms
Modified version:
2017/10/17 14:56:38 Total items 3000, Clone avg. 119.899µs, DeepCopy avg. 462.608µs, Total Difference -1.02812648s
As you can see, in the original version JSON is faster, but in the modified version gob became faster, as the cost of transmitting type info amortized.
Testing / benching method
Now on to your testing method. This way of measuring performance is bad and can yield quite inaccurate results. Instead you should use Go's built-in testing and benchmark tools. For details, read Order of the code and performance.
Caveats of these cloning
These methods work with reflection and thus can only "clone" fields that are accessible via reflection, that is: exported. Also they often don't manage pointer equality. By this I mean if you have 2 pointer fields in a struct, both pointing to the same object (pointers being equal), after marshaling and unmarshaling, you'll get 2 different pointers pointing to 2 different values. This may even cause problems in certain situations.
The "proper" way of cloning
Considering the caveats mentioned above, often the proper way of cloning needs help from the "inside". That is, cloning a specific type is often only possible if that type (or the package of that type) provides this functionality.
Yes, providing a "manual" cloning functionality is not convenient, but on the other side it will outperform the above methods (maybe even by orders of magnitude), and will require the least amount of "working" memory required for the cloning process.
这篇关于在Golang中快速复制对象的更快方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
