使用cython序列化一组整数 [英] Serialize a group of integers using cython

问题描述

我从 Pyrobuf 页的序列化速度比通过 struct.pack 快3倍：

I saw this sample code from the Pyrobuf page for serializing an integer ~3 times faster than via struct.pack:

def ser2(): 
    cdef int x = 42 
    return (<char *>&x)[:sizeof(int)]

我想知道如何才能处理一组整数。
我看到cython具有 int [：] 和 array.array 类型，但是我仍然没有了解如何使用整数列表，并通过 struct.pack（'i'，* num_list）获得相同（但更快）的结果。 map（）似乎对我来说工作不快，我想知道应该怎么做。

I was wondering how this could be done for a group of integers. I saw cython has int[:] and array.array types, but I still don't understand how do I take a list of integers for example, and get the same (but faster) result as via struct.pack('i', *num_list). map() didn't seem work faster for me, and I'm wondering how this should be done.

推荐答案

我假设您想加快以下操作（Python3）：

I assume you want to speed up the following (Python3):

import struct 
lst=list(range(100))  #any other size
struct.pack('i'*len(lst), *lst)

没有 struct 和cython，您可以在python中实现以下目标：

without struct and cython you could achieve it as following in python:

import array
bytes(array.array('i', lst))

但这比 struct -module慢一些：

this is however somewhat slower than the struct-module:

>>> %timeit struct.pack('i'*len(lst), *lst)
2.38 µs ± 9.48 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
>>> %timeit bytes(array.array('i',lst))
3.94 µs ± 92 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

但是，cython可用于加快创建数组的速度，文档请参见此处（数组）和此处（str / bytes）：

However, cython can be used to speed up the creation of the array, for the documentation see here(arrays) and here(str/bytes):

 %%cython
 import array
 from cpython cimport array
 def ser_int_list(lst):
    cdef Py_ssize_t n=len(lst)
    cdef array.array res=array.array('i')
    array.resize(res, n)  #preallocate memory
    for i in range(n):
       res.data.as_ints[i]=lst[i]            #lst.__get__() needs Python-Integer, so let i be a python-integer (not cdef)
    return res.data.as_chars[:n*sizeof(int)] # str for python2, bytes for python3

时间显示以下性能：

#list_size    struct-code    cython-code    speed-up
     1           343 ns         238 ns        1.5
    10           619 ns         283 ns         2
   100          2.38 µs        2.38 µs        3.5
  1000          21.6 µs        5.11 µs         4
 10000           266 µs        47.5 µs        5.5 

ie cython可以提高速度，从小型列表的 1.5 到大型列表的 5.5 。

i.e. cython provides some speed-up, from 1.5 for small list up to 5.5 for large lists.

也许可以进一步调整，但我希望您能理解。

Probably this could be tweaked even further, but I hope you get the idea.

测试代码：

import struct
for n in [1, 10,10**2, 10**3, 10**4]:
    print ("N=",n)
    lst=list(range(n))
    print("struct:")
    %timeit struct.pack('i'*len(lst), *lst)
    print("cython:")
    %timeit ser_int_list(lst)

这篇关于使用cython序列化一组整数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！