numarray速度问题 [英] numarray speed question
问题描述
因此，如果有一个存储在二进制文件中的500万
整数的数组应该非常快，并且确实需要这样做。
a = numarray。 fromfile（''filename''，（2,2,2））
numarray.add（a，9，a）
但是那是怎么回事比将整个文件读入内存更快然后在C中使用for循环获得
：
（循环范围）{
* p ++ + = 9}
或者本质上是什么？
So it is supposed to be very fast to have an array of say 5 million
integers stored in a binary file and do
a = numarray.fromfile(''filename'', (2, 2, 2))
numarray.add(a, 9, a)
but how is that faster than reading the entire file into memory and then
having a for loop in C:
(loop over range) {
*p++ += 9 }
or is that essentially what''s going on?
推荐答案
周三， 2004年8月4日，grv写道：
On Wed, 4 Aug 2004, grv wrote:
因此，如果有一个存储在二进制文件中的500万个整数的数组应该非常快并且
a = numarray.fromfile（''filename''，（2,2,2））
numarray.add（a，9，a）
但是怎么样比将整个文件读入内存更快，然后在C中有一个for循环：
（循环范围）{
* p ++ + = 9}
要么 这基本上是在发生什么？
So it is supposed to be very fast to have an array of say 5 million
integers stored in a binary file and do
a = numarray.fromfile(''filename'', (2, 2, 2))
numarray.add(a, 9, a)
but how is that faster than reading the entire file into memory and then
having a for loop in C:
(loop over range) {
*p++ += 9 }
or is that essentially what''s going on?
这基本上是'正在发生的事情;）numarray的重点不是
超快，但要与等效的C（或Fortran，或者说是实际拥有者）实现一样快。在很多情况下，它的速度更快，因为
numarray在设计时考虑了几种速度，但它没有什么可以做到的（b
$ b） C中没有一点工作。
That''s essentially what''s going on ;) The point of numarray isn''t to be
hyper-fast, but to be as fast as the equivalent C (or Fortran, or
what-have-you) implementation. In many cases, it''s faster, because
numarray is designed with several speed hacks in mind, but it''s nothing
you can''t do (without a little work) in C.
sq ** ****@WPI.EDU （Christopher T King）写道：
< Pi ******************** ******************@ccc6.wpi .edu>：
sq******@WPI.EDU (Christopher T King) wrote in
<Pi**************************************@ccc6.wpi .edu>:
2004年8月4日星期三，grv写道：
On Wed, 4 Aug 2004, grv wrote:
因此，如果将一个500万个整数的数组存储在一个二进制文件中，那么它应该非常快，并且可以使用a = numarray。 fromfile（''filename''，（2,2,2））
numarray.add（a，9，a）
但是如何比将整个文件读入内存更快然后在C中有一个for循环：
（循环范围）{
* p ++ + = 9}
或者基本上是什么在？
So it is supposed to be very fast to have an array of say 5 million
integers stored in a binary file and do
a = numarray.fromfile(''filename'', (2, 2, 2))
numarray.add(a, 9, a)
but how is that faster than reading the entire file into memory and
then having a for loop in C:
(loop over range) {
*p++ += 9 }
or is that essentially what''s going on?
这基本上就是'goi'但是，numarray的重点并不是超快，而是要与等效的C（或Fortran，或者什么是你）实现一样快。在许多情况下，它的速度更快，因为numarray在设计时考虑了几种速度，但在C中你没有什么可以做的（没有一点工作）。
That''s essentially what''s going on ;) The point of numarray isn''t to be
hyper-fast, but to be as fast as the equivalent C (or Fortran, or
what-have-you) implementation. In many cases, it''s faster, because
numarray is designed with several speed hacks in mind, but it''s nothing
you can''t do (without a little work) in C.
是的，但是看到我对实际可以实现的速度黑客感兴趣
改进上面的代码。我只是没有看到任何可以迭代的内容，并且更快地在内存区域上添加
。
Yes but see I''m interested in what speed hacks can actually be done to
improve the above code. I just don''t see anything that can iterate and add
over that memory region faster.
在某些时候， gr **** @ hotmail.com （grv）写道：
At some point, gr****@hotmail.com (grv) wrote:
sq ****** @ WPI.EDU （Christopher T King）在
< Pi *******写道*******************************@ccc6.wpi .edu>：
sq******@WPI.EDU (Christopher T King) wrote in
<Pi**************************************@ccc6.wpi .edu>:
2004年8月4日星期三，grv写道：
On Wed, 4 Aug 2004, grv wrote:
因此，如果有一个存储在二进制文件中的500万个整数的数组应该非常快做
a = numarray.fromfile（''filename''，（2,2,2））
numarray.add（a，9，a）
>但是如何比将整个文件读入内存更快，然后在C中使用for循环：
（循环范围）{
* p ++ + = 9}
或者基本上是什么？
So it is supposed to be very fast to have an array of say 5 million
integers stored in a binary file and do
a = numarray.fromfile(''filename'', (2, 2, 2))
numarray.add(a, 9, a)
but how is that faster than reading the entire file into memory and
then having a for loop in C:
(loop over range) {
*p++ += 9 }
or is that essentially what''s going on?
那是'sss正在发生什么事情;）numarray的重点不是超快，而是要像同等的C（或Fortran，或者什么是你）一样快）实施。在许多情况下，它的速度更快，因为numarray在设计时考虑了几种速度，但在C中你没有什么可以做的（没有一点工作）。
That''s essentially what''s going on ;) The point of numarray isn''t to be
hyper-fast, but to be as fast as the equivalent C (or Fortran, or
what-have-you) implementation. In many cases, it''s faster, because
numarray is designed with several speed hacks in mind, but it''s nothing
you can''t do (without a little work) in C.
是的，但看到我对实际可以通过哪些速度黑客来改进上述代码感兴趣。我只是没有看到任何可以迭代并更快地添加内存区域的内容。
Yes but see I''m interested in what speed hacks can actually be done to
improve the above code. I just don''t see anything that can iterate and add
over that memory region faster.
好吧，numarray可能不会更快case（将标量添加到
a矢量）。事实上，相关的numarray代码如下所示：
static int add_Float64_vector_scalar（long niter，long ninargs，long noutargs，vo
id ** buffers， long * bsizes）{
long i;
Float64 * tin1 =（Float64 *）缓冲区[0];
Float64 tscalar = *（ Float64 *）缓冲区[1];
Float64 * tout =（Float64 *）缓冲区[2];
for（i = 0; i< niter ; i ++，tin1 ++，tout ++）{
* tout = * tin1 + tscalar;
}
返回0;
}
你用* numarray做的是：
1）透明处理byteswapped，misaligned，discontiguous，
类型不匹配的数据（例如，来自在
系统上生成的内存映射文件，其中不同的字节顺序为单精度而不是
双精度）。
2）易用性。上面两行python代码是_it_（除了
用于''import numarray''语句）。你的C代码几乎没有任何地方可以使用.b $ b几乎完全可以使用。你需要添加例程来
读取文件等。
3）交互式使用。您可以在Python命令行中完成所有这些操作。如果
你想要乘以而不是添加，一个向上箭头和一些编辑
就可以做到这一点。使用C，你必须重新编译。
如果你需要尽可能快的速度（在numarray中完成之后和
发现它不是''' （足够快），你可以编写一个扩展模块，以便在C中执行该操作，或者查看scipy.weave以获取内联C代码，或者查看
f2py进行链接Fortran的Python代码。
-
|> | \ / |<
/ ---- -------------------------------------------------- -------------------- \
| David M. Cooke
| cookedm（at）physics（ dot）mcmaster（dot）ca
Well, numarray probably isn''t faster for this case (adding a scalar to
a vector). In fact, the relevant numarray code looks like this:
static int add_Float64_vector_scalar(long niter, long ninargs, long noutargs, vo
id **buffers, long *bsizes) {
long i;
Float64 *tin1 = (Float64 *) buffers[0];
Float64 tscalar = *(Float64 *) buffers[1];
Float64 *tout = (Float64 *) buffers[2];
for (i=0; i<niter; i++, tin1++, tout++) {
*tout = *tin1 + tscalar;
}
return 0;
}
What you *do* get with numarray is:
1) transparent handling of byteswapped, misaligned, discontiguous,
type-mismatched data (say, from a memory-mapped file generated on a
system with a different byte order as single-precision instead of
double-precision).
2) ease-of-use. That two lines of python code above is _it_ (except
for an ''import numarray'' statement). Your C code isn''t anywhere
nearly complete enough to use. You would need to add routines to
read the file, etc.
3) interactive use. You can do all this in the Python command line. If
you want to multiply instead of add, an up-arrow and some editing
will do that. With C, you''d have to recompile.
If you need the best possible speed (after doing it in numarray and
finding it isn''t fast enough), you can write an extension module to
do that bit in C, or look into scipy.weave for inlining C code, or into
f2py for linking Fortran code to Python.
--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
这篇关于numarray速度问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
