查看 numpy 数组? [英] View onto a numpy array?

问题描述

我有一个二维 numpy 数组.有没有办法在其上创建一个包含前 k 行和所有列的视图?

I have a 2D numpy array. Is there a way to create a view onto it that would include the first k rows and all columns?

关键是要避免复制底层数据(数组太大以至于无法进行部分复制.)

The point is to avoid copying the underlying data (the array is so large that making partial copies is not feasible.)

推荐答案

当然，只需像往常一样索引它.例如.y = x[:k, :] 这将返回原始数组的视图.不会复制任何数据，对 y 所做的任何更新都将反映在 x 中，反之亦然.

Sure, just index it as you normally would. E.g. y = x[:k, :] This will return a view into the original array. No data will be copied, and any updates made to y will be reflected in x and vice versa.

我通常使用 uint8 的 >10GB 3D 数组，所以我很担心这个……如果你记住一些事情，Numpy 在内存管理方面可以非常有效.以下是避免在内存中复制数组的一些技巧:

I commonly work with >10GB 3D arrays of uint8's, so I worry about this a lot... Numpy can be very efficient at memory management if you keep a few things in mind. Here are a few tips on avoiding making copies of arrays in memory:

使用 +=、-=、*= 等来避免复制数组.例如.x += 10 将就地修改数组，而 x = x + 10 将复制并修改它.(另外，看看numexpr)

Use +=, -=, *=, etc to avoid making a copy of the array. E.g. x += 10 will modify the array in place, while x = x + 10 will make a copy and modify it. (also, have a look at numexpr)

如果你确实想用 x = x + 10 复制，请注意 x = x + 10.0 将导致 x自动向上转换为浮点数组，如果它还没有的话.但是，x += 10.0，其中 x 是一个整数数组，将导致 10.0 向下转换为相同的 int精度作为数组，而不是.

If you do want to make a copy with x = x + 10, be aware that x = x + 10.0 will cause x to automatically be up-casted to a floating point array, if it wasn't already. However, x += 10.0, where x is an integer array, will cause the 10.0 to be down-casted to an int of the same precision as the array, instead.

此外，许多 numpy 函数都带有 out 参数，因此您可以执行诸如 np.abs(x, x) 之类的操作来获取 的绝对值x 就地.

Additionally, many numpy functions take an out parameter, so you can do things like np.abs(x, x) to take the absolute value of x in-place.

作为第二次编辑，这里有一些关于使用 numpy 数组的视图与副本的更多提示:

As a second edit, here's few more tips on views vs. copies with numpy arrays:

与 python 列表不同，y = x[:] 不返回副本，它返回一个视图.如果您确实想要一个副本(当然，这将使您使用的内存量增加一倍)使用 y = x.copy()

Unlike python lists, y = x[:] does not return a copy, it returns a view. If you do want a copy (which will, of course, double the amount of memory you're using) use y = x.copy()

您会经常听到 numpy 数组的花式索引".使用列表(或整数数组)作为索引是花式索引".它可能非常有用，但会复制数据.

You'll often hear about "fancy indexing" of numpy arrays. Using a list (or integer array) as an index is "fancy indexing". It can be very useful, but copies the data.

举个例子:y = x[[0, 1, 2], :] 返回一个副本，而 y = x[:3,:]> 会返回一个视图.

As an example of this: y = x[[0, 1, 2], :] returns a copy, while y = x[:3,:] would return a view.

即使像 x[4:100:5, :-10:-1, None] 这样非常疯狂的索引也是正常"索引并且会返回一个视图，所以不要害怕在大型数组上使用各种切片技巧.

Even really crazy indexing like x[4:100:5, :-10:-1, None] is "normal" indexing and will return a view, though, so don't be afraid to use all kinds of slicing tricks on large arrays.

x.astype() 将返回数据的副本作为新类型，而 x.view() 将返回一个看法.

x.astype(<dtype>) will return a copy of the data as the new type, whilex.view(<dtype>) will return a view.

但是要小心...它非常强大和有用，但您需要了解底层数据如何存储在内存中.如果您有一个浮点数组，并将它们视为整数，(反之亦然)numpy 会将数组的底层 bits 解释为整数.

Be careful with this, however... It's extremely powerful and useful, but you need to understand how the underlying data is stored in memory. If you have an array of floats, and view them as ints, (or vice versa) numpy will interpret the underlying bits of the array as ints.

例如，这意味着 1.0 在 little-endian 系统上作为 64 位浮点数在被视为 64 位 int 时将是 4607182418800017408，而 数组>[ 0, 0, 0, 0, 0, 0, 240, 63] 如果被视为 uint8.当您需要对大型数组进行某种类型的位处理时，这真的很好，不过……您对内存缓冲区的解释方式有低级控制.

For example, this means that 1.0 as a 64bit float on a little-endian system will be 4607182418800017408 when viewed as a 64bit int, and an array of [ 0, 0, 0, 0, 0, 0, 240, 63] if viewed as a uint8. This is really nice when you need to do bit-twiddling of some sort on large arrays, though... You have low level control over how the memory buffer is interpreted.

这篇关于查看 numpy 数组?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！