PyTorch 内存模型:“torch.from_numpy()"；与“torch.Tensor()" [英] PyTorch memory model: "torch.from_numpy()" vs "torch.Tensor()"

问题描述

我正在尝试深入了解 PyTorch Tensor 内存模型的工作原理.

I'm trying to have an in-depth understanding of how PyTorch Tensor memory model works.

# input numpy array
In [91]: arr = np.arange(10, dtype=float32).reshape(5, 2)

# input tensors in two different ways
In [92]: t1, t2 = torch.Tensor(arr), torch.from_numpy(arr)

# their types
In [93]: type(arr), type(t1), type(t2)
Out[93]: (numpy.ndarray, torch.FloatTensor, torch.FloatTensor)

# ndarray 
In [94]: arr
Out[94]: 
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.],
       [ 6.,  7.],
       [ 8.,  9.]], dtype=float32)

<小时>

我知道 PyTorch 张量共享 NumPy ndarrays 的内存缓冲区.因此，改变一个将反映在另一个中.所以，我在这里切片和更新张量 t2

---

I know that PyTorch tensors share the memory buffer of NumPy ndarrays. Thus, changing one will be reflected in the other. So, here I'm slicing and updating some values in the Tensor t2

In [98]: t2[:, 1] = 23.0

正如预期的那样，它在 t2 和 arr 中更新，因为它们共享相同的内存缓冲区.

And as expected, it's updated in t2 and arr since they share the same memory buffer.

In [99]: t2
Out[99]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]


In [101]: arr
Out[101]: 
array([[  0.,  23.],
       [  2.,  23.],
       [  4.,  23.],
       [  6.,  23.],
       [  8.,  23.]], dtype=float32)

但是，t1 也更新了.请记住，t1 是使用 torch.Tensor() 构建的，而 t2 是使用 torch.from_numpy() 构建的/p>

But, t1 is also updated. Remember that t1 was constructed using torch.Tensor() whereas t2 was constructed using torch.from_numpy()

In [100]: t1
Out[100]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]

所以，无论我们是否使用torch.from_numpy() 或 torch.Tensor() 从一个 ndarray 构造一个张量，所有这样的张量和 ndarrays 共享相同的内存缓冲区.

So, no matter whether we use torch.from_numpy() or torch.Tensor() to construct a tensor from an ndarray, all such tensors and ndarrays share the same memory buffer.

基于这种理解，我的问题是为什么一个专用函数 torch.from_numpy() 仅存在于 torch.Tensor() 可以完成这项工作吗?

Based on this understanding, my question is why does a dedicated function torch.from_numpy() exists when simply torch.Tensor() can do the job?

我查看了 PyTorch 文档，但它没有提及任何相关内容?有什么想法/建议吗?

I looked at the PyTorch documentation but it doesn't mention anything about this? Any ideas/suggestions?

推荐答案

from_numpy() 自动继承输入数组 dtype.另一方面，torch.Tensor 是 torch.FloatTensor 的别名.

from_numpy() automatically inherits input array dtype. On the other hand, torch.Tensor is an alias for torch.FloatTensor.

因此，如果您将 int64 数组传递给 torch.Tensor，则输出张量是浮点张量，它们不会共享存储.torch.from_numpy 按预期为您提供 torch.LongTensor.

Therefore, if you pass int64 array to torch.Tensor, output tensor is float tensor and they wouldn't share the storage. torch.from_numpy gives you torch.LongTensor as expected.

a = np.arange(10)
ft = torch.Tensor(a)  # same as torch.FloatTensor
it = torch.from_numpy(a)

a.dtype  # == dtype('int64')
ft.dtype  # == torch.float32
it.dtype  # == torch.int64

这篇关于PyTorch 内存模型:“torch.from_numpy()"；与“torch.Tensor()"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！