PyTorch内存模型:对比"torch.Tensor()" [英] PyTorch memory model: "torch.from_numpy()" vs "torch.Tensor()"

问题描述

我试图深入了解PyTorch Tensor内存模型的工作原理.

I'm trying to have an in-depth understanding of how PyTorch Tensor memory model works.

# input numpy array
In [91]: arr = np.arange(10, dtype=float32).reshape(5, 2)

# input tensors in two different ways
In [92]: t1, t2 = torch.Tensor(arr), torch.from_numpy(arr)

# their types
In [93]: type(arr), type(t1), type(t2)
Out[93]: (numpy.ndarray, torch.FloatTensor, torch.FloatTensor)

# ndarray 
In [94]: arr
Out[94]: 
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.],
       [ 6.,  7.],
       [ 8.,  9.]], dtype=float32)

---

我知道PyTorch张量共享NumPy ndarrays的内存缓冲区.因此，改变一个将反映在另一个中.因此，在这里我切片并更新了Tensor t2

---

I know that PyTorch tensors share the memory buffer of NumPy ndarrays. Thus, changing one will be reflected in the other. So, here I'm slicing and updating some values in the Tensor t2

In [98]: t2[:, 1] = 23.0

并且正如预期的那样，由于它们共享相同的内存缓冲区，因此在t2和arr中进行了更新.

And as expected, it's updated in t2 and arr since they share the same memory buffer.

In [99]: t2
Out[99]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]


In [101]: arr
Out[101]: 
array([[  0.,  23.],
       [  2.,  23.],
       [  4.,  23.],
       [  6.,  23.],
       [  8.,  23.]], dtype=float32)

但是， t1也已更新.请记住，t1是使用torch.Tensor()构造的，而t2是使用torch.from_numpy()

But, t1 is also updated. Remember that t1 was constructed using torch.Tensor() whereas t2 was constructed using torch.from_numpy()

In [100]: t1
Out[100]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]

因此，无论我们使用 torch.from_numpy() 还是 torch.Tensor() 从ndarray all 这样的张量和ndarray共享相同的内存缓冲区.

So, no matter whether we use torch.from_numpy() or torch.Tensor() to construct a tensor from an ndarray, all such tensors and ndarrays share the same memory buffer.

基于这种理解，我的问题是为什么要使用专用功能 torch.from_numpy() 存在，只是 torch.Tensor() 可以完成这项工作?

Based on this understanding, my question is why does a dedicated function torch.from_numpy() exists when simply torch.Tensor() can do the job?

我查看了PyTorch文档，但没有提及任何内容?有任何想法/建议吗?

I looked at the PyTorch documentation but it doesn't mention anything about this? Any ideas/suggestions?

推荐答案

from_numpy()自动继承输入数组dtype.另一方面，torch.Tensor是torch.FloatTensor的别名.

from_numpy() automatically inherits input array dtype. On the other hand, torch.Tensor is an alias for torch.FloatTensor.

因此，如果将int64数组传递给torch.Tensor，则输出张量为浮点张量，它们将不会共享存储. torch.from_numpy为您提供了torch.LongTensor的期望.

Therefore, if you pass int64 array to torch.Tensor, output tensor is float tensor and they wouldn't share the storage. torch.from_numpy gives you torch.LongTensor as expected.

a = np.arange(10)
ft = torch.Tensor(a)  # same as torch.FloatTensor
it = torch.from_numpy(a)

a.dtype  # == dtype('int64')
ft.dtype  # == torch.float32
it.dtype  # == torch.int64

这篇关于PyTorch内存模型:对比"torch.Tensor()"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！