23일차 [ pytorch (중요) ]

Dtype

• rank 0,1,2,3 -> scalar, vector, matrix, tensor
• cpu는 코어만 있음. / gpu는 코어 +전용 램 메모리 있음. 
• cpu는 메모리와 스토리지 간에 데이터 스왑 가능, gpu와 cpu는 상호간에 데이터 스왑 불가

Tensor 연산

indexing and slicing

data = [
    [1, 2, 3], [4, 5, 6], [7, 8, 9]
]
tensor = torch.tensor(data, dtype=torch.float)

print(tensor)
print('-'*50)

print('First row: ',tensor[0])
print('Last row: ',tensor[-1])
print('First column: ', tensor[:, 0])
print('Last column1:', tensor[:, -1])
print('Last column2:', tensor[..., -1])


"""
[6, 4]
[9, 7]
"""
tensor[1:,[-1,0]]

합치기

• cat :concat
• stack : 새로운 차원 형성

t1 = torch.tensor([[1,2],[3,4]])
t2 = torch.tensor([[5,6],[7,9]])
t1.shape, t2.shape
(torch.Size([2, 2]), torch.Size([2, 2]))

torch.cat((t1,t2),dim=0) # shape: (4,2)
tensor([[1, 2],
        [3, 4],
        [5, 6],
        [7, 9]])
        
torch.cat((t1,t2),dim=1) # shape: (2,4)
tensor([[1, 2, 5, 6],
        [3, 4, 7, 9]])
        
tmp = torch.stack((t1,t2),dim=0)
tmp.shape # (2,2,2) # (2 matrix, 2 vector, 2 scalar)
tmp

tmp1 = torch.stack((t1,t2),dim=1)
tmp1.shape # (2,2,2) # (2 matrix, 2 vector, 2 scalar)
tmp1

tensor([[[1, 2],
         [5, 6]],

        [[3, 4],
         [7, 9]]])

곱하기

 

• torch.mm() (일반 곱셈) vs torch.mul() (broadcasting)

>>> a = torch.randn(3)
>>> a
tensor([ 0.2015, -0.4255,  2.6087])
>>> torch.mul(a, 100)
tensor([  20.1494,  -42.5491,  260.8663])

>>> b = torch.randn(4, 1)
>>> b
tensor([[ 1.1207],
        [-0.3137],
        [ 0.0700],
        [ 0.8378]])
>>> c = torch.randn(1, 4)
>>> c
tensor([[ 0.5146,  0.1216, -0.5244,  2.2382]])
>>> torch.mul(b, c)
tensor([[ 0.5767,  0.1363, -0.5877,  2.5083],
        [-0.1614, -0.0382,  0.1645, -0.7021],
        [ 0.0360,  0.0085, -0.0367,  0.1567],
        [ 0.4312,  0.1019, -0.4394,  1.8753]])

• Batch 끼린 연산이 안된다.

# 딥러닝에서 0번째 차원은 batch 차원이다.
a = torch.zeros(256,5,6) # (batch, row,column)
b = torch.zeros(256,6,7) # (batch, row,column)
torch.bmm(a,b).shape # batch 끼린 연산 안함
torch.Size([256, 5, 7])

c=a @ b
c.shape
torch.Size([256, 5, 7])

• matmul(): broadcasting이 되는 배치 연산

# broadcasting이 되는 배치 연산
a  = torch.zeros((256,5,6)) # tensor
b = torch.zeros((6,7)) # matrix
torch.matmul(a,b).shape
torch.Size([256, 5, 7])

 

Single-Element Tensors

• sum (agg)

tensor = torch.randn(4)

agg = tensor.sum()
tensor, agg, type(agg)
(tensor([-1.0184,  1.3498,  0.5666, -1.0979]), tensor(-0.1998), torch.Tensor)

• .item()  # get a value

agg_item = agg.to('cpu').item() # GPU -> CPU -> python
agg_item, type(agg_item)
(-0.19984161853790283, float)

 

in-place operations

• add_, subtract_ 

tensor
tmp = tensor.add_(5)
tensor, tmp, tmp
(tensor([3.9816, 6.3498, 5.5666, 3.9021]),
 tensor([3.9816, 6.3498, 5.5666, 3.9021]),
 tensor([3.9816, 6.3498, 5.5666, 3.9021]))

• reshape(), permute()

tensor = torch.rand(2,3,4)
tensor.reshape(2,12).shape
torch.Size([2, 12])

tensor.permute(2,1,0).shape # 차원의 위치를 바꿈.
torch.Size([4, 3, 2])

• squeeze(), unsqueeze()

# squeeze(), 차원 크기 1인 차원 제거
tensor = torch.rand(1,3,1,20,1)
tensor.shape
torch.Size([1, 3, 1, 20, 1])

tensor.squeeze().shape
torch.Size([3, 20])

tensor.squeeze(dim=4).shape
torch.Size([1, 3, 1, 20])