pytorch 梯度NAN异常值的解决方案

import torch
# 异常检测开启
torch.autograd.set_detect_anomaly(True)
# 反向传播时检测是否有异常值，定位code
with torch.autograd.detect_anomaly():
 loss.backward()

a = torch.Tensor([[1, 2, np.nan], [np.inf, np.nan, 4], [3, 4, 5]])
 
a
Out[158]: 
tensor([[1., 2., nan],
  [inf, nan, 4.],
  [3., 4., 5.]])

a = torch.where(torch.isnan(a), torch.full_like(a, 0), a)
 
a
Out[160]: 
tensor([[1., 2., 0.],
  [inf, 0., 4.],
  [3., 4., 5.]])

a = torch.where(torch.isinf(a), torch.full_like(a, 0), a)
 
a
Out[162]: 
tensor([[1., 2., 0.],
  [0., 0., 4.],
  [3., 4., 5.]])

torch.full_like(input, fill_value, …) 返回与input相同size，单位值为fill_value的矩阵
 
#如下面这个例子，a为3*3的tensor
b =torch.full_like(a, 0,)
 
b
Out[165]: 
tensor([[0., 0., 0.],
  [0., 0., 0.],
  [0., 0., 0.]])