在Tensorflow中实现自定义损失函数时，输入张量必须至少为2D: [] [Op:BiasAddGrad]

Question

在我的项目中，我必须自定义损失函数，这是匈牙利损失函数。

def hungarian_algorithm_with_None(y_true, y_pred):
from scipy.optimize import linear_sum_assignment
BATCH_SIZE = 32
final_cost = []
for batch in range(BATCH_SIZE):
y_true_ = y_true[batch,:]
y_pred_ = y_pred[batch,:]
store_element = np.ones(shape=(20,20))
y_true_ = tf.cast(y_true_,dtype=tf.double)
y_pred_ = tf.cast(y_pred_,dtype=tf.double)
for i in range(y_true_.shape[0]):
for j in range(y_pred_.shape[0]):     
if not np.isnan(y_true_[i]) and not np.isnan(y_pred_[j]):
element = tf.square(y_true_[i] - y_pred_[j])            
store_element[i,j] = element
# store_element_None.append(element)
cost = store_element # cost here is the values inside the table
row_ind, col_ind = linear_sum_assignment(cost)
chosen_elements = set(zip(row_ind.tolist(), col_ind.tolist()))
for sample in range(cost.shape[0]):
for sirna in range(cost.shape[1]):
if cost[sample, sirna] == 1:
cost[sample, sirna] = np.nan

cost = np.nanmin(cost,axis=1)

cost = tf.cast(cost,dtype=tf.float32)

final_cost.append(cost)
return tf.math.reduce_mean(tf.experimental.numpy.nanmean(tf.reshape(final_cost,shape=(BATCH_SIZE,20)),axis=-1))
def hungarian_loss(y_true, y_pred):
cost = tf.py_function(func=hungarian_algorithm_with_None, inp=[y_true,y_pred], Tout=tf.float32,)
cost_tensor = tf.convert_to_tensor(cost)
# print(cost_tensor)
return cost

当我用样本输入和输出测试它时

test_y_pred = tf.random.uniform(shape=(32,20,))
test_y_true = tf.random.uniform(shape=(32,20,))

工作正常，返回结果

<tf.Tensor: shape=(), dtype=float32, numpy=0.0017042245>

然而，当我使用它进行训练时，它在

行出现错误

grads = tape.gradient(loss_value, test_model.trainable_weights)

这是完整的定制训练循环

x = np.random.random((1000, 32))
y = np.random.random((1000, 10))
train_dataset = tf.data.Dataset.from_tensor_slices((x, y))
loss_tracker = tf.keras.metrics.Mean(name="loss")
mae_metric = tf.keras.metrics.MeanAbsoluteError(name="mae")
optimizer = tf.keras.optimizers.SGD(learning_rate=1e-2)
for epoch in range(10):
print("nStart of epoch %d" % (epoch,))

for step, (x_batch_train, y_batch_train) in enumerate(train_dataset.batch(32)):
with tf.GradientTape() as tape:
logits = test_model(x_batch_train, training=True)  # Logits for this minibatch
loss_value = hungarian_loss(y_batch_train, logits)
# loss_value = custom_mse(y_batch_train, logits)

tf.print(loss_value.shape)

grads = tape.gradient(loss_value, test_model.trainable_weights) **--> error is here**
optimizer.apply_gradients(zip(grads, test_model.trainable_weights))
loss_tracker.update_state(loss_value)
mae_metric.update_state(y_batch_train, logits)

if step % 32 == 0:
print(
"Training loss (for one batch) at step %d: %.4f"
% (step, float(loss_value))
)

我不明白是什么原因导致这个错误。如您所见，我注释了
custom_mse(y_batch_train, logits)&;． 该函数的形状输出与函数hungarian_loss相同，但它适用于上面的训练循环。下面是custom_mse

的代码

import tensorflow.keras.backend as K
def custom_mse(y_true, y_pred):
y_true = tf.cast(y_true,dtype=tf.float32)
y_pred = tf.cast(y_pred,dtype=tf.float32)
# calculating squared difference between target and predicted values 
loss = K.square(y_pred - y_true)  # (batch_size, 2)
loss = tf.experimental.numpy.nanmean(loss,axis=-1)
return tf.reduce_mean(loss)

Answer 1

谁在寻找答案。我通过删除

解决了这个问题

def hungarian_loss(y_true, y_pred):
cost = tf.py_function(func=hungarian_algorithm_with_None, inp=[y_true,y_pred], Tout=tf.float32,)
cost_tensor = tf.convert_to_tensor(cost)
# print(cost_tensor)
return cost

我得到这个错误的原因是因为当它计算反向传播时，函数tf.py_function没有调用函数

def hungarian_algorithm_with_None(y_true, y_pred):

这就是为什么它至少需要二维张量

另外，在自定义损失函数时不要使用numpy，它会使变量与tensorflow图断开，从而无法计算反向传播。