如何在 TensorFlow 中将 HSV 张量转换为 RGB 张量?

Question

我需要一個可以將HSV張量(形狀：[batch_size, image_width, image_height, num_channels]，通道表示h, s, v(轉換為RGB張量([batch_size, image_width, image_height,num_channels]，通道表示r，g，b]的操作(。而且我知道存在tf.image.hsv_to_rgb，虽然它似乎是一个图像预处理功能，无法获得渐变。对于创建对话操作，我们是否需要使用一些元操作从头开始编写代码(例如。tf.multiply()、tf.add()等(

new_image_hsv = tf.random_normal(shape=[32,32,3]) 
new_image_rgb = tf.image.hsv_to_rgb(new_image_hsv, name='hsv2rgb') 
hsv2rgb = tf.get_default_graph().get_operation_by_name('hsv2rgb') 
print('hsv2rgb: ', get_gradient_function(hsv2rgb)) #hsv2rgb: None

Answer 1

目前HSV到RGB的转换功能没有注册的梯度，您可以考虑打开有关它的问题。但是，从内核的实现来看，完全可以使用具有定义梯度的基本 TensorFlow 操作来复制计算：

import tensorflow as tf
def my_hsv_to_rgb(tensor):
h = tensor[..., 0]
s = tensor[..., 1]
v = tensor[..., 2]
c = s * v;
m = v - c;
dh = h * 6
h_category = tf.cast(dh, tf.int32)
fmodu = tf.mod(dh, 2)
x = c * (1 - tf.abs(fmodu - 1))
component_shape = tf.shape(tensor)[:-1]
dtype = tensor.dtype
rr = tf.zeros(component_shape, dtype=dtype)
gg = tf.zeros(component_shape, dtype=dtype)
bb = tf.zeros(component_shape, dtype=dtype)
h0 = tf.equal(h_category, 0)
rr = tf.where(h0, c, rr)
gg = tf.where(h0, x, gg)
h1 = tf.equal(h_category, 1)
rr = tf.where(h1, x, rr)
gg = tf.where(h1, c, gg)
h2 = tf.equal(h_category, 2)
gg = tf.where(h2, c, gg)
bb = tf.where(h2, x, bb)
h3 = tf.equal(h_category, 3)
gg = tf.where(h3, x, gg)
bb = tf.where(h3, c, bb)
h4 = tf.equal(h_category, 4)
rr = tf.where(h4, x, rr)
bb = tf.where(h4, c, bb)
h5 = tf.equal(h_category, 5)
rr = tf.where(h5, c, rr)
bb = tf.where(h5, x, bb)
r = rr + m
g = gg + m
b = bb + m
return tf.stack([r, g, b], axis=-1)

测试它：

import tensorflow as tf
import numpy as np
img = tf.placeholder(tf.float32, (None, None, 3))
# Compute builtin conversion to check that our conversion is correct
tf_conversion = tf.image.hsv_to_rgb(img)
my_conversion = my_hsv_to_rgb(img)
# Difference between the builtin conversion and ours
error = tf.losses.mean_squared_error(tf_conversion, my_conversion)
# Take gradients of the conversion
my_conversion_grad = tf.gradients(my_conversion, img)[0]
# Test it
with tf.Session() as sess:
np.random.seed(100)
random_img = np.random.rand(10, 10, 3)
error_val, grad_val = sess.run([error, my_conversion_grad],
feed_dict={img: random_img})
print(error_val)
print(grad_val)

输出：

Error:
1.914486e-16
Gradient:
[[[-7.0903623e-01 -5.3507453e-01  2.6491349e+00]
[-3.4420034e-03 -1.2991568e-01  2.9949572e+00]
[ 6.7739707e-01 -2.7006465e-01  1.3685231e+00]
[-1.1187987e+00 -3.0346024e-01  1.7070839e+00]
[-1.4286079e-01 -2.4429685e-01  2.8794882e+00]
[-8.3736974e-01 -3.2182935e-01  1.4807379e+00]
[ 7.0991272e-01 -4.7601932e-01  2.6977921e+00]
[-1.6489303e+00 -5.5128366e-01  1.6589090e+00]
[-1.2725148e-02 -5.9869420e-03  2.6076081e+00]
[-7.2826855e-02 -2.3104541e-02  1.7949229e+00]]
#...

但是，请注意，HSV 到 RGB 的转换不是一个连续函数(见这里(，这可能是它没有定义梯度的原因。这意味着梯度可能并不总是指示正确的优化方向，因为转换定义中的这种"跳跃"。