无法连接嵌入层和类别编码层

Question

>问题

按照 GCP 顶点 AI 示例和下面的代码进行操作。

for key in input_layers:
feature_name = features.original_name(key)
if feature_name in features.EMBEDDING_CATEGORICAL_FEATURES:
vocab_size = feature_vocab_sizes[feature_name]
embedding_size = features.EMBEDDING_CATEGORICAL_FEATURES[feature_name]

embedding_output = keras.layers.Embedding(
input_dim=vocab_size + 1,
output_dim=embedding_size,
name=f"{key}_embedding",
)(input_layers[key])
print(f"Shape of embed layer [{key}] has None [{embedding_output.shape}] output_dim is {embedding_size}")

layers.append(embedding_output)
elif feature_name in features.ONEHOT_CATEGORICAL_FEATURE_NAMES:
vocab_size = feature_vocab_sizes[feature_name]
onehot_layer = keras.layers.experimental.preprocessing.CategoryEncoding(
max_tokens=vocab_size,
output_mode="binary",
name=f"{key}_onehot",
)(input_layers[key])
onehot_layer = tf.keras.layers.CategoryEncoding(
num_tokens=vocab_size, 
output_mode="one_hot",
name=f"{key}_onehot",
)(input_layers[key])

print(f"Shape of one hot layer [{key}] has None [{onehot_layer.shape}]")

layers.append(onehot_layer)
elif feature_name in features.NUMERICAL_FEATURE_NAMES:
numeric_layer = tf.expand_dims(input_layers[key], -1)
print(f"Shape of numeric layer [{key}] has None [{numeric_layer.shape}]")

layers.append(numeric_layer)
else:
pass
#--------------------------------------------------------------------------------
# Concatenate Embedding and CategoryEncoding layers fails
#--------------------------------------------------------------------------------
joined = keras.layers.Concatenate(name="combines_inputs")(layers)

该代码会导致无法连接层的错误。

Shape of embed layer [trip_month_xf] has None [(None, 2)]
Shape of embed layer [trip_day_xf] has None [(None, 4)]
Shape of one hot layer [trip_day_of_week_xf] has None [(7,)]
Shape of embed layer [trip_hour_xf] has None [(None, 3)]
Shape of numeric layer [trip_seconds_xf] has None [(None, 1)]
Shape of numeric layer [trip_miles_xf] has None [(None, 1)]
Shape of one hot layer [payment_type_xf] has None [(5,)]
Shape of embed layer [pickup_grid_xf] has None [(None, 3)]
Shape of embed layer [dropoff_grid_xf] has None [(None, 3)]
Shape of numeric layer [euclidean_xf] has None [(None, 1)]
Shape of embed layer [loc_cross_xf] has None [(None, 10)]
ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concatenation axis. Received: input_shape=[(None, 2), (None, 4), (7,), (None, 3), (None, 1), (None, 1), (5,), (None, 3), (None, 3), (None, 1), (None, 10)]

问题

原因是什么以及如何解决？

Answer 1

原因

keras.layers.experimental.preprocessing.CategoryEncoding图层生成形状(N, )。

试图重塑为(None, N)但没有奏效

from keras.layers.core import Reshape
onehot_layer = Reshape((-1, vocab_size))(onehot_layer)
----------
ValueError: Exception encountered when calling layer "reshape_13" (type Reshape).
Dimension size must be evenly divisible by 49 but is 7 for '{{node reshape_13/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](Placeholder, reshape_13/Reshape/shape)' with input shapes: [7], [3] and with input tensors computed as partial shapes: input[1] = [7,?,7].
Call arguments received:
• inputs=tf.Tensor(shape=(7,), dtype=float32)

修复

使用tf.keras.layers.CategoryEncoding.

elif feature_name in features.ONEHOT_CATEGORICAL_FEATURE_NAMES:
vocab_size = feature_vocab_sizes[feature_name]
# onehot_layer = keras.layers.experimental.preprocessing.CategoryEncoding(
#     max_tokens=vocab_size,
#     output_mode="binary",
#     name=f"{key}_onehot",
# )(input_layers[key])
onehot_layer = tf.keras.layers.CategoryEncoding(
num_tokens=vocab_size, 
output_mode="one_hot",
name=f"{key}_onehot",
)(input_layers[key])

结果：

Shape of embed layer [trip_month_xf] has None [(None, 2)]
Shape of embed layer [trip_day_xf] has None [(None, 4)]
Shape of one hot layer [trip_day_of_week_xf] has None [(None, 7)]
Shape of embed layer [trip_hour_xf] has None [(None, 3)]
Shape of numeric layer [trip_seconds_xf] has None [(None, 1)]
Shape of numeric layer [trip_miles_xf] has None [(None, 1)]
Shape of one hot layer [payment_type_xf] has None [(None, 5)]
Shape of embed layer [pickup_grid_xf] has None [(None, 3)]
Shape of embed layer [dropoff_grid_xf] has None [(None, 3)]
Shape of numeric layer [euclidean_xf] has None [(None, 1)]
Shape of embed layer [loc_cross_xf] has None [(None, 10)]

引用

• shape=(None，) 或 (None，12) 的输入层实际上是什么意思？

在 tf.keras 中，None 维度意味着它可以是任何标量数，因此您可以使用此模型推断任意长的输入。此维度不会影响网络的大小，它只是表示您可以在测试期间自由选择输入的长度(样本数)。

• 值错误：尝试将"形状"转换为张量并失败。错误：不支持无值。#1070

如果你想使用 tf.keras

或从 tensorflow import keras，你所要做的就是用 -1 替换 tf.keras.layers.Reshape 函数中的第一个值 (s[1] == None)：

替换为以下代码：

mrcnn_bbox = layers.Reshape((-1, num_classes, 4), name="mrcnn_bbox")(x)

• 如何在 Keras 中将嵌入与可变长度输入连接起来？

modifier_input_ = Input(shape=(None,), name='modifier_type_in')
...
statistics_inputs =[Input(shape=(None, len(statistics),), name='statistics')] # Input(shape=(1,))

• 如何将One-Hot Encoding layer添加到Tensorflow模型中？

numerical_input = tf.keras.layers.Input(shape=(2,), dtype=tf.float32)

categorical_input = tf.keras.layers.Input(shape=(1,), dtype=tf.int32)
encoded = tf.keras.layers.CategoryEncoding( num_tokens=3, output_mode="one_hot")(categorical_input)
concat = tf.keras.layers.concatenate([numerical_input, encoded])