如何从tensorflow数据集中获取标签

Question

ds_test = tf.data.experimental.make_csv_dataset(
file_pattern = "./dfj_test/part-*.csv.gz",
batch_size=batch_size, num_epochs=1, 
#column_names=use_cols, 
label_name='label_id',
#select_columns= select_cols,
num_parallel_reads=30, compression_type='GZIP',
shuffle_buffer_size=12800)

这是我训练时的训练套。在完成模型之后，我想压缩df_test的预测和标签列。

preds = model.predict(df_test)

获取预测非常简单，而且是numpy数组格式。然而，我不知道如何从df_test中获得相应的标签。我想压缩(preds，标签(以进行进一步分析。有什么提示吗？谢谢

(tf版本2.3.1(

Answer 1

您可以映射每个示例以返回您想要的字段

# load some exemplary data
TRAIN_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/train.csv"
train_file_path = tf.keras.utils.get_file("train.csv", TRAIN_DATA_URL)
dataset = tf.data.experimental.make_csv_dataset(train_file_path, batch_size=100, num_epochs=1) 

# get field by unbatching 
labels_iterator= dataset.unbatch().map(lambda x: x['survived']).as_numpy_iterator()
labels = np.array(list(labels_iterator))

# get field by concatenating batches
labels_iterator= dataset.map(lambda x: x['survived']).as_numpy_iterator()
labels = np.concatenate(list(labels_iterator))