将空值替换为字符串值

Question

我尝试将空值转换为字符串变量x。原因是该数据框架应该导入到power Bi中以进行可视化。我们的目标是计算一个箱形图，我的想法是，如果有一个x，那么这些值将不包括在计算中。这样，我们可以避免在计算中高估。用x替换空值是个好主意吗?还是有更好的方法?

data = [["1", "Amit", "DU", "I", "8", "6"],
["2", "Mohit", "DU", "I", "4", "2"],
["3", "rohith", "BHU", "I", "5", "3"],
["4", "sridevi", "LPU", "I", "1", "6"],
["1", "sravan", "KLMP", "M", "2", "4"],
["5", "gnanesh", "IIT", "M", "null", "8"],
["6", "gnadesh", "KLM", "c", "10", "null"]]
columns = ['ID', 'NAME', 'college', 'metric', 'x', 'y']

dataframe = spark.createDataFrame(data, columns)

实际输出

+---+-------+-------+------+----+-----+
| ID|   NAME|college|metric|  x |  y  |
+---+-------+-------+------+----+----+
|  1|   Amit|     DU|     I|  8 |  6 |
|  2|  Mohit|     DU|     I|  4 |  2 |
|  3| rohith|    BHU|     I|  5 |  3 |
|  4|sridevi|    LPU|     I|  1 |  6 |
|  1| sravan|   KLMP|     M|  2 |  4 |
|  5|gnanesh|    IIT|     M|null|  8 |
|  6|gnadesh|    KLM|     c| 10 |null|
+---+-------+-------+------+----+----+

所需输出

+---+-------+-------+------+----+-----+
| ID|   NAME|college|metric|  x |  y  |
+---+-------+-------+------+----+----+
|  1|   Amit|     DU|     I|  8 |  6 |
|  2|  Mohit|     DU|     I|  4 |  2 |
|  3| rohith|    BHU|     I|  5 |  3 |
|  4|sridevi|    LPU|     I|  1 |  6 |
|  1| sravan|   KLMP|     M|  2 |  4 |
|  5|gnanesh|    IIT|     M|  x |  8 |
|  6|gnadesh|    KLM|     c| 10 |  x |
+---+-------+-------+------+----+----+

我试过这个代码，但它不工作使用字符串，但只能使用数字

data = data.fillna({'y':'x'})

Answer 1

您可以使用以下命令:

data = data.fillna('x')

请确保您的列x和y都是Stringtype():

data = data.withColumns('x',col(('x').cast('string'))