如何克服"ValueError: Resolve param in estimatorParamMaps failed" PySpark 错误？

Question

我试图保存一个网格搜索的PySparkTrainValidationSplitModel对象，在调整逻辑回归的正则化时，我得到了以下奇怪的错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-104-8e6b86f1e92c> in <module>
1 # Save model, or upload if already saved
2 if not os.path.isdir(drive_path + 'lr_2_model'):
----> 3     lr_2_model.save(drive_path + 'lr_2_model')
4 else:
5     lr_2_model = TrainValidationSplitModel.load(drive_path + 'lr_2_model')
5 frames
/content/spark-3.3.0-bin-hadoop3/python/pyspark/ml/tuning.py in meta_estimator_transfer_param_maps_to_java(pyEstimator, pyParamMaps)
324                         break
325                 if javaParam is None:
--> 326                     raise ValueError("Resolve param in estimatorParamMaps failed: " + str(pyParam))
327                 if isinstance(pyValue, Params) and hasattr(pyValue, "_to_java"):
328                     javaValue = cast(JavaParams, pyValue)._to_java()
ValueError: Resolve param in estimatorParamMaps failed: LogisticRegression_87f4bc317e0b__regParam

这是导致错误的代码。这段代码使用了以前的LogisticRegressionPySpark模型，在该模型中我调整了maxIter参数。

# Save model, or upload if already saved
if not os.path.isdir(drive_path + 'lr_2_model'):
lr_2_model.save(drive_path + 'lr_2_model')
else:
lr_2_model = TrainValidationSplitModel.load(drive_path + 'lr_2_model')

这是我定义lr_2_model的代码(grid_search是我编写的自定义函数。错误不可能是因为它与其他模型一起工作(：

# Run grid search
%%time
if not os.path.isdir(drive_path + 'lr_2_model'):
lr_2_model = grid_search(stages_with_classifier=lr_2_stages, 
train_df=train_df_preprocessed, 
model_grid=lr_2_grid, 
parallelism=5)

这是我定义lr_2_grid、lr_2_stages和lr_2的代码。

lr_2 = LogisticRegression(
featuresCol='scaled_features',
labelCol='Anomalous',
weightCol='Weight',
standardization=False)
lr_2_stages = stages + [lr_2]
# Specify parameter grid
lr_2_grid = ParamGridBuilder()
.addGrid(lr_1.regParam, list(np.linspace(0.001, 0.1, 5)))
.build()

Answer 1

我解决了这个问题。我在ParamGridBuilder中调用了以前的LogisticRegression模型：.addGrid(**lr_1**.regParam, list(np.linspace(0.001, 0.1, 5)))

facepalm