我正在尝试在EMR(Jupyter(笔记本上使用带有PySpark的SageMaker Python SDK。 当尝试使用XGBoostSageMakerEstimator时
，如下所示，from sagemaker_pyspark.algorithms import XGBoostSageMakerEstimator
xgboost_estimator = XGBoostSageMakerEstimator(
sagemakerRole=IAMRole(someRoleArn),
trainingInstanceType='ml.m4.xlarge',
trainingInstanceCount=1,
endpointInstanceType='ml.m4.xlarge',
endpointInitialInstanceCount=1)
我收到以下错误，我无法找到解决方案。
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.ScalaMap object at 0x7fd3d9e96240>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'ScalaMap' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.ScalaMap object at 0x7fd3d9e96240>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'ScalaMap' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d3c8>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d128>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d0f0>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d080>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e96ef0>>
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
任何解决此问题的帮助将不胜感激。
用：
- EMR (emr-5.26.0( 集群与 Spark 2.4.3
- 附加到集群的 EMR 笔记本
- sagemaker_pyspark预装了 EMR-5.26.0
我遇到了同样的错误。我相信sagemaker_pyspark与 Spark 版本> 2.3.2 不兼容(来源：https://github.com/aws/sagemaker-spark/commit/4055f1e05be7d5e764f2abc8b3d6fc2c252ae272(。我能够与为该项目做出贡献的人确认这一点。
我使用 Spark 2.3.2 运行了我的代码，但不再看到异常。