当我通过以下命令启动Spark-Shell时
bin/spark-shell -packages org.apache.bahir:spark-streaming-mqtt_2.11:2.3.0 - repositories http://central.maven.org/maven2/maven2/org/org/apache/bahir/bahir/bahir/spark/spark-Streaming-mqtt_2.11/2.3.0/
发生了两个错误。
服务器访问错误在url https://central.maven.org/org/apache/bahir/bahir-parent_2.11/2.3.2/bahir-parent-parent_2.11-2.3.2.3.2.jar(javax.net。ssl.sslhandshakeexception:java.security.cert.certificateException:找到匹配central.maven.org的主题替代DNS名称。)
和
url https://central.maven.org/org/apache/bahir/spark-streaming-mqtt_2.11/2.11/2.11/2.11/2.2/spark-streaming-mqtt_2.11-2.11-2.2.2.2-javadoc。jar(javax.net.ssl.sslhandhandshakeexception:java.security.cert.certificateException:找不到匹配central.maven.org的主题替代dns名称。)
)
在这里,我给了存储库作为http://central.maven.org/maven2/org/apache/bahir/spark-streaming-mqtt_2.11/2.11/2.3.0/,但它会自动连接到https://central.maven。
如何在我的火花壳中添加这两个模块?我的目的是构建一个可以处理多个主题的火花流MQTT应用程序。
这将是您的系统问题。关于错误相关,导致javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException
的原因很多。原因之一是由于请求主机URL(包括IP地址)与证书(通常包括DNS HostName)之间的不匹配,请求失败。这是由于证书缺少别名(主题替代名称)主机使用默认名称不同的服务器时。
问题可以通过多种方式解决。请在以下链接中找到一些替代方案:
https://support.mulesoft.com/s/article/certificateexception-no-subject-ablet-names-names-present
https://support.cloudbees.com/hc/hc/en-us/articles/360017693231-WHY-WHY-AM---- getting-no-subject-no-subject-alternative-alternative-dns-name-name-name-matching-xxx-xxx-when-when-when-when-connecting-through--ldaps-
https://confluence.atlassian.com/confkb/java-security-cert-certcertificateexception exception-no-subject-aubject-anternative-dns-name-name-matching-hostname-hostname-found-452100730.html
https://confluence.atlassian.com/jirakb/java-security-cert-cert-certificateexception-no-subject-aubject-anternative-dns-name-name-matching-hostname-hostname-found-297669411.html
我能够在Spark-Shell中添加模块。请在下面找到摘要。
C:UsersXYzUser>spark-shell --repositories http://central.maven.org/maven2/org/apache/bahir/spark-streaming-mqtt_2.11/2.3.0/ --packages org.apache.bahir:spark-streaming-mqtt_2.11:2.3.0
http://central.maven.org/maven2/org/apache/bahir/spark-streaming-mqtt_2.11/2.3.0/ added as a remote repository with the name: repo-1
Ivy Default Cache set to: C:Users...ivy2cache
The jars for the packages stored in: C:Users...ivy2jars
:: loading settings :: url = jar:file:/C:/Tools/spark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
org.apache.bahir#spark-streaming-mqtt_2.11 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-73c724b4-c15c-45a8-89df-f492b2eb6feb;1.0
confs: [default]
found org.apache.bahir#spark-streaming-mqtt_2.11;2.3.0 in central
found org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.1.0 in central
found org.spark-project.spark#unused;1.0.0 in user-list
:: resolution report :: resolve 7200ms :: artifacts dl 16ms
:: modules in use:
org.apache.bahir#spark-streaming-mqtt_2.11;2.3.0 from central in [default]
org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.1.0 from central in [default]
org.spark-project.spark#unused;1.0.0 from user-list in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 3 | 1 | 1 | 0 || 3 | 0 |
---------------------------------------------------------------------
:: problems summary ::
:::: ERRORS
unknown resolver null
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:: retrieving :: org.apache.spark#spark-submit-parent-73c724b4-c15c-45a8-89df-f492b2eb6feb
confs: [default]
0 artifacts copied, 3 already retrieved (0kB/31ms)
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://...:4040
Spark context available as 'sc' (master = local[*], app id = local-1552454258705).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_ / _ / _ `/ __/ '_/
/___/ .__/_,_/_/ /_/_ version 2.4.0
/_/