SBT无法导入apache库

Question

我正在尝试建立一个简单的kafka消费者应用程序，它从安全的HTTPS kafka集群中消费消息。

这是我的SBT构建文件。

version := "0.1"
//libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "3.2.0" % "provided"
// https://mvnrepository.com/artifact/org.apache.kafka/kafka
libraryDependencies += "org.apache.kafka" %% "kafka" % "6.1.0-ccs"
d
scalaVersion := "2.13.6"

我的实际消费代码是…

package main.scala.kafka
import java.util
import java.util.Properties
import org.apache.kafka.clients.consumer.KafkaConsumer
object consumer extends App {


val TOPIC="amg-dev-time"
val  props = new Properties()
props.put("bootstrap.servers", "kafka-localhost.net:9093")
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("group.id", "amlng-dev-realtime")
val consumer = new KafkaConsumer[String, String](props)
consumer.subscribe(util.Collections.singletonList(TOPIC))
while(true){
val records=consumer.poll(100)
for (record<-records.asScala){
println(record)
}
}
}

当我运行上面的设置时，我得到对象apache不是包org的成员进口org.apache.kafka.clients.consumer.KafkaConsumer

帮助我解决这个问题，并帮助我提供一个关于如何连接到安全kafka集群以在scala中消费消息的示例。

在刷新SBT构建文件时…

[error]   Not found
[error]   Not found
[error]   not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error]   download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error]   not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.kafka:kafka_2.13:6.1.0-ccs
[error]   Not found
[error]   Not found
[error]   not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error]   download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error]   not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] Total time: 30 s, completed 19-Oct-2021, 6:21:47 pm
[info] shutting down sbt server```

Answer 1

Kafka 6.1.0-ccs托管在Confluent上。

尝试添加:

resolvers += "confluent" at "https://packages.confluent.io/maven/"

应该能解决这个问题(以scastie为例)