如何使用<Row> Java API 将 DataSet ds 插入 cassandra

Question

Spark Cassandra连接器所需的示例代码2.11-2.0.5，无法将数据集直接插入Cassandra DB

Answer 1

1. 添加以下Cassandra依赖项

   <dependency>
       <groupId>com.datastax.spark</groupId>
       <artifactId>spark-cassandra-connector_2.10</artifactId>
       <version>1.2.1</version>
   </dependency>
   <dependency>
       <groupId>com.datastax.spark</groupId>
       <artifactId>spark-cassandra-connector-java_2.10</artifactId>
       <version>1.2.1</version>
   </dependency>
   

2. 将您的Cassandra数据库配置详细信息设置为Spark

   SparkConf sparkConf = new SparkConf();
   sparkConf.setAppName("Spark-Cassandra Integration");
   sparkConf.setMaster("local[4]");
   sparkConf.set("spark.cassandra.connection.host", "127.0.0.1");
   sparkConf.set("spark.cassandra.connection.native.port", "9042");
   sparkConf.set("spark.cassandra.connection.rpc.port", "9160");
   sparkConf.set("spark.cassandra.connection.timeout_ms", "5000");
   sparkConf.set("spark.cassandra.read.timeout_ms", "200000");
   sparkConf.set("spark.cassandra.auth.username", "test_user");
   sparkConf.set("spark.cassandra.auth.password", "test_password");
   

3. 创建火花上下文

   JavaSparkContext javaSparkContext = new JavaSparkContext(sparkConf);
   

4. 拥有RDD后，通过提供键空名称＆amp;表名

   CassandraJavaUtil.javaFunctions(userRDD).writerBuilder(keySpaceName, tableName, CassandraJavaUtil.mapToRow(UserBean.class)).saveToCassandra();
   

有关更多详细信息，您可以使用Apache Spark