Cassandra Server在一致性ONE读查询时超时(0个副本(s)响应超过1个要求)

Question

我使用的是cqlsh v5.0.1，其中我有一个8节点cassandra集群，其中有几个表，在我扫描一个表，并有一个简单的逻辑，如果一行比我删除它的年龄大于6个月，如果它小于6个月，那么我更新该行的ttl，为了做到这一点，我使用express-cassandra npm并使用每行方法流式传输表的行，但我经常得到关于服务器超时的错误，并且我的程序终止，因为我没有得到进一步处理所需的下一页。

下面我附上了我的表配置和代码

Keyspace: events
Read Count: 550349
Read Latency: 14.500334253355598 ms.
Write Count: 46644769
Write Latency: 0.2615331485294739 ms.
Pending Flushes: 0
Table: track
SSTable count: 18
Space used (live): 1.56 TB
Space used (total): 1.56 TB
Space used by snapshots (total): 0 bytes
Off heap memory used (total): 2.66 GB
SSTable Compression Ratio: 0.12156681850176397
Number of partitions (estimate): 222854730
Memtable cell count: 4092
Memtable data size: 8.04 MB
Memtable off heap memory used: 0 bytes
Memtable switch count: 1828
Local read count: 550349
Local read latency: 12.668 ms
Local write count: 46644784
Local write latency: 0.201 ms
Pending flushes: 0
Bloom filter false positives: 5
Bloom filter false ratio: 0.00000
Bloom filter space used: 417.49 MB
Bloom filter off heap memory used: 570.87 MB
Index summary off heap memory used: 211.54 MB
Compression metadata off heap memory used: 1.89 GB
Compacted partition minimum bytes: 43 bytes
Compacted partition maximum bytes: 765.03 MB
Compacted partition mean bytes: 44.5 KB
Average live cells per slice (last five minutes): 10.050420168067227
Maximum live cells per slice (last five minutes): 124
Average tombstones per slice (last five minutes): 9.004201680672269
Maximum tombstones per slice (last five minutes): 1597 

模式:

CREATE TABLE events.track (
"profileId" text,
"projectId" text,
"sessionId" bigint,
"anonymousId" text,
"appBuild" text,
"appName" text,
"appNamespace" text,
"appVersion" text,
attributes list<text>,
channels list<text>,
"deviceId" text,
"deviceManufacturer" text,
"deviceModel" text,
"deviceName" text,
"eventTypes" list<text>,
ip text,
"libraryName" text,
"libraryVersion" text,
locale text,
"networkCarrier" text,
"osName" text,
"osVersion" text,
"propertyIds" list<text>,
referrer text,
"screenDensity" int,
"screenHeight" int,
"screenWidth" int,
"sessionAttributes" map<text, text>,
texts list<text>,
timestamps list<timestamp>,
timezone text,
"userAgent" text,
"writeKey" text,
PRIMARY KEY (("profileId", "projectId"), "sessionId")
) WITH CLUSTERING ORDER BY ("sessionId" DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';

流代码

EventModel.eachRow({}, { fetchSize: 10000 }, function (n, row) {
eventsChunk.push(row);
},
function (err, result) {
// error handling and business logic here
});

任何帮助都将是非常感激的。

Answer 1

超时是因为全表扫描的开销很大，特别是当一个节点上有超过2亿个分区时。

你没有指定你的表模式和你正在运行的查询，但我想象你正在使用ALLOW FILTERING进行范围查询，这会使节点处于负载状态，因此它们变得无响应。

Cassandra是为OLTP工作负载而设计的，当你想要非常非常快地检索单个分区时。全表扫描是OLAP工作负载，所以你需要一个分析解决方案，比如使用Apache Spark和Spark -cassandra-connector。

连接器优化了针对Cassandra的查询，因此连接器不是进行全表扫描，而是将它们分解为令牌范围的段，一次只请求一小部分。干杯!