范围查询过程中出现Cassandra内存不足错误



我有一个包含1MB Blob的表。

创建表格blobs_1(关键文本,版本bigint,块int,object_blob blob,object_ size int,PRIMARY KEY(密钥、版本、区块))

每个lob分布在大约100块上。以下查询导致OutOfMemory错误:

从blobs_1中选择object_size,其中key='key1',version=1;

错误如下:

java.lang.OutOfMemoryError:java堆空间网址:org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccess Reader.java:344)网址:org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUti.java:392)网址:org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUti.java:355)网址:org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)网址:org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)网址:org.apache.cassandra.db.Column$1.computeNext(Column.java:75)网址:org.apache.cassandra.db.Column$1.computeNext(Column.java:64)网址:com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterater.java:143)网址:com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)网址:org.apache.cassandra.db.colum迭代器.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)网址:org.apache.cassandra.db.colum迭代器.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)网址:com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterater.java:143)网址:com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)网址:org.apache.casandra.db.columndirator.STableSliceIterator.hasNext(SSTableSliceIt erator.java:82)网址:org.apache.cassandra.db.columiterator.LazColumnDescriptor.counteNext(LazyColumnDescriptor.java:82)网址:org.apache.cassandra.db.columiterator.LazColumnDescriptor.counteNext(LazyColumnDescriptor.java:59)网址:com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterater.java:143)网址:com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)网址:org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)网址:org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)网址:org.apache.cassandra.utils.MergeIterator$Candidate.advanced(MergeIterar.java:144)网址:org.apache.cassandra.utils.MergeIterator$ManyToOne.advanced(MergeIterar.java:123)网址:org.apache.cassandra.utils.MergeIterator$ManyToOne.counteNext(MergeItator.java:97)网址:com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterater.java:143)网址:com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)网址:org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)网址:org.apache.cassandra.db.filter.QueryFilter.correctColumns(QueryFilter.java:122)网址:org.apache.cassandra.db.filter.QueryFilter.collectionOnDiskAtom(QueryFilter.java:80)网址:org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)网址:org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)网址:org.apache.cassandra.utils.MergeIterator$ManyToOne.conse(MergeIterar.java:115)网址:org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeItator.java:98)

您需要缩小页面大小。默认的分页大小适用于正常的小列/行。对于较大的Blob,您需要缩小页面大小。

https://github.com/datastax/java-driver/blob/2.0/driver-core/src/main/java/com/datastax/driver/core/Statement.java#L234

发生此错误是因为Cassandra在读取表的单列时反序列化的数据超过了所需的数据量(至少是Cassandra 1.2,可能在2.0分支中有所改进)。

为了解决这个问题,您可以为元数据(大小等)引入一个单独的表。它会稍微降低写入速度,但会大大提高读取性能。

最新更新