卡桑德拉读取过程



比如说,我有一个表格,有 4 列。我在里面写了一些数据。如果我尝试读取数据,过程是这样的。我想了解一个特定的场景,其中所有列(我尝试读取的行)都存在于内存表中。是否会检查此类行的数据 SSTables?我认为,在这种情况下,没有必要检查 SSTables,因为显然内存表中存在的数据将是最新的副本。因此,在这种情况下,与 memtable 没有行或仅包含部分数据时的读取相比,读取应该更快。

我创建了一个表(user_data),并输入了一些数据,从而创建了 2 个 SSTable。在此之后,我插入了一行新行。我检查了数据目录,并确保 SSTable 计数仍然是 2。这意味着我输入的新数据位于内存表中。我在 cqlsh 中设置了"跟踪",然后选择了同一行。下面是输出:

Tracing session: de2e8ce0-cf1e-11e6-9318-a131a78ce29a
activity                                                                                     | timestamp                  | source        | source_elapsed | client
----------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+---------------
     Execute CQL3 query | 2016-12-31 11:33:36.494000 | 172.16.129.67 |              0 | 172.16.129.67
Parsing select address,age from user_data where name='Kishan'; [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            182 | 172.16.129.67
Preparing statement [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            340 | 172.16.129.67
Executing single-partition query on user_data [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            693 | 172.16.129.67
Acquiring sstable references [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            765 | 172.16.129.67
Merging memtable contents [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            821 | 172.16.129.67
Read 1 live rows and 0 tombstone cells [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |           1028 | 172.16.129.67
       Request complete | 2016-12-31 11:33:36.495225 | 172.16.129.67 |           1225 | 172.16.129.67

我不明白这里"获取稳定引用"的含义。由于完整的数据位于Memtable中,因此,据我了解,没有必要检查SSTables。那么,这些参考资料究竟是为了什么?

所有列(我试图读取的行)都存在于内存表中。是否会检查此类行的数据 SSTables?

在这种特殊情况下,它还将沿内存可视差检查稳定数据。

它只会转到 sstable(实际上首先在行缓存中,然后是布隆过滤器,然后是sstable),对于该列,该列在 memtable 中不存在。

编辑:

要了解有关读取过程如何在此处工作的更多信息,让我们深入了解Cassandra源代码。让我们从跟踪日志开始,我们将逐行完成这些步骤:

让我们从这里开始:

Executing single-partition query on user_data [ReadStage-2]

您的选择查询是单分区行查询,这很明显。Cassandra只需要从单个分区读取数据。让我们跳到相应的方法和java-doc这里,是自我解释的:

/**
* Queries both memtable and sstables to fetch the result of this query.
* <p>
* Please note that this method:
*   1) does not check the row cache.
*   2) does not apply the query limit, nor the row filter (and so ignore 2ndary indexes).
*      Those are applied in {@link ReadCommand#executeLocally}.
*   3) does not record some of the read metrics (latency, scanned cells histograms) nor
*      throws TombstoneOverwhelmingException.
* It is publicly exposed because there is a few places where that is exactly what we want,
* but it should be used only where you know you don't need thoses things.
* <p>
* Also note that one must have created a {@code ReadExecutionController} on the queried table and we require it as
* a parameter to enforce that fact, even though it's not explicitlly used by the method.
*/
public UnfilteredRowIterator queryMemtableAndDisk(ColumnFamilyStore cfs, ReadExecutionController executionController)
{
assert executionController != null && executionController.validForReadOn(cfs);
Tracing.trace("Executing single-partition query on {}", cfs.name);
return queryMemtableAndDiskInternal(cfs);
}

从 avobe 步骤中,我们发现对于您的查询,它将调用此方法queryMemtableAndDiskInternal(cfs);

private UnfilteredRowIterator queryMemtableAndDiskInternal(ColumnFamilyStore cfs)
{
/*
* We have 2 main strategies:
*   1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use
*      unless we have a names filter that we know we can optimize futher.
*   2) If we have a name filter (so we query specific rows), we can make a bet: that all column for all queried row
*      will have data in the most recent sstable(s), thus saving us from reading older ones. This does imply we
*      have a way to guarantee we have all the data for what is queried, which is only possible for name queries
*      and if we have neither non-frozen collections/UDTs nor counters (indeed, for a non-frozen collection or UDT,
*      we can't guarantee an older sstable won't have some elements that weren't in the most recent sstables,
*      and counters are intrinsically a collection of shards and so have the same problem).
*/
if (clusteringIndexFilter() instanceof ClusteringIndexNamesFilter && !queriesMulticellType())
return queryMemtableAndSSTablesInTimestampOrder(cfs, (ClusteringIndexNamesFilter)clusteringIndexFilter());
...
...

在这里,我们从这条评论中找到了答案:

We have 2 main strategies: 1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use........

Cassandra同时查询记忆表和马厩。

之后,如果我们跳入queryMemtableAndSSTablesInTimestampOrder方法,我们发现:

/**
* Do a read by querying the memtable(s) first, and then each relevant sstables sequentially by order of the sstable
* max timestamp.
*
* This is used for names query in the hope of only having to query the 1 or 2 most recent query and then knowing nothing
* more recent could be in the older sstables (which we can only guarantee if we know exactly which row we queries, and if
* no collection or counters are included).
* This method assumes the filter is a {@code ClusteringIndexNamesFilter}.
*/
private UnfilteredRowIterator queryMemtableAndSSTablesInTimestampOrder(ColumnFamilyStore cfs, ClusteringIndexNamesFilter filter)
{
Tracing.trace("Acquiring sstable references");
ColumnFamilyStore.ViewFragment view = cfs.select(View.select(SSTableSet.LIVE, partitionKey()));
ImmutableBTreePartition result = null;
Tracing.trace("Merging memtable contents");
.... // then it also looks into sstable on timestamp order.

从上面的部分中,我们已经找到了最后两个跟踪日志:

Acquiring sstable references [ReadStage-2]

Merging memtable contents [ReadStage-2]


希望这有帮助。

相关链接: 来源: 单分区读取命令.java

最新更新