卡桑德拉(Cassandra)长时间查询时间,并在钥匙完全限制时增加记忆



我有一个卡桑德拉表,钥匙看起来像这样:

主键((" K1"," K2")," C1"," C2"),带有聚类顺序 (" C1" desc," C2" desc);

当我完全限制查询时,它比忽略最后一个聚类键的时间要长得多。它还预示了"添加到可记忆的添加",而无约束的查询不会。为什么是这样?我以前知道此查询不会将条目添加到可记忆的内容中,因为当我将物品添加到记忆中时,我的自定义代码运行。此代码只有在插入或修改的内容时才能运行,但在我只查询项目时就开始运行。

编辑:我应该提到两个查询返回1行,这是相同的记录。

  activity                                                                                                                                                                          | timestamp                  | source        | source_elapsed | client
 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+------------
                                                                                                                                                                 Execute CQL3 query | 2017-09-05 18:09:37.456000 | **.***.**.237 |              0 | ***.**.*.4
                                              Parsing select c2 from feed where k1 = 'AAA' and k2 = 'BBB' and c1 = '2017-09-05T16:09:00.222Z' and c2 = 'CCC'; [SharedPool-Worker-1] | 2017-09-05 18:09:37.456000 | **.***.**.237 |            267 | ***.**.*.4
                                                                                                                                          Preparing statement [SharedPool-Worker-1] | 2017-09-05 18:09:37.456000 | **.***.**.237 |            452 | ***.**.*.4
                                                                                                                     Executing single-partition query on feed [SharedPool-Worker-3] | 2017-09-05 18:09:37.457000 | **.***.**.237 |           1253 | ***.**.*.4
                                                                                                                                 Acquiring sstable references [SharedPool-Worker-3] | 2017-09-05 18:09:37.457000 | **.***.**.237 |           1312 | ***.**.*.4
                                                                                                                                    Merging memtable contents [SharedPool-Worker-3] | 2017-09-05 18:09:37.457000 | **.***.**.237 |           1370 | ***.**.*.4
                                                                                                                                 Key cache hit for sstable 22 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           6939 | ***.**.*.4
                                                                                                                                 Key cache hit for sstable 21 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           7077 | ***.**.*.4
                                                                                                                                 Key cache hit for sstable 12 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           7137 | ***.**.*.4
                                                                                                                                  Key cache hit for sstable 6 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           7194 | ***.**.*.4
                                                                                                                                  Key cache hit for sstable 3 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           7249 | ***.**.*.4
                                                                                                                                 Merging data from sstable 10 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463000 | **.***.**.237 |           7362 | ***.**.*.4
                                                                                                                                 Key cache hit for sstable 10 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463001 | **.***.**.237 |           7429 | ***.**.*.4
                                                                                                                                  Key cache hit for sstable 9 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463001 | **.***.**.237 |           7489 | ***.**.*.4
                                                                                                                                  Key cache hit for sstable 4 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463001 | **.***.**.237 |           7628 | ***.**.*.4
                                                                                                                                  Key cache hit for sstable 7 [SharedPool-Worker-3] | 2017-09-05 18:09:37.463001 | **.***.**.237 |           7720 | ***.**.*.4
                                                                                                                                 Defragmenting requested data [SharedPool-Worker-3] | 2017-09-05 18:09:37.463001 | **.***.**.237 |           7779 | ***.**.*.4
                                                                                                                                      Adding to feed memtable [SharedPool-Worker-4] | 2017-09-05 18:09:37.464000 | **.***.**.237 |           7896 | ***.**.*.4
                                                                                                                            Read 1 live and 4 tombstone cells [SharedPool-Worker-3] | 2017-09-05 18:09:37.464000 | **.***.**.237 |           7932 | ***.**.*.4
                                                                                                                                                                   Request complete | 2017-09-05 18:09:37.464092 | **.***.**.237 |           8092 | ***.**.*.4
activity                                                                                                                                              | timestamp                  | source        | source_elapsed | client
-------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+------------
                                                                                                                                    Execute CQL3 query | 2017-09-05 18:09:44.703000 | **.***.**.237 |              0 | ***.**.*.4
                                Parsing select c2 from feed where k1 = 'AAA' and k2 = 'BBB' and c1 = '2017-09-05T16:09:00.222Z'; [SharedPool-Worker-1] | 2017-09-05 18:09:44.704000 | **.***.**.237 |            508 | ***.**.*.4
                                                                                                             Preparing statement [SharedPool-Worker-1] | 2017-09-05 18:09:44.704000 | **.***.**.237 |            717 | ***.**.*.4
                                                                                        Executing single-partition query on feed [SharedPool-Worker-2] | 2017-09-05 18:09:44.704000 | **.***.**.237 |           1377 | ***.**.*.4
                                                                                                    Acquiring sstable references [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1499 | ***.**.*.4
                                                                                                    Key cache hit for sstable 10 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1730 | ***.**.*.4
                                                       Skipped 8/9 non-slice-intersecting sstables, included 5 due to tombstones [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1804 | ***.**.*.4
                                                                                                    Key cache hit for sstable 22 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1858 | ***.**.*.4
                                                                                                    Key cache hit for sstable 21 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1908 | ***.**.*.4
                                                                                                    Key cache hit for sstable 12 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705000 | **.***.**.237 |           1951 | ***.**.*.4
                                                                                                     Key cache hit for sstable 6 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705001 | **.***.**.237 |           2002 | ***.**.*.4
                                                                                                     Key cache hit for sstable 3 [SharedPool-Worker-2] | 2017-09-05 18:09:44.705001 | **.***.**.237 |           2037 | ***.**.*.4
                                                                                       Merged data from memtables and 6 sstables [SharedPool-Worker-2] | 2017-09-05 18:09:44.705001 | **.***.**.237 |           2252 | ***.**.*.4
                                                                                               Read 1 live and 4 tombstone cells [SharedPool-Worker-2] | 2017-09-05 18:09:44.705001 | **.***.**.237 |           2307 | ***.**.*.4
                                                                                                                                      Request complete | 2017-09-05 18:09:44.705458 | **.***.**.237 |           2458 | ***.**.*.4
cqlsh> show version [cqlsh 5.0.1 | Cassandra 3.7 | CQL spec 3.4.2 |
Native protocol v4]

这是一个很好的问题,您(有益地)提供了我们需要回答的所有信息!

您的第一个查询是一个查找(因为您指定了两个群集键)。第二个是切片。

如果我们看迹线,则痕迹的明显区别是:

Skipped 8/9 non-slice-intersecting sstables, included 5 due to tombstones

这是一个很好的暗示,我们正在采用两种不同的阅读路径。您可以使用它来编码潜水,但是长话短说,您用于观点读取的过滤器意味着您会以不同的顺序查询可记忆的/sstables-对于点读取,我们按时间戳进行排序,为了切片,我们会尝试首先消除非交流的Sstables。

代码中的注释提示 - 第一个:

/**
 * Do a read by querying the memtable(s) first, and then each relevant sstables sequentially by order of the sstable
 * max timestamp.
 *
 * This is used for names query in the hope of only having to query the 1 or 2 most recent query and then knowing nothing
 * more recent could be in the older sstables (which we can only guarantee if we know exactly which row we queries, and if
 * no collection or counters are included).
 * This method assumes the filter is a {@code ClusteringIndexNamesFilter}.
 */

和第二个:

    /*
     * We have 2 main strategies:
     *   1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use
     *      unless we have a names filter that we know we can optimize futher.
     *   2) If we have a name filter (so we query specific rows), we can make a bet: that all column for all queried row
     *      will have data in the most recent sstable(s), thus saving us from reading older ones. This does imply we
     *      have a way to guarantee we have all the data for what is queried, which is only possible for name queries
     *      and if we have neither collections nor counters (indeed, for a collection, we can't guarantee an older sstable
     *      won't have some elements that weren't in the most recent sstables, and counters are intrinsically a collection
     *      of shards so have the same problem).
     */

在您的情况下,如果返回的行恰好在记忆中,则第一个(点)读取将更快。另外,由于您有8个Sstables,因此您可能使用STC或TWC-如果使用LCS,则可能会将该分区压缩到〜5个Sstables中,并且(再次)具有更可预测的读取性能。

我以前知道此查询不会将条目添加到可记忆的内容中,因为当我将物品添加到记忆中时,我的自定义代码运行。此代码只有在插入或修改的内容时才能运行,但仅在我只查询项目时就开始运行。

否则默认情况下,

都不应将任何内容添加到记忆中,除非您正在阅读修复(也就是说,除非副本之间的值不匹配或背景读取修复机会是触发的)。请注意,切片查询比点查询更可能不匹配,因为它是基于扫描的 - 您将读取任何/所有/所有删除标记(墓碑),具有c1 = '2017-09-05T16:09:00.222Z'

的匹配值

编辑:我错过了跟踪中的一行:

Defragmenting requested data

这表明您使用的是STC并触摸了太多的Sstables,因此将整个分区复制到可记忆中,以使未来的读数更快。这是STC中鲜为人知的优化,当您开始触摸太多的Sstables时,您可以使用LCS围绕它进行工作。

您将苹果与橙色进行比较

  • 您要求所有行匹配条件的第一个查询 k1 = 'AAA' and k2 = 'BBB' and c1 = '2017-09-05T16:09:00.222Z' and c2 = 'CCC' 这里的额外条件是C2 ='CCC',因此Cassandra需要做更多的工作来返回与这些条件相匹配的行。

  • 在第二个查询中,您正在放松C2上匹配的条件,因此您可以看到不同的性能行为。

假设您有1000行与条件K1 ='AAA'和K2 ='BBB'和C1 ='2017-09-05T16:09:00.2222Z'。添加C2的条件可能仅返回4行(可能需要检查所有行中的所有行中的所有行),其中删除条件将在匹配K1,K2和C1时开始流式传输结果。

  • 如果您真的想比较,则可以比较
  • 之间的性能

k1 = 'AAA' and k2 = 'BBB' and c1 = '2017-09-05T16:09:00.222Z' and c2 = 'CCC' OR k1 = 'AAA' and k2 = 'BBB' and c1 = '2017-09-05T16:09:00.222Z' and c2 = 'XXX'

检查性能时,您需要多次运行相同的查询以避免任何缓存行为。

最新更新