有关Cassandra压力测试用户模式的备忘录
0. 首先
・本条款中介绍的是关于Cassandra-stress的内容，基于分发版本和版本号 = DSE 4.8.16进行撰写。
・由于本说明已经过时，并且存在更多子命令和选项，因此请详细参阅Apache或DataStax的指南。
1. 用户模式概览
通过使用用户模式，可以对任意表格和针对该表格的任意查询进行压力测试。
2. yaml文件定义
yaml文件的定义说明
键空间: 键空间定义。
只需一个选项，将以下内容用中文进行释义：
– 键空间指定
– 在CREATE KEYSPACE中定义键空间
※如果要使用已经创建的键空间，则可以省略键空间定义。
<定义说明示例>
keyspace: stest01
keyspace_definition:
CREATE KEYSPACE stest01 WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1': 3};
②表格：表格定义：
・表名的定义
・使用CREATE TABLE进行表的定义
※若已经创建好的表，则可以省略表的定义
<定义的示例>
table: test01
table_definition: |
CREATE TABLE test01 (
name text,
date timestamp,
address inet,
ino int,
iname text,
uid timeuuid,
PRIMARY KEY(name, date)
) WITH compaction = { 'class':'SizeTieredCompactionStrategy' }
列规范:
指定表的列字段长度、值的范围以及每个分区的聚类行数的定制化。
< 定義的示例記述 >
columnspec:
- name: name
size: uniform(1..20)
- name: date
cluster: fixed(1)
- name: address
population: exp(1..4)
- name: ino
population: exp(1..1000)
- name: iname
size: fixed(8)
population: uniform(1..5)
< 数据模式指定项目 >
パラメタ意味デフォルトsizetext,blob,set,listのcolumnのフィールドの長さを定義、例えば、fixed(5)指定すると、固定で長さ5になるし、uniform(1..10)とすると長さ1～10の分布となる。UNIFORM(4..8)population列の値が変動する範囲を定義、テキスト項目だと母集団の数の指定になるし、数値的な項目だと指定した範囲の値となる。例えば、text型でuniform(1..10)とすると値の母集団10とおりとなるし、int型でfixed(10)だと値がすべて10になる。UNIFORM(1..100B)cluster各partitionごとのclustering keyの数を制御。例えば、fixed(4)指定すると、partitionごとにclustering key 4レコードずつ生成FIXED(1)
< 配置指定 >
插入
指定应该在执行压力测试时插入数据的方法。例如，如果指定为fixed(10)，那么将执行插入固定行数的操作。
<定义的记载示例>
insert:
partitions: fixed(10)
batchtype: UNLOGGED
指定项目
指定意味デフォルトPartitionsinsertするパーティションの数FIXED(1)Batch type使用するCQLバッチのタイプ UNLOGGED指定でパフォーマンスを向上可能LOGGED
⑤ 询问：
・指定CQL查询中的表
・fields指定是从绑定变量所在行选择还是从分区内的所有行选择
<定义的示例描述>
queries:
select1:
cql: select address from test01 where name = ? and date = ?
fields: samerow
3. 如何运行
・由于指定了YAML文件，需要指定user选项，并使用profile =来指定yaml文件。
・通过ops指定要执行的操作以及每个操作的操作数。※在”(“和”)”处需要使用转义斜杠。
・可以使用duration=和n=来指定执行时间和操作次数。
< 副选项，附加参数的一部分 >
パラメタパラメタ種別意味デフォルト備考-rateサブオプションthreads=?とかthreads<=?とかで実行スレッド数を指定4スレッドから開始し、スループットが飽和するまでスレッドを増加して実行していく(最大1000まで)・「4.実行例」を参照profile=?追加パラメタuserオプション指定時、YAMLファイルを指定-
truncate=?追加パラメタ作成したテーブルをtruncateするnever(once,alwaysの選択可)
duration=?追加パラメタ1回あたりの実行時間を指定 秒/分/時間で指定-・n=と同時指定はできないn=?追加パラメタ実行する操作回数-・durationと同時指定はできない。例えばn=100でinsert=3,simpe1=1なら、insert:75回、simple1操作:25回実行n>?追加パラメタ不確定収束を受け入れる前に実行する最小実行回数を指定-・durationと同時指定はできないn<?追加パラメタ不確定収束を受け入れる前に実行する最大実行回数を指定-・durationと同時指定はできない
truncate=?追加パラメタ作成したテーブルをtruncateするnever(once,alwaysの選択可)
duration=?追加パラメタ1回あたりの実行時間を指定 秒/分/時間で指定-・n=と同時指定はできないn=?追加パラメタ実行する操作回数-・durationと同時指定はできない。例えばn=100でinsert=3,simpe1=1なら、insert:75回、simple1操作:25回実行n>?追加パラメタ不確定収束を受け入れる前に実行する最小実行回数を指定-・durationと同時指定はできないn<?追加パラメタ不確定収束を受け入れる前に実行する最大実行回数を指定-・durationと同時指定はできない
4. 实际执行的例子
将100次插入操作固定在一个线程中执行。
# cassandra-stress user profile=stest02.yaml ops\(insert=1\) n=100 -rate threads=1
Connected to cluster: Test Cluster
Datatacenter: DC1; Host: localhost/127.0.0.1; Rack: RAC1
Created schema. Sleeping 1s for propagation.
Sleeping 2s...
Running [insert] with 1 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Generating batches with [5..5] partitions and [20..20] rows (of [20..20] total rows in the partitions)
total, 20, 90, 452, 1809, 10.5, 8.1, 21.0, 21.0, 21.0, 21.0, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 90 [insert:90]
partition rate : 452 [insert:452]
row rate : 1809 [insert:1809]
latency mean : 10.5 [insert:10.5]
latency median : 8.1 [insert:8.1]
latency 95th percentile : 21.0 [insert:21.0]
latency 99th percentile : 21.0 [insert:21.0]
latency 99.9th percentile : 21.0 [insert:21.0]
latency max : 21.0 [insert:21.0]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
END
#
在增加线程条件的情况下，执行100个插入操作，最多可达到16个线程。
# cassandra-stress user profile=stest02.yaml ops\(insert=1\) n=100 -rate threads\<=16
Connected to cluster: Test Cluster
Datatacenter: DC1; Host: localhost/127.0.0.1; Rack: RAC1
Created schema. Sleeping 1s for propagation.
Sleeping 2s...
Running with 4 threadCount
Running [insert] with 4 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Generating batches with [5..5] partitions and [20..20] rows (of [20..20] total rows in the partitions)
total, 20, 95, 477, 1906, 39.4, 31.7, 125.7, 125.7, 125.7, 125.7, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 95 [insert:95]
partition rate : 477 [insert:477]
row rate : 1906 [insert:1906]
latency mean : 39.4 [insert:39.4]
latency median : 31.7 [insert:31.7]
latency 95th percentile : 125.7 [insert:125.7]
latency 99th percentile : 125.7 [insert:125.7]
latency 99.9th percentile : 125.7 [insert:125.7]
latency max : 125.7 [insert:125.7]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Sleeping for 15s
Running with 8 threadCount
Running [insert] with 8 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
total, 20, 247, 1234, 4937, 51.8, 53.0, 78.6, 78.6, 78.6, 78.6, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 247 [insert:247]
partition rate : 1234 [insert:1234]
row rate : 4937 [insert:4937]
latency mean : 51.8 [insert:51.8]
latency median : 53.0 [insert:53.0]
latency 95th percentile : 78.6 [insert:78.6]
latency 99th percentile : 78.6 [insert:78.6]
latency 99.9th percentile : 78.6 [insert:78.6]
latency max : 78.6 [insert:78.6]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 4 threadCount: 159%
Sleeping for 15s
Running with 16 threadCount
Running [insert] with 16 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
total, 20, 208, 1042, 4166, 109.1, 118.0, 148.4, 148.4, 148.4, 148.4, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 208 [insert:208]
partition rate : 1042 [insert:1042]
row rate : 4166 [insert:4166]
latency mean : 109.1 [insert:109.1]
latency median : 118.0 [insert:118.0]
latency 95th percentile : 148.4 [insert:148.4]
latency 99th percentile : 148.4 [insert:148.4]
latency 99.9th percentile : 148.4 [insert:148.4]
latency max : 148.4 [insert:148.4]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 8 threadCount: -15%
id, type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
4 threadCount, insert, 20, 95, 477, 1906, 39.4, 31.7, 125.7, 125.7, 125.7, 125.7, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
4 threadCount, total, 20, 95, 477, 1906, 39.4, 31.7, 125.7, 125.7, 125.7, 125.7, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
8 threadCount, insert, 20, 247, 1234, 4937, 51.8, 53.0, 78.6, 78.6, 78.6, 78.6, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
8 threadCount, total, 20, 247, 1234, 4937, 51.8, 53.0, 78.6, 78.6, 78.6, 78.6, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
16 threadCount, insert, 20, 208, 1042, 4166, 109.1, 118.0, 148.4, 148.4, 148.4, 148.4, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
16 threadCount, total, 20, 208, 1042, 4166, 109.1, 118.0, 148.4, 148.4, 148.4, 148.4, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
END
#
②插入100次执行线程条件递增执行（直至饱和）。
可能与“rate auto指定一样”
# cassandra-stress user profile=stest02.yaml ops\(insert=1\) n=100
Connected to cluster: Test Cluster
Datatacenter: DC1; Host: localhost/127.0.0.1; Rack: RAC1
Created schema. Sleeping 1s for propagation.
Sleeping 2s...
Running with 4 threadCount
Running [insert] with 4 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Generating batches with [5..5] partitions and [20..20] rows (of [20..20] total rows in the partitions)
total, 20, 97, 486, 1943, 38.0, 24.9, 150.4, 150.4, 150.4, 150.4, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 97 [insert:97]
partition rate : 486 [insert:486]
row rate : 1943 [insert:1943]
latency mean : 38.0 [insert:38.0]
latency median : 24.9 [insert:24.9]
latency 95th percentile : 150.4 [insert:150.4]
latency 99th percentile : 150.4 [insert:150.4]
latency 99.9th percentile : 150.4 [insert:150.4]
latency max : 150.4 [insert:150.4]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Sleeping for 15s
Running with 8 threadCount
Running [insert] with 8 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
total, 20, 263, 1317, 5270, 44.3, 47.8, 75.5, 75.5, 75.5, 75.5, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 263 [insert:263]
partition rate : 1317 [insert:1317]
row rate : 5270 [insert:5270]
latency mean : 44.3 [insert:44.3]
latency median : 47.8 [insert:47.8]
latency 95th percentile : 75.5 [insert:75.5]
latency 99th percentile : 75.5 [insert:75.5]
latency 99.9th percentile : 75.5 [insert:75.5]
latency max : 75.5 [insert:75.5]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 4 threadCount: 171%
Sleeping for 15s
Running with 16 threadCount
Running [insert] with 16 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
total, 20, 259, 1296, 5185, 83.1, 85.4, 122.9, 122.9, 122.9, 122.9, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 259 [insert:259]
partition rate : 1296 [insert:1296]
row rate : 5185 [insert:5185]
latency mean : 83.1 [insert:83.1]
latency median : 85.4 [insert:85.4]
latency 95th percentile : 122.9 [insert:122.9]
latency 99th percentile : 122.9 [insert:122.9]
latency 99.9th percentile : 122.9 [insert:122.9]
latency max : 122.9 [insert:122.9]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 8 threadCount: -2%
Sleeping for 15s
Running with 24 threadCount
Running [insert] with 24 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
total, 20, 4137, 20685, 82738, 134.0, 132.2, 159.2, 159.2, 159.2, 159.2, 0.0, -0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 4137 [insert:4137]
partition rate : 20685 [insert:20685]
row rate : 82738 [insert:82738]
latency mean : 134.0 [insert:134.0]
latency median : 132.2 [insert:132.2]
latency 95th percentile : 159.2 [insert:159.2]
latency 99th percentile : 159.2 [insert:159.2]
latency 99.9th percentile : 159.2 [insert:159.2]
latency max : 159.2 [insert:159.2]
Total partitions : 100 [insert:100]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 16 threadCount: 1497%
Sleeping for 15s
Running with 36 threadCount
Running [insert] with 36 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Results:
op rate : NaN [insert:NaN]
partition rate : NaN [insert:NaN]
row rate : NaN [insert:NaN]
latency mean : NaN [insert:NaN]
latency median : 0.0 [insert:0.0]
latency 95th percentile : 0.0 [insert:0.0]
latency 99th percentile : 0.0 [insert:0.0]
latency 99.9th percentile : 0.0 [insert:0.0]
latency max : 0.0 [insert:0.0]
Total partitions : 0 [insert:0]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 24 threadCount: -100%
Sleeping for 15s
Running with 54 threadCount
Running [insert] with 54 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Results:
op rate : NaN [insert:NaN]
partition rate : NaN [insert:NaN]
row rate : NaN [insert:NaN]
latency mean : NaN [insert:NaN]
latency median : 0.0 [insert:0.0]
latency 95th percentile : 0.0 [insert:0.0]
latency 99th percentile : 0.0 [insert:0.0]
latency 99.9th percentile : 0.0 [insert:0.0]
latency max : 0.0 [insert:0.0]
Total partitions : 0 [insert:0]
Total errors : 0 [insert:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
Improvement over 36 threadCount: NaN%
Sleeping for 15s
id, type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
4 threadCount, insert, 20, 97, 486, 1943, 38.0, 24.9, 150.4, 150.4, 150.4, 150.4, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
4 threadCount, total, 20, 97, 486, 1943, 38.0, 24.9, 150.4, 150.4, 150.4, 150.4, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
8 threadCount, insert, 20, 263, 1317, 5270, 44.3, 47.8, 75.5, 75.5, 75.5, 75.5, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
8 threadCount, total, 20, 263, 1317, 5270, 44.3, 47.8, 75.5, 75.5, 75.5, 75.5, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
16 threadCount, insert, 20, 259, 1296, 5185, 83.1, 85.4, 122.9, 122.9, 122.9, 122.9, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
16 threadCount, total, 20, 259, 1296, 5185, 83.1, 85.4, 122.9, 122.9, 122.9, 122.9, 0.1, 0.00000, 0, 0, 0, 0, 0, 0
24 threadCount, insert, 20, 4137, 20685, 82738, 134.0, 132.2, 159.2, 159.2, 159.2, 159.2, 0.0, -0.00000, 0, 0, 0, 0, 0, 0
24 threadCount, total, 20, 4137, 20685, 82738, 134.0, 132.2, 159.2, 159.2, 159.2, 159.2, 0.0, -0.00000, 0, 0, 0, 0, 0, 0
36 threadCount, insert, 0, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00000, 0, 0, 0, 0, 0, 0
36 threadCount, total, 0, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00000, 0, 0, 0, 0, 0, 0
54 threadCount, insert, 0, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00000, 0, 0, 0, 0, 0, 0
54 threadCount, total, 0, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00000, 0, 0, 0, 0, 0, 0
END
[root@cs7ds4001 tools]#
用混合插入和查询的方式执行
以指定数量的线程重复执行总共100次。
# cassandra-stress user profile=stest02.yaml ops\(insert=1,select1=1\) n=100 -rate threads=1
Connected to cluster: Test Cluster
Datatacenter: DC1; Host: localhost/127.0.0.1; Rack: RAC1
Created schema. Sleeping 1s for propagation.
Sleeping 2s...
Running [insert, select1] with 1 threads for 100 iteration
type, total ops, op/s, pk/s, row/s, mean, med, .95, .99, .999, max, time, stderr, errors, gc: #, max ms, sum ms, sdv ms, mb
Generating batches with [5..5] partitions and [20..20] rows (of [20..20] total rows in the partitions)
insert, 17, 73, 366, 1465, 9.7, 8.2, 22.3, 22.3, 22.3, 22.3, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
select1, 15, 83, 83, 83, 3.7, 2.7, 11.3, 11.3, 11.3, 11.3, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
total, 32, 138, 431, 1529, 6.9, 5.7, 15.7, 22.3, 22.3, 22.3, 0.2, 0.00000, 0, 0, 0, 0, 0, 0
Results:
op rate : 138 [insert:73, select1:83]
partition rate : 431 [insert:366, select1:83]
row rate : 1529 [insert:1465, select1:83]
latency mean : 6.9 [insert:9.7, select1:3.7]
latency median : 5.7 [insert:8.2, select1:2.7]
latency 95th percentile : 15.7 [insert:22.3, select1:11.3]
latency 99th percentile : 22.3 [insert:22.3, select1:11.3]
latency 99.9th percentile : 22.3 [insert:22.3, select1:11.3]
latency max : 22.3 [insert:22.3, select1:11.3]
Total partitions : 100 [insert:85, select1:15]
Total errors : 0 [insert:0, select1:0]
total gc count : 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:00:00
END
#
