用fio测试,这也是各种云所推荐的基准测试方法.
具体链接:https://help.aliyun.com/document_detail/25382.html
测试盘自身是NVME的,本身性能是超过树莓派USB 3.0 5Gbps瓶颈的,转接卡是自身10Gbps的,直接盘直写,盘自身性能不成问题.
测试主体是:
- 树莓派4 USB3.0
- 树莓派4 USB2.0
注:硬盘是MLC的,无缓存外掉速的说法,这是连接雷电后在电脑测速.
注:这个测试是会绕过缓存机制的.
本文主要是记录,不讲解任何东西.(怀疑树莓派上是假USB3.0)
USB3.0结果:
$ sudo fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Rand_Write_Testing
Rand_Write_Testing: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.16
Starting 1 process
Rand_Write_Testing: Laying out IO file (1 file / 10240MiB)
Jobs: 1 (f=1): [w(1)][100.0%][w=30.5MiB/s][w=7797 IOPS][eta 00m:00s]
Rand_Write_Testing: (groupid=0, jobs=1): err= 0: pid=1174: Wed Oct 16 03:02:41 2019
write: IOPS=7190, BW=28.1MiB/s (29.5MB/s)(10.0GiB/364574msec); 0 zone resets
slat (usec): min=35, max=23520, avg=121.39, stdev=88.87
clat (usec): min=128, max=84778, avg=17670.49, stdev=4219.25
lat (usec): min=183, max=84985, avg=17793.71, stdev=4243.86
clat percentiles (usec):
| 1.00th=[15008], 5.00th=[15270], 10.00th=[15401], 20.00th=[15664],
| 30.00th=[15926], 40.00th=[16188], 50.00th=[16581], 60.00th=[16909],
| 70.00th=[17171], 80.00th=[19006], 90.00th=[20055], 95.00th=[20579],
| 99.00th=[40109], 99.50th=[42206], 99.90th=[51119], 99.95th=[56361],
| 99.99th=[66323]
bw ( KiB/s): min=21568, max=33728, per=99.98%, avg=28755.88, stdev=2178.56, samples=729
iops : min= 5392, max= 8432, avg=7188.95, stdev=544.64, samples=729
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=90.55%, 50=9.34%
lat (msec) : 100=0.11%
cpu : usr=6.00%, sys=63.75%, ctx=4048040, majf=0, minf=20
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=0,2621440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
WRITE: bw=28.1MiB/s (29.5MB/s), 28.1MiB/s-28.1MiB/s (29.5MB/s-29.5MB/s), io=10.0GiB (10.7GB), run=364574-364574msec
Disk stats (read/write):
sda: ios=0/2631507, merge=0/354397, ticks=0/308972, in_queue=270180, util=60.53%
$ sudo fio -direct=1 -iodepth=128 -rw=randread -ioengine=libaio -bs=4k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Rand_Read_Testing
Rand_Read_Testing: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.16
Starting 1 process
Jobs: 1 (f=0): [f(1)][100.0%][r=73.7MiB/s][r=18.9k IOPS][eta 00m:00s]
Rand_Read_Testing: (groupid=0, jobs=1): err= 0: pid=1358: Wed Oct 16 03:05:23 2019
read: IOPS=20.4k, BW=79.5MiB/s (83.4MB/s)(10.0GiB/128801msec)
slat (usec): min=12, max=8880, avg=33.96, stdev= 9.70
clat (usec): min=159, max=17736, avg=6248.19, stdev=403.31
lat (usec): min=189, max=17769, avg=6283.19, stdev=405.50
clat percentiles (usec):
| 1.00th=[ 6128], 5.00th=[ 6128], 10.00th=[ 6128], 20.00th=[ 6194],
| 30.00th=[ 6194], 40.00th=[ 6194], 50.00th=[ 6194], 60.00th=[ 6194],
| 70.00th=[ 6194], 80.00th=[ 6194], 90.00th=[ 6259], 95.00th=[ 6259],
| 99.00th=[ 8979], 99.50th=[ 9110], 99.90th=[ 9241], 99.95th=[ 9241],
| 99.99th=[12125]
bw ( KiB/s): min=56192, max=82584, per=99.98%, avg=81396.35, stdev=4048.37, samples=257
iops : min=14048, max=20646, avg=20349.07, stdev=1012.09, samples=257
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=99.98%, 20=0.02%
cpu : usr=14.97%, sys=84.04%, ctx=43826, majf=0, minf=147
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
READ: bw=79.5MiB/s (83.4MB/s), 79.5MiB/s-79.5MiB/s (83.4MB/s-83.4MB/s), io=10.0GiB (10.7GB), run=128801-128801msec
Disk stats (read/write):
sda: ios=2618013/4, merge=0/1, ticks=498529/4, in_queue=472690, util=99.98%
$ sudo fio -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=1024k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Write_PPS_Testing
Write_PPS_Testing: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][w=225MiB/s][w=225 IOPS][eta 00m:00s]
Write_PPS_Testing: (groupid=0, jobs=1): err= 0: pid=975: Wed Oct 16 02:58:44 2019
write: IOPS=236, BW=236MiB/s (248MB/s)(10.0GiB/43307msec); 0 zone resets
slat (usec): min=1547, max=109358, avg=4204.03, stdev=3859.30
clat (msec): min=2, max=430, avg=265.76, stdev=31.34
lat (msec): min=14, max=433, avg=269.96, stdev=31.61
clat percentiles (msec):
| 1.00th=[ 215], 5.00th=[ 226], 10.00th=[ 232], 20.00th=[ 241],
| 30.00th=[ 249], 40.00th=[ 257], 50.00th=[ 264], 60.00th=[ 271],
| 70.00th=[ 279], 80.00th=[ 288], 90.00th=[ 300], 95.00th=[ 313],
| 99.00th=[ 351], 99.50th=[ 393], 99.90th=[ 418], 99.95th=[ 422],
| 99.99th=[ 430]
bw ( KiB/s): min=143360, max=287617, per=97.75%, avg=236678.31, stdev=25417.95, samples=86
iops : min= 140, max= 280, avg=230.94, stdev=24.77, samples=86
lat (msec) : 4=0.01%, 20=0.03%, 50=0.07%, 100=0.11%, 250=30.67%
lat (msec) : 500=69.11%
cpu : usr=4.63%, sys=45.93%, ctx=4405, majf=0, minf=22
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.3%, >=64=99.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=0,10240,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
WRITE: bw=236MiB/s (248MB/s), 236MiB/s-236MiB/s (248MB/s-248MB/s), io=10.0GiB (10.7GB), run=43307-43307msec
Disk stats (read/write):
sda: ios=4217/20430, merge=0/9, ticks=21280/172285, in_queue=192430, util=98.49%
$ sudo fio -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=1024k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Read_PPS_Testing
Read_PPS_Testing: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=1): [R(1)][97.1%][r=302MiB/s][r=302 IOPS][eta 00m:01s]
Read_PPS_Testing: (groupid=0, jobs=1): err= 0: pid=970: Wed Oct 16 03:12:45 2019
read: IOPS=303, BW=304MiB/s (318MB/s)(10.0GiB/33726msec)
slat (usec): min=711, max=12519, avg=3278.05, stdev=2466.73
clat (msec): min=3, max=314, avg=206.49, stdev=11.03
lat (msec): min=10, max=322, avg=209.77, stdev=11.09
clat percentiles (msec):
| 1.00th=[ 199], 5.00th=[ 201], 10.00th=[ 203], 20.00th=[ 205],
| 30.00th=[ 205], 40.00th=[ 207], 50.00th=[ 207], 60.00th=[ 207],
| 70.00th=[ 209], 80.00th=[ 211], 90.00th=[ 211], 95.00th=[ 213],
| 99.00th=[ 218], 99.50th=[ 228], 99.90th=[ 296], 99.95th=[ 305],
| 99.99th=[ 313]
bw ( KiB/s): min=110592, max=319488, per=99.18%, avg=308349.70, stdev=24759.81, samples=67
iops : min= 108, max= 312, avg=300.94, stdev=24.15, samples=67
lat (msec) : 4=0.01%, 20=0.04%, 50=0.10%, 100=0.16%, 250=99.36%
lat (msec) : 500=0.34%
cpu : usr=0.23%, sys=30.09%, ctx=6509, majf=0, minf=16408
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.3%, >=64=99.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=10240,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=304MiB/s (318MB/s), 304MiB/s-304MiB/s (318MB/s-318MB/s), io=10.0GiB (10.7GB), run=33726-33726msec
Disk stats (read/write):
sda: ios=26702/4, merge=0/1, ticks=119480/9, in_queue=114010, util=96.61%
USB2.0结果:
$ sudo fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Rand_Write_Testing
Rand_Write_Testing: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.16
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][w=16.6MiB/s][w=4250 IOPS][eta 00m:00s]
Rand_Write_Testing: (groupid=0, jobs=1): err= 0: pid=979: Wed Oct 16 04:53:20 2019
write: IOPS=3920, BW=15.3MiB/s (16.1MB/s)(10.0GiB/668718msec); 0 zone resets
slat (usec): min=7, max=12466, avg=234.53, stdev=552.68
clat (msec): min=14, max=107, avg=32.41, stdev= 4.95
lat (msec): min=14, max=107, avg=32.64, stdev= 4.98
clat percentiles (usec):
| 1.00th=[25560], 5.00th=[26346], 10.00th=[27132], 20.00th=[28181],
| 30.00th=[29754], 40.00th=[30802], 50.00th=[31851], 60.00th=[33162],
| 70.00th=[34341], 80.00th=[35390], 90.00th=[37487], 95.00th=[39060],
| 99.00th=[51643], 99.50th=[57410], 99.90th=[68682], 99.95th=[72877],
| 99.99th=[80217]
bw ( KiB/s): min=14560, max=18072, per=99.97%, avg=15676.07, stdev=288.64, samples=1337
iops : min= 3640, max= 4518, avg=3919.00, stdev=72.16, samples=1337
lat (msec) : 20=0.01%, 50=98.80%, 100=1.19%, 250=0.01%
cpu : usr=3.51%, sys=10.30%, ctx=542689, majf=0, minf=18
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=0,2621440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
WRITE: bw=15.3MiB/s (16.1MB/s), 15.3MiB/s-15.3MiB/s (16.1MB/s-16.1MB/s), io=10.0GiB (10.7GB), run=668718-668718msec
Disk stats (read/write):
sda: ios=63/2616582, merge=0/3831, ticks=108/37844595, in_queue=20044300, util=100.00%
$ sudo fio -direct=1 -iodepth=128 -rw=randread -ioengine=libaio -bs=4k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Rand_Read_Testing
Rand_Read_Testing: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.16
Starting 1 process
Jobs: 1 (f=0): [f(1)][100.0%][r=19.5MiB/s][r=5001 IOPS][eta 00m:00s]
Rand_Read_Testing: (groupid=0, jobs=1): err= 0: pid=1036: Wed Oct 16 05:07:03 2019
read: IOPS=4161, BW=16.3MiB/s (17.0MB/s)(10.0GiB/629880msec)
slat (usec): min=7, max=733172, avg=206.44, stdev=674.46
clat (msec): min=9, max=30841, avg=30.53, stdev=214.42
lat (msec): min=9, max=30841, avg=30.75, stdev=215.26
clat percentiles (usec):
| 1.00th=[16581], 5.00th=[19792], 10.00th=[23462], 20.00th=[25035],
| 30.00th=[26346], 40.00th=[27657], 50.00th=[28967], 60.00th=[30278],
| 70.00th=[31327], 80.00th=[32900], 90.00th=[34866], 95.00th=[36439],
| 99.00th=[47449], 99.50th=[52691], 99.90th=[63177], 99.95th=[67634],
| 99.99th=[77071]
bw ( KiB/s): min= 8248, max=25520, per=100.00%, avg=17490.84, stdev=2167.25, samples=1198
iops : min= 2062, max= 6380, avg=4372.70, stdev=541.82, samples=1198
lat (msec) : 10=0.01%, 20=5.38%, 50=93.91%, 100=0.70%, 250=0.01%
lat (msec) : >=2000=0.01%
cpu : usr=3.84%, sys=12.18%, ctx=668680, majf=0, minf=151
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
READ: bw=16.3MiB/s (17.0MB/s), 16.3MiB/s-16.3MiB/s (17.0MB/s-17.0MB/s), io=10.0GiB (10.7GB), run=629880-629880msec
Disk stats (read/write):
sda: ios=2616403/0, merge=3797/0, ticks=35782389/0, in_queue=17962220, util=95.18%
$ sudo fio -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=1024k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Write_PPS_Testing
Write_PPS_Testing: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=0): [f(1)][100.0%][w=63.0MiB/s][w=63 IOPS][eta 00m:00s]
Write_PPS_Testing: (groupid=0, jobs=1): err= 0: pid=1077: Wed Oct 16 05:13:26 2019
write: IOPS=28, BW=28.7MiB/s (30.1MB/s)(10.0GiB/357064msec); 0 zone resets
slat (usec): min=127, max=132153, avg=34743.75, stdev=54481.92
clat (msec): min=940, max=3140, avg=2194.86, stdev=61.38
lat (msec): min=1061, max=3140, avg=2229.61, stdev=82.20
clat percentiles (msec):
| 1.00th=[ 2123], 5.00th=[ 2198], 10.00th=[ 2198], 20.00th=[ 2198],
| 30.00th=[ 2198], 40.00th=[ 2198], 50.00th=[ 2198], 60.00th=[ 2198],
| 70.00th=[ 2198], 80.00th=[ 2198], 90.00th=[ 2198], 95.00th=[ 2198],
| 99.00th=[ 2198], 99.50th=[ 2265], 99.90th=[ 2769], 99.95th=[ 3004],
| 99.99th=[ 3138]
bw ( KiB/s): min= 8192, max=36864, per=99.82%, avg=29313.03, stdev=2288.16, samples=711
iops : min= 8, max= 36, avg=28.61, stdev= 2.23, samples=711
lat (msec) : 1000=0.01%, 2000=0.53%, >=2000=99.46%
cpu : usr=0.67%, sys=0.45%, ctx=3203, majf=0, minf=20
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.3%, >=64=99.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=0,10240,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
WRITE: bw=28.7MiB/s (30.1MB/s), 28.7MiB/s-28.7MiB/s (30.1MB/s-30.1MB/s), io=10.0GiB (10.7GB), run=357064-357064msec
Disk stats (read/write):
sda: ios=48/20472, merge=0/0, ticks=75/20299678, in_queue=10248960, util=95.73%
$ sudo fio -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=1024k -size=10G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sda -name=Read_PPS_Testing
Read_PPS_Testing: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=1): [R(1)][99.5%][r=26.0MiB/s][r=26 IOPS][eta 00m:02s]
Read_PPS_Testing: (groupid=0, jobs=1): err= 0: pid=1104: Wed Oct 16 05:20:03 2019
read: IOPS=28, BW=28.1MiB/s (29.4MB/s)(10.0GiB/364926msec)
slat (usec): min=173, max=382384, avg=32582.64, stdev=51367.62
clat (msec): min=897, max=33993, avg=2243.17, stdev=2382.57
lat (msec): min=1011, max=34221, avg=2278.70, stdev=2403.89
clat percentiles (msec):
| 1.00th=[ 1838], 5.00th=[ 2056], 10.00th=[ 2056], 20.00th=[ 2056],
| 30.00th=[ 2056], 40.00th=[ 2056], 50.00th=[ 2056], 60.00th=[ 2056],
| 70.00th=[ 2056], 80.00th=[ 2056], 90.00th=[ 2056], 95.00th=[ 2056],
| 99.00th=[ 2056], 99.50th=[17113], 99.90th=[17113], 99.95th=[17113],
| 99.99th=[17113]
bw ( KiB/s): min= 2048, max=36864, per=100.00%, avg=31200.17, stdev=3897.42, samples=668
iops : min= 2, max= 36, avg=30.45, stdev= 3.80, samples=668
lat (msec) : 1000=0.01%, 2000=1.51%, >=2000=98.48%
cpu : usr=0.06%, sys=0.63%, ctx=3106, majf=0, minf=16408
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.3%, >=64=99.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=10240,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=28.1MiB/s (29.4MB/s), 28.1MiB/s-28.1MiB/s (29.4MB/s-29.4MB/s), io=10.0GiB (10.7GB), run=364926-364926msec
Disk stats (read/write):
sda: ios=20477/0, merge=0/0, ticks=20760164/0, in_queue=9607270, util=87.85%