Difference between revisions of "IO statistics"

From OpenVZ Virtuozzo Containers Wiki
Jump to: navigation, search
(created)
 
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page describes the IO statistics that is collected at the IO-scheduler level. It describes the information about the container's real work with disks.
+
This page describes the IO statistics that is collected at the IO-scheduler level. It describes the information about the container's real work with disks. This is different from what shown by [[IO accounting]].
  
 
== Kernel interface  ==
 
== Kernel interface  ==
  
The stats in question are reported via the proc files. Currently it is available in kernels starting from 028stab069.1.
+
The stats are reported via the proc files. Currently it is available in kernels starting from 028stab069.1.
  
 
=== Files ===
 
=== Files ===
  
* /proc/bc/$id/iostat  
+
* <code>/proc/bc/$id/iostat</code>
 
: statistics for beancounter $id
 
: statistics for beancounter $id
* /proc/bc/iostat  
+
* <code>/proc/bc/iostat</code>
 
: statistics for all beancounters
 
: statistics for all beancounters
  
Line 20: Line 20:
 
! N !! name !! type !! description
 
! N !! name !! type !! description
 
|-
 
|-
| 1 || disk || string || Disk device name, e.g. sda or hda
+
| 1 || disk || string || Disk device name, e.g. sda or hda, or a special queue (like fuse or flush)
 
|-
 
|-
 
| 2 || ub id || integer || Beancounter id
 
| 2 || ub id || integer || Beancounter id
 
|-
 
|-
| 3 || state || char || currently unused
+
| 3 || state || char || currently unused (always '.')
 
|-
 
|-
 
| 4 || busy queues || integer || The number of queues with requests (see below)
 
| 4 || busy queues || integer || The number of queues with requests (see below)
 
|-
 
|-
| 5 || on dispatch || integer || The number of requests dispatched to a hardware
+
| 5 || on dispatch || integer || currently unused (always '0')
 
|-
 
|-
| 6 || activations count || integer || The number of switches to active state
+
| 6 || activations count || integer || currently unused (always '0')
 
|-
 
|-
 
| 7 || wait time || integer || Total time in waiting state in milliseconds
 
| 7 || wait time || integer || Total time in waiting state in milliseconds
Line 41: Line 41:
 
|}
 
|}
  
New columns might be added at the end of row in future!
+
New columns might be added at the end of row in future.
 +
 
 +
Separate stats exist for fuse and flush, that only report requests and sectors stats (others are always 0).
 +
 
 +
Example of parsing code: parse_proc_iostat() function in [https://src.openvz.org/projects/OVZ/repos/vzstat/browse/src/vzstat.c vzstat.c]
 +
 
 +
=== I/O schedulers ===
 +
Check available/active I/O schedulers for block device "sda":
 +
# cat /sys/block/sda/queue/scheduler
 +
noop deadline [cfq]
 +
 
 +
* for "cfq" I/O scheduler: a separate block device line is added in iostat proc file
 +
# cat /proc/bc/100/iostat
 +
flush 100 . 0 0 0 0 0 7389 1893968 0 0
 +
fuse 100 . 0 0 0 0 0 0 0 0 0
 +
sda 100 . 0 0 0 9000 1843380 245216 55845488 245028 188
 +
 
 +
* for "deadline" I/O scheduler: no additional per-device line is added, iops counters for such devices are added to "flush" line counters (iops limit works)
 +
 
 +
* for "noop" I/O scheduler: iops are not counted (iops limit does not work)
 +
 
 +
* for devices with no I/O scheduler (like logical devices, ceph rbd devices, etc): iops are not counted (iops limit does not work)
 +
# cat /sys/block/dm-0/queue/scheduler
 +
none
 +
 
 +
# cat /sys/block/rbd0/queue/scheduler
 +
none
  
 
=== Queues ===
 
=== Queues ===
Line 60: Line 86:
  
 
  active  = sum(b.used_time - a.used_time) * 100 / interval;
 
  active  = sum(b.used_time - a.used_time) * 100 / interval;
  waiting = sum(b.wait_time - b.wait_time) * 100 / interval;
+
  waiting = sum(b.wait_time - a.wait_time) * 100 / interval;
 
  idle    = 100 - (active + waiting);
 
  idle    = 100 - (active + waiting);
  
Line 82: Line 108:
  
 
denotes the average request size for a beancounter to a particular disk.
 
denotes the average request size for a beancounter to a particular disk.
 +
 +
== See also ==
 +
* [[IO accounting]]
 +
* [[I/O priorities]]
 +
* [[I/O limits]]

Latest revision as of 10:52, 12 May 2017

This page describes the IO statistics that is collected at the IO-scheduler level. It describes the information about the container's real work with disks. This is different from what shown by IO accounting.

Kernel interface[edit]

The stats are reported via the proc files. Currently it is available in kernels starting from 028stab069.1.

Files[edit]

  • /proc/bc/$id/iostat
statistics for beancounter $id
  • /proc/bc/iostat
statistics for all beancounters

Format[edit]

Files contains one row for each disk-beancounter pair.

Columns are:

N name type description
1 disk string Disk device name, e.g. sda or hda, or a special queue (like fuse or flush)
2 ub id integer Beancounter id
3 state char currently unused (always '.')
4 busy queues integer The number of queues with requests (see below)
5 on dispatch integer currently unused (always '0')
6 activations count integer currently unused (always '0')
7 wait time integer Total time in waiting state in milliseconds
8 used time integer Total time in active state in milliseconds.
9 requests completed integer The number of completed requests
10 sectors transferred integer The number of 512 sectors transferred (includes both read and write)

New columns might be added at the end of row in future.

Separate stats exist for fuse and flush, that only report requests and sectors stats (others are always 0).

Example of parsing code: parse_proc_iostat() function in vzstat.c

I/O schedulers[edit]

Check available/active I/O schedulers for block device "sda":

# cat /sys/block/sda/queue/scheduler
noop deadline [cfq]
  • for "cfq" I/O scheduler: a separate block device line is added in iostat proc file
# cat /proc/bc/100/iostat
flush 100 . 0 0 0 0 0 7389 1893968 0 0
fuse 100 . 0 0 0 0 0 0 0 0 0
sda 100 . 0 0 0 9000 1843380 245216 55845488 245028 188
  • for "deadline" I/O scheduler: no additional per-device line is added, iops counters for such devices are added to "flush" line counters (iops limit works)
  • for "noop" I/O scheduler: iops are not counted (iops limit does not work)
  • for devices with no I/O scheduler (like logical devices, ceph rbd devices, etc): iops are not counted (iops limit does not work)
# cat /sys/block/dm-0/queue/scheduler
none
# cat /sys/block/rbd0/queue/scheduler
none

Queues[edit]

Each beancounter may have many queues with requests. Typically there's one queue for each task with synchronous (e.g. reads) requests and and the fixed amount of them for asynchronous requests (e.g. cached writes) for each beancounter.

Interpretation[edit]

Disk usage times[edit]

The disk usage should be reported in a top-like style. Consider the following code

read_iostat(&a);
sleep(interval);
read_iostat(&b);

Now the following numbers should be calculated and shown.

active  = sum(b.used_time - a.used_time) * 100 / interval;
waiting = sum(b.wait_time - a.wait_time) * 100 / interval;
idle    = 100 - (active + waiting);

The sum function sums up the times for all disk for the beancounter.

Additionally two more values should be shown for beancounter.

IO speed[edit]

The value

sum(b.transfered_sectors - a.transfered_sectors) * 512 / interval

denotes the speed of the IO performed by the beancounter.

Average request size[edit]

The value

(b.transfered_sectors - a.transfered_sectors)/(b.requests_completed - a.requests_completed)

denotes the average request size for a beancounter to a particular disk.

See also[edit]