6,534
edits
Changes
→Other reasons of strange numbers: +ext4
The aim of this article is to understand where the numbers that are shown by <code>stat</code>/<code>df</code> utils in [[VEcontainer]] come from.
Consider typical OpenVZ setup, where <code>ext2</code> separate file system is mounted on <code>/vz</code>. <code>ext2</code> is called ''underlying file system'' in such situation. Linux VFS design allows every filesystem file system to export to userspace user space the following information concerning disk space (here and further I use <math>{subscript }</math> is used to specify paricular filesystem particular file system type):
* <math>total_{ext2}</math> - total amount of disk space that potentially can be acquired (e.g. HDD capacity)
* <math>avail_{ext2}</math> - amount of disk space that is still available for non-root users
Note, that not all free blocks can be used by non-root users: some amount of disk space is reserved for root. For example on <code>ext2</code> filesystem file system only root can use last free 5 percent (by default) of disk space. This is the difference between <math>avail_{ext2}</math> and <math>free_{ext2}</math>. Also marknote, that the following inequality is always true:
: <math>avail_{ext2} \le free_{ext2}</math> (1)
Inside a [[VEcontainer]] , a special filesystem file system type is used: , called <code>simfs</code>. This filesystem file system allows to isolate a particular [[VECT]] from other [[VE]]sCTs. Hence, when <code>df</code> or <code>stat</code> utils utilities are invoked , they get information from <code>simfs</code>, which exports to them the following values (by analogy with <code>ext2</code>):
* <math>total_{simfs}</math>
* <math>avail_{simfs}</math>
This article is in fact devoted to how simfs filesystem file system calculates the values above.
To produce any calculations , input data are required. What are input data for <code>simfs</code>? Except Aside from already discussed mentioned information from underlying filesystem file system (<math>total_{ext2}</math>, <math>total_{ext2}</math>, <math>total_{ext2}</math>) , one more element comes into force in OpenVZ environment. It is OpenVZ ''per-VE container disk quotas''. The values that provide this element are:
* <math>quota_{used}</math> - the number of blocks currently used by a [[VECT]]* <math>quota_{barrier}</math> - the number of blocks this VE CT can potentially can obtain.
OpenVZ disk quota counts the number of blocks currently used by VE a CT and prevents this number to be greater than the limit/barrier set.
<!-- TODO: Uncoment Uncomment after adding examples.
First let's use
<pre>
command to get information we need, and later discuss df output.
-->
:: <math>total_{simfs}</math> = <math>total_{ext2}</math>
:: <math>free_{simfs}</math> = <math>free_{ext2}</math>
:: <math>avail_{simfs} = avail_{ext2}</math>
:: <math>total_{simfs} = quota_{barrier}</math>
: Amount The amount of free space in this case should logically be the following:
:: <math>free_{simfs} = quota_{barrier} - quota_{used}</math>
: However here is a pitfall. Suppose that the amount of free disk space actually on the underlying filesystem is less than it is estimated from quota using the formule formula above, i.e.::: <math>free_{ext2} < quota_{barrier} - quota_{used} > free_{ext2}</math>: Then, definitely, the amount of free disk space reported by <code>simfs </code> should be other! different. This situation will be considered in the next point and in this point examined later; here we assume that there is enough space on partition, i.e:: <math>free_{ext2} \ge quota_{barrier} - quota_{used} \le free_{ext2}</math> (2): As concerns for amount of disk space available for non-root users, if there is enough disk space::: <math>avail_{ext2} \ge quota_{barrier} - quota_{used} \le avail_{ext2}</math>: and this is right in current point due to assumption (2) and the inequality (1), then amount of disk space available for non-root users in [[VE]] a CT equals the free space estimated from quota:
:: <math>free_{simfs} = quota_{barrier} - quota_{used}</math>
{{Clear}}
The table below summarize all same reason as with <math>free_{simfs}</math> fits for calculating <math>avail_simfs</math>. Two cases are possible cases.If: <math>avail_{ext2} \ge quota_{barrier} - quota_{used}</math>then : <math>avail_{simfs} = free_{ext2}</math>and if: <math>avail_{ext2} < quota_{barrier} - quota_{used}</math>then: <math>avail_{simfs} = quota_{barrier} - quota_{used}</math>
== Cases Conclusion ==So there are three basic variants. Variant number one is not good, because a container's administrator can not get information about CT disk usage and the [[host system]] administrator can't limit CT disk usage. Variant three is not good because we have some weird (but logical) values in <mathcode>Sdf</mathcode> - /<code>stat</code> output in CT, e.g. total amount of disk space on underlying filesystemcan decrease. Variant two is perfect. How can we make sure that this variant always take place? Here is the simple rule:
Note that if you install a template, you decrease <math>s</math>. This is bad because, ideally, after each template
installation you have to check inequality (5). To avoid this I suggest to mount separate partition on <code>/vz/private</code>, rather than on <code>/vz</code>. In such case <math>s</math> always equals <math>0</math>.
== Cases Summarizing Table ==
{| class="wikitable" align="center"
| colspan="2" | Quota off
| <math>total_{simfs} = total_{ext2}</math>
<math>free_{simfs} = free_{ext2}</math>
<math>avail_{simfs} = avail_{ext2}</math>
|-
| rowspan = 3 | Quota on
| <math>free_{ext2} \ge quota_{limit} - quota_{used}</math>
<math>avail_{ext2} \ge quota_{limit} - quota_{used}</math>
| <math>total_{simfs} = quota_{barrier}</math>
<math>free_{simfs} = quota_{limit} - quota_{used}</math>
<math>avail_{simfs} = quota_{limit} - quota_{used}</math>
|-
| <math>free_{ext2} \ge quota_{limit} - quota_{used}</math>
<math>avail_{ext2} \le quota_{limit} - quota_{used}</math>
| <math>total_{simfs} = quota_{barrier}</math>
<math>free_{simfs} = quota_{limit} - quota_{used}</math>
<math>avail_{simfs} = avail_{ext2}</math>
|-
| <math>free_{ext2} \le quota_{limit} - quota_{used}</math>
<math>avail_{ext2} \le quota_{limit} - quota_{used}</math>
| <math>total_{simfs} = quota_{usage} + free_{ext2}</math>
<math>free_{simfs} = free_{ext2}</math>
<math>avail_{simfs} = avail_{ext2}</math>
|}
== Other reasons of strange numbers ==
At the moment I see only two more reasons why numbers in <code>df</code>/<code>stat</code> output can confuse you.
# The quota is inconsistent. This can happen if you turned quota off for some time, if you wrote directly to private area (<code>/vz/private</code>), but not through <code>simfs</code>, etc. When you have doubts whether your quota is consistent or not, just drop quota (<code>vzquota drop <ctid></code>, where <code><ctid></code> is the id of a stopped [[CT]]). While starting [[CT]], <code>vzctl</code> will automatically initialize quota.
# Unsupported underlying filesystem. Currently OpenVZ quota only supports <code>ext2</code>, <code>ext3</code> and <code>ext4</code>. With other file system types you can have unpredictable results. Praemonitus praemunitus!
== TODO ==
TODO: Add examples with stat/df
[[Category: Troubleshooting]]
[[Category: Resource management]]
[[Category: Disk quota]]