Performance analysis of the new version of pidstat's %wait and mpstat's %iowait, top's wa

A few days ago, I discussed the issue with a person in the WeChat group. It mentioned whether %wait in pidstat is waiting for io.

At that time, I had some ambiguities that there seemed to be no wait counter in pidstat. But based on years of judgment experience in top wa and %iowait in mpstat. I said I was waiting for io.

I also said at the time that if I made a wrong judgment, next time I meet, I will invite him to drink flower wine (not the flower wine you think,

）。
In order to verify what I said, I checked it on the server. It doesn't matter at first sight, the result is not waiting for this counter! !

I'm getting confused now, why don't I have this counter?

But the interface for students asking questions is like this.

One more wait. And this wait is indeed output in the -u CPU parameter of pidstat.

Although I see the wait of the CPU output by pidstat, it does not correspond to the iowait of mpstat and the wa in top.

As can be seen from the above figure, the %iowait output by mpstat has no value, but the %wait in pidstat has a relatively high value. The data output at the same time differs so much, which is indeed strange.

So I asked what the version in his environment was. He said that the version of sysstat 11.5.5 can be seen. The pidstat I output in my environment is 10.1.5. If I use yum to install in Centos 8 and 7, there is no such a high version.

Analysis process

There is no way, in line with the paranoia of technology, I think we have to find out if there is any explanation wrong!

Take a look at the analysis process.

Counter explanation

To understand why the counter values are not correct, you must first know whether they have the same meaning.

Because the parameter values of several commands are different, top and mpstat have been used for many years. Then first check how the help of these counters is defined.
The wa in top is explained in this way.

wa, IO-wait : time waiting for I/O completion

The iowait in mpstat is like this.

%iowait
 Show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.

This is why I judged that the wait in the CPU is waiting for IO, because the above two counters clearly say waiting for IO.
but! ! ! The turning point of my misjudgment has come.
After I installed the latest version of sysstat, I saw the explanation of wait in pidstat.