How to profile the IO activities triggered by the java process?

Question

Currently, I know there are some tools for profiling the CPU/memory for the java process.

When I wanted to track how the java process interacts with IO, I didn't find any tool for checking the details, especially for which threads/call-stacks, usages, and files.

The tools I can use are some OS-level commands like dstat.

Is there any tool for profiling DISK IO for java?

Answer 1

Is there any tool for profiling DISK IO for java?

Some JVM profilers have probes that deal with file access. JProfiler for example has a file probe that measures read and write operations for both java.io and java.nio.

Measurements are resolved by file:

For performance considerations, the hot spots view of the file probe shows which files are responsible for most of the time spent in disk I/O and gives you expandable back traces for each file.

Disclaimer: My company develops JProfiler

Answer 2

It's unconventional but easy. Just do this.

It's hard to find a profiler that a) works on wall-clock time rather than CPU time, b) samples the call stack, not just the IP, c) takes a small number of samples rather than a large number.

Remember, your overriding goal is locating the problem, so you can fix it, not just timing it but wondering what it is. The bigger the problem is, the more likely a random halt will land in it and tell you exactly what it's doing and why.

How big are problems? If a problem takes X% of time, X% is the most you could save by fixing it. So if X is 10% or less, why bother? Supposing X% is bigger than 10%, then every sample has that probability of landing in it. So if you take N samples, the number of times it lands in the problem is N times X% (average).

How many samples do you need? Well, if you see the program doing something wasteful on one sample, it's doubtful that you've found it. But if you see it doing the same wasteful thing on two samples, statistics say you've found the problem! And the smaller N is before you see the problem, the bigger it is! Do you know exactly how big it is? No, and you don't care, because you know it's big enough to fix. (If you want more information, take more samples. I never take more than 20.) (You can always measure the speedup afterwards, with a simple timer.)

Is there more than one problem? You bet there is! If you fix one problem, it makes the remaining ones bigger! Example, there could be two problems, problem A costing 50%, and problem B costing 25%. The first time you look, you find problem A, fix it, and now problem B costs 50%, not 25%. Fix both of them, and you're 4 times faster. (This is the magnification effect.) (If you happen to fix B first, no problem. Now A is 67% of the time.)
Moral: Don't stop looking and fixing until you fail to find things you can fix.

What typically happens with profilers is they find things that don't amount to much and/or you can't fix anyway. Like 5% in some obscure library routine. Does this leave programmers disappointed? NO. It makes them happy, because it seemingly tells them their code is optimal! Is that a paradox? Yes. Profilers are popular precisely because they don't work!