How to analyze Snap.*trc file?

Question

I'm now investigating OOM on WebSphere, I got a file Snap.*.trc. I referenced another SO question's comment, How to analyse Websphere core*.dmp file and Snap*.trc files? , and I created Snap.*.trc.fmt file.

However, I can't understand what this file is. For example, GC log (native_stderr.log), I can see what garbage collector is doing by using IBM Pattern Modeling and Analysis Tool(PMAT), but it seems there is no tool for analyzing Snap.*.trc file. Is there any tool for this?

Answer 1

Generally, Snap*trc files are used by support rather than by customers. They contain any tracepoint data held in memory at the time of the dump. They can be useful in some OOM cases to check whether, for example, the OOM occurred because of exhausting native memory. It seems you've already figured out how to format them and the resulting *trc.fmt text file is just a set of trace points, so it's the same as analyzing any trace (which generally means you need to understand the code, and thus why Snap*trc files tend to be limited to use by support). You can find a bit more info on Snap files here: https://publib.boulder.ibm.com/httpserv/cookbook/Troubleshooting-Troubleshooting_Java-Troubleshooting_IBM_Java.html#Troubleshooting-Troubleshooting_IBM_Java-Snap_Traces

With that said, generally, here's how I analyze OOMs:

1. Look at the 1TISIGINFO in the javacore*txt file. This will tell you whether it's a Java OOM or a native OOM.
2. If it's a Java OOM, then load the core*dmp file into the IBM Memory Analyzer Tool. Note that the other question you referenced says that you must run jextract on the core*dmp file to analyze it, and this is no longer the case with recent versions of Java - simply load the core*dmp file into the IBM MAT tool.
3. If it's a native OOM, then that gets more complicated, so you can post back here with details.

As always, you can also open a support case with IBM and they can help you with some of this analysis.

Answer 2

You can always use the GMCV Health Center. Download and install it via eclipse market place. It is able to read trc files. Also see the user guide here. It is able to give you information about the following things:

• Classes: Information about classes being loaded
• CPU: Processor usage for the application and the system on which it is running
• Environment: Details of the configuration and system of the monitored application
• Garbage Collection: Information about the Java heap and pause times
• I/O: Information about I/O activities that take place
• Locking: Information about contention on inflated locks
• Method Trace: Information about method use over time
• Native Memory: Information about the native memory usage
• Profiling: Provides a sampling profile of Java methods including call paths
• Threads: Information about the live threads of the monitored JVM