Is the JMeter Load Server affecting my results?

Question

I'm running a JMeter test using an amazon EC2 instance (large) as the load server using 1,000 threads. The load server CPU is steady at about 90% utilization and memory is at 70%.

Is there a rule of thumb regarding at what point does load server not have enough resources (memory or CPU) which causes the load on the load server itself to impact test results?

Regarding CPU would you say 90%? 95% 99%? Regarding Memory would you say 90%? 95% 99%?

Thanks Ophir

Update: I asked on the official JMeter mailing list and received some great answers: http://jmeter.512774.n5.nabble.com/Is-my-load-server-causing-results-to-be-in-accurate-td5718385.html

Answer 1

It depends.

First, if you use JMeter only to generate heavy load on a server an you not use the JMeter results because you use the server log files like tomcat access.log on your target server, then you can drive your JMeter server up to 80% CPU utilization. Higher load reduce the ability to switch between process and thread clearly.

Second, if you need the JMeter results for analysis, the you shoud reduce the CPU utilization to 40 - 50% because the high CPU usage adulterate your JMeter results. This meet also the recommendation of Xceptance, mother of XLT, a JMeter like project. High memory usage is not the problem directly, you have to check your GC times, because the GC adulterate your JMeter results.

Hint: if you make only short HTTP request in your JMeter test, then select the HTTPClient3.1 as HTTP Request Implementation. This client is faster an produces less load for small HTTP Requests (see Links german Blog, Atlassian Blog).