Jmeter is an open source automation testing tool .It is a Java application and can run on any system that has a Java implementation. It works under Windows (98, NT, XP, etc), Unix (Solaris, Linux, etc) and OpenVMS Alpha 7.3+. It will test both web application and web services .It is used to do load(How much volume system can process?) and performance testing(how fast is the system?) which measures response time, all other server resources such as memory usage, CPU loads, Load test of FTP servers, database servers and java servlets .It is used to simulate heavy load on the server, it can assign many users simultaneously. It can do load and performance test on many different server types:
- Database via JDBC
- SOAP
- Web – HTTP, HTTPS
- LDAP
- Mail – POP3(S) and IMAP(S)
- JMS
Download and installation steps
Download the latest version from (http://jakarta.apache.org) and unpack the archive to a local folder .The release contains all of the files we need to build and run most types of tests, e.g. FTP, Web (HTTP/HTTPS), JDBC,LDAP, Java, and JUnit. These files are found in the bin directory. There are some additional jar files which are not included after installation if we are using JDBC, JMS or JavaMail, also to test web services we need the mail.jar and activation.jar .After downloading keep these two jars into the JMeter installation lib directory.To run JMeter, run jmeter (for Unix) file or jmeter.bat (for Windows). After a short pause, the JMeter GUI will appear.
Jmeter GUI
Description
- Test plan describes series of steps.
- Workbench-Workbench act as temporary workspace to store test elements.
Functions & Features
Thread: It is used to specify the number of running threads.
Test Plan>add>Threads(Users)
Controllers: Two types of controllers are samplers and logic controllers.
Samplers: Sampler do the actual work in jmeter and interacts with the server we are loading.(send request to server)
Thread Group>add>Sampler>Webservice(Soap)Request
Logic controller: Decide when to send request .It can change the order of request coming from their child elements .It can modify the request themselves and cause Jmeter to repeat requests. Controller determines the order in which samplers are processed.
Thread Group>add>Logic controllers>Loop controller
Listeners: The information produced by samplers used by listeners .Few listeners are Graph Results,View Result tree,Spline visualizer, Simple Data Writer, Statistical Aggregate graph.
Thread Group>add>Listener>Name of Listener
Assertions: Assertions allows to validate the responses received from the server being tested.For example we can assert that the response to a query will contain particular text also we can specify that it should match the whole response.we can add an assertion to a HTTP request that checks for the text is ” <HTML>”. Jmeter then checks that text is present in the HTTP response.If Jmeter cannot find the text then it will mark it as failed request. To view the result add assertion listener to the Thread group.
Thread Group>add>Assertions>Name of the Assertion
Config element: Config element does not send the request.It can add or modify request.
Thread Group>add>Config Element>add Element according to requirement.
Timer: Jmeter send request without pausing between each request. The timer will cause Jmeter to delay a certain amount of time before each request that a thread makes. If we add more than one timer to a thread group Jmeter takes the sum of the timers and pauses for the amount of time before executing the samplers to which the timers apply.
Thread Group>add>Timer>add Timer
Pre-processor element: It perform actions prior to a sample request being made.If a pre-processor element added to a sampler element then it will execute just prior to that sampler element running .It is used to modify settings of sample request.
Thread Group>add>Pre-processors
Post-processor element: It perform actions after sample request has been made. It will execute just after the sample element runs .It is used to process the response data.
Thread Group>add>Post-processors
Building a web test plan for web services
STEP 1: Start Jmeter
Run the jmeter.bat file for windows or the Jmeter on unix from the Jmeter bin directory.
STEP 2: Create test plan
- Select test plan
- Right click test plan and add>thread group
Assign values to corresponding parameters coming under Test Plan.
For example: Start a Thread with a value 10 and assign Ramp up period 10 (It keeps a delay between starting each user .if we enter a Ramp-Up period of 5 seconds, JMeter will finish starting all of users by the end of 5 seconds.)
Ramp up period can be determined by following below steps:
1.The first sampler’s hit rate should be close to the average hit rate of other samplers thereby preventing a small ramp up period.
2.The first thread that finishes does indeed finishes after the last thread starts and keep its as far apart as possible thereby preventing large ramp up period.
To compare the server response time increase the number of threads and according to that assign the Ramp up period also.
STEP 3:
Add a sampler for web service .In this case Thread Group>add>sampler>
Web services(SOAP) Request.
1. If the URL to the WSDL file is available then paste the link in the WSDL URL field and click on load WSDL. The available web
methods will be available in the web method combo-box.
2. Click on configure so that Server name or IP, Port number, Path and action will be populated . Also manually we can type the values in.
3. Fill up the SOAP request in SOAP/XML-RPC data .we can also fill this area by loading file with SOAP XML data with the help of browse button specified in the GUI.
STEP 4:
Running the load test
We can run the test by clicking on run->start and can view the result in graph format with the help of listeners added to the UI.
1. Aggregate Graph
Aggregate graph will generate bar graphs and save the graph.
It comes up with following parameters:-
Samples – The number of samples with the same label.
Average – The average time of a set of results.
Median – Middle values taken from list of observations.
90% line – is the value below which 90% of the samples fall.
Min -The shortest time for the samples with the same label.
Max – The longest time for the samples with the same label.
Error % – Percent of requests with errors.
Kb/sec – The throughput measured in Kilobytes per second .
Throughput – The Throughput is measured in requests per second /minute/hour.
We can view minimum, maximum, median average and 90% response time. Select the column, enter the width, height and titles to get a view of graph.
2. Aggregate Report
Aggregate report is same as aggregate graph .The only difference is aggregate graph provides bar graph.
3. Spline visualizer
In this graph responsiveness of the application changes over time.It comes up with a continuous line which is drawn across 10 points where each point represents 10 percent of the sample.
4. View results in tree
In this tree we can view the response from the server for any sample.It also contains a request panel which shows the header added by Jmeter. we can view the response in different format with the help of drop-down box specified at the bottom side of the GUI.
It contains
Sample result
- Thread number and details
- Response time
- Server response codes and headers
Request
- Request sent to server
Response data in different format:
- HTML
- HTML(download embedded resources)
- JSON
- Regexp Tester
- Text
- XML
- The HTML view pass down the response as HTML.
- HTML(download embedded resources) is selected. It download images and style sheets.
- The XML view give response in tree style.
- JSON also show response in tree style.
- Regexp Tester view only works for text responses.
It shows the plain text in the upper panel. The “Test” button allows the user to apply the Regular Expression to the upper panel and the results will be displayed in the lower panel.
5. Graph Result
It creates a simple graph that plots all sample times . At the bottom of the graph we can view throughput(Green), Average of all samples (blue), standard deviation(red), sample(black) and median(purple) which can be displayed in milliseconds.
Throughput: The throughput number represents the actual number of request/minute server handled.
Median: Median is the number that represents the time, where half of server response time is lower than this number and half is higher. If median is much lower than Average then it indicates that few request are behaving badly .The median with list of numbers can be found by arranging the observations from lowest to highest value and consider the middle value as median value .If there are odd number of observations then middle value is the median,if there are even number of observations then median is the mean of the two middle values.
Standard Deviation: It is the variation in server’s response time It measures how widely values are dispersed from the average.The lower the difference between the average and median the lower will be the standard deviation and higher the difference between the average and median higher will be the standard deviation.If standard deviation is very high then there is some problem causing some request to behave differently . If it starts out small but keeps getting higher as time passes (perhaps there are memory leak). It should be small and graph should be fairly flat also we can conclude that system is fine at the load we are testing.
Average: is the total time running divided by number of request sent to the server.
6. Distribution Graph
The distribution graph will show bar for every unique response time .If the application had wild fluctuations then its not a good application with memory leak .If the application is not good then threshold lines will be beyond the width of graph .The graph draws two threshold lines 50% and 90%, 90 threshold should be within the the width of the graph.
7. Statistical aggregate graph
Its a Jmeter plugin .To install the graph only have to unzip StatAggVisualizer.zip into the jmeter installation dir .The zip file has four files, three of them go into jmeter_HOME/lib directory and remaining goes into the jmeter_HOME/lib/ext. It will display graph with throughput and response time as parameters.
Throughput: It is used to see how many request/second system can handle .Higher values are good if we keep on increasing the load we can check the throughput variations .It should increase for sometime then flatten and then finally fall. At light load zone as the number of user increases throughput increases linearly .Throughput max shows some bottlenecks in website .Downloading files require higher throughput than browsing text based web pages.
Response time: Its time between the request and the display of all the page content .Its a graph with Response Time versus Elapsed time which reports the average response time of the transactions over the elapsed time of the load test. It shows the time server took for responding as the load is increased and the peak indicates that server has reached its capacity to serve data, and is unable to scale further .If we compare with throughput as throughput increases response time also increases .As load increases throughput starts leveling .The leveling out of throughput shows that server reached its capacity to serve the data .The point at which graph increases sharply indicates beyond this load server cannot serve the request and users will see very slow response .once the load goes beyond certain point then average response time growth is more than linear .If there are sharp variations then it shows unstable system behavior.
Response time criteria:
- Users do not notice a delay of less than 0.1 second.
- A delay of less than 1 second does not interrupt user’s work.
- Users will still wait for the response if it is delayed by less than 10 seconds.
- After 10 seconds, users lose focus and start doing something else
Summary
Jmeter is a open source tool that not only allows to test web services but also performs load testing of web application.This article describes that by using existing functionality of Jmeter and its UI it is able to simulate load of users hitting the server by placing delay with the help of timer .It tells us how this tool can be used for measuring response time of web services. The results and charts shown in this article gives a clear picture about the response time to web service requests.The most important thing is to get a better load test output through comparative result which have value and also help to understand server performance. As a tester point of view i can say that while load and performance testing we must have knowledge about impact of memory, CPU, caching and bandwidth.Keeping in mind of all these things we have to perform testing.
Hi!
for graph result listener
what if median is much higher than average thne what does it mean?
Please..I really need your answer..
thanks,
jam