Checking responses in JMeter tests
In this recipe, we'll use some techniques to validate JMeter test results rather than just checking manually for an HTTP 200
response code.
Getting ready
You will need to install Oracle SOA Suite 11g for this recipe.
This recipe assumes that you have a loaded test plan. You can complete the Creating a web service test using JMeter and Running JMeter on multiple servers recipes, in order to have access to a working test plan against a composite. Alternatively you can use these steps with your own working test plan.
How to do it…
By following these steps, you can use assertions to check that the responses contain the correct data.
- In the test plan, right-click on the Test Users thread group, and select Add | Assertions | Response Assertion.
- In Assertion, ensure that the checkboxes Main sample only, Test Response, and Contains are selected.
- Click on Add in Patterns to Test in the textbox, and enter
<result>2</result>
. Your test plan should look similar to the following screenshot: - In step 3, we've purposely added an assertion that will fail for the input to our test composite. Run the test by clicking on the green run icon, and navigate to Result Listener Tree View.
- Expand the error result tree and view the assertion error. We can see why our response check failed—there was no result of
2
in the response. - Change the assertion to check that it matches the expected result. Do this by double-clicking on the pattern to test for in Response Assertion, and change it to
<result>1</result>
. - Run the test again and view the response listener; it will now contain our valid web service test result as well as the one for which the assertion failed.
- Response assertions are Perl 5 style regular expressions; we'll add another pattern to test for a
<result>
element that contains any number. Click on Add, and in the Patterns To Test box, add the following:(?i)<result>\d</result>
- Run the test again, and the result should contain no errors if the pattern is correct.
How it works…
Assertions can be used in JMeter to check the responses in our tests. In the event that an assertion fails, we can determine the cause of the failure in a results listener. We purposefully set an incorrect assertion to begin with, and saw this in action. Assertions are relatively intensive for JMeter to process, so they should be used sparingly, checking for key text that you expect to be in a successful response, rather than trying to perform complex regular expressions or analysis on the returned data. If you use assertions, be sure to monitor the CPU usage of the host running JMeter; if it maxes out, your tests are limited by the performance of the testing host, rather than the SOA Suite host. If this happens, you should add extra JMeter hosts to increase the load, or save the response data and run assertions after the test by using scripting or some other framework.
The assertion type that you can add to JMeter tests can interrogate responses in one of the two ways. Selecting the Contains or Matches checkboxes enables the use of Perl5-style regular expressions. Choosing Equals or Substring will match plaintext only, and will do so in a case-sensitive way.
For the assertions added in this recipe, we not only exclusively used the Contains checkbox, but also used both plaintext and regular expressions—this is perfectly valid.
If you want to use Matches and Contains response checks, you can do this by using a second assertion test element.
There's more…
There are a large number of assertion elements that can be added to our JMeter tests; they are documented at http://jmeter.apache.org/usermanual/component_reference.html#assertions. Of particular note are XML schema assertions, which check that a response conforms to a DTD or XML schema. XPath assertions can also be used to extract values from our SOAP responses, and BSF assertions that can use a scripting language, such as groovy, to check the returned data. These complex assertion types should be used sparingly and with care, as they are very CPU-intensive.
See also
- The Running JMeter on multiple servers recipe