JMeter User interface can conduct any type of performance test execution. But we can also conduct a performance test using JMeter CLI or Command Line Interface (CLI). In this article on JMeter command line execution, you can learn how to execute a performance test using command line interface and its advantages.
We wrote this article specifically for those using Windows Operating systems.
Advantages of Jmeter command line execution
resource optimization
It reduces overhead of running JMeter using user interface, since it consumes comparatively less resource usage like CPU and memory in comparison.
headless performance testing
We can do a headless performance testing by running it using command line interface. This is helpful specially while we are running a test in a server machine or any other headless environment.
performance test automation
By running performance test using CLI, it can be integrated with any kind of Continuous Integration /Continuous deployment (CI/CD) pipelines. Hence it becomes useful in using this feature while scripts are running in any CI/CD pipelines as part of automating them.
Running jmeter command line execution
Now since we know advantages of running a performance test using CLI , let us see the details on steps to run a test using Command LineInterface. We can run a test using CLI in a simple 2 steps process.
Step 1-Place JMeter script and Test Reports/results
First Step is to place JMeter script and reports in any folder and place this folder in JMeter bin directory. JMeter script file will have a JMX extension and plan to keep results or test reports preferably in an excel with csv extension.
Placing script and result file in Bin directory
step 2-running in command line interface(cli)
In the second step, open the command prompt and navigate to JMeter bin directory using the change directory (CD command).
Now execute below mentioned command prompt to start executing the script in command prompt
jmeter -n -t Mention the Script location -l Mention the result file location
Explanation to above mentioned command is
-n indicates non graphical user interface mode of execution
-t indicate the location of the script
-l indicates location of the results file
CLI execution using the command
Conclusion
As you can see here , jmeter execution using command line is a very easy way of doing it .Many issue related to high resource intensive operations happening during test execution can be easily managed by planning to run it using JMeter command line execution.
If you are interested to learn more about related article, then find it below.
This article is easy installation steps of installing JMeter tool in your system having Windows operating system.
JMeter Installation-Windows Operating System
Prerequisites
Since JMeter is a java based application its mandatory to have Java Development Kit installed in your system before you proceed with JMeter installation. You can download and install JDK here Java Downloads | Oracle India
Post installation of JDK follow below mentioned steps to verify successful installation of JDK and also to continue JMeter installation in your windows Operating system.
Verify Java JDK
Open command prompt and type java -version. Your JDK is successfully installed if you get java version details as mentioned below (I have installed java version 18.0.1.1, but feel free to install Java 8 or higher versions). Incase if you do not get the java version details then ensure the correctness of JDK installation along with java path settings.
In this article on JMeter API testing, we will be discussing the details on API’s, different tools that is used for API testing and also on how to develop an API script for testing using JMeter tool. But before discussing about JMeter API testing, mentioning few details about API’s and tools available to do an API testing.
What is API?
Application Program Interface (API) is a communication layer between different components in a multi-tier application architecture. API’s plays a key role in ensuring an effective communication between different components. Let us try to understand APIs with an example of an ecommerce website. Let’s assume there is an ecommerce website and multiple customers purchases products from this website. Here let us assume that API 1 is a product price related Api and API 2 is product quantity related Api.
Now while customers trying to purchase a product(s), they get information related to price and quantity related to the products on the website is because these Api’s are always interacting with the database to fetch this information and providing it to the customer through the website interface on a real time basis. In this example we have mentioned only two Api’s but in a real-world scenario many API’s will be interacting with multiple components throughout the day.
What is API Testing?
API Testing is a testing practice performed to validate the Api’s are working well in terms of its functionality,performance,security and reliability. It plays an important part of testing life cycle specially during integration testing.
API Testing Tools
Many tools are available in the market to do API testing. Few of the tools are free tools which can be used to test your Api’s. Few of the widely used tools to do API testing is mentioned below.
SoapUI
Postman
Apache JMeter
Katalon Studio
Apigee
JMeter API Testing
As mentioned earlier in this article API testing is important to validate an API’s functionality, performance, security and reliability. Hence it is again very important to focus on API’s performance because high performing or faster APIs are always needed to make faster communication between the software components. JMeter API testing can be useful particularly while we test API to understand the performance of it.
Below mentioned is a step-by-step approach in developing a JMeter script as part of your JMeter API testing. For creating a JMeter api script making use of some API’s available in Reqres website.
JMeter API Testing – Step1 -Setup JMeter Test plan
To begin with launch JMeter and add below mentioned JMeter components.
Right click on Test plan and select threads<<>>Thread groups
Right click on Thread groups and then select add sampler<<>>HTTP Request
Right click on Thread groups and then select add Listener<<>>View Result trees
Right click on Thread groups and then select add Listener<<>>Aggregate reports
Once the above listed JMeter components are added your JMeter screen will look similar to below mentioned screenshot.
JMeter API Testing – Step2 -Setup JMeter HTTP Request
Click on HTTP request and will select an api request details from Reqres website. In this example using a Create User request API having a POST method. Ideally an api request will have a request URL, request body, request header and a method. The api used here have a request URL, request body and a method.
In this example request details is as follows.
Request URL is https://reqres.in/api/users
Request body is
{
"name": "morpheus",
"job": "leader"
}
Request method is POST
Now in the HTTP request add the details of the request as mentioned below .
HTTP Request Name – Give any suitable meaningful name of your choice
Protocol – HTTP or HTTP(s), In this example it is https:
Server name – domain name, In this example it is reqres.in
Method – Select the API method, In this example it is POST
Path – Add the subdomain, In this example it is /api/users
Request Body – Add the request body, In this example it is { “name”: “morpheus”, “job”: “leader” }
Note – Incase if your API request have a request header then you need to make sure to add the request header along with your http request. For adding it, right click HTTP Request, select Add config element <<>> HTTP Header Manager.
JMeter API Testing – Step3 -Run HTTP Request and Validate Response.
Now the JMeter API script is ready to be executed. Execute the script clicking on start button (green play button) in the JMeter icons. Validate your response by clicking View results Tree. The performance summary report of API test execution is available in the Aggregate Report (Ref screenshot below)
In this article you can find three different ways of recording an application user flow as part of JMeter script recording and enhancements. Either of this method can be followed in recording your application under test.
JMeter recording using JMeter HTTP’s Test Script Recorder
Recording using Blazemeter extension
Recording using Fiddler
JMeter recording using JMeter HTTP’s Test Script Recorder
Recording an application using HTTP’s test script recorder is the most widely and preferable way of recording an application. Preferred browsers for most of the applications are chrome and firefox.
Prerequisite Steps
In Chrome Browser
Proxy settings
Navigate to proxy settings of your system and change proxy address to localhost and port to 8888 and save the changes.
Configure JMeter root certificate In Chrome
In chrome browser navigate to settings <<>> Privacy and Security<<>>Manage Device Certificate<<>>click on Import<<>>Browse and select ApacheJMeterTemporaryRootCA.crt from JMeter bin directory and get it added to your browser trusted certificate directory. ApacheJMeterTemporaryRootCA.crt is listed in certificate list after the successful import.
In Firefox Browser
Proxy settings
Navigate to Firefox settings <<>>options<<>>Connection settings<<>>Manual proxy configurations<<>>Change HTTP proxy to localhost and port to 8888. Click on the option “Also use this proxy for FTP and HTTPS“.
Configure JMeter root certificate In Firefox
In Firefox browser navigate to settings <<>> Search for certificate<<>>Click on view certificate<<>>click on Import<<>>Browse and select ApacheJMeterTemporaryRootCA.crt from JMeter bin directory and get it added to your browser trusted certificate directory.
After completing prerequisites open JMeter and get ready to record your application.
Adding Recording Controller And HTTP(s) Test Script Recorder
Follow below mentioned steps to add necessary controllers to begin recording your application workflow using JMeter.
Right click Test plan<<>>Add<<>>thread groups
Right click Test plan <<>>Add<<>>Non-Test Elements<<>>HTTP(s) Test Script Recorder.
Right Thread group <<>>Add<<>>Logic controller<<>>Recording controller
Adding recording controller is an optional step but will help in naming and organizing different user flow of application recording accordingly. Rename the recording controllers based on the user flow of your recording.
Test script recorder(http(s)) setup
Mention the port in global setting as 8888 (note that this port number should be the same as mentioned in proxy setting) and select target controller as “Test plan>Http(s) Test Script Recorder”. Click on start in Http(s) Test Script Recorder. Click on “OK” while the Root CA certificate notification pops up. Open browser, launch the application and start recording the user flow.
Recorded User flow
Once the recording is completed, click on stop and the recorded Api’s will be available for further script enhancement.
Recording using Blazemeter extension
Recording an application user flow can be done using blaze meter extension. This method is useful if application is recorded using chrome browser
Navigate to chrome extensions and enable blaze meter extension to start recording the application user flow
Signup & SignIn Blazemeter Account
Signup and create a blaze meter account. Login to blazemeter account and start recording the application by clicking on blazemeter extension available in chrome extensions. Signing up or creating an account in blaze meter is important because signing in is needed to save the recorded script.
Start Recording
Record application user flow by clicking record button in blazemeter extension
Save Recorded Script
Save the script by selecting JMeter script option and finally the downloaded jmx file can be used for further script development.
Recording using Fiddler
Download & Install Fiddler
Fiddler is a useful tool in capturing and debugging network traffics. Navigate to fiddler to download fiddler. Post downloading complete the installation of fiddler tool.
Recording Using Fiddler
Open fiddler and click on File and select capture traffic and start recording application user flow.
Exporting File
After completing the recording click on File<<>>Export Sessions<<>>All Sessions. Select the export format to HTTP Archive v1.1 Save the file with. HAR extension. Now open blazemeter converter and convert .HAR file to .JMX file .
Correlation is the method of capturing dynamic values which comes as a server response and sending the captured value to subsequent request. In this article on JMeter correlation, Iam mentioning about different types of extractors available in JMeter and how to use them in correlating dynamic values.
In JMeter, you can use extractors of different types to perform correlation. The extractor, as the name indicates, extracts the dynamic value and stores it in a variable if you need to pass the stored value in the next requests. Below mentioned are different types of extractors available in JMeter.
CSS Selector Extractor
JSON Extractor
Boundary Extractor
Regular Expression Extractor
XPath Extractor
How to add extractor?
Step1-Identify request wherein the extractor needs to be added. Step2-Right click on the request select add, select Post processors, and click on the type of extractor whichever needs to be added.
when to do correlation?
Response for a request will have different values. For easiness let us categorize these values as static values and dynamic values. Since static value remains the same, you do not need to capture it using any extractor. Dynamic values change in every time. If a value is dynamic and it is coming in any subsequent request, you should correlate it.
Now let us go through the details on each extractor available in JMeter
Types of extractors
CSS Selector Extractor
You can plan to use CSS selectors or XPath extractors if you need to capture a dynamic value from an HTML or XML response. The CSS extractor in the example mentioned below extracts all headings of a webpage. Once you have tested the extraction using the CSS selector tester, apply the CSS selector extractor.
CSS Selector TesterExtracting 5th heading content using CSS Selector Extractor.
JSON Extractor
You can use JSON extractors particularly when you need to capture a dynamic value from a JSON response. In the below example created data is a dynamic response value. As shown in the screenshot, we use a JSON path expression to capture the created value using JSON extractors.
Json extractorValidating Json path expression using json path tester before applying it
Boundary Extractor
We can capture the dynamic value using the boundary extractor by identifying its left and right boundary. In the below mentioned example id number is a dynamic value. Now to capture this dynamic value we need to identify left and right boundaries and add it in respective fields as mentioned in screenshot.
Here in below mentioned example left boundary is {“id”:” and right boundary is “,”created At
Dynamic value from a response can be extracted used using regular expression extractor. Here in this example dynamic id number is captured using regular expression “(.)” In this regular expression “.” indicates any character or integer value and “” indicates any numbers of values following the first value.
XPath Extractor
XPath extractor becomes handy specially during other extractors like boundary and regular expressions extractors fails to capture the desired dynamic value from the response. It is useful in getting values from HTML tags similar to CSS extractors.
Xpath testerXPath Extractor
Interested in learning more about Xpath ? , visit this website to learn more on Xpath Tutorial Point
checking extractor before applying
It is important to validate correlation before applying it. You can do it in either of the ways mentioned below.
1)Navigate to response for a particular request in “View Result Tree”. Apply the regular expression in search bar and select “Regular exp” option and click Find.
2)Navigate to response for a particular request in “View Result Tree”. Select from the dropdown extractor tester of your choice. In this example it is boundary extractor tester. Place left and right boundary accordingly and click test. If you fill the input information appropriately, then only you will get the results.
Passing captured Correlated value to next request
To pass the captured or correlated value you need to pass the variable name as mentioned below (in screenshot). In the subsequent request to create user2, we use the ID value captured in the create user1 response. This is applicable to values captured using any JMeter extractors.
Format to keep the value in request is ${Variable Name}.
Boundary extractor with a variable name as C_Id1The variable ${C_Id1} is used in the second request body create User2.ID having dynamic value 6 is captured in variable C_Id1Dynamic Id value “6” is sent in second request create User2
In this article on JMeter thread group, you will learn on what is a thread groups. different types of thread groups, thread group properties, scheduler configuration and most importantly how will you decide which thread groups to use for your performance test execution
Thread Groups
Thread groups is a group of threads executing similar kind of actions during a test execution. Here in JMeter tool each thread represents a virtual user. As part of a performance test execution if you need to design a scenario wherein a set of users performs similar kind of actions then we add thread groups in test plan. Precisely saying thread groups are user groups and if you use JMeter for any kind of performance test execution we need to add thread groups.
Types of thread groups and how to add them
Mentioning 8 different types of thread groups available in JMeter. Few thread groups like thread groups, setUp thread group and teardown thread groups are available as a default option.
JMeter plugins used to add remaining thread group mentioned below.
Thread Group
bzm – Arrivals Thread Group
bzm – Concurrency Thread Group
bzm – Free-Form Arrivals Thread Group
jp@gc – Stepping Thread Group
jp@gc – Ultimate Thread Group
setup Thread Group
teardown Thread Group
Right click on Test plan, select add and then select either of these thread groups based on your choice.
Options in Thread groups
Action to be taken after a Sampler Error
Thread properties
Scheduler configuration
Action to be taken after a Sampler Error
Action to be taken after a sampler error is a common option across all the thread groups available in JMeter. Based on option we select here JMeter will decide on the actions to be taken if it encounters with any errors during the execution.
Continue – To continue with the test execution even in case of any errors
Start Next Thread Loop – It will stop the currently running thread group and proceed with other thread groups (if any)
Stop Thread – It will stop the running thread in thread group.
Stop Test – It will stop the test completely.
Stop Test Now – Almost the same option available with stop test. The test will stop more abruptly.
Thread Properties
Pattern of distributing the load is based on the options selected in thread properties. Below mentioned are properties available in JMeter thread groups.
Number of threads (users) – Mention number of virtual users or threads for load distribution
Ramp up period(sec)-Time in seconds needed to ramp up all the virtual users while running a test. Example if you’re running a test with 100 users and if you give 10 seconds in ramp up period, then all 100 users will be ramping up and running in your test in a time duration of 10 seconds.
Loop count – Indicates number of times execution iterates. Check the forever option if you want to run the test for a specific duration.
Same User on each Iteration – Check this option if same set of users to be used during the execution.
Specify thread lifetime – Total duration of test execution in seconds.
Scheduler Configuration
In scheduler configuration additional information on load distribution and its duration included. Below mentioned are the scheduler configuration available in JMeter thread groups. Scheduler configuration information available totally depends on the thread group chosen for an execution.
Target Rate (arrivals/min) – Targeted request/sample rate in minutes mentioned here. Target rate represents the sample / request rate in a minute and is not the user rate per minute. This option is available in bzm – Arrivals Thread Group
Ramp Up Time (min)-Time in minutes it takes to push the request load during the execution. This option is available in bzm – Arrivals Thread Group
Ramp-up Step Count-Mention total number of steps needed while ramping up the request load. This option is available in bzm – Arrivals Thread Group
Hold Target Rate Time (min)-This is the total test execution duration. This option is available in bzm – Arrivals Thread Group
Target Concurrency-Targeted number of concurrent users needed during the test.
This group will start-Mention the number of users needed during the start of the test. This option is available in jp@gc – Stepping Thread Group
First, wait for – Wait time in seconds needed before ramping the next set of users. This option is available in jp@gc – Stepping Thread Group
Then Start – New set of users gradually added. This option is available in jp@gc – Stepping Thread Group
Next, add – Next set of users added in the time intervals of seconds. This option is available in jp@gc – Stepping Thread Group
Then, hold load for – Overall test execution duration. This option is available in jp@gc – Stepping Thread Group
Which Thread Group Should I Use?
Based on the way you decide to design the load of the application you can decide to choose the thread group. Including few information on how each thread groups is different from one another. Going through the details on each thread group you can decide to opt either of these thread groups which suits the user load design or pattern.
Thread Group
Thread group is the most commonly used compared to other thread groups available in JMeter. Suppose you want to load test your application by using multiple concurrent users for a specific duration then this thread group can be opted. In this example mentioned below 100 users will ramp up in 10 seconds and the test will run for 1 hour duration.
bzm – Arrivals Thread Group
Bzm – Arrivals Thread Group used if we want to trigger a test with a specific number of request rate rather than number of users . In below example 100 requests simulated every minute and will run for 1 hour duration. Total of 6333 request generated by this thread group. Ideally 6000 requests in 1 hour duration, remaining 333 requests getting added in 5 min ramp up duration.
bzm – Concurrency Thread Group
Bzm – Concurrency Thread Group can be used to simulate concurrency in load. In below example, all 12 users will be ramping up in 2 steps by around 5 minutes and will do the execution for 1 hour duration. The total load test execution duration is 1 hour 5 min which is 1 hour of hold target rate plus 5 minutes of ramp up.
bzm – Free-Form Arrivals Thread Group
This thread group is similar to bzm – Arrivals Thread Group and you can design user load on the basis of total number of requests. Below mentioned user load design will run the test for 1 hour 5 min duration and will generate around 340 requests.
jp@gc – Stepping Thread Group
In this thread group we can ramp up the user load gradually. Step by step progression of user load is possible using stepping thread group. In the below example total of 100 users will be running. 20 users ramp up initially and 10 users added gradually in every 30 seconds. This test will run for a duration of 1 hour 5 min which includes 1 hour steady load with 100 users and 5 min of user ramp up.
jp@gc – Ultimate Thread Group
In JP@GC – Ultimate thread group, load can be designed with many more parameters or inputs. This thread can be used in designing spike test. In the below example a total of 60 users will simulate the load which starts with an initial delay of 10 sec. The first thread scheduled will run for 600 seconds and second thread scheduled will run for 1 hour duration.
setup Thread Group & teardown Thread Group
Both setup Thread Group & teardown Thread Group are having the same option and is similar to basic thread group. Setup threads are like a prerequisite thread group and executed before the actual test execution.
Teardown thread groups executed post the actual test execution, generally executed for any activities needed after the actual execution. Below mentioned are some use cases of setup and tear down thread groups.
Few use cases of setup Thread Group:
Extracting User details data set information from database
Test data loaded as data set to database before load test execution.
Few use cases of Tear down Thread Group:
Deleting the dataset from database post the load test execution.
Clearing data from cache server post the load test execution.
