JMeter User interface can conduct any type of performance test execution. But we can also conduct a performance test using JMeter CLI or Command Line Interface (CLI). In this article on JMeter command line execution, you can learn how to execute a performance test using command line interface and its advantages.
We wrote this article specifically for those using Windows Operating systems.
Advantages of Jmeter command line execution
resource optimization
It reduces overhead of running JMeter using user interface, since it consumes comparatively less resource usage like CPU and memory in comparison.
headless performance testing
We can do a headless performance testing by running it using command line interface. This is helpful specially while we are running a test in a server machine or any other headless environment.
performance test automation
By running performance test using CLI, it can be integrated with any kind of Continuous Integration /Continuous deployment (CI/CD) pipelines. Hence it becomes useful in using this feature while scripts are running in any CI/CD pipelines as part of automating them.
Running jmeter command line execution
Now since we know advantages of running a performance test using CLI , let us see the details on steps to run a test using Command LineInterface. We can run a test using CLI in a simple 2 steps process.
Step 1-Place JMeter script and Test Reports/results
First Step is to place JMeter script and reports in any folder and place this folder in JMeter bin directory. JMeter script file will have a JMX extension and plan to keep results or test reports preferably in an excel with csv extension.
Placing script and result file in Bin directory
step 2-running in command line interface(cli)
In the second step, open the command prompt and navigate to JMeter bin directory using the change directory (CD command).
Now execute below mentioned command prompt to start executing the script in command prompt
jmeter -n -t Mention the Script location -l Mention the result file location
Explanation to above mentioned command is
-n indicates non graphical user interface mode of execution
-t indicate the location of the script
-l indicates location of the results file
CLI execution using the command
Conclusion
As you can see here , jmeter execution using command line is a very easy way of doing it .Many issue related to high resource intensive operations happening during test execution can be easily managed by planning to run it using JMeter command line execution.
If you are interested to learn more about related article, then find it below.
Performance Testing is a software testing practice to evaluate a system or applications performance in terms of its speed, stability, scalability and reliability. It’s a part of non-functional testing ensuring that application performs at its best in every situation. There are multiple factors in a distributed application which decides performance of an application. Below mentioned are few of them.
Server-side metrics – Server Response time
Server -side resource utilizations – CPU, Memory, Disk.
Network latencies
Client-side performance metrics – Transaction Response Time, Throughput, Failures and Error rates
Front end Metrics – Page load time
In this modern era, we expect everything to be operating in lightning speed. Hence it is a must rather than a luxury in having performance engineering and testing while developing a solution. Imagine use case of an ecommerce website. You plan to purchase a product and while you are searching for the product it is taking a long time to fetch your product search results, would you prefer to continue shopping using this website? . Business face a huge loss if they ignore the need of it.
Performance Testing Vs Performance Engineering
Performance engineering is a discipline of designing, developing, testing and tuning an application in creating a high performing and scalable application while performance testing is a subset of performance engineering wherein the practices of testing an application by mimicking the user load as expected in production is conducted and after evaluating execution results, performance optimizations are applied.
Now looking into performance engineering the efforts starts from application design phase itself. Appropriate measures are taken care in the beginning of a software development life cycle to deliver a high performance and scalable solution. Performance engineering phase also gets involved in tuning and optimizing the developed solutions based on the test execution results.
Performance engineering scope also includes in the process of resource capacity planning which fulfills the need of a delivering the need of a highly scalable application which withstands with growing business needs.
When to start?
Organizations follow a practice of staring performance effort during or after system integration test is completed. But there are many flaws in starting performance effort in later stages of a testing life cycle
Less time for the developers to make code changes in optimizing performance.
Technical architect or solution architects have to make cost and time intensive modifications in system design sometimes from the scratch
Less time for performance test engineer to understand the application architecture and build an effective Test strategy.
Poor capacity planning
As you can see there are many challenges while involving performance testing effort in a last-minute hurry. Hence to overcome these challenges its always ideal in involving performance engineering efforts in the design phase of a release cycle. By involving performance engineering in design phase, we gain a lot of benefits like
Developers get ample amount of time in optimizing code for a higher performance
System designing based on test execution results which makes the system architecture scalable, robust and stable with higher resilience.
Performance engineer can include more scenarios and test strategies in testing the application
We can effectively do capacity planning in terms of resource scalability, cost optimizations etc.
This article is easy installation steps of installing JMeter tool in your system having Windows operating system.
JMeter Installation-Windows Operating System
Prerequisites
Since JMeter is a java based application its mandatory to have Java Development Kit installed in your system before you proceed with JMeter installation. You can download and install JDK here Java Downloads | Oracle India
Post installation of JDK follow below mentioned steps to verify successful installation of JDK and also to continue JMeter installation in your windows Operating system.
Verify Java JDK
Open command prompt and type java -version. Your JDK is successfully installed if you get java version details as mentioned below (I have installed java version 18.0.1.1, but feel free to install Java 8 or higher versions). Incase if you do not get the java version details then ensure the correctness of JDK installation along with java path settings.
In this article about performance testing process, you can read about end-to-end life cycle of performance testing starting from requirement gathering to final sign off. Below mentioned are the stages of performance testing.
Requirement Gathering
Test Planning
Test Environment, Test Data & Monitoring Readiness
Scripting
Workload Modelling
Test Execution & Monitoring
Result Preparation & Analysis
Performance Bottleneck / Defect Fix, Retest
Performance Signoff
Requirement Gathering
All information required to start a performance testing is collected in requirement gathering phase.The key focus in requirement gathering phase will be in getting an understanding on system architecture, different system components available in the architecture and its integration with each other.
In requirement gathering phase information on scenarios, workflows ,components involved etc are collected. Volume used for performance testing is another important criteria.This discussion happens with developers,product owners,architects,infrastructure owners etc.
test planning
Test strategy and planning is the next phase after requirement gathering for an application. In this step, a test plan document prepared with below details by a performance test engineer.
Overall Test Strategy
Test Objective
Scenarios or User flows
Estimations – Resource and Timeline
Risk & Mitigations
Infrastructure and Capacity details
In scope & out of scope
Test Environment, Tools, Test Data and Monitoring details
Timelines
Internal and external stakeholders will review and approve the test plan document before next process in performance testing life cycle.
Test ENVIRONMENT, Test data readiness and monitoring setups
Performance test environment should be similar to production environment. This helps in maintaining accuracy and reliability of performance results. Data volume for performance testing should be based on infrastructure capacity specially incase if performance test environment is not similar to production environment in terms of its capacity.
Performance testing needs adequate volume of data to test a scenario more realistically. A full-fledged end – end monitoring setup has to created which helps a performance engineer triage a performance bottleneck. Multiple monitoring and log analysis tools are available in the market like Dynatrace, AppDynamics, New relics, Splunk etc . These tools will help to identify performance bottleneck quicker and easier.
scripting
In scripting phase, different tools like Loadrunner , JMeter, Neo load etc is used to develop performance scripts. These scripts might be an API based or a user interface based script. Scripts usually mimics the real user actions or load pattern. If you wish to read more about articles about JMeter, then refer JMeter Blogs
workload modelling
In workload modelling, number of users needed calculation done to simulate a load. It is calculated using little ‘s law. It helps performance engineer to design a realistic load while running a performance test.
test execution and monitoring
In execution phase different types of performance testing like Load test, endurance test, stress test etc are conducted along with a real time monitoring of client and server side metrics to evaluate applications performance. Based on the type of application and its business usage, performance engineer will decide about the types of execution needed for testing. To know more about types of performance test execution read this article on Types of Performance Testing
result preparations and analysis
In performance testing process, result preparation and its analysis is the crucial phase of entire performance testing life cycle. Performance engineer looks into different performance metrics like response time, throughput, error rates, resource usages, downtimes, outages and report them to different teams like development, infrastructure, devops and many more.
Performance Bottleneck / Defect FIX, Retest
In result analysis phase if a performance engineer is identifying a performance bottleneck, then performance test engineer will collect more data related to performance bottleneck and communicate with the relevant team with a defect. Developers will always be benefit by a detailed analysis and report on the identified bottleneck. Defects will help performance engineers to track the progress on fix for the identified performance bottleneck.
Performance Signoff
In the final stage of performance testing life cycle, an application is certified stating as “GOOD TO GO” to production followed by production monitoring.
In this article on JMeter API testing, we will be discussing the details on API’s, different tools that is used for API testing and also on how to develop an API script for testing using JMeter tool. But before discussing about JMeter API testing, mentioning few details about API’s and tools available to do an API testing.
What is API?
Application Program Interface (API) is a communication layer between different components in a multi-tier application architecture. API’s plays a key role in ensuring an effective communication between different components. Let us try to understand APIs with an example of an ecommerce website. Let’s assume there is an ecommerce website and multiple customers purchases products from this website. Here let us assume that API 1 is a product price related Api and API 2 is product quantity related Api.
Now while customers trying to purchase a product(s), they get information related to price and quantity related to the products on the website is because these Api’s are always interacting with the database to fetch this information and providing it to the customer through the website interface on a real time basis. In this example we have mentioned only two Api’s but in a real-world scenario many API’s will be interacting with multiple components throughout the day.
What is API Testing?
API Testing is a testing practice performed to validate the Api’s are working well in terms of its functionality,performance,security and reliability. It plays an important part of testing life cycle specially during integration testing.
API Testing Tools
Many tools are available in the market to do API testing. Few of the tools are free tools which can be used to test your Api’s. Few of the widely used tools to do API testing is mentioned below.
SoapUI
Postman
Apache JMeter
Katalon Studio
Apigee
JMeter API Testing
As mentioned earlier in this article API testing is important to validate an API’s functionality, performance, security and reliability. Hence it is again very important to focus on API’s performance because high performing or faster APIs are always needed to make faster communication between the software components. JMeter API testing can be useful particularly while we test API to understand the performance of it.
Below mentioned is a step-by-step approach in developing a JMeter script as part of your JMeter API testing. For creating a JMeter api script making use of some API’s available in Reqres website.
JMeter API Testing – Step1 -Setup JMeter Test plan
To begin with launch JMeter and add below mentioned JMeter components.
Right click on Test plan and select threads<<>>Thread groups
Right click on Thread groups and then select add sampler<<>>HTTP Request
Right click on Thread groups and then select add Listener<<>>View Result trees
Right click on Thread groups and then select add Listener<<>>Aggregate reports
Once the above listed JMeter components are added your JMeter screen will look similar to below mentioned screenshot.
JMeter API Testing – Step2 -Setup JMeter HTTP Request
Click on HTTP request and will select an api request details from Reqres website. In this example using a Create User request API having a POST method. Ideally an api request will have a request URL, request body, request header and a method. The api used here have a request URL, request body and a method.
In this example request details is as follows.
Request URL is https://reqres.in/api/users
Request body is
{
"name": "morpheus",
"job": "leader"
}
Request method is POST
Now in the HTTP request add the details of the request as mentioned below .
HTTP Request Name – Give any suitable meaningful name of your choice
Protocol – HTTP or HTTP(s), In this example it is https:
Server name – domain name, In this example it is reqres.in
Method – Select the API method, In this example it is POST
Path – Add the subdomain, In this example it is /api/users
Request Body – Add the request body, In this example it is { “name”: “morpheus”, “job”: “leader” }
Note – Incase if your API request have a request header then you need to make sure to add the request header along with your http request. For adding it, right click HTTP Request, select Add config element <<>> HTTP Header Manager.
JMeter API Testing – Step3 -Run HTTP Request and Validate Response.
Now the JMeter API script is ready to be executed. Execute the script clicking on start button (green play button) in the JMeter icons. Validate your response by clicking View results Tree. The performance summary report of API test execution is available in the Aggregate Report (Ref screenshot below)
