Tuesday, November 13, 2018

Distributed (Remote) Testing Using Jmeter

JMeter: Distributed (Remote) Testing

JMeter is popular open source load testing software from Apache and it is a Java Graphical application. The easiest way to run it in Amazon EC2 is in a windows server so it could be easily access via RDP.

JMeter has a master/slave architecture where the client is the master and multiple JMeter servers are slaves running tests in a scalable fashion. I tried to set up an environment where the master instance would be hosted by my workstation behind corporate firewall but due to the corporate polices with regards to setting up inbound rules in the firewall it proved to be a challenge.

So, I gave up and went with the approach where I set up  both the master and client(s) in Amazon EC2, the master on a large instance so it has the CPU to process the graphs and the slave(s) on medium instances that have the network capacity to run the tests.
In order to be able to set up distributed environment you have to have some basic knowledge of how to spin up and Amazon server image and set up security groups (and the concept of AWS security groups). You have to be logged into AWS console in order to accomplish it.

Before we dive into the step-by-step instructions, it's a good idea to define the terms and make sure the definition is clear.
Master– the system running Jmeter GUI, which controls the test
Slave– the system running jmeter-server, which takes commands from the GUI and send requests to
the target system(s)
Target– the webserver to stress test

Important notes

Make sure that all the nodes (client and servers) :
  • are running exactly the same version of JMeter.
  • are using the same version of Java on all systems. Using different versions of Java may work but is discouraged. I had to download in install the same version of JDK across all servers that I used in the test.
If the test uses any data files, note that these are not sent across by the client so make sure that these are available in the appropriate directory on each server. In my test, on the server machine I had to define the following path, that I had defined in the 'User Defined Variables' section of my Java script.

Step-by-step guide

  • Log in AWS console.
  • Define two security groups, one to be used by the master and another one by the slave. Go with default values, you will tailor them later once you have spun up the server images.
  • Pick an Amazon Windows Image. In my case I selected  64 bit Windows 2008 R2 Server image (the same was selected for the second image). Launch a large instance (for the master). Take note of its public IP addresses. In my case I had ''. Make sure that port 3389 (security group) open for RDP, the same would apply for the second image (see below)
  • Launch a medium instance to be used as a host for a slave. In a case when a few slaves are required then more instances should be launched. Take note of their public IP addresses. I spun a single instance with the following IP address: ''
  • Edit the (master) security group with the following inbound rule (see the screenshot below). The IP address of the slave's host should be used with the CIDR notation, that is the /32 prefix should be appended to the IP address. Note, I picked an arbitrary port 51000. Could be any port as long as it is not in use by any other process. Once you have edited it, the changes will be applied on the fly to the created image.
  • This port would be used to transfer test results from the slave server over to the master.
  • Edit the (slave) security group with the following inbound rule (see the screenshot below). The IP address of the slave's host should be used with the CIDR notation, that is the '/32' prefix should be added to the IP address. Note, I picked an arbitrary port 4000. Could be any port as long as it is not in use by any other process.
  • This port will be used for the master server to pass a set of test instructions to the slave machine. The master will be listening to this one port only to receive results. That means that all of the slaves will be sending their results back through this port.
  • Now, navigate to the slave host and into the [JMETER_HOME]/bin directory. Open jmeter.properties file and make sure to edit the following properties in the Remote hosts and RMI configuraion section of this file:
  • Now, on the slave host in the same [JMETER_HOME]/bin directory launch the JMeter application via command line, making sure to pass the public IP address of the slave's host. YOu are going to end up with the following output:

  • On the master machine similar to step one edit a few properties as indicated below in the jmeter.properties file
  • On the master machine launch JMeter GUI and select the  slave's host in the shortcut menu which should get the test going:


Hope this information will be useful, please provide your valuable feedback/suggestions !! Happy Testing.

Wednesday, March 21, 2018

Essential Skills for Performance Tester

Based on the discussion we had among the experienced performance testers/ engineers we came up with the essential skills that is required to be in Performance Testers

17 essential skills for growing web performance engineers
Requirement Gathering and System Level Skills:

1) Able to Understand the System Architecture (Knowledge of components ,OS , techology etc)
2) Have Basic understanding of System hardware Resource Like CPU , Memory, Network, Disc etc
3) Should Know about the Diffrent Types of Test Environments
4) Able to Know and understand  Functional Level Workflows
5) Able to gather information about Scenario of Currnt Production Load and Expected Load
6) Able to gather the information about SLAs

Scripting and Testing Skills:

1) Able to Install and Configure testing tools
2) Able to record and enhance the scripts
3) Understand and Prepare the Test Data
4) Able to create different scenarios (Load,Stress, real Life ) based on the expectations
5) Running the load test executions for the above created scenarios

Monitoring and Result Analysis Skills:

1) Able to analyze the load test tool results
2) Configure server monitoring counters
3) Bottleneck Identification
4) Reporting

Feel free to add in comments if i have missed anything

Tuesday, January 9, 2018

Using JSON extractor in Jmeter: How to deal with dynamic values in JSON response

How To use JSON Extractor in Jmeter

Json extractor is used like regular expression extractor to capture some dynamic values from the response but used for json only.

By using this we can fetch the values of variables in the Json.

Let me take an example

Below is the Json and its corresponding response:

Here we want to extract the value of variable name "label" or "val"
 which are changing dynamically

We can use Json extractor to fulfil thie purpose here

below is the JSON values that can be used

In this Json extractor blow are the parameters and their corresponding values

Variable Name: Name of the variable (can be used anything logical, here we have used the same as we are getting in response)

JSON Path expression: It is the most important part for JSON configurations, We should use the path extractor expression here.
 Here we have used $..lable , Here two dot means we have json variable in two nested blocks, For normal JSON where there is only one tag , we used single dot.

Match Number: 0 means any random value, here we can use 1 or 2 to get first value or second value and so on, we can also use -1 for getting all possible values to be stored in array (with variable name as mentioned above)

Default Values: this values that can be passed when we didnt got the expected value.

TO varify the value captured in Json extractor:

We can use the Debug Sampler on the thread group to validate the value captured using Json extarctor, Debug sample is used to print all the used defined configured variable in jmeter.

Here we can see that variable "label": captured SpiceJet , and lable_matchNr having value as 10 because there are 10 occurrence of the values corresponding to the expression we have defined for variable label in JSON extractor.

Feel Free to add if i have missed anything.

Tuesday, December 12, 2017

Client Side Response Time/Rendering Time and Server Side Time of Mobile app/Web Application

When we talk about the performance testing of any application weather it is web application of Mobile App. First question comes in our mind is about the user experience of the application.

Most of the tools that are available in the market gives the stats of only one component of this scenario i.e. they only provide the server-side response time. 

But client-side loading time/rendering time is also point of interest if we are taking about the end to end response time.

though not very useful information is available over the google from where you can get the client-side response time and server-side response time as well., but when there is an issue with the performance of the application then it becomes necessary to segregate even if the issue is at server side of at client side.

We can achieve this by a simple formula:

Capture over all page load time
Capture TTFB [Time to first buffer] 

1) There is an issue in page at server side if TTFB is high and difference of load time and TTFB is very low
2) There is an issue at client side if TTFB is very low and high page load time is there

Even we can get the segregation of server-side response time as well as client side response time using the Fiddler while capturing the Mobile App/Web App requestes:

You can easily understand these from the below screenshot:

Below parameters will give us this segregate results:

·  FiddlerBeginRequest – The time at which Fiddler began sending the HTTP request to the server.
·  ServerGotRequest - Exact time that Fiddler finished (re)sending the HTTP request to the server.
·  ServerBeginResponse - Exact time that Fiddler got the first bytes of the server's HTTP response.
·  ServerDoneResponse - Exact time that Fiddler got the last bytes of the server's HTTP response.
·  ClientBeginResponse - Exact time that Fiddler began transmitting the HTTP response to the client browser.
·  ClientDoneResponse- Exact time that Fiddler finished transmitting the HTTP response to the client browser.

Capturing Response time for Andriod App Launcher

Below are the scenario that we have tried to capture the response time of Google Now Launcher activities .

We have tried to capture the response time of below activities taking it as Minor Project : 

Minor Project – QA
Evaluate the performance of Google Now Launcher from Play Store.
• Install the latest version of Google Now Launcher
• Report the average application opening time of Hot-seat icons
• Report the average time to navigate to App Drawer
• Report the average application opening time of App Drawer icons
• Test that on long press the Launcher enters overflow mode
• Test that user is able to change wallpaper of Launcher

All the activities that are mentioned above are client side only and leg time for the same depend on the RAM in the test device, Though we can easily get the matrix for CPU, Memory,Network utilization on device but for capturing leg time, I have tried the following scenarios

1) Manually Using Stopwatch:
                 Major drawback of capturing Response time/Lag Time using this scenario is the non accuracy of the results as response time is in less than a second.
2) Using ADB Logs
                  We were expecting the time to be logged in Activity manager line item in the ADB logs, but for application(Purely Client side) like Launchers will not have any activity Manager line item in Logs., This is possible for Native app to get the response time of user activity 

Inline image 1

3) Using Debug Proxy : Not able to record any client side request in Google Now Launcher

4) Using Fiddler : Seems it can give the client side response time, we have tried for some web apps, (refer to attached snapshot), we are facing some issue while downloading and installing fiddler certificate for Andriod and trying to get the same.

5) Using Automation tools(Thought):
                 We can capture the client side response time using  the customization logs in the automation scripts to capture the time stamps , But when we tried this solution for app launhers, we were not able to get the response time for the same.

Apart from these , i have also tried many other things that i found on google, but was unabale to capture the same.

Thursday, October 26, 2017

Performance and Load testing using Neoload - Mobile Application -Part 1

Neoload Record Any Mobile App
·       Direct recording from any device or emulator
·       Recording of native, hybrid and mobile browser apps

Steps for Recording Mobile Application
·       Mobile and your laptop should be on same network on which neoload is working.
·       Configure the cell phone to use Neoload as a proxy.
·       In Mobile, Open Settings à Go To WIFI à Long press the connected networkà Go for Modify network àShow advance Setting àManual à Set Proxy (Input the laptop ip and Neoload port i.e. 8090).
·       Start recording in Neoload
·       Create a project, Below screen will appear.
Recording window

·       Click on start recording .Below screen will appear.

·       Neoload supports client to behave as different option, For mobile select identify as.

·       For Recording through mobile proxy remove the selection from start client and click on OK.

·       Scenario Considered
o   Open xyz App
o   App Dashboard appear
·       Mention the transaction name , Request will get recording in the same as you proceed.
·       Click on stop after recording the scenario the script get recorded.

Let me know in case of any query and keep connected for next part .

Tuesday, June 13, 2017

Mobile App Performance Testing -Part 3

Mobile App Android Client Side Performance Profiling

We have already studied about "How we can perform the server side load testing of Mobile app

Now in this pot we will see the basic concepts and steps to perform the client side profiling of Mobile App

Below techniques can be used:

 Reading android logs using Mlogcat : 

1) Download the MLogcat tool
2) Connect Your Mobile(having debugged app) with system
3) Logs are started in front of you
4) Apply filter based on the process id
5) Copy -paste the logs in excel & delimite to find the elapsed time and queries
6) Report these slow running/duplicate queries and time to developers

Capture Device Hardware Utilisation  Mlogcat :

You can capture the real life CPU Usage/RAM Usage and Network Usage of device

Monitor Thread and Heap Using DDMS:

1) Download Andriod Studio
2) Attached Mobile with Debug build of application
3) Go to Preferences --> DDMS
4) Select your app from left pane
5) Go to Heap tab and monitor the heap
6) Take dump when heap usage reaches close to 100%
7) Analyse the heap using Eclipse Memory analyser for Prime suspects

Feel Free to contact us in case of any query

Join "Performance Testing Expert" Group On Facebook