14 February 2012

Remote Java Profiling using Visual Vm

This blog entry details how to profile a JVM running on a remote box using Visual Vm. This enables you to monitor the performances of all the Java processes running on the remote JVM.


What is Visual VM?
Visual VM is an open-source profiling tool that comes shipped with the JDK nowadays (JDK7+). It can be used to profile JDK1.5+. Using the tool is very self-explanatory if you have used other profilers, check out the vsiual vm getting started / introduction documentation if you haven't used a java profiling tool before. 


Setting up your localhost
Download and install VM on your machine, important GOTCHA ensure you start visual VM with the same version of the JDK that the remote host is using. To specify the JDK start with the following argument:


--jdkhome JDK_HOME visualvm (where JDK_HOME is the path to your JDK install directory)


Ok that's the easy part now we need to get Jstatd running on the remote host to be able to hook into the JVM.

Adding a remote host in Visual VM


Setting up JStatd
Jstatd is an RMI service that provides an interface to hook into/ monitor JVM's. WWe will create a basic set up with all security switched off for the quickest of set ups. If you want to enable security read the Java docs which explain how to set up policies.

Jstatd is provided with your jdk, to use it we need to create a policy file which will contain the security policy details. Create a file called jstatd.all.policy and enter the following syntax:




Save the file in JDK_HOME/bin. Now start Jstatd from the command promptly using the following command:

jstatd -J-Djava.security.policy=jstatd.all.policy.

 If you receive the error message detailed below then it means the Jstatd RMI server was unable to find your policy file ensure the  java.security.policy jvm argument matches your filename and that the file name lives in the correct JDK bin directory e.g. C:\PROGRA~1\Java\jdk1.6.0_29\bin.



Once working a Jstatd connection should have automatically appeared on your remote host that you set up in visual vm. When a process is started on the remote vm this should also appear under the remote host with a PID. You can now profile each PID, however to get the full profiling capabilities you will have to set up a JMX connection.


Creating a JMX Connection for more profile information
JMX (Java Management Extensions) has been around since JDK1.5 and is also part of your JDK/JRE.It allows remote monitoring of the JVM using Managed Beans which you connect to via RMI and other network protocols such as http/soap. Most Application Servers support this and you can even run a standalone Java application with JMX.

Again we are going for a basic set up with security switched off. You also need to specify a port number, in my example I have used 6643 but you can choose any port just ensure there isn't another service already using this. To use jmx start your remote process with the following java arguments:

-Dcom.sun.management.jmxremote.port=6643 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Once the process has started go back to Visual Vm and add a JMX connection on the remote host, the connection string should be hostname:port number e.g. 163.189.94.16:6643. If you receieve the error messsage 'Cannot connect to hostname:port using service:jmx:rmi:///jndi/rmi://hostname:port/jmxrmi'. This means that the JMX did not start correctly on your remote process, ensure the port number and hostname is correct. You can always run JConsole on the remote machine to check if JMX has started. To start it simply run JDK_HOME/bin/jconsole.

CPU Sampling on a Remote JMX Connection in Visual VM


12 February 2012

JMeter: Load testing a Java Web Application which requires multiple User Login's

Background: What JMeter is and what problem it solved for me

Jmeter is an open-source load testing tool from Apache which can be used to load test Web Applications amongst other things (Web Services, Databases etc). This blog entry gives an overview of what I learnt from my first experience of using JMeter and is designed to be read by anyone trying to test a Web Application that requires a log in with multiple different users.

The problem
One of my clients Web Application's was having performance issues around a piece of functionality running a sysbase stored procedure which caused a high number of locks under heavy load. To re-create this scenario JMeter needed to log into the application as 10 different users, go through a short web flow (which included filling in a form), this was then repeated (forever over a period of 2 hours). So if you need something similar hopefully this will help. If you just want a basic static web flow go to the Jmeter web starter tutorial.

Creating the basic Test Plan fundamentals

Setting up a Thread Group to represent your users
The first thing you'll need to do is create a thread group to represent the users - here you define the number of users, repetitions and delay time between each user. Then add a HTTP Request Defaults (which will contain shared default data for your http requests) and a HTTP Cookie Manager to take care of cookie handling to this thread group. Next you'll want to see some output from your load test so add a Graph Results listener to your user group to see a simple graphical representation of load time and also add a Tree Results listener which will list all the request/response data (very handy for debugging).


Creating a thread group: this group has 10 users with a second pause in between each user looping continuously for 2 hours


Making the HTTP requests
Set up your Http defaults to contain your web server/host (any ports), also set the protocol to default to https (if needed) and To create a basic HTTP GET request add a HTTP Request Sampler to your thread group and enter your relative path e.g. / for homepage (this assumes that you have your host/domain entered in your http request defaults) and that's it. Run the test and look in the tree results to ensure you are getting a valid 200 response back. You can add extra Http Headers to the request (e.g. Refer or User-Agent) by right-clicking on your http request Sample and adding a Http Headers Manager - here you just add key-value pairs for the headers.



To make a POST request simply set the Method drop down to POST in your Http Request Sampler. For your POST to work correctly for a form set the path to the target of the form and add the form field values as key-value pairs (by clicking on add under the ' send parameters with request' section) ensuring the field name matches that in the html (also include any hidden fields). Ensure that the 'Use multipart/form-data for POST' is ticked. Again when you run this the tree results are very handy to check that all the correct data is being sent in the request.

Making a HTTP Post Request: this request logs into a web application using data in a CSV file


Making the POST request data unique for each user (thread) allowing multiple different logins
You can use placeholders in your parameter value as opposed to a hardcoded value these placeholders are then replaced with a value found in a row within a given CVS file. To do this create a common delimited CSV file for example for log in details:

username1,password1
username2,password2

Then add a CSV Data Set Config to your Http Request Sampler, specify your filename and columns (Variable Names) e.g. username,password. You then put the variable name placeholder in your post param data e.g. $(username) $(password). For a more detailed explanation go here xxx.

Increasing the load
Once everything’s in place simply tweak the loop values in your User Group settings or increase the number of values, you can even schedule the job to run.

Running behind a proxy?
If you need to run Jmeter behind a proxy then simply start Jmeter with the following parameters  like so;  jmeter -H proxy.rta.nsw.gov.au -P 8888 -u username -a password -N localhost 

Conclusion
JMeter was quick and easy to set up as a developer and can be a very useful tool to try and re-create a performance issue/load problems when used along side a Java profiler such as visual vm.