|home |articles |cartoons |site map |contact us |
Tools: | GC log analysers| Multi-tenancy tools| Books| SizeOf| Thread analysers|

Our valued sponsors who help make this site possible
New Relic: Try free w/ production profiling and get a free shirt! 

Site24x7: Java Method-Level Tracing into Transactions @ $12/Month/JVM. Sign Up! 

Hotpatching A Java 6 Application: page 1 of 2

jKool for DevOps
Light up your Apps & get a cool t-shirt

Get rid of your performance problems and memory leaks!

Java Performance Tuning, 2nd ed
The classic and most comprehensive book on tuning Java

Java Performance Tuning Newsletter
Your source of Java performance news. Subscribe now!
Enter email:

New Relic
New Relic: Try free w/ production profiling and get a free shirt!

Site24x7: Java Method-Level Tracing into Transactions @ $12/Month/JVM. Sign Up!

Get rid of your performance problems and memory leaks!

jKool for DevOps
Light up your Apps & get a cool t-shirt

Hotpatching a Java application allows you to fix arbitrary code-level problems in a running application without terminating that application. Here Jack Shirazi covers the basics of how to hotpatch a Java 6 application
Published February 2007, Author Jack Shirazi

Page 1 of 2
next page: Finishing the hotpatching process

Last month the Java Performance Tuning newsletter special on Java 6 lead to an interesting discussion between myself, Kirk Pepperdine the and TheServerSide columnist, and Heinz Kabutz, the author of the excellent Java Specialists Newsletter, about how to monitor things using the new dynamic attach facility available in Java 6 we announced in last month's newsletter.

During the discussion, we realized some interesting things become possible with the combination of features in Java 6, including the rather amazing capability that you can patch a running Java process without stopping it. In this article, I'm going to walk through the basic procedure to show you one way how this could be achieved. I'll use a simple program which adds and deletes objects to a stack to illustrate the technique. My program adds and deletes objects at the same average rate, and queuing theory indicates that such a queue is potentially unbounded. The example is specific, but the technique I'll use to monitor and fix my app is generic. To keep the code as readable as possible, I've defined everything in the default package and used the current working directory as my classpath in all cases. All code, with unix & windows batch files, can be downloaded here

Initial Diagnosis

My process is up and running, and there is something wrong. I'm not getting the output I was expecting. My app is supposed to be reporting the first ten elements in the stack every once in a while, and I was expecting to see a variable number of elements listed, but I only see one element in the stack every time I get the report.

The output from Main is just one element from the stack
Figure 1: Only one element is being output

For an abitrary application I would follow standard practice and turn up the debug level in the area of functionality that I think should tell me what is happening, and I would also generate stack dumps (send kill -3 to the process in linux/unix or type cntrl-\ in the Java command console in unix/linux or cntrl-break in the Java command console in Windows or generate from jconsole). That is the normal technique to narrow down the problem, but in this case even if I do apply those techniques it doesn't help me to narrow down the problem - and that is not untypical for many problems I've seen in production.

public class Main {
    static Stack Pile = new Stack();

In this case, the Stack that I'm reporting is actually a static object in my Main class. If only I could look at the elements in that stack. Maybe my understanding of queuing theory is all wrong and there should only be one element in the stack. Of course I can kill the process, recompile it, and dump the whole object next time round, but that is not the aim of this article. We want to see how to examine the object without having to stop the process.

Attaching To The Process

So how do I dump and examine that stack? First I need to attach to my process. The basic technique for attaching to a Java 6 process and executing code is detailed in Using Mustang's Attach API (A. Sundararajan). I'll detail it again here though. First I need an "attacher" process - that's straightforward:

public class Attach {
    public static void main(String[] args) throws Exception {
        // This program accepts two parameters:
        //    1. The pid of the JVM on which we want to load an agent
        //    2. The full path of the agent jar file to be loaded

        //Attach to the target Java process
        VirtualMachine vm = VirtualMachine.attach(args[0]);

        // load a specified agent onto the JVM
        vm.loadAgent(args[1], null);

Given the process ID of the target process to attach to, a single call attaches us to the process - VirtualMachine.attach(PID). Note that VirtualMachine is Sun specific class in the package, so this code only works with the Sun JVM, other JVMs should have equivalent attach methods.

The only tricky bit to remember about the attacher is that to compile and run it you need to include the tools.jar from the java distribution in the classpath (e.g. from %JAVAHOME%\lib\tools.jar or $JAVAHOME/lib/tools.jar).

Once attached, a second VirtualMachine method call loads our agent, and we are into our own code from there. That's all there is to attaching and starting our agent!

Getting State

What I need next is the actual agent that will execute inside my process. This can be any class, I just need to keep in mind the entry point - when the class is loaded into my application it will get its agentmain() method called. The basic technique to access the object is some simple reflection. The static I'm after is Main.Pile. To access that using reflection is easy enough: Class.forName("Main").getDeclaredField("Pile").get(null).

So my agent class is actually quite simple (I just make it defensive with a generic try catch just in case I've made a mistake somewhere - I don't want my investigation to unintentionally crash the app):

public class DumpPile {
    public static void agentmain(String agentArgs, Instrumentation inst)
            System.out.println("Dumping Main.Pile ...");
            Object o = Class.forName("Main").getDeclaredField("Pile").get(null);
            System.out.println("Pile.size() == "+ ((java.util.Stack)o).size());
        catch(Exception e)

Running this against my server results in the stack getting dumped to stdout, and I'm not really surprised to find that my stack is much bigger than one element! The subsequent output line from my agent of "Pile.size() == 124" confirms that the list is significantly more than one element.

output from Main now shows a dump of the Stack object with 146 elements
Figure 2: We get the dump of the Stack

Page 1 of 2
next page: Finishing the hotpatching process

Last Updated: 2017-01-04
Copyright © 2007-2017 All Rights Reserved.
All trademarks and registered trademarks appearing on are the property of their respective owners.
RSS Feed:
Trouble with this page? Please contact us