JVM Runtime Data Areas

The Java virtual machine defines various runtime data areas that are used during execution of a program. Some of these data areas are created on Java virtual machine start-up and are destroyed only when the Java virtual machine exits. Other data areas are per thread. Per-thread data areas are created when a thread is created and destroyed when the thread exits Stack Stack is a memory place where the methods and the local variables are stored. Variable references (either primitive or object references) are stored in the Stack. Each thread has a private stack, which stores frames. The following exceptional conditions are associated with Java virtual machine stacks

If the computation in a thread requires a larger Java virtual machine stack than is permitted, the Java virtual machine throws a StackOverflowError.

If Java virtual machine stacks can be dynamically expanded, and expansion is attempted but insufficient memory can be made available to effect the expansion, or if insufficient memory can be made available to create the initial Java virtual machine stack for a new thread, the Java virtual machine throws an OutOfMemoryError

Heap Heap is a memory place where the objects and its instance variable are stored. Each time an object is created in Java it goes into the area of memory known as heap. Information about HEAP The heap is created on virtual machine start-up. Heap storage for objects is reclaimed by an automatic storage management system (known as a garbage collector); objects are never explicitly deallocated. The heap may be of a fixed size or may be expanded as required by the computation and may be contracted if a larger heap becomes unnecessary. The memory for the heap does not need to be contiguous. A Java virtual machine implementation may provide the programmer or the user control over the initial size of the heap, as well as, if the heap can be dynamically expanded or contracted, control over the maximum and minimum heap size. The following exceptional condition is associated with the heap: If a computation requires more heap than can be made available by the automatic storage management system, the Java virtual machine throws an OutOfMemoryError

Heap and stack are the two main memories that JVM is concerned as far as memory allocation by the OS (operating system) is concerned. Stack and heap are the memories allocated by the OS to the JVM that runs in the system.

JVM Internals & Runtime Data Areas.doc

Page 1

The primitive variables like int and double are allocated in the stack, if they are local method variables and in the heap if they are member variables (i.e. fields of a class). In Java methods local variables are pushed into stack when a method is invoked and stack pointer is decremented when a method call is completed. In a multi-threaded application each thread will have its own stack but will share the same heap. This is why care should be taken in your code to avoid any concurrent access issues in the heap space. The stack is threadsafe (each thread will have its own stack) but the heap is not threadsafe unless guarded with synchronization through your code. Method Area It is shared among all threads. It stores per-class structures such as the runtime constant pool, field and method code. It is logically part of the heap. Memory for class (static) variables and methods declared in the class is also taken from it. Runtime Constant Pool Or Constant Pool It is a per-class or per-interface runtime representation of the constant_pool table. The JVM maintains a per-type constant pool, including literals (string, integer, and floating point constants). Each runtime constant pool is allocated from the JVM method area. Frame or Stack Frame Frame holds local variables (including parameters), operand stack and partial results, and plays a part in method invocation and return. A new frame is created each time a method is invoked and is destroyed on method completion. Each frame has a reference to the runtime constant pool of the class of the current method. PC Register The Java virtual machine can support many threads of execution at once. Each Java virtual machine thread has its own pc (program counter) register. At any point, each Java virtual machine thread is executing the code of a single method, the current method for that thread. If that method is not native, the pc register contains the address of the Java virtual machine instruction currently being executed. If the method currently being executed by the thread is native, the value of the Java virtual machine's pc register is undefined. The Java virtual machine's pc register is wide enough to hold a returnAddress or a native pointer on the specific platform. An Example class Memory { static int x; public static void main(String[] args){ Memory memory = new Memory(); int y =0;

/* static stack storage*/ /* dynamic heap storage*/

/* dynamic stack storage */ String myString = new String(Memory); /* dynamic heap storage

*/
} }
When you create an object using the new operator, for example memory = new Memory ();, it allocates memory for the memory object on the heap. The stack memory space is used when you declare automatic variables. Note, when you do a string initialization, for example String myString;, it is a reference to an object so it will be created using new and hence it will be placed on the heap.

JVM Internals & Runtime Data Areas.doc

Page 2

Memory space for objects is always allocated in heap. Objects are placed on the heap. Built-in datatypes like int, double, float and parameters to methods are allocated on the stack.

JVM Performance Tuning

The application server, being a Java process, requires a Java virtual machine (JVM) to run, and to support the Java applications running on it. As part of configuring an application server, you can fine-tune settings that enhance system use of the JVM

Heap Size
The allocation of memory for the JVM is specified using -X options Options -Xms -Xmx -Xmn Meaning initial java heap size maximum java heap size the size of the heap for the young generation

It is good practice with server-side Java applications like Resin to set the minimum -Xms and maximum -Xmx heap sizes to the same value. For efficient garbage collection, the -Xmn value should be lower than the -Xmx value. Heap size does not determine the amount of memory your process uses. If you monitor your java process with an OS tool like top or taskmanager, you may see the amount of memory you use exceed the amount you have specified for -Xmx. -Xmx limits the java heap size, java will allocate memory for other things, including a stack for each thread. It is not unusual for the total memory consumption of the VM to exceed the value of -Xmx.

Stack Size
Each thread in the VM get's a stack. The stack size will limit the number of threads that you can have, too big of a stack size and you will run out of memory as each thread is allocated more memory than it needs. Options -Xss Meaning the stack size for each thread

-Xss determines the size of the stack: -Xss1024k. If the stack space is too small, eventually you will see an exception java.lang.StackOverflowError.

JVM Internals & Runtime Data Areas.doc

Page 3

Internal architecture of the Java virtual machine

JVM Internals & Runtime Data Areas.doc

Page 4

Runtime Data Areas

Native Method Stack Shared among all threads

JVM
THREAD 1 PC STACK Per method Area Frame Frame Operand Stack Local Variables Referenc es Partial Results HEAP Class instances and Arrays Instance Variable Per class basis Method Area
Runtime Constant Pool Method Code Static Variables

THREAD 2 PC STACK Per method Area Frame Frame

Local Variables Operand Stack Partial Results

Dealing with Memory Leaks

What is it?
Memory leaks occur when a program never stops using an object, thus keeping a permanent reference to it. For Example class MemoryLeak { private static List<Double> memoryLeakList = new ArrayList<Double>(); public static void main(String[] args){ double d = 0.0; // infinite loop would eventually run out of memory while(true){ Double dbl = new Double(d); memoryLeakList.add(dbl); d++; } } } If you run this program, When no more memory is remaining, an OutOfMemoryError alert will be thrown and generate an exception like this -

JVM Internals & Runtime Data Areas.doc

Page 5

Exception in thread main java.lang.OutOfMemoryError: Java heap space at MemoryLeak.main(MemoryLeak.java:7) In the example above, we continue adding new elements to the list memoryLeakArea without ever removing them. In addition, we keep references to the memoryLeakArea, thereby preventing GC from collecting the list itself. So although there is GC available, it cannot help because we are still using memory. The more time passes the more memory we use, which in effect requires an infinite amount memory for this program to continue running. This is an example of unbounded memory leakthe longer the program runs, the more memory it takes. So even if the memory size is increased, the application will still run out of memory at a later date.

Determining if an application has a memory leak

To understand what is going on, we need to familiarize ourselves with how the JVM uses system memory for its heap. When running java.exe, you can use certain options to control the startup and maximum size of the garbage-collected heap (-ms and -mx, respectively). The Sun JDK 1.1.8 uses a default 1 MB startup setting and a 16 MB maximum setting. The IBM JDK 1.1.8 uses a default maximum setting of one-half the total physical memory size of the machine. These memory settings have a direct impact on what the JVM does when it runs out of memory. The JVM may continue growing the heap rather than wait for a garbage collection cycle to complete. So for the purposes of finding and eventually eliminating a memory leak, we are going to need better tools than task monitoring utility programs. Memory debugging programs can come in handy when you're trying to detect memory leaks. These programs typically give you information about the number of objects in the heap, the number of instances of each object, and the memory being using by the objects. In addition, they may also provide useful views showing each object's references and referrers so that you can track down the source of a memory leak. This example I have taken from http://java.dzone.com Not every OutOfMemoryError alert indicates that a program is suffering from a memory leak. Some programs simply need more memory to run. In other words, some OutOfMemoryError alerts are caused by the load, not by the passage of time, and as a result they indicate the need for more memory in a program rather than a memory leak. To distinguish between a memory leak and an application that simply needs more memory, we need to look at the peak load concept. When program has just started no users have yet used it, and as a result it typically needs much less memory then when thousands of users are interacting with it. Thus, measuring memory usage immediately after a program starts is not the best way to gauge how much memory it needs! To measure how much memory an application needs, memory size measurements should be taken at the time of peak loadwhen it is most heavily used.

JVM Internals & Runtime Data Areas.doc

Page 6

The graph below shows the memory usage in a healthy Java application that does not suffer from memory leaks, with the peak load occurring around 10 AM and application usage drastically decreasing at 5 PM. Naturally, the peak load on business applications often correlates with normal business hours.

The application illustrated by the chart above reaches its peak load around 10 AM and needs around 900MB of memory to run. This is normal behavior for an application suffering from no memory leaks; the difference in memory requirements throughout the day is caused solely by the user load. Now, lets suppose that we have a memory leak in the application. The primary characteristic of memory leaks is that memory requirements increase as a function of time, not as a function of the load. Lets see how the application would look after running for a few days with a memory leak and the same peak user loads reached around 10 AM every day:

JVM Internals & Runtime Data Areas.doc

Page 7

Because peak loads on the system are similar every morning but memory usage is growing over a period of a few days, this picture indicates a strong possibility of memory leaks. If the program eventually started suffering from OutOfMemory exceptions, it would be a very strong indication that theres a problem with memory leaks. The picture above shows a memory leak of about 100MB per day. Note that the key to this example is that the only thing changing is the amount of time the system is upthe system peak load doesnt change over time. This is not the case for all businesses. For example, the peak load for a tax preparation service is seasonal, as there are likely more users on the system in April than July. There is one special case that should be noted here: a program that needs to be restarted periodically in order to prevent it from crashing with an OutOfMemoryError alert. Imagine that on the previous graph the max memory size was 1100MB. If the program started with about 900MB of memory used, it would take about 48 hours to crash because it leaks about 100MB of memory per day. Similarly, if the max memory size was set to 1000MB, the program would crash every 24 hours. However, if the program was regularly restarted more often than this interval, it would appear that all is fine. Regularly scheduled restarts may appear to help, but also might make upward sloping memory use (as shown in the previous graph) more difficult to notice because the graph is cut short before the pattern emerges. In a case like this, youll need to look more carefully at the memory usage, or try to increase the available memory so that its easier to see the pattern.

JVM Internals & Runtime Data Areas.doc

Page 8

Monitoring Memory in Java

There are a couple of options for measuring the amount of memory a program uses. The simplest one, which does not require any tools and works even with production systems, is the Verbose GC log. Verbose GC Log The Verbose GC log is defined when the JVM process is started. There are a couple of switches that can be used: 1. 2. 3. 4. -verbose:gc prints basic information about GC to the standard output -XX:+PrintGCTimeStamps prints the times that GC executes -XX:+PrintGCDetails prints statistics about different regions of memory in the JVM -Xloggc:<file> logs the results of GC in the given file

The following is an example of the output generated for Tomcat running in the default configuration with all of the previous switches enabled:
1.854: [GC 1.854: [DefNew: 570K->62K(576K), 0.0012355 secs] 2623K->2175K(3980K), 0.0012922 secs] 1.871: [GC 1.871: [DefNew: 574K->55K(576K), 0.0009810 secs] 2687K->2229K(3980K), 0.0010752 secs] 1.881: [GC 1.881: [DefNew: 567K->30K(576K), 0.0007417 secs] 2741K->2257K(3980K), 0.0007947 secs] 1.890: [GC 1.890: [DefNew: 542K->64K(576K), 0.0012155 secs] 2769K->2295K(3980K), 0.0012808 secs]

The most important set of numbers is located in the second column after the second -> (e.g., in the top line shown it is 2623K->2175K(3980K). These numbers indicate that as a result of GC, we are using around 2200K of memory at the end of each GC cycle. This trace is not an indication of a memory leakit shows a short-term trend with less then a second between samples, and thats why we must observe long-term trends. However, if the Verbose GC log showed that the program was using around 2200K of memory after running for two days, and after running for 10 days it was using 2GB of memory (even after GC had just run), we could then conclude that theres a memory leak. All the information that needs to be collected in order to determine if a memory leak exists can be found in the results of the Verbose GC logs. Monitoring the Java Process The following approach works for any Java process, including standalone clients as well as application servers like JBoss and servlet containers like Tomcat. It is based on starting the Java process with JMX monitoring enabled and attaching with the JMX monitoring tools. Well use Tomcat in the following example. To start Tomcat or the Java process with JMX monitoring enabled, use the following options when starting JVM: -Dcom.sun.management.jmxremote enables JMX monitoring

JVM Internals & Runtime Data Areas.doc

Page 9

-Dcom.sun.management.jmxremote.port=<port> controls the port for JMX monitoring

Note that if youre on a production system, youll most likely want to secure your JVM before running it with these parameters. For that, you can specify these additional options: com.sun.management.jmxremote.ssl com.sun.management.jmxremote.authenticate

Once started, you can use JConsole or VisualVM to attach to the process. Note that later JDK 6 versions include VisualVM.

Heap Dump
A heap dump is a list of objects in the memory of JVM as well as the content of the memory occupied by those objects. It preserves the value of any attributes of the objects, including references to other objects. In other words, a heap dump gives you a complete picture of the memory. There are multiple tools that allow you to dump heap in a Java process:

If youre using JDK 6, you can use tool called jmap on any platform. If youre using JDK 5, the situation is slightly more complex: If youre running UNIX (Linux, Solaris, OS X) with JDK 5 you can use jmap. If youre using JDK 5 update 14 or later, you can use the -XX: +HeapDumpOnCtrlBreak option when starting JVM, then use the CTRL+BREAK key combination on Windows (or CTRL + \ on UNIX) to dump the heap. If youre running Windows and using JDK 5 pre-update 14, youll soon wish you werent. Trying to reproduce the problem with a more recent JDK is probably the best bet here.

Some tools like VisualVM and memory profilers allow you to initiate a heap dump from the GUI, but you dont need any fancy tools herejmap will do just fine. As it provides the most general case, well use jmap in the next example. Before you dump heap, be sure to keep the following issues in mind: Programs in the JVM should be paused for the duration of the heap dump, which might take anywhere from ten seconds to several minutes. Many enterprise applicationsand users of those applicationsdont take kindly to pauses that long, which may cause various timeouts to expire. So dont try this at home or in production (unless the application is already a goner)! Heap dumps are saved on disk, and the files might be fairly large. A good rule is to make sure that you have at least twice the size of the physical memory free on the disk before you initiate a memory dump.

JVM Internals & Runtime Data Areas.doc

Page 10

With those final words of caution out of the way, you should now be ready to run the following command: jmap -heap:live,format=b,file=FILENAME PID Note that the -F option, which will dump non-responsive programs, might be useful on UNIX systems, but is not available on Windows. Note also that JDK 6 includes the option +XX: +HeapDumpOnOutOfMemoryError that will dump heap whenever the OutOfMemoryError alert is encountered. This can be a useful option, but keep in mind that it has the potential to consume significant amounts of disk space. You now have a heap dump in the file FILENAME and are ready to analyze it.

Whats In Leaked Memory?

With the heap dump complete, we can now take a look at the memory and find out whats really causing the memory leak. Suppose that objects are holding references to each other as illustrated by the picture below. For the sake of easy calculation, lets assume that each object is 100 bytes, so that all of them together occupy 600 bytes of memory.

Now, suppose that the program holds reference to object A for a prolonged period of time. As a result, objects B, C, D, E, and F are all ineligible for garbage collection, and we have the following amount of memory leaking: 100 bytes for object A 500 bytes for objects B, C, D, E and F that are retained due to the retention of object A

So, holding reference to object A causes a memory leak of 600 bytes. The shallow heap of object A is 100 bytes (object A itself), and the retained heap of object A is 600 bytes.

JVM Internals & Runtime Data Areas.doc

Page 11

Although objects A through F are all leaked, the real cause of the memory leak is the program holding reference to object A. So how can we fix the root cause of this leak? If we first identify that object F is leaked, we can follow the reference chain back through objects D, C and A to find the cause of the memory leak. However, there are some complications to this follow the reference chain process: Reference chains can be really long, so manually following them can be time consuming An object is sometimes retained by more than one object, and there can even be circles involved as shown in the picture below:

If we start following inbound references from object F in this example, we have to choose between following object C or object D. In addition, theres the possibility of getting caught in a circle by repeatedly following the path between objects D, E and B. On this small diagram its easy to see that the root cause is holding object A, but when youre dealing with a situation that involves hundreds of thousands of objects (as any self-respecting memory leak does) you quickly realize that manually following the reference chain be very complex and time consuming. This is where some shortcuts can come in handy:

If we had a tool that allowed us to play a what would happen if I remove this reference type of guessing game, we could run experiments that help locate the cause. For example, we could see that if we removed reference from Cause of Leak to A in the diagram above, objects A through F would all be freed. Some tools (like Quests JProbe) have this capability. If the memory leak is large and we have a tool that allows us to sort objects by retained heap, well get an even greater head start because the objects with the largest retained heap are usually the cause of large memory leaks.

Now that we understand what memory leaks are and how they can be corrected, lets find out how to fix them by analyzing heap dumps.

JVM Internals & Runtime Data Areas.doc

Page 12

Tools for Dealing with Heap Dumps

Tools often provide a few extra helpful features Present a better summary of heap statistics. Sort objects by retained heap. In other words, some tools can tell you the memory usage of an object and all other objects that are referenced by it, as well as list the objects referenced by other objects. This makes it much faster to diagnose the cause of a memory leak. Function on machines that have less memory then the size of the heap dump. For example, theyll allow you to analyze a 16GB heap dump from your server on a machine with only 1GB of physical memory.

VisualVM is nice tool that gives you just enough to resolve memory leaks, and it shows heap dumps and relations between objects in graphical form. Feature-wise, one step above VisualVM is the Eclipse Memory Analyzer Tool (MAT), a free tool that includes a lot of additional options. Although its still in incubation phase as of publication of this article, MAT is free and weve found it to be extremely useful. Commercial products like JProfiler, YourKit, and JProbe are also excellent tools for debugging memory leaks. These applications include a few options that go above and beyond VisualVM and MAT, but theyre certainly not necessary to successfully debug memory leaks.

JVM Internals & Runtime Data Areas.doc

Page 13