Java Development without GC

All products developed by Coral Blocks have the very important feature of leaving ZERO garbage behind. Because the latency imposed by the Java Garbage Collector (i.e. GC) is unacceptable for high-performance systems and because it is impossible to turn off the GC, the best option for real-time systems in Java is to not produce any garbage at all so that the GC never kicks in. Imagine a high-performance matching engine operating in the microsecond level, sending and receiving hundreds of thousands messages per second. If at any given time the GC decides to kick in with its 1+ millisecond latencies, the disruption in the system will be huge. Therefore, if you want to develop real-time systems in Java with minimal variance and latency, the best option is to do it right without creating any garbage for the GC. In this article we will discuss best practices and how you can use Coral Blocks’ MemorySampler utility class to help you accomplish this critical goal.

The MemorySampler utility class

Garbage or dereferenced Java instances can show up in three different places: third-party libraries, JDK classes and your code. Before we discuss each one of them, let’s take a break to introduce Coral Blocks’ MemorySampler utility class which will allow you not just to tell your boss you are producing no trash but to prove it. (Note: MemorySampler is part of the CoralBits component)

		MemorySampler.start();
		// do nothing!
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 0
		 Memory allocated total: 0
		 */
		
		MemorySampler.start();
		// allocate nothing...
		int x = 10;
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 0
		 Memory allocated total: 0
		 */
		
		MemorySampler.start();
		// allocate an object...
		String s = new String("trash");
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 24
		 Memory allocated total: 24

		 Stack Trace for java.lang.String
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:20)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:33)
		 */
		
		MemorySampler.start();
		// allocate 10 objects...
		for(int i = 0; i < 10; i++) new String("trash!");
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 240
	 	 Memory allocated total: 264

		 Stack Trace for java.lang.String
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:26)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:33)
		 */

		Map<String, String> map = new HashMap<String, String>();
		String key = "key";
		String value = "value";
		
		MemorySampler.start();
		map.put(key, value);
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 112
		 Memory allocated total: 376

		 Stack Trace for [Ljava.util.HashMap$Entry;
    		java.util.HashMap.inflateTable(HashMap.java:320)
    		java.util.HashMap.put(HashMap.java:492)
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:69)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:76)

		 Stack Trace for java.util.HashMap$Entry
    		java.util.HashMap.createEntry(HashMap.java:901)
    		java.util.HashMap.addEntry(HashMap.java:888)
    		java.util.HashMap.put(HashMap.java:509)
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:69)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:76)
		 */

As you can see by the output above, MemorySampler can tell you some important things:

  • The amount of memory allocated on the last pass
  • The total memory allocated so far
  • Who allocated the memory in the last pass with the source code line number
  • The stack trace leading to the allocation call

Of course the fact that the code is allocating memory does not necessarily mean it is creating garbage as references can be pooled for re-use. For example the code below:

		Map<String, String> map = new HashMap<String, String>();
		
		for(int i = 0; i < 100; i++) {

			MemorySampler.start();
			
			map.put("key", "value");
			map.remove("key");
			
			MemorySampler.end();
			if (MemorySampler.wasMemoryAllocated(true)) { // true => ignore the first pass (init)
				MemorySampler.printSituation();
			}
		}

Prints the output below 99 times:

         Stack Trace for java.util.HashMap$Entry
             java.util.HashMap.createEntry(HashMap.java:901)
             java.util.HashMap.addEntry(HashMap.java:888)
             java.util.HashMap.put(HashMap.java:509)
             com.coralblocks.coralutils.gcutils.Basics.test2(Basics.java:104)
             com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:116)

The total memory allocated is incrementing linearly with iterations, in other words, the memory allocated per pass is always 32 and the total memory allocated increases from 144 to 3280:

Memory allocated on last pass: 32
Memory allocated total: 144

(...)

Memory allocated on last pass: 32
Memory allocated total: 3184

At this point it is clear that java.util.HashMap.createEntry is not pooling its objects, it is creating garbage and it is just a matter of enough iterations before it triggers the GC. We will soon see how to fix that.

Warming up, Checking the GC and Sampling

The key to make sure your system is not creating any garbage is to warm up your critical path from start to finish a couple of million times and then check for memory allocation another couple of million times. If it is allocating memory linearly as the number of iterations increases, it is most likely creating garbage and you should use the stack trace provided by MemorySampler to investigate it further. It might sound more complicated than it really is as it will often be straightforward to verify that as you iterate doing the same thing, the same object is allocated over and over again indicating a garbage leak, as it was the case with the java.util.HashMap.

Real-time applications usually have a critical loop that is executed non-stop by a high-priority thread. You can plug the MemorySampler in that loop to embody your whole application. For example, CoralReactor, which is a high-performance asynchronous network library, has the following code at the top of its critical selector (i.e. reactor) thread:

			while (isRunning) {
				
				if (traceAllocation) {
					
    				MemorySampler.end();
    				
    				if (MemorySampler.wasMemoryAllocated()) {

    					MemorySampler.printSituation();
    				}
    				
    				MemorySampler.start();
				}

				// that's the critical path, in other words, all branches of your application
				// can be reached from this point

				// (...)
			}

When using CoralReactor or any other application with a MemorySampler configured in the critical path as above, the standard procedure to detect garbage creation or Java garbage leaks is:

  • Turn off the MemorySampler and run your application with -verbose:gc
  • Send a couple of million messages or exercise your code the same way a couple of million times and check if the GC kicks in.
  • If it doesn’t you most likely do not have a garbage leak but it does not hurt to turn on the MemorySampler and check for any memory allocation.
  • If it does then you must turn on the MemorySampler and investigate to see who is the culprit.

A good system will have the MemorySampler on and not allocate anything after warming up. At this point you can be sure that at least the most important branches of your application that you warmed up are free from garbage leaks and you can execute them a billion times without any GC overhead.

Getting rid of the trash

Once you have determined that your system has a garbage leak, you have to fix it. As we said in the beginning, there are three scenarios where you can find a garbage leak:

  • In a third-party library

If you are using a third-party library that is producing a lot of garbage you should consider writing your own, contacting the author or the company or fixing the code yourself if it is an open-source project. If that cannot be done, then you should start looking for a better alternative or another way to perform the same task in a more efficient way. Some libraries claim they are real-time libraries that produce no garbage. You should favor these ones and of course test them to see if they are really leaving no trash behind.

  • In the JDK classes

Please contact us for more information on how to accomplish that.

  • In your own code

When coding your own lines of code you must have the discipline and right libraries and tools not to leave a mess for the GC to clean. For example, you should know that autoboxing produces garbage as well as varargs. Some JDK data structures like java.util.HashMap produce garbage as we saw in an earlier example and are not very efficient to store primitive keys. You must have your own set of high-performance, clean and fast data-structures if you want to do real-time Java development efficiently. You can find open-source real-time libraries or make the ones that are not real-time better by patching their code.

Another important tool to have in hand is a good, clean and fast object pool. By making your objects mutable and pooling them for re-use you can eliminate most of the garbage leaks in Java. For example, it is not difficult to pool the entry objects in the java.util.HashMap to make it garbage-free.

Conclusion

Java Development without GC overhead is very possible and you can use a memory sampler to make sure your application is not leaving any garbage behind. By using real-time libraries you can build a high-performance system from the ground up with minimal variance and latency. If you are in a hurry to build your real-time codebase foundation, you can count on Coral Blocks to help you. We have all the libraries and tools to build any real-time ultra-low-latency system from the ground up. You can use one of our components and we can provide you with our utility classes and data structures that will not just simplify your applications but leave them shiny and clean of garbage. They will never see the GC again.