New benchmark results show Java ready for prime time on servers

Performance has improved 10x

By Michael O'Connell

July 1998

Mail this
article to
a friend

San Francisco (July 16, 1998) -- Recently conducted benchmark tests show that Java has enjoyed impressive improvements (up to 10 times) in the past several months, and is a viable option for commercial software running on servers. With the new crop of Java virtual machines (VMs) due to ship this fall, Java on the server will become even more appealing.

So says John Neffenger, CTO and co-founder of Volano LLC. Neffenger recently conducted an extensive series of benchmark tests measuring performance and scalability of a dozen Java VMs. The results confirm his company's decision to be a 100% Pure Java software company.

"These tests show enormous improvement. Things are looking so much better, even compared to six months ago," Neffenger says. "The pace of improvements is relentless. These Java platforms are now more than 10 times faster, vastly more stable, more than twice as scalable when adding processors, and able to handle roughly 10 times the number of network connections [compared to] just 18 months ago."

With performance of the various Java VMs now ready for prime time, Java software developers can now target customers on a wider variety of platforms. And the arrival of new VMs that support additional operating systems (such as Novell NetWare) means developers now have a greater number of potential customers than ever before.

"We're getting 4 million additional potential customers for free this fall thanks to Novell," Neffenger says.

The benchmarks
Neffenger's tests employed VolanoMark, a 100% Pure Java server benchmark characterized by long-lasting network connections and high thread counts. Volano has submitted this benchmark to the SPEC Open Systems Group for possible use in the group's server-side Java benchmark suite.

While the latest version of VolanoMark measures scalability, it also measures raw performance in terms of number of messages per second. Performance-wise, the winner is a pre-release version of Tower Technology's TowerJ compiler, which generates a native executable from pure Java code that runs almost 25 percent faster than the fastest Java interpreter, even when the interpreter has a just-in-time compiler. Along with Tower, Microsoft, Novell, Sun, and IBM all offer VMs that offer good performance, handling more than 1,000 messages per second.

This chart compares the performance of 12 VMs. Except for Apple Computer's Macintosh Runtime for Java (MRJ), all of the JVMs were tested on the same hardware and with the same Java application code.

Of the currently available Java VMs he tested, Neffenger identified clear leaders in terms of speed, stability, and network connection scalability:

Java virtual machines available now

Category	Leading Java Vendor
Speed	Microsoft SDK for Java Version 2.02
Stability	Sun JDK 1.1.5 for Solaris Production Release
Network Scalability	Microsoft SDK for Java Version 2.02

The high quality of the Java VMs that will be released later this year made it more difficult to choose clear winners among that group, Neffenger said. But based on the preliminary tests, these early releases give a hint as to who may lead by the end of this year:

Java virtual machines to be released later this year

Category	Potential Leading Java Vendor
Speed	Tower TowerJ 2.1.2 Java Compiler for Linux
Stability	Sun JDK 1.2 for Solaris Production Release
Network Scalability	Novell JDK 1.1.5 for NetWare 5
Processor Scalability	Sun JDK 1.2 for Solaris Production Release

For the complete test results and more details, see "Which Java VM scales best?" in JavaWorld's current issue.

--Michael O'Connell is editor-in-chief of JavaWorld magazine, a SunWorld sister publication

Resources

"New Java server-side benchmark puts NT's VM ahead of Solaris's -- on Intel," December 1997 SunWorld news story http://www.sunworld.com/swol-12-1997/swol-12-volano.html
"Results of the first-ever JVM server benchmark revealed," December 1997 JavaWorld news story http://www.javaworld.com/javaworld/jw-12-1997/jw-12-volanomark.html
"SunSoft's project Speedway delivers," July 1997 SunWorld news story http://www.sunworld.com/swol-07-1997/swol-07-speedway.html
Technical details about the VolanoMark benchmark http://www.volano.net/guide/mark.html
The SPEC Open System Group is considering VolanoMark 2.0 for use in its forthcoming server-side Java benchmark suite http://www.spec.org/osg/
The Volano LLC Home Page http://www.volano.com
Microsoft SDK 2.02 http://www.microsoft.com/java/sdk/20/default.htm
Novell JDK 1.1.5 http://www.novell.com/netware5/index.html
SunSoft JDK 1.1.5 http://www.sun.com/solaris/java/
SunSoft JDK 1.2 Dev 3 http://developer.java.sun.com/developer/earlyAccess/jdk1.2/index.html
Tower Technology TowerJ 2.1.2 http://www.twr.com/

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-07-1998/swol-07-javabenchmark.html
Last modified:

Comments:
Name:
Email:
Company Name: