New Java server-side benchmark puts NT's VM ahead of Solaris's -- on Intel VolanoMark shows a surprising disparity in Java Virtual Machine implementations By Stephanie Steenbergen

December  1997

Mail this article to a friend

Volano tested Java Virtual Machines on NT, Solaris, OS/2 Warp, and Red Hat Linux, all on the same Intel hardware, and found that NT scored a VolanoMark of 756 (higher numbers are faster), followed by 334 for Solaris, 239 for OS/2, and 223 for Red Hat.

Volano did not conduct tests on the UltraSPARC platform. Mark Elderkin, Volano's President says that all the benchmarks were done on Intel to keep the hardware standard throughout the testing.

Elderkin says he expects to have SPARC numbers posted on the Volano Web site within the week.

How does VolanoMark compare to the CaffeineMark benchmark? John Neffenger, Volano's founder and CTO says, "CaffeineMark mostly tests base Java interpreter speed. It runs a bunch of tests on the graphics and things like that." He adds that even some of the most basic server-side tests in VolanoMark are not included in other Java benchmarks.

"So that's why we wrote VolanoMark," Neffenger says. "We were seeing some of these really high, fantastic CaffeineMark scores and then getting those Java Virtual Machines and not even being able to support 10 concurrent connections. We wrote VolanoMark just so we could try to evaluate these Java Virtual Machines."

Neffenger published an explanation of his benchmark today in JavaWorld Magazine (see Resources below for a link to the story). VolanoMark consists of three test areas:

1. A direct performance comparison of the more popular Java virtual machines on an Intel server environment
2. A look at how fast a Java licensee's release is compared to the reference release from JavaSoft
3. A comparison of today's server-side Java performance with that of a year ago

Is Sun surprised by the results? "They've (Sun) known for a long time," says Neffenger. "We've had this tool and the vendors have had it for months. [Sun is] working really hard to change all that around."

Neffenger adds, "I think it's really awkward for both Sun and Microsoft to be caught in this situation where Microsoft is Java's biggest opponent, and yet it winds up being (at least from our viewpoint) its best implementer."

"Even though the scores show clear performance advantages on one over the other," says Neffenger, "all the machines we've evaluated in the article are plenty fast enough. We've realized performance is no longer an issue at all. Now were really focusing on stability and scalability from all the vendors."

Neffenger says that main goal in getting the benchmark out is not only to reward the Java implementors and vendors who've done good jobs, but also to encourage some of the other vendors who should have amazing Java Virtual Machine implementations but don't realize how important the Java server-side issues are. Neffenger also notes the "surprising disparity" in the performance, stability, and scalability of all the different Java Virtual Machine implementations, which ranged from a VolanoMark of 149 for the JDK 1.1.4 on Windows NT to a high of 756 for the Microsoft JVM on NT.

Resources

• Volano http://www.volano.com
• Explanation of VolanoMark in our sister publication JavaWorld http://www.javaworld.com/javaworld/jw-12-1997/jw-12-volanomark.html
• CaffeineMark information http://www.webfayre.com/pendragon/cm3/index.html

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-12-1997/swol-12-volano.html
Last modified: