OpenJDK 8/11 vs. GraalVM 20 vs. Amazon Corretto JVM Benchmarks

[phoronix]

Phoronix: OpenJDK 8/11 vs. GraalVM 20 vs. Amazon Corretto JVM Benchmarks

Following last week's benchmarks of OpenJDK 8 through the newly-released OpenJDK 14 JVM benchmarks, some Phoronix readers expressed interest in seeing Java benchmarks with Oracle's GraalVM as well as Amazon's Corretto JVM implementations. Here are some benchmarks of those benchmarks up against OpenJDK both for Java 8 and Java 11 releases.

http://www.phoronix.com/vr.php?view=28985

[andyprough]

I keep thinking of Dave Grohl of the Foo Fighters and Nirvana. Maybe the GraalVM people should ask Dave for the rights to his name.

[tildearrow]

I thought GraalVM was going to be much faster than that...

[andrei_me]

Graal seems optimized for a specific use-case or for something not measured by the benchmark used (throughput, memory usage, latency, whatever)

[Leprechaunius]

Well, those are some slightly disappointing results...
Talking to people behind GraalVM, it seems it is specially tuned to offer fast performance when doing streams and lambdas - something I really doubt is used in the java benchmarks, since streams and lambdas cause much overhead when running in traditional Java environment, so they are usually avoided when writing computation-heavy algorithms.
However, for nonblocking I/O, they are a better fit; and indeed, it shows in the IO benchmarks here as well.

However, What is the most interesting is the drop in performance when going from 8 to 11. I have no idea why the performance plummets down so much. Even if you somehow tune your benchmark with the Java 8 GCs, it can't explain so huge difference. Even the Unsafe class cannot have such an impact, I'm really baffled.

[cynical]

I thought GraalVM was going to be much faster than that...

Why would you think that? HotSpot is very mature and heavily optimized. It will take quite some time for Graal to catch up. The current use case would be something like AWS Lambda where you desire the AOT compilation for low memory footprint and fast startup.

[cynical]

Well, those are some slightly disappointing results...
Talking to people behind GraalVM, it seems it is specially tuned to offer fast performance when doing streams and lambdas

Yeah exactly. I can't wait for it to catch up, not just for better performance with functional Java, but for other languages on the JVM like Scala, Kotlin, and Clojure.

[Guest]

Well, those are some slightly disappointing results...
Talking to people behind GraalVM, it seems it is specially tuned to offer fast performance when doing streams and lambdas - something I really doubt is used in the java benchmarks, since streams and lambdas cause much overhead when running in traditional Java environment, so they are usually avoided when writing computation-heavy algorithms.
However, for nonblocking I/O, they are a better fit; and indeed, it shows in the IO benchmarks here as well.

However, What is the most interesting is the drop in performance when going from 8 to 11. I have no idea why the performance plummets down so much. Even if you somehow tune your benchmark with the Java 8 GCs, it can't explain so huge difference. Even the Unsafe class cannot have such an impact, I'm really baffled.

Doesn't always really matter though, whether it's 50% slower... For example, when delivering SaaS, where every customer has got separate virtualized/dockerized instance, and load per customer is only ~5%, then having native binary, instead of JIT execution in JVM, will make RAM usage much lower,... And that reduces costs a lot,...

I'm toying with golang, and developing full featured web application with server side rendering (not just REST API). When docker container starts, it uses only 7MB RÁM, and under some load testing it keeps usage below 20MB. PostgreSQL database has got few times higher memory usage.... I believe, golang could provide nice future for SaaS.

[tildearrow]

Why would you think that? HotSpot is very mature and heavily optimized. It will take quite some time for Graal to catch up. The current use case would be something like AWS Lambda where you desire the AOT compilation for low memory footprint and fast startup.

Because I thought GraalVM was like a full compiler for Java...

[Michael_S]

Because I thought GraalVM was like a full compiler for Java...

Graal gives you much faster startup and much smaller file size (since it only bundles what you use and not the entire Java standard library). But it only uses on garbage collector instead of the four or more (?) that are available in the JVM and can't do runtime optimizations like the JVM.