GraalVM 20.3 Released With Many Small Performance Optimizations

[phoronix]

Phoronix: GraalVM 20.3 Released With Many Small Performance Optimizations

GraalVM continues its quest as the virtual machine not only supporting Java but also additional languages and execution modes with a focus on stellar performance and speedy startups. GraalVM CE 20.3 was released on Tuesday as the latest for this open-source package supporting Java, Node.js, an LLVM runtime, and more...

http://www.phoronix.com/scan.php?page=news_item&px=GraalVM-20.3-Released

[Michael_S]

For a while I was excited by Graal as a way to make Java programs, and JVM-ecosystem programs, that have faster startup and lower memory overhead.

But recently I'm finding myself convinced by argument that for fast programs that make efficient use of resources, the space to watch is a new batch of attempted replacements for C. All attempt to be C with some of the warts addressed, and all attempt to be simpler than D or Rust and way, way simpler than C++.

Andrew Kelley is working on the Zig language.
Alex Medvednikov is working on the V language.
Jonathan Blow is working on the JAI language.
Drew DeVault has said he's working on his own C replacement.

Only the first two are in public alpha, and it's too early to use either for a business. But maybe someday I can write useful programs that don't need 6, 10, or 35MB of RAM just to print "Hello World".

[bug77]

...a focus on stellar performance...

It's not yet conclusively faster than other JDKs.

...and speedy startups

Currently, if you want to start fast, you need native-image, which kinda bypasses GraalVM altogether. Otherwise, the right way to improve startup time in Java is to get rid of Spring and use something like Micronaut or Quarkus.

I'm being mean, of course, the goal of GraalVM is intriguing. But it's awkward to see v20.3 for something that's essentially a beta.
And I'm with Michael_S , if you really need speed, Java is about the last place you should look.

[mdedetrich]

It's not yet conclusively faster than other JDKs.

It is, especially for JVM languages that create JVM bytecode which the JVM isn't optimized for. i.e. current JVM is really bad when it comes to methods that have bimorphic (or worse, i.e. megamorphic) dispatch, GraalVM handles this much better. This greatly benefits languages like Scala

Also most of the improvements to AOT optimization are going into GraalVM rather than the JVM because its a lot nicer to work in (its written in java versus old style C++)

Currently, if you want to start fast, you need native-image, which kinda bypasses GraalVM altogether. Otherwise, the right way to improve startup time in Java is to get rid of Spring and use something like Micronaut or Quarkus.

I'm being mean, of course, the goal of GraalVM is intriguing. But it's awkward to see v20.3 for something that's essentially a beta.
And I'm with Michael_S , if you really need speed, Java is about the last place you should look.

Even if you don't use Spring the startup with JVM is still really slow (even when using the -client flag) . For example scalafmt (https://scalameta.org/scalafmt/) which is a tool written in Scala that formats Scala code is much faster when using a native graalvm image versus the standard JVM .jar. This is also a code formatting tool, so there isn't any Spring webserver running inside of it, its basically algorithmic code which is the happy case for JVM optimizations.

The JVM has a huge amount of baggage and cruft, and one of the biggest general issues with JVM is that it prioritizes optimization for typical "Java style" code rather than the generic JVM bytecode (contrasting to something like LLVM which tries to optimize all use cases that you can throw at it).

This is even noticeable when Java 1.8 introduce SAM (Single Abstract Methods) which allowed much easier use of lambda's. A lot of libraries/frameworks started using this feature and noticed that its slower compared to the typical imperative style Java because the JVM wasn't optimized for it. This problem afaik still exists (actually the bimorphic/megamorphic problem stated earlier occurs a lot more frequently when using lambda's).

[bug77]

It is, especially for JVM languages that create JVM bytecode which the JVM isn't optimized for. i.e. current JVM is really bad when it comes to methods that have bimorphic (or worse, i.e. megamorphic) dispatch, GraalVM handles this much better. This greatly benefits languages like Scala

Also most of the improvements to AOT optimization are going into GraalVM rather than the JVM because its a lot nicer to work in (its written in java versus old style C++)

Well, the most popular language for the JVM is still Java. Until they make Java run consistently faster, I'll maintain my assertion.
Btw, how is Scala doing these days, have they figured out Java 9 with modules and everything?

Even if you don't use Spring the startup with JVM is still really slow (even when using the -client flag) . For example scalafmt (https://scalameta.org/scalafmt/) which is a tool written in Scala that formats Scala code is much faster when using a native graalvm image versus the standard JVM `.jar`. This is also a code formatting tool, so there isn't any Spring webserver running inside of it, its basically algorithmic code which is the happy case for JVM optimizations.

So because something that uses native image is faster, that means GraalVM is faster?

The distinction I was hoping you'd make is between reflection heavy vs code that makes use of annotation processors instead.

I agree JVM implementations have a lot of baggage, but GraalVM not being (miles) faster running Java with its clean-slate, modern implementation is food for thought.

[mdedetrich]

Well, the most popular language for the JVM is still Java. Until they make Java run consistently faster, I'll maintain my assertion.

True, but my point is that these cases also crop up in Java code, its just less common. Again re-iterating, the functional style of code which are seeing in modern Java (with SAM's) is less performant because the JVM hasn't been optimized well for these cases.

The only difference here between Scala and Java (as an example) is that idiomatic Scala code uses these features a lot more often so the difference is more noticable.

Btw, how is Scala doing these days, have they figured out Java 9 with modules and everything?

Scala has kinda ignored Java modules because its not that necessary. Scala itself has its own module system (if you can call it that) due to just being more expressive language so the community doesn't really see a point in Java 9 modules yet (although Java 9 modules has more advantages, i.e. DCE with GraalVM).

So because something that uses native image is faster, that means GraalVM is faster?

The distinction I was hoping you'd make is between reflection heavy vs code that makes use of annotation processors instead.

Well yeah I was simplifying things, there are different areas of performance. One is startup time, another is the JIT's optimization at runtime etc etc. So as a specific example of runtime JIT optimization, GraalVM CE currently handles inlining of bimorphic/megamorphic methods where as the JVM doesn't so this is an example of a runtime performance optimization.

Regarding reflection/annotations, this is considered un-idiomatic Scala code. Modern idiomatic Scala tends to stay away from reflection/annotations as much as possible, instead there are alternative solutions that use Scala's type system which is expressive enough that reflection is often not required. Scala will then generate the compatible JVM bytecode at compile time.

A good talk about this is here https://www.youtube.com/watch?v=a6dK3eXhO7k

I agree JVM implementations have a lot of baggage, but GraalVM not being (miles) faster running Java with its clean-slate, modern implementation is food for thought.

Of course but it is the future, its like the Wayland of Linux (although the difference is not as extreme since GraalVM at least supports almost all of the use cases of JVM so its easier to migrate).

[bug77]

mdedetrich I guess we agree overall (GraalVM handles some things better then other JVMs, some worse). My point was I was expecting GraalVM to match other JVMs in most areas and improve in others. That's not the case and that, for me, is a disappointment.

I can see you're hyped up about Scala, it seems to me that's on the way out. You say Scala avoided modules because it didn't need them. Well, the JDK closed access to internal APIs and that hurt Scala's performance, that's the reason last I checked their web page still said they support JDK 9+ (11, actually) with caveats. It is my impression that modules and closing off internal APIs hit the Scala devs rather hard.
And since we're off topic, people seem to be moving more towards Kotlin these days, not Scala. Kotlin is theoretically more capable than Scala, but reading Kotlin makes me want to gouge my eyes out.

[Michael_S]

Kotlin is theoretically more capable than Scala, but reading Kotlin makes me want to gouge my eyes out.

More capable how? My impression of Kotlin was that someone looked at Scala and then tried to adopt the 50-80% of its features that wouldn't terrify someone coming from Java 7.

My wild, incoherent, poorly informed impression with Scala is that it was on the rise but got hamstrung by two problems. The first is that SBT might be the worst build tool invented in the last twenty years. The second is that Scala language revisions keep changing major features and ABIs. How many potential new Scala projects are on hold while people wait for the dust to settle around Dotty? (Maybe Dotty has been released, I stopped paying attention.)

I've got about four months of professional Scala development under my belt across two jobs. Both times, I spent more time and experienced more pain trying to get SBT to do something simple that I needed than I did working on actual Scala code. Both times, my teammates and I gave up and hacked around the problem. "Run this shell script. Then run the SBT command. Then run that shell script..." We've got one Scala webapp in production, and a rewrite is on the roadmap.

[mdedetrich]

Disclaimer: I have been coding professionally in Scala for ~10 years now

More capable how? My impression of Kotlin was that someone looked at Scala and then tried to adopt the 50-80% of its features that wouldn't terrify someone coming from Java 7.

Agreed, Scala is actually much more capable than Kotlin. Kotlin is basically Java++, where as Scala has really becomes its own language. Idiomatic modern Scala is much closer to a more ergonomic OCaml that at the end of the purely functional programming spectrum feels like a strict Haskell with modules/classes/subtyping.

Scala has constructs such as HKT (higher kinded types) that when combined with OCaml/Modula style modules (i.e. treating sets of functions as values) provide much more abstraction power than is possible in Kotlin.

My wild, incoherent, poorly informed impression with Scala is that it was on the rise but got hamstrung by two problems. The first is that SBT might be the worst build tool invented in the last twenty years. The second is that Scala language revisions keep changing major features and ABIs.

So I have spent an unreasonable amount of time complaining about SBT and using it for literally years but in my opinion the major issue with SBT is not that its "terrible" or "bad". SBT's is actually one of the most correct and comprehensive build tools and this is basically a side effect of SBT being purely functional (under the hood). In other words, SBT directly tracks state which gives it a lot of advantages in builds since it can precisely track the state of files which makes it possible to make very complicated builds that are relatively bug free (i.e. a common issues in build tools such as invalidating build caches and extending the builds are easier to do in SBT while not being hacky). As an example, sbt-android https://github.com/scala-android/sbt-android has a lot less issues/bugs than Android's default gradle build system and is even able to do things that gradle cannot.

The biggest issue with SBT however is that it created its own internal DSL/syntax/constructs rather than using Scala's already existing idioms/constructs. For example SBT uses the in syntax to simulate composition, even though Scala has traits that can be mixed with other traits. SBT Task's are basically lazy values (and Scala already has lazy).

What this means is that when learning SBT, there is a massive (and unnecessary) cognitive overhead because you essentially have to learn yet another language (and Scala is already difficult enough, its like the C++ of high level programming languages).

This is in contrast to build tools like CBT (https://github.com/cvogt/cbt) which try to use Scala's own constructs as much as possible rather than re-inventing the wheel. Ontop of this, when working with SBT you have to deal with build graphs directly (most build tools actually evaluate to a graph where nodes can be arbitrarily overwritten). There are easier ways to handle working with such graph structures in Scala (the creator of CBT actually made a great video on this here https://www.youtube.com/watch?v=d4QheVMJk7E).

Unfortunately SBT has already gained so much traction in the Scala ecosystem that its very hard to come up with better build tools that has enough inertia.

How many potential new Scala projects are on hold while people wait for the dust to settle around Dotty? (Maybe Dotty has been released, I stopped paying attention.)

Regarding the bincompat problem, this is indeed a problem in Scala (although lately much less of an issue because Scala stopped breaking binary compatibility so frequently, now the Scala binary compatibility is much more stable, lasting roughly a couple of years, SBT also handles this very well). Dotty however solves this problem, and it does that by storing TASTY (typed abstracted syntax tree) inside of .jar files rather than standard java .class files. This means that once people start using these jars, there aren't going to be these binary compatibility issues. TASTY ast's are guaranteed to be stable even amongst different Scala versions.

I've got about four months of professional Scala development under my belt across two jobs. Both times, I spent more time and experienced more pain trying to get SBT to do something simple that I needed than I did working on actual Scala code. Both times, my teammates and I gave up and hacked around the problem. "Run this shell script. Then run the SBT command. Then run that shell script..." We've got one Scala webapp in production, and a rewrite is on the roadmap.

Yeah this is frequently an issue, and also the same reason why I am so vocal about how damaging SBT is to Scala's ecosystem. Having to learn SBT ontop of the Scala language itself is reasonably way too much for a lot of people.

Also another historical problem with Scala is that the tooling (outside of SBT) needed a lot of help, particularly in IDE's. As an example, Intellij ended up implementing its own type system rather than using scalac's (scala compiler) one. This means that you ended up getting divergent results of what is "correct" in your source code (i.e. Intellij would give you red lines but scalac would compile it fine). Scala's type system is also so complex that Intellij ends up using godly amounts of memory because it ends up having to cache a stupidly large amount of permutations of AST's to provide decent inspection/completion options.. This has however gotten a lot better recently, and with metals/dotty the IDE'/editors actually use the compilers type checker rather than re-implementing it.

Dottys compiler actually is much closer to a server than a traditional c style compiler, IDEs/Editors can hook into a persistent running session of dotty and communicate with it to get details about completions. Currently its done using LSP (language server protocol).

[Michael_S]

Disclaimer: I have been coding professionally in Scala for ~10 years now

Cool. Aside from small projects here and there that at one point or another involved Scala, Python, Perl, and JS, my actual day job has been Java for fifteen years. I'm sick of it, but my current position is too cushy to leave.

Thanks for the detailed response to my post, I appreciate it. I may give Scala another look.

Agreed, Scala is actually much more capable than Kotlin. Kotlin is basically Java++, where as Scala has really becomes its own language. Idiomatic modern Scala is much closer to a more ergonomic OCaml that at the end of the purely functional programming spectrum feels like a strict Haskell with modules/classes/subtyping.

Scala has constructs such as HKT (higher kinded types) that when combined with OCaml/Modula style modules (i.e. treating sets of functions as values) provide much more abstraction power than is possible in Kotlin.

Right. I learned Scala around 2010 and was quite comfortable with a lot of the example code, but once I dug into Scala in real projects I started seeing some type annotations that made my head spin. I haven't used Kotlin for anything serious, but the Kotlin code I've played with was easier to understand - as you say, Java++.

So I have spent an unreasonable amount of time complaining about SBT and using it for literally years but in my opinion the major issue with SBT is not that its "terrible" or "bad". SBT's is actually one of the most correct and comprehensive build tools and this is basically a side effect of SBT being purely functional (under the hood). In other words, SBT directly tracks state which gives it a lot of advantages in builds since it can precisely track the state of files which makes it possible to make very complicated builds that are relatively bug free (i.e. a common issues in build tools such as invalidating build caches and extending the builds are easier to do in SBT while not being hacky). As an example, sbt-android https://github.com/scala-android/sbt-android has a lot less issues/bugs than Android's default gradle build system and is even able to do things that gradle cannot.

The biggest issue with SBT however is that it created its own internal DSL/syntax/constructs rather than using Scala's already existing idioms/constructs. For example SBT uses the in syntax to simulate composition, even though Scala has traits that can be mixed with other traits. SBT Task's are basically lazy values (and Scala already has lazy).

What this means is that when learning SBT, there is a massive (and unnecessary) cognitive overhead because you essentially have to learn yet another language (and Scala is already difficult enough, its like the C++ of high level programming languages).

Right. When I was wrestling with SBT, I read a lot on what features it had and why. And the feature set is impressive, and as you said ahead of all or almost all other build systems. But it's its own language and they went berserk with operator overloading. I know it's an old joke to any Scala veteran, but when I read a complex SBT file I start expecting to see a NO CARRIER in the line noise. I understand newer versions have moved away from the operator overloading, but the last version of SBT I used (barely managed to use) was from 2014 or earlier.

This is in contrast to build tools like CBT (https://github.com/cvogt/cbt) which try to use Scala's own constructs as much as possible rather than re-inventing the wheel.

Interesting. As I said, I stopped following Scala a few years ago but at the time it looked like Mill and maybe Fury had potential as SBT alternatives. I can't find it now, but I watched a video on the Fury build tool that was really impressive.

Regarding the bincompat problem, this is indeed a problem in Scala (although lately much less of an issue because Scala stopped breaking binary compatibility so frequently, now the Scala binary compatibility is much more stable, lasting roughly a couple of years, SBT also handles this very well). Dotty however solves this problem, and it does that by storing TASTY (typed abstracted syntax tree) inside of .jar files rather than standard java .class files. This means that once people start using these jars, there aren't going to be these binary compatibility issues. TASTY ast's are guaranteed to be stable even amongst different Scala versions.

That's good to hear.

Also another historical problem with Scala is that the tooling (outside of SBT) needed a lot of help, particularly in IDE's.

I think that's an unavoidable problem with complex languages. Raku, formerly known as Perl 6, has a colossal set of features: optional static types (that are enforced), multi-methods, full object-oriented with multiple inheritance, generics, operator overloading, pattern matching, junctions - if you're bored, check it out. They took Perl 5, stripped out the junk, and then covered a surprising portion of the territory between what was left and Scala. ...and editor support for Raku isn't so hot either. One of the core language architects built a version of IntelliJ for it, called Comma, and Comma chokes on the syntax highlighting for medium size Raku files.