GCC's Conversion To Git Is Being Held Up By RAM, a.k.a. Crazy DDR4 Prices

After converting the GNU Emacs repository to Git a few years back, Eric S Raymond has been working on the massive undertaking of transferring the GCC (GNU Compiler Collection) repository in full over to Git. But the transition to GCC Git is being hampered since due to the massive size of the repository, Raymond's system is running under extreme memory pressure with 64GB of RAM.

ESR provided an update on the GCC repository conversion process. He has managed to solve the only known remaining technical bug that's been blocking the repository, but now he can't get the process completed since he's over-running memory capacity. His primary workstation has 64GB of DDR4 memory and that's turned out to not be enough for the GNU Compiler Collection repository with more than a quarter million commits over the past three decades.

Due to more DDR4 memory being "hideously expensive", he's figuring out alternatives. While the GNU has the GCC compile farm with multiple systems having 128GB+ of RAM, ESR believes his system is faster and better tuned for the memory-heavy workloads. Among the measures being now attempted are running the process with his web-browser shutdown to conserve memory and potentially shifting some code from Python to the Go programming language, but that could prove to be very tedious.

Eric further elaborated, "The truth is we're near the bleeding edge of what conventional tools and hardware can handle gracefully. Most jobs with working sets as big as this one's do only comparatively dumb operations that can be parallellized and thrown on a GPU or supercomputer. Most jobs with the algorithmic complexity of repository surgery have *much* smaller working sets. The combination of both extrema is hard."

Some developers have talked about chipping in money so ESR can upgrade the RAM in his system and there is also the fact that the GNU/FSF does collect money for different causes (such as the FSF GNU Toolchain Fund) could potentially be tapped. We'll see what happens but he wants to act on it sooner rather than later as the GCC repository only continues getting larger by the day.