LLVM Lands Initial Support For Fat LTO Objects

[phoronix]

Phoronix: LLVM Lands Initial Support For Fat LTO Objects

In a move similar to GCC's implementation, LLVM Git landed this week initial support for fat LTO objects. This "-ffat-lto-objects" support will be found with the LLVM/Clang 17 release this autumn...

https://www.phoronix.com/news/LLVM-Fat-LTO-Objects

[FireBurn]

So what does Clangs current non-thin LTO do?

[equeim]

So what does Clangs current non-thin LTO do?

I think "thin" LTO is just an improved version of "old" LTO, and "old" LTO is not "fat" either (i.e. its object files contain only IR bitcode too). At least that what I gather from Clang documentation: https://clang.llvm.org/docs/ThinLTO.html#id4

[Weasel]

Object files normally contain compiled code that the linker simply links. The linker, traditionally, has no notion of compilation or optimization and just links already compiled together. This was the standard practice of linkers without LTO, of course.

With LTO, object files don't store compiled code, they store intermediate representation by the compiler (depends on compiler). The linker then invokes the compiler to compile this code from the object files, effectively "optimizing the program as a whole, rather than one object file at a time" so it can do inter-object optimizations (note: the proper term is "translation unit" but whatever). So the linker with LTO acts as a compiler, too.

Here's the problem though: a normal linker without LTO expects compiled code, and doesn't compile. So it can't use the LTO object files. Or an incompatible linker LTO that doesn't support the LTO of the object file (say, different version of compiler, or different compiler altogether, e.g. GCC vs LLVM). So it simply fail.

"Fat" LTO objects means that the object file contains both the normal compiled code that traditional linker expects without LTO, and the IR (compiler-specific for LTO). The linker then can use the normal object file without LTO, or use the IR if it supports it.

It's not really interesting if you just compile the whole thing once yourself with same compiler and everything.

[brucethemoose]

There was an ffmpeg bug related to this:

https://github.com/InBetweenNames/gentooLTO/issues/354

FFMPEG does build nicely with the link-time optimizer, but putting -flto in the flags or configuring with --enable-lto tends to cause the build to fail with lots of undefined symbols. Instead, put -ffat-lto-objects in the flags (already there if you use the default CFLAGS that comes with Clear) so that the linker has a fallback. Do be sure to include --extra-ldflags='-flto -fuse-linker-plugin' --ar=gcc-ar.

[RJVB]

It's not really interesting if you just compile the whole thing once yourself with same compiler and everything.

It is very interesting if you want to be able to use static libraries (archives) with different compilers/linkers. I've even seen LTO builds of a single project fail if those archives were created by the generic `ar` archiver rather than with the `llvm-ar` archiver from the same (or possibly a newer) LLVM release.