A New Project To Run Mac OS X Binaries On Linux

[ryao]

Hi, first of all I'd like to say I'm happy I made it to Phoronix :-)


I started the project in August this year. When working on something of this size, you have to start small. There's just no other way.

The next big thing (which is quite close by the way) is to have Apple LLVM-GCC fully working. That would basically mean you could compile applications for iOS on other platform than OS X. This is currently not possible. Anybody who wants to do iOS development has to buy a Mac. That sucks big time. iOS is not very popular in this part of Europe, but at least in the US I believe it's still the platform no. 1 for mobile apps.

Getting iOS apps running on... say Android is a different story. I'm planning on supporting ARM (and good old PPC too, while I'm at it) in the dynamic loader and the ObjC runtime. The frameworks itself is I think a job for someone else, because as I've said, I don't see that many iOS devices around so I lack the motivation.

Just for OS X apps, there is lots and lots and lots of work to do. Anyone who's ever written a piece of software for OS X could start mentioning loads of frameworks specific to this platform. While I'm enjoying the work, I also realize that I don't stand much chance to have anything suitable for end users in a short time frame. So if there are any guys out there willing to help out, be it with GNUstep or something that I actually write as part of Darling (IOKit userspace libs, Apple Events support, etc....), help is always welcome. You don't even need to have any experience with writing software for OS X...

Luboš

I have a few comments that you might or might not appreciate:

1. You should pay strong attention to getting the syscall layer correct. If you implement the syscall layer well, it should be possible to use Mac OS X's userland on top of the lower level layers. That would enable you and others to compare the Mac OS X userland against the OSS userland components. That should make this significantly easier. It would also enable you to try a userland from Apple's Darwin images.
2. You can use Gentoo Prefix to assist you in development. It is a userland package manager that lets you build a fairly large body of existing UNIX software for Mac OS X. What should be particularly interesting to you is that its dependencies are fairly light. Installing it should require only standard UNIX utilities, a toolchain, libc and the system headers. Once it is installed, it depends on only libc and the system headers. There might be a few other libraries that I missed, although they would be things like libm and libdl.
3. If you are interested in seeing widespread adoption, it might be better to pick a less restrictive license. The WINE project uses the LGPL, which would probably work well for you. It is important to make a decision on this before you start receiving contributions, because it will be a pain to switch should you change your mind afterward. This is especially true with the GPLv3 because people tend to be very litigation-happy when things involve it.
4. This is an awesome idea. Ignore people that say that this is a waste of time. Their opinions are not worth your attention.

WINE is a failure. After like 20 years of development, millions in donations, and an army of developers, the best it can do is (maybe) run your Windows applications(with some extra bugs and worse performance).

A couple of years ago I evaluated running half a dozen enterprise Windows applications in WINE. WINE failed to run any of them adequately, most couldn't even start, and it was always because of some unimplemented function in a DLL somewhere... Maybe after 40 years of development WINE will adequately run 90% of applications written for Windows 2000.

If that doesn't tell you that "Not An Emulators" are a waste of time, I don't know what will.

It would be interesting to run WINE on top of Darling. Anyway, there are two approaches to running foreign binaries on a system. One is to implement the syscall layer and rely on the foreign operating system's userland to enable it to work (the OS/2 approach). The other is to try to implement both the kernel and userland (the WINE approach). The former was very successful in its time and does not require many resources. The latter is a nightmare.

I doubt that Darling would result in making Linux into a perfect replacement for Mac OS X, but I think Darling would make Mac OS X development on Linux easier. If Darling can obtain near perfect operation when paired with Mac OS X's actual userland, that would be a nice bonus.

[rudolph_steinberg]

It would be interesting to run WINE on top of Darling. Anyway, there are two approaches to running foreign binaries on a system. One is to implement the syscall layer and rely on the foreign operating system's userland to enable it to work (the OS/2 approach). The other is to try to implement both the kernel and userland (the WINE approach). The former was very successful in its time and does not require many resources. The latter is a nightmare.

I doubt that Darling would result in making Linux into a perfect replacement for Mac OS X, but I think Darling would make Mac OS X development on Linux easier. If Darling can obtain near perfect operation when paired with Mac OS X's actual userland, that would be a nice bonus.

A reasonable sentiment, but when you take into account that most OSX applications these days are crappy ports of Windows applications that have half of the performance and numerous additional bugs compared to their Windows counterpart, then it really start looking bleak. For example, look at any of the Windows/OSX audio applications, everything from workstations to plugins; they all perform so badly compared to Windows that you actually need that dual socket Xeon configuration just to break even with mainstream quad cores in Windows, presumably due to crappy BSDisms that Mac inherited, as well as general developer apathy to that 5% of their customer base. Had they based OSX on Linux, which is actually scalable and performant, it might have been a different story.

[RayDonnelly]

Great project, congratulations on the progress you've made.

To prevent some duplication, here's another project with similar goals (apologies if you've already aware of it):

https://github.com/shinh/maloader
http://shinh.skr.jp/slide/ldmac/000.html

I started the project in August this year. When working on something of this size, you have to start small. There's just no other way.

The next big thing (which is quite close by the way) is to have Apple LLVM-GCC fully working. That would basically mean you could compile applications for iOS on other platform than OS X. This is currently not possible. Anybody who wants to do iOS development has to buy a Mac. That sucks big time. iOS is not very popular in this part of Europe, but at least in the US I believe it's still the platform no. 1 for mobile apps.

I agree that not having cross compilers (for any platform) sucks, but it's not strictly true that there are none for iOS/OSX; there's been a few projects dedicated to making cross compilers for Darwin (xchain, toolchain4, puredarwin and opendarwin). I've adopted, modernised and improved them, forking from javacom's toolchain4:

Binaries:
http://mingw-and-ndk.googlecode.com/...-120724.tar.xz
http://mingw-and-ndk.googlecode.com/...dows-120614.7z

Source:
https://github.com/mingwandroid/toolchain4

Using these you can build both iOS and OSX software using either gcc or llvmgcc on either Linux or Windows. You need to bring your own SDK of course. I've not yet looked into the feasibility of building Darwin libc or any of the other system libs (nor the legality of distributing these). I think there's definitely a gap for the OSX/iOS equivalent of MinGW-w64.

My build scripts and patches are a bit untidy, I'm currently engaged in an effort to merge this work into crosstool-ng which will force me to clean things up.

Another project that's fairly closely related is Cocotron, an effort to make open versions of Cocoa for Windows:

http://www.cocotron.org/

I've a feeling that with all these bits lying around, something really cool could be cobbled together.

Ray.

[LubosD]

It would be interesting to run WINE on top of Darling. Anyway, there are two approaches to running foreign binaries on a system. One is to implement the syscall layer and rely on the foreign operating system's userland to enable it to work (the OS/2 approach). The other is to try to implement both the kernel and userland (the WINE approach). The former was very successful in its time and does not require many resources. The latter is a nightmare.

This is something I think about every now and then. So far, I'm using the Wine approach.

There is a problem with the OS/2 approach: unlike what you believe, it requires a *huge* investment of time in developing all the layers, which leads to basically recreating the original system. And that doesn't integrate very well with a Linux (desktop) environment. There will always have to be a line where you stop immitating and start integrating with "standard Linux layers".

Also given how much Apple's APIs rely on Mach kernel stuff, this is certainly not something I'd like to be putting into the Linux kernel. Not to mention how crappy Mach is. In certain synthetic tests, programs for OS X run faster under Darling than on the original system and Mach is to blame for that. We're talking orders of magnitude faster. But before anyone starts saying "Darling is faster than the original system", everything that comes from GNUstep seems to be way slower actually.

With the Wine approach, the similarities between the OS X and Linux userspace are in my favor. In my loader, OS X binaries are being dynamically linked against native ELF libraries where applicable.

Also, note that I am not doing this for any specific OS X applications. I think GNU/Linux is a very good platform when it comes to application availability and quality. By the way, I use Wine only like couple of times a year for stuff I could absolutely do without.

I'm doing this to give people a choice and to make Linux an even more "universal" operating system. I also do this because it's *not* something that exists in ten different variants and has been done by hunderds of people before.

As for LGPL/BSD: I do not use the BSD or similar license for a purpose. You wan't to make money off this? You either give me a cut or share your code with others. I find it fair.

[ryao]

This is something I think about every now and then. So far, I'm using the Wine approach.

There is a problem with the OS/2 approach: unlike what you believe, it requires a *huge* investment of time in developing all the layers, which leads to basically recreating the original system. And that doesn't integrate very well with a Linux (desktop) environment. There will always have to be a line where you stop immitating and start integrating with "standard Linux layers".

Also given how much Apple's APIs rely on Mach kernel stuff, this is certainly not something I'd like to be putting into the Linux kernel. Not to mention how crappy Mach is. In certain synthetic tests, programs for OS X run faster under Darling than on the original system and Mach is to blame for that. We're talking orders of magnitude faster. But before anyone starts saying "Darling is faster than the original system", everything that comes from GNUstep seems to be way slower actually.

With the Wine approach, the similarities between the OS X and Linux userspace are in my favor. In my loader, OS X binaries are being dynamically linked against native ELF libraries where applicable.

Also, note that I am not doing this for any specific OS X applications. I think GNU/Linux is a very good platform when it comes to application availability and quality. By the way, I use Wine only like couple of times a year for stuff I could absolutely do without.

I'm doing this to give people a choice and to make Linux an even more "universal" operating system. I also do this because it's *not* something that exists in ten different variants and has been done by hunderds of people before.

As for LGPL/BSD: I do not use the BSD or similar license for a purpose. You wan't to make money off this? You either give me a cut or share your code with others. I find it fair.

You might want to read the following paper:

http://research.microsoft.com/apps/p...aspx?id=141071

Anyway, have fun with your effort. It will be nice if it results in something.

[schmidtbag]

I think this is fantastic and I think it will become a greater success than wine and in less time, as long as enough people pitch in. Keep in mind, it shouldn't take as long for this project to work because Mac uses OpenGL, it can run X11 programs, it is BSD based, and there's just overall more similarities to the 2 OSes than there are with Windows. That being said, many games that are on Mac (which there are actually a lot of now) should run much better on linux than running the windows version of that game in Wine. It could get programs like the Adobe CS to operate nearly flawlessly, whereas getting the Windows version to work has overall been a complete failure. Programs like iTunes or Xcode could be used, which would be handy for any iOS developers.

While I'm sure some of you may disagree with me on this, just note that comparing Windows to Linux is like comparing a fish to a rodent. They're both animals, they both have circulatory, digestive, nervous, sexual, etc systems, but they're otherwise nothing alike. Mac vs Linux is like comparing a cat to a rodent. Still blatantly different but far more comparable than a fish. Comparable to the point that a rodent could wear a cat costume as a disguise and get away with it.

EDIT:
Somewhere down the line, I think it'd be cool if there was a PPC version of this that could run the old mac programs. I would only expect this to work on a PPC processor of course.

[ryao]

I think this is fantastic and I think it will become a greater success than wine and in less time, as long as enough people pitch in. Keep in mind, it shouldn't take as long for this project to work because Mac uses OpenGL, it can run X11 programs, it is BSD based, and there's just overall more similarities to the 2 OSes than there are with Windows. That being said, many games that are on Mac (which there are actually a lot of now) should run much better on linux than running the windows version of that game in Wine. It could get programs like the Adobe CS to operate nearly flawlessly, whereas getting the Windows version to work has overall been a complete failure. Programs like iTunes or Xcode could be used, which would be handy for any iOS developers.

While I'm sure some of you may disagree with me on this, just note that comparing Windows to Linux is like comparing a fish to a rodent. They're both animals, they both have circulatory, digestive, nervous, sexual, etc systems, but they're otherwise nothing alike. Mac vs Linux is like comparing a cat to a rodent. Still blatantly different but far more comparable than a fish. Comparable to the point that a rodent could wear a cat costume as a disguise and get away with it.

EDIT:
Somewhere down the line, I think it'd be cool if there was a PPC version of this that could run the old mac programs. I would only expect this to work on a PPC processor of course.

It will be interesting to watch how many people contribute. I do not imagine many people would find the idea that contributing to LubosD's project would enable him to enrich himself very appealing. The project might encounter legal problems should he fail to clarify that to contributors.

[n3wu53r]

I think this is fantastic and I think it will become a greater success than wine and in less time, as long as enough people pitch in. Keep in mind, it shouldn't take as long for this project to work because Mac uses OpenGL, it can run X11 programs, it is BSD based, and there's just overall more similarities to the 2 OSes than there are with Windows. That being said, many games that are on Mac (which there are actually a lot of now) should run much better on linux than running the windows version of that game in Wine. It could get programs like the Adobe CS to operate nearly flawlessly, whereas getting the Windows version to work has overall been a complete failure. Programs like iTunes or Xcode could be used, which would be handy for any iOS developers.

While I'm sure some of you may disagree with me on this, just note that comparing Windows to Linux is like comparing a fish to a rodent. They're both animals, they both have circulatory, digestive, nervous, sexual, etc systems, but they're otherwise nothing alike. Mac vs Linux is like comparing a cat to a rodent. Still blatantly different but far more comparable than a fish. Comparable to the point that a rodent could wear a cat costume as a disguise and get away with it.

EDIT:
Somewhere down the line, I think it'd be cool if there was a PPC version of this that could run the old mac programs. I would only expect this to work on a PPC processor of course.

It's worth noting, many Mac game ports are using wrappers around things like wine. For example, Transgaming cider http://transgaming.com/cider . I believe cider might actually have been a fork of wine (like cedega, which si from the same company) from when it was BSD licensed.

[boltronics]

Awesome! I looked around a few years ago for a project like this, and was surprised when I couldn't find anything. Good luck - I really hope this project takes off.

[CorkyAgain]

... it shouldn't take as long for this project to work because Mac uses OpenGL, it can run X11 programs, it is BSD based ...

But none of those things are related to the reasons people are interested in running apps from OS X.

Yes, you can run X11 and OpenGL programs on a Mac. But guess what? As far as I know, every one of those programs already runs on Linux. The reason people wanted X11 on Macs was so they could run open source applications!

BSD, X11 and OpenGL aren't going to help you, however, if what you want to do is run an app that uses Quartz and/or any of the other Mac-specific frameworks.