I know several people switched from Linux to OpenSolaris, mainly because of ZFS. With other solutions, your data is at risk because of silent corruption. The problem is when your hardware silently corrupts the bits, without telling you, without noticing it themselves. Then you need ECC functionality. ECC scans and corrects flipped bits in RAM. ZFS does the same thing on disk. And hardware raid doesnt help against problem of flipped bits, because HWraid wont even notice if a bit has been flipped. You need ECC to detect that. No filesystem utilizes ECC, except ZFS. The larger the discs, the more probable of some bits gets corrupted. Also, ZFS doesnt need fsck, and Linux fsck does only check the metadata, the data is not checked.
One of the best textst on why to use ZFS, on future filesystems and the new problems hyper modern filesystems face. A new disc has 20% of the surface dedicated to error correcting codes, and still there are errors!
CERN investigates silent corruption and presents very interesting conclusions:
It is time to abandon RAID-5, because the discs are so big now that very often bits will be silently flipped. The larger the raid, the more flipped bits, eventually you will always see flipped bits. Only ZFS fixes this problem:
"Electrical or magnetic interference inside a computer system can cause a single bit of DRAM to spontaneously flip to the opposite state."
Other things I like with OpenSolaris is that it is more robust than Linux, it scales better, etc. Solaris is an OS for big computers and big loads. On the desktop it doesnt shine.
For instance, you dont face code of varying quality in Solaris, as Linux kernel developer Andrew Morton says about Linux:
Q:Is it your opinion that the quality of the kernel is in decline? Most developers seem to be pretty sanguine about the overall quality problem. Assuming there's a difference of opinion here, where do you think it comes from? How can we resolve it?
A:I used to think it was in decline, and I think that I might think that it still is. I see so many regressions which we never fix."
That is why see people switching from Linux to Solaris:
Linux RAM overcommit it not a good strategy:
Linux doesnt scale well, when used as a file server:
And the lack of stable API/ABI is a bit of a pain. Linux is a moving target, your old drivers will not necessarily work. With Solaris is a different thing, the API and ABI has been frozen since way back. SUN guarantees binary compatibility back to Solaris v2.6, now Solaris is v5.10.