FreeBSD Gets A Much Faster Sendfile Thanks To Netflix

[phoronix]

Phoronix: FreeBSD Gets A Much Faster Sendfile Thanks To Netflix

Thanks to work done by Netflix and NGINX, a new, drop-in-replacement sendfile syscall has been written for FreeBSD that is much faster...

http://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-Faster-Sendfile

[Imroy]

How does Linux's sendfile() compare?

[jacob]

How does Linux's sendfile() compare?

I think that on Linux, sendfile() is not a syscall but a library function implemented using splice() and some other things. No idea how it compares to FreeBSD's new sendfile.

[nranger]

I think that on Linux, sendfile() is not a syscall but a library function implemented using splice() and some other things. No idea how it compares to FreeBSD's new sendfile.

Nope, Linux sendfile() has been a regular syscall since kernel 2.2 back around 1999. It was implemented as a response to other systems like Windows NT having zero-copy methods of getting file contents out to a network link. FreeBSD got the ball rolling with the 3.0 release in 1998. When Linux added the splice() system call in 2006, sendfile got implemented on top of it, but it's still part of the kernel interface.

Typical uses of sendfile() in both FreeBSD and Linux are pretty similar, though the former has I think always been more flexible. You'll probably hit some performance edge cases in more extreme circumstances just do to the differences in network stack. The particular innovation Netflix has given to FreeBSD in this case is to make the call asynchronous -- meaning the calling application can continue without waiting. I think Linux's sendfile is still synchronous, so to accomplish the same thing your application has to do IO on separate threads. Either way the caller is going has to be more sophisticated (complicated) to deal with when the IO might (or might not) return. The FreeBSD approach though probably has some advantages in context switching if the server is being hit with many thousands of connections.

Another thing Netflix is doing which is not mentioned in this commit is how they moved TLS handling from the web server into the kernel. Regular sendfile is going to be useless if you have to copy the data out to userspace to be encrypted for an HTTPS session. I think their current approach is pretty specific to how Netflix uses NGINX, so I'm not sure how generally useful it's going to be and if the Linux folks would be keen on moving a big chunk of web serving functionality into kernel space to compete. More info here: https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf

[Mike Frett]

That's a great explanation nranger. Thank You! =)

[Jumbotron]

Kudos to Netflix for giving back.

[zdzichu]

nranger: In-kernel TLS for Linux is being worked on by Facebook: https://lwn.net/Articles/666509/

[bpetty]

FreeBSD, and MacOSX for that matter, both have zero copy, non-blocking, asynch sendfile implementations. It almost sounds like the article is making it sound like Netflix make some masterful change to sendfile. I haven't looked into the changes, but sendfile already could do most of the things on this list. I think they added a couple of extra flags. The changes may be game changing when you compare it to the Linux sendfile call.

[Guest]

How does Linux's sendfile() compare?

Why don't you ask Linus?

And I have to say that I absolutely despise the BSD people. They did sendfile() after both Linux and HP-UX had done it, and they must have known about both implementations. And they chose the HP-UX braindamage, and even brag about the fact that they were stupid and didn't understand TCP_CORK (they don't say so in those exact words, of course - they just show that they were stupid and clueless by the things they brag about).

[nils_]

I think their current approach is pretty specific to how Netflix uses NGINX, so I'm not sure how generally useful it's going to be and if the Linux folks would be keen on moving a big chunk of web serving functionality into kernel space to compete. More info here: https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf

There once was an in kernel webserver called tux if I remember correctly. It might be more interesting to move more of the functionality into nginx instead, if you look at what Cloudflare does for example they do some packet handling in userspace.