Facebook Releases Folly C++ Open-Source Library

[phoronix]

Phoronix: Facebook Releases Folly C++ Open-Source Library

Facebook this week released Folly, an open-source C++ library...

http://www.phoronix.com/vr.php?view=MTExMjQ

[gururise]

What functionality do these libraries provide? Doesn't seem to be mentioned in the article.

[yogi_berra]

What functionality do these libraries provide? Doesn't seem to be mentioned in the article.

It drops your stock price 30%

[smitty3268]

You should read the link Michael hid. https://www.facebook.com/notes/faceb...50864656793920

These utilities are loosely connected, but the over-arching theme for all of the components is high performance at scale. Some of them will show a fairly specialized focus, like reducing contention or packing things into small amounts of memory. Others, such as our in-memory JSON manipulation library or our string-formatting library, have a larger scope. But in either case, our motivation was to build components that were faster and more efficient than what we previously used.

As an example, consider AtomicHashMap. At Facebook, many of our services have large, concurrent in-memory indexes where lookups and insertions are common, but deletion is a rare (or unnecessary) operation. Often the keys in these indexes are 32 or 64 bit integers or can be hashed or converted to such integers. With these constraints, we've very efficiently addressed these types of workloads with a hashtable class that is wait-free for lookups, and has fine-grained locking to protect insertions. This component has nearly eliminated contention-related performance bottlenecks in several of our search services.

Performance is highly important, but since we have a couple hundred engineers working primarily in C++, our code also needs to be fast and reasonably easy to work with. In light of this, we make heavy use of modern C++, which has allowed us to retain a lot of programming convenience without spending much of our hardware resources on it. We believe some of these libraries even offer a step forward in terms of programmer convenience.

In particular, our string utilities are easier to use, faster, and safer than a number of existing options. Using folly::to instead of boost::lexical_cast wastes fewer CPU cycles and makes code more terse. Similarly, folly::format is less clumsy than iostream manipulators, and, unlike other options available to C++, does not sacrifice either type safety or performance. Many services spend significant portions of their CPU time manipulating strings, and programmers also tend to spend a non-trivial fraction of their time writing string manipulation code. We think these tools optimize for both processes simultaneously.

Practicality pervades Facebook engineering, and we have a low tolerance for "Not Invented Here" syndrome (after all, the company is built on open source software). So, you'll find Folly complements some existing high quality C++ libraries, such as Boost or the Standard Library, both of which we use heavily. The cases where you'll see Folly alternatives to an existing library are motivated by real-world drawbacks we ran into after trying existing solutions (generally performance related), or in some cases just a slightly more modern take on an old story (yes, we have a new version of the venerable ScopeGuard).

Better link, to github: https://github.com/facebook/folly/bl...cs/Overview.md

Below is a list of Folly components in alphabetical order, along with a brief description of each.
Arena.h, ThreadCachedArena.h

Simple arena for memory allocation: multiple allocations get freed all at once. With threaded version.
AtomicHashMap.h, AtomicHashArray.h

High-performance atomic hash map with almost lock-free operation.
Benchmark.h

A small framework for benchmarking code. Client code registers benchmarks, optionally with an argument that dictates the scale of the benchmark (iterations, working set size etc). The framework runs benchmarks (subject to a command-line flag) and produces formatted output with timing information.
Bits.h

Various bit manipulation utilities optimized for speed.
Bits.h

Bit-twiddling functions that wrap the ffsl(l) primitives in a uniform interface.
ConcurrentSkipList.h

An implementation of the structure described in A Provably Correct Scalable Concurrent Skip List by Herlihy et al.
Conv.h

A variety of data conversion routines (notably to and from string), optimized for speed and safety.
DiscriminatedPtr.h

Similar to boost::variant, but restricted to pointers only. Uses the highest-order unused 16 bits in a pointer as discriminator. So sizeof(DiscriminatedPtr<int, string, Widget>) == sizeof(void*).
dynamic.h

Dynamically-typed object, created with JSON objects in mind.
Endian.h

Endian conversion primitives.
Escape.h

Escapes a string in C style.
eventfd.h

Wrapper around the eventfd system call.
FBString.h

A drop-in implementation of std::string with a variety of optimizations.
FBVector.h

A mostly drop-in implementation of std::vector with a variety of optimizations.
Foreach.h

Pseudo-statements (implemented as macros) for iteration.
Format.h

Python-style formatting utilities.
GroupVarint.h

Group Varint encoding for 32-bit values.
Hash.h

Various popular hash function implementations.
Histogram.h

A simple class for collecting histogram data.
IntrusiveList.h

Convenience type definitions for using boost::intrusive_list.
json.h

JSON serializer and deserializer. Uses dynamic.h.
Likely.h

Wrappers around __builtin_expect.
Malloc.h

Memory allocation helpers, particularly when using jemalloc.
MapUtil.h

Helpers for finding items in associative containers (such as std::map and std::unordered_map).
PackedSyncPtr.h

A highly specialized data structure consisting of a pointer, a 1-bit spin lock, and a 15-bit integral, all inside one 64-bit word.
Preprocessor.h

Necessarily evil stuff.
PrettyPrint.h

Pretty-printer for numbers that appends suffixes of unit used: bytes (kb, MB, ...), metric suffixes (k, M, G, ...), and time (s, ms, us, ns, ...).
ProducerConsumerQueue.h

Lock free single-reader, single-writer queue.
Random.h

Defines only one function---randomNumberSeed().
Range.h

Boost-style range facility and the StringPiece specialization.
RWSpinLock.h

Fast and compact reader-writer spin lock.
ScopeGuard.h

C++11 incarnation of the old ScopeGuard idiom.
SmallLocks.h

Very small spin locks (1 byte and 1 bit).
small_vector.h

Vector with the small buffer optimization and an ptional embedded PicoSpinLock.
sorted_vector_types.h

Collections similar to std::map but implemented as sorted vectors.
StlAllocator.h

STL allocator wrapping a simple allocate/deallocate interface.
String.h

String utilities that connect folly::fbstring with std::string.
Synchronized.h

High-level synchronization library.
System.h

Demangling and errno utilities.
ThreadCachedInt.h

High-performance atomic increment using thread caching.
ThreadLocal.h

Improved thread local storage for non-trivial types.
TimeoutQueue.h

Queue with per-item timeout.
Traits.h

Type traits that complement those defined in the standard C++11 header <traits>.
Unicode.h

Defines the codePointToUtf8 function.

[uid313]

Not a fan of Facebook, but thanks for contributing back to the community!