An introduction to C++'s variadic templates: a thread-safe multi-type map

Posted on Mon 01 February 2016 in C++ • Tagged with C++11, C++14, variadic templates, meta programmingLeave a comment

Trivia:

One of our favorite motto in our C++ team at work is: you shall use dependency injections instead of singletons! It actually comes with our unit-testing strategy. If the various components of your architecture are too tightly coupled, it becomes a tremendous effort to deeply test small critical chunks of your code. Singletons are that kind of beast that revives itself without your permission and comes from hell to haunt your lovely unit-tests. Our main project being multi-threaded (hence highly bug-prone) and vital for the company, "singleton" became a forbidden word. Yet, our team recently started going down the dark path. Thanks to C++11 and its variadic templates, I carefully crafted a thread-safe multi-type map container that simplified our configuration reloading system and saved us from the dark side of the coder force. If you always wondered what are variadic templates, how C++11's tuples can be implemented, I am going to present these concepts in this post using my container as a cobaye.

Note: for the sake of your sanity and the fact that errare humanum est, this article might not be 100% accurate!

Why would I use a thread-safe multi-type map?

Let me explain our odyssey: we are working on a highly modular and multi-threaded application. One of its core feature is the ability to reload various configuration files or assets used by some components spread accross many threads and a giant hierarchy of objects. The reloading process is automic using Linux's inotify monitoring filesystem events. One thread is dedicated to the reception of filesystem events and must react accordingly by parsing any changes and pushing them to other threads. At first, we used, to pass-by any newly parsed asset, some thread-safe queues or something analog to go channels. Since we did not want to use singletons, we had to pass references to our queues all along our object hierarchy. Sadly, our queue implementation is one to one and supports only one type, none of our config/asset types share the same base-type. For each asset type and each component using this asset, we had to create a new queue and pass-it all along our hierarchy. That is certainely not convenient! What we really wanted was a hybrid class between a std::map and a std::tuple.

We could have used a std::map with Boost.Variant to store our items, using a type like the following "std::map< std::string, std::shared_ptr< Boost.Variant < ConfigType1, ConfigType2>>>". Boost.Variant permits to encapsulate a heterogeneous set of types without common base-type or base-class, which solves one of our point. Another solution would be to encapsulate manually all our configuration classes in the same familly of classes, that is pretty cumbersome. But anyway, std::map does not guaranty any safety if you are writing and reading at the same time on a map slot. Secondly, std::shared_ptr does guaranty a thread-safe destruction of the pointee object (i.e: the reference counter is thread-safe) but nothing for the std::shared_ptr object itself. It means that copying a std::shared_ptr that could potentially be modified from another thread, might lead to an undefined behaviour. Even if we were to encapsulate all these unsafe accesses with mutexes, we are still lacking a nice mechanism to get update notifications for our objects. We do not want to constantly poll the latest version and propagate it through our code. And finally, if that solution were elegant enough, why would I currently write this blog post?

C++11 brings another collection type called std::tuple. It permits to store a set of elements of heterogeneous types. Take a look at this short example:

auto myTuple = std::make_tuple("Foo", 1337, 42);

std::cout << std::get<0>(myTuple) << std::endl; // Access element by index: "Foo"
std::cout << std::get<1>(myTuple) << std::endl; // Access element by index: 1337
std::cout << std::get<2>(myTuple) << std::endl; // Access element by index: 42
std::cout << std::get<const char*>(myTuple) << std::endl; // Access element by type: "Foo"

// compilation error: static_assert failed "tuple_element index out of range"
std::cout << std::get<3>(myTuple) << std::endl;

// compilation error: static_assert failed "type can only occur once in type list"
std::cout << std::get<int>(myTuple) << std::endl;

Tuples are that kind of C++11 jewelry that should decide your old-fashioned boss to upgrade your team's compiler (and his ugly tie). Not only I could store a const char* and two ints without any compiling error, but I could also access them using compile-time mechanisms. In some way, you can see tuples as a compile-time map using indexes or types as keys to reach its elements. You cannot use an index out of bands, it will be catched at compile-time anyway! Sadly, using a type as a key to retrieve an element is only possible if the type is unique in the tuple. At my work, we do have few config objects sharing the same class. Anyway, tuples weren't fitting our needs regarding thread safety and update events. Let's see what we could create using tasty tuples as an inspiration.

Note that some tuples implementations were already available before C++11, notably in boost. C++11 variadic templates are just very handy, as you will see, to construct such a class.

A teaser for my repository class:

To keep your attention for the rest of this post, here is my thread-safe multi-type map in action:

#include <iostream>
#include <memory>
#include <string>

#include "repository.hpp"

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

// Create a type for our repository.
using MyRepository = Repository
    <
        Slot<std::string>, // One slot for std::string.
        Slot<int, Key1>, // Two slots for int.
        Slot<int, Key2> // Must be differentiate using "type keys" (Key1, Key2).
    >;

int main()
{
    MyRepository myRepository;

    myRepository.emplace<std::string>("test"); // Construct the shared_ptr within the repository.
    myRepository.emplace<int, Key1>(1337);
    myRepository.set<int, Key2>(std::make_shared<int>(42)); // Set the shared_ptr manually.

    // Note: I use '*' as get returns a shared_ptr.
    std::cout << *myRepository.get<std::string>() << std::endl; // Print "test".
    std::cout << *myRepository.get<int, Key1>() << std::endl; // Print 1337.
    std::cout << *myRepository.get<int, Key2>() << std::endl; // Print 42.

    std::cout << *myRepository.get<int>() << std::endl;
    //             ^^^ Compilation error: which int shall be selected? Key1 or Key2?

    auto watcher = myRepository.getWatcher<std::string>(); // Create a watcher object to observe changes on std::string.
    std::cout << watcher->hasBeenChanged() << std::endl; // 0: no changes since the watcher creation.

    myRepository.emplace<std::string>("yo"); // Emplace a new value into the std::string slot.
    std::cout << watcher->hasBeenChanged() << std::endl; // 1: the std::string slot has been changed.

    std::cout << *watcher->get() << std::endl; // Poll the value and print "yo".
    std::cout << watcher->hasBeenChanged() << std::endl; // 0: no changes since the last polling.

    return EXIT_SUCCESS;
}

First and foremost, its name repository might not be well-suited for its responsibility. If your native language is the same as shakespeare and come-up with a better term, please feel free to submit it. In our internal usage, config repository sounded great!

I start by describing the slots necessary for my application by creating a new type MyRepository using a type alias. As you can see, I use the type of the slots as a key for accessing elements. But in case of contention, I must use a second key: an "empty type" ; like Key1 and Key2 in this example. If using types as keys seems odd for you, fear not! Here is the most rational explanation I can share with you: we are trying to benefit from our "know-it-all compiler". Your compiler is mainly manipulating types, one can change its flow using these types during the compilation process. Note that these structs are not even complete (no definition), it has no impact for the runtime memory or runtime execution and that's the amazing part of meta-programming. The dispatch of an expression such as "myRepository.get< int, Key1>()" is done during your build-time.

You may also notice that every slot is actually a std::shared_ptr. It enforces a clean ressource management: in a multithreaded application, one must be really careful of the lifetime of heap objects. std::shared_ptr in this case permits me to ensure that even if someone replaces a value in a slot, other components on other threads manipulating the old value won't end up with a dangling pointer/reference bomb in their hands. Another solution would be to use plain value objects, but not only it would require copying big objects in every other components but it would also remove polymorphism.

As for the updates signalisation, you first create a watcher object that establishes a contract between a desired slot to watch and your context. You can thereafter query in thread-safe way weither an update has been made and, if so, poll the latest changes. The watcher object is actually a std::unique_ptr for a special class, it cannot be moved nor copied without your permission and will automagically disable the signalisation contract between the slot and your context, once destroyed. We will dive deeper in this topic in the comming sections.

Within our application, the repository object is encapsulated into a RuntimeContext object. This RuntimeContext object is created explicitely within our main entry point and passed as a reference to a great part of our components. We therefore keep the possibility to test our code easily by setting this RuntimeContext with different implementations. Here is a simplified version of our usage:

// runtimecontext.hpp
#include "repository.hpp"

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

class ConfigType1; // Defined in another file.
class ConfigType2; // Defined in another file.

// Create a type for our repository.
using ConfigRepository = Repository
    <
        Slot<ConfigType1>,
        Slot<ConfigType2, Key1>,
        Slot<ConfigType2, Key2>
    >;

struct RuntimeContext
{
    ILogger* logger;
    // ...
    ConfigRepository configRepository;
};

// Main.cpp

#include "runtimecontext.hpp"

int main()
{
    RuntimeContext runtimeContext;
    // Setup:
    runtimeContext.logger = new StdOutLogger();
    // ...

    // Let's take a reference to the context and change the configuration repository when necessary. 
    startConfigurationMonitorThread(runtimeContext);

    // Let's take a reference and pass it down to all our components in various threads.
    startOurApplicationLogic(runtimeContext);

    return EXIT_SUCCESS;
}

Time for a C++11 implementation:

We can decompose the solution in 3 steps: at first we need to implement a map that accepts multiple types, we then need to work on the thread safety and finish by the watcher mechanism. Let's first fulfill the mission of this post: introducing you to variadic templates to solve the multiple-type problem.

Variadic templates:

You may not have heard of variadic templates in C++11 but I bet that you already used variadic functions like printf in C (maybe in a previous unsafe life). As wikipedia kindly explains "a variadic function is a function of indefinite which accepts a variable number of arguments". In other words, a variadic function has potentially an infinite number of parameters. Likewise, a variadic template has potentially an infinite number of parameters. Let's see how to use them!

Usage for variadic function templates:

Let's say that you wish to create a template that accept an infinite number of class as arguments. You will use the following notation:

template <class... T>

You specify a group of template parameters using the ellipsis notation named T. Note that this ellipsis notation is consistent with C's variadic function notation. This group of parameters, called a parameter-pack, can then be used in your function template or your class template by expanding them. One must use the ellipsis notation again (this time after T) to expand the parameter pack T:

template <class... T> void f(T...)
//              ^ pack T       ^expansion
{
    // Your function content.
}

Now that we have expanded T, what can we do Sir? Well, first you give to your expanded parameter types, a fancy name like t.

template <class... T> void f(T... t)
//                                ^ your fancy t.
{
    // Your function content.
}

If T = T1, T2, then T... t = T1 t1, T2 t2 and t = t1, t2. Brilliant, but is that all? Sure no! You can then expand again t using an "suffix-ellipsis" again:

template <class... T> void f(T... t)
{
    anotherFunction(t...);
    //                ^ t is expanded here! 
}

Finally, you can call this function f as you would with a normal function template:

template <class... T> void f(T... t)
{
    anotherFunction(t...);
}

f(1, "foo", "bar"); // Note: the argument deduction avoids us to use f<int, const char*, const char*>
// f(1, "foo", "bar") calls a generated f(int t1, const char* t2, const char* t3)
// with T1 = int, T2 = const char* and T3 = const char*,
// that itself calls anotherFunction(t1, t2, t3) equivalent to call anotherFunction(1, "foo", "bar");

Actually, the expansion mechanism is creating comma-separated replication of the pattern you apply the ellipsis onto. If you think I am tripping out with template-related wording, here is a much more concret example:

template <class... T> void g(T... t)
{
    anotherFunction(t...);
}

template <class... T> void f(T*... t)
{
    g(static_cast<double>(*t)...);
}

int main()
{
    int a = 2;
    int b = 3;

    f(&a, &b); // Call f(int* t1, int* t2).
    // Do a subcall to g(static_cast<double>(*t1), static_cast<double>(*t2)).

    return EXIT_SUCCESS;
}

I could use the pattern '*' for f parameters and therefore take them as a pointer! In the same manner, I applied the pattern 'static_cast< double>(*) to get the value of each arguments and cast them as doubles before forwarding them to g.

One last example before moving to variadic class templates. One can combine "normal" template parameters with parameter packs and initiate a compile recursion on function templates. Let's take a look at this printing function:

#include <iostream>

template <class HEAD> void print(HEAD head)
{
    std::cout << "Stop: " << head << std::endl;
}

template <class HEAD, class... TAIL> void print(HEAD head, TAIL... tail)
{
    std::cout << "Recurse: " << head << std::endl;
    print(tail...);
}

int main()
{
    print(42, 1337, "foo");

    // Print:
    // Recurse: 42
    // Recurse: 1337
    // Stop: foo

    // Call print<int, int, const char*> (second version of print).
    // The first int (head) is printed and we call print<int, const char*> (second version of print).
    // The second int (head again) is printed and we call print<const char*> (first version of print).
    // We reach recursion stopping condition, only one element left.

    return EXIT_SUCCESS;
}

Variadic templates are very interesting and I wouldn't be able to cover all their features within this post. It roughly feels like functional programming using your compiler, and even some Haskellers might listen to you if you bring that topic during a dinner. For those interested, I would challenge them to write a type-safe version of printf using variadic templates with the help of this reference. After that, you will run and scream of fear at the precense of C's vargs.

"Variadic" inheritance:

Sometimes during my programming sessions, I have a very awkward sensation that my crazy code will never compile and, yet, I finally see "build finished" in my terminal. I am talking about that kind of Frankenstein constructions:

struct A { };

struct B { };

template <class... T>
struct C: public T... // Variadic inheritance
{

};

C<A, B> c;

Yes, we can now create a class inheriting of an infinite number of bases. If you remember my explanation about pattern replications separated by commas, you can imaginge that struct C: public T... will be "transformed" in struct C: public A, public B, public T being the pattern. We start to be able to combine multiple types, each exposing a small amount of methods, to create a flexible concret type. That's one step closer to our multi-type map, and if you are interested in this concept, take a look at mixins.

Instead of inheriting directly from multiple types, couldn't we inherit from some types that encapsulate our types? Absolutely! A traditional map has some slots accessible using keys and these slots contain a value. If you give me base-class you are looking for, I can give you access to the value it contains:

#include <iostream>

struct SlotA
{
    int value;
};

struct SlotB
{
    std::string value;
};

// Note: private inheritance, no one can access directly to the slots other than C itself.
struct Repository: private SlotA, private SlotB
{

    void setSlotA(const int& value)
    {
        // I access the base-class's value
        // Since we have multiple base with a value field, we need to "force" the access to SlotA.
        SlotA::value = value;
    }

    int getSlotA()
    {
        return SlotA::value;
    }

    void setSlotB(const std::string& b)
    {
        SlotB::value = b;
    }

    std::string getSlotB()
    {
        return SlotB::value;
    }
};


int main()
{
    Repository r;

    r.setSlotA(42);
    std::cout << r.getSlotA() << std::endl; // Print: 42.

    r.setSlotB(std::string("toto"));
    std::cout << r.getSlotB() << std::endl; // Print: "toto".

    return EXIT_SUCCESS;
}

This code is not generic at all! We know how to create a generic Slot using a simple template, and we acquired the magic "create varidiac inheritance" skill. If my Repository class inherit from Slot< TypeA> and you call a method template with TypeA as a template argument, I can call the doGet method of the Slot< TypeA> base-class and give you back the value of TypeA in that repository. Let's fix the previous ugly copy-paste code:

#include <iostream>
#include <string>

template <class Type>
class Slot
{
protected:
    Type& doGet() // A nice encapsulation, that will be usefull later on.
    {
        return value_;
    }

    void doSet(const Type& value) // Same encapsulation.
    {
        value_ = value;
    }
private:
    Type value_;
};

template <class... Slots>
class Repository : private Slots... // inherit from our slots...
{
public:
    template <class Type> // Give me a type and,
    Type& get()
    {
        return Slot<Type>::doGet(); // I can select the Base class.
    }

    template <class Type>
    void set(const Type& value)
    {
        Slot<Type>::doSet(value);
    }
};

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

// Create a type for our repository.
using MyRepository = Repository
        <
                Slot<int>,       // Let's pick the type of our slots.
                Slot<std::string>
        >;

int main()
{
    MyRepository myRepository;

    myRepository.set<std::string>("toto");
    myRepository.set(42); // Notice the type deduction: we pass an int, so it writes in the int slot.

    std::cout << myRepository.get<int>() << std::endl; // Print: "toto".
    std::cout << myRepository.get<std::string>() << std::endl; // Print: 42.

    return EXIT_SUCCESS;
}

This repository starts to take shape, but we are not yet done! If you try to have two int slots, you will raise a compilation error: "base class 'Slot' specified more than once as a direct base class". We need to add another key-type to our slot class with a default value and we need to modify our repository methods to handle it:

struct DefaultSlotKey; // No needs for a definition

template <class T, class Key = DefaultSlotKey> // The Key type will never be trully used. 
class Slot
{
    // ...
};

template <class... Slots>
class Repository : private Slots...
{
public:
    template <class Type, class Key = DefaultSlotKey> // The default key must be here too.
    Type& get()
    {
        return Slot<Type, Key>::doGet();
    }

    template <class Type, class Key = DefaultSlotKey>
    void set(const Type& value)
    {
        Slot<Type, Key>::doSet(value);
    }
};

struct Key1; // No need for definition.
struct Key2;

// Now you can do:
using MyRepository = Repository
    <
            Slot<int>,       // Let's pick the type of our slots.
            Slot<std::string, Key1>,
            Slot<std::string, Key2>
    >;

Here is a UML representation of this Repository using distinct Keys for the type std::string: A nice UML diagram of my classes

Our repository class is missing an emplace method, right? emplace is taking a variable number of arguments with different types and forward them to create an object within one of our slots. A variable number of arguments and types must remind you something... variadic templates! Let's create this variadic emplace method as well as its equivalent in the Slot class:

// In class Slot:

template <class... Args>
void doEmplace(const Args&... args) // Here the pattern is const  &.
{
    value_ = Type(args...); // copy-operator (might use move semantics).
}

// In class Repository:
template <class Type, class Key = DefaultSlotKey, class... Args>
void emplace(const Args&... args) // Here the pattern is const  &.
{
    Slot<Type, Key>::doEmplace(args...);
}

// Usage:
myRepository.emplace<std::string>(4, 'a'); // Create a std::string "aaaa".

One last improvement for the future users of your repositories! If one morning, badly awake, a coworker of yours is trying to get a type or key that doesn't exist (like myRepository.get< double>();), he might be welcomed by such a message:

/home/jguegant/Coding/ConfigsRepo/main.cpp:36:33: error: call to non-static member function without an object argument
    return Slot<Type, Key>::doGet();
           ~~~~~~~~~~~~~~~~~^~~~~
/home/jguegant/Coding/ConfigsRepo/main.cpp:67:18: note: in instantiation of function template specialization 'Repository<Slot<int, DefaultSlotKey>, Slot<std::__1::basic_string<char>, DefaultSlotKey> >::get<double, DefaultSlotKey>' requested here
    myRepository.get<double>();
                 ^
/home/jguegant/Coding/ConfigsRepo/main.cpp:36:33: error: 'doGet' is a protected member of 'Slot<double, DefaultSlotKey>'
        return Slot<Type, Key>::doGet();
                                ^
/home/jguegant/Coding/ConfigsRepo/main.cpp:10:11: note: declared protected here
    Type& doGet()
          ^
2 errors generated.

This message is very confusing, our class does not inherit from Slot< double, DefaultSlotKey>! And we are talking about a clang output, I wonder what gcc or MSVC could produce... If you do not want to be assinated from your moody colleague with a spoon, here is a nice solution using C++11's static_asserts. Static asserts give you the possibility to generate your own compiler error messages in the same fashion as normal asserts but at compile-time. Using a the trait like std::is_base_of, you can suggest the user of your repository to check twice his type. Let's put this static_assert at the beggining of all the methods of Repository:

static_assert(std::is_base_of<Slot<Type, Key>, Repository<Slots...>>::value, 
          "Please ensure that this type or this key exists in this repository");

We are done for this part (finally...), time to think about multi-threading! If you want to know more about the magic behind std::is_base_of, I would suggest you to read my previous post on SFINAE, it might give you few hints. Here is a gist of what we achieved so far. Did you notice the change on emplace? If you do not understand it, have a look at this explanation on perfect forwarding. Sadly, it would be a way too long topic for this post (trust me on that point!) and has a minor impact on our repository right now.

Let's play safe:

The repository we just succeeded to craft can now be used in a single-thread environment without further investigation. But the initial decision was to make this class manipulable from multiple-threads without any worries considering the safety of our operations. As explained in the beginning of this post, we will not use direct values as we currently do, but instead allocate our objects on the heap and use some shared pointers to strictly control their lifetime. No matter which version (recent or deprecated) of the object a thread is manipulating, it's lifetime will be extended until the last thread using it definitely release it. It also implies that the objects themselves are thread-safe. In the case of read-only objects like configs or assets, it shouldn't be too much a burden. In this gist, you will find a repository version using std::shared_ptrs.

std::shared_ptr is an amazing feature of C++11 when dealing with multi-threading, but has its weakness. Within my code (in the previous gist link) a race condition can occur:

// What if I try to copy value_ at the return point...
std::shared_ptr<Type> doGet() const
{
    return value_;
}

// ... meanwhile another thread is changing value_ to value?
void doSet(const std::shared_ptr<Type> &value)
{
    value_ = value;
}

As specified: "If multiple threads of execution access the same std::shared_ptr object without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur". Note that we are talking about the same shared pointer. Multiple shared pointer copies pointing to the same object are fine, as long as these copies originated from the same shared pointer in first place. Copies are sharing the same control block, where the reference counters (one for shared_ptr and one for weak_ptr) are located, and the specification says "the control block of a shared_ptr is thread-safe: different std::shared_ptr objects can be accessed using mutable operations, such as operator= or reset, simultaneously by multiple threads, even when these instances are copies, and share the same control block internally.".

Depending on the age of your compiler and its standard library, I suggest two solutions:

1) A global mutex:

A straightforward solution relies on a std::mutex that we lock during doGet and doSet execution:

...
    std::shared_ptr<Type> doGet()
    {
        // The lock is enabled until value_ has been copied!
        std::lock_guard<std::mutex> lock(mutex_);
        return value_;
    }

    void doSet(const std::shared_ptr<Type> &value)
    {
        // The lock is enabled until value has been copied into value!
        std::lock_guard<std::mutex> lock(mutex_);
        value_ = value;
    }

private:
    std::mutex mutex_;
...

This solution is ideal if you have a Linux distribution that only ships gcc 4.8.x like mine. While not particularly elegant, it doesn't have a great impact on performances compared to the next solution.

2) Atomic access functions:

Starting from gcc 4.9, one can use atomic access functions to manipulate shared pointers. I dream of a day where a specialisation for std::atomic< std::shared_ptr> exists, but from now, we will resort to use std::atomic_load and std::atomic_exchange:

...
    std::shared_ptr<Type> doGet() const
    {
        return std::atomic_load(&value_);
    }

    void doSet(const std::shared_ptr<Type> &value)
    {
        std::atomic_exchange(&value_, value);
    }

private:
    std::shared_ptr<Type> value_;
...

Atomics are elegants and can often bring a great increase of performances if using lock-free instructions internally. Sadly, in the case of shared_ptrs, atomic_is_lock_free will return you false. By digging in libstdc++ and libc++, you will find some mutexes. gcc seems to use a fixed size "pool" of mutexes attributed to a shared_ptr according to a hash of its pointee address, when dealing with atomic operations. In other words, no rocket-science for atomic shared pointers until now.

Our own watchers:

"...I shall live and die at my post. I am the sword in the darkness. I am the watcher on the walls. I am the shield that guards the realms of men..." -- The Night's Watch oath

We want to be able to seal a bond between one of the slot and a context. By context, I mean the lifetime of an object in a thread, a function or a method. If an update has been made on that slot, we must be signaled in that context and to retrieve the new update. The bond must be destroyed if the context does not exist anymore. It should reminds you the Night's Watch oath ... as well as the RAII idiom: "holding a resource is tied to object lifetime: resource acquisition is done during object creation, by the constructor, while resource deallocation is done during object destruction, by the destructor. If objects are destroyed properly, resource leaks do not occur.". A strong ownership policy can be obtained with the help of a std::unique_ptr and the signalisation can be done using a boolean flag.

We will, therefore, encapsulate a std::atomic_bool into a class Watcher automagically registered to a slot once created, and unregistered once destructed. This Watcher class also takes as a reference the slot in order to query its value as you can see:

template <class Type, class Key>
class Watcher
{
public:
    Watcher(Slot<Type, Key>& slot):
            slot_(slot),
            hasBeenChanged_(false)
    {
    }

    Watcher(const Watcher&) = delete;               // Impossible to copy that class.

    Watcher & operator=(const Watcher&) = delete;   // Impossible to copy that class.

    bool hasBeenChanged() const
    {
        return hasBeenChanged_;
    }

    void triggerChanges()
    {
        hasBeenChanged_ = true;
    }

    auto get() -> decltype(std::declval<Slot<Type, Key>>().doGet())
    {
        hasBeenChanged_ = false; // Note: even if there is an update of the value between this line and the getValue one,
        // we will still have the latest version.
        // Note 2: atomic_bool automatically use a barrier and the two operations can't be inversed.
        return slot_.doGet();
    }

private:
    Slot<Type, Key>& slot_;
    std::atomic_bool hasBeenChanged_;
};

As for the automatic registration, we will add two private methods registerWatcher and unregisterWatcher to our Slot class that add or remove a watcher from an internal list. The list is always protected, when accessed, with a std::mutex and tracks all the current watchers that must be signaled when set is called on that slot.

template <class Type, class Key>
class Slot
{
public:
    using ThisType = Slot<Type, Key>;
    using WatcherType = Watcher<Type, Key>;

...
private:
    void registerWatcher(WatcherType* newWatcher)
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        watchers_.push_back(newWatcher);
    }

    void unregisterWatcher(WatcherType *toBeDelete)
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        watchers_.erase(std::remove(watchers_.begin(), watchers_.end(), toBeDelete), watchers_.end());

        delete toBeDelete; // Now that we removed the watcher from the list, we can proceed to delete it.
    }

    void signal()
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        for (auto watcher : watchers_) {
            watcher->triggerChanges(); // Let's raise the hasBeenChanged_ atomic boolean flag. 
        }
    }

private:
    std::vector<WatcherType*> watchers_; // All the registered watchers are in that list.

...
};

You may have notice that we are passing a bare WatcherType pointers. The ownership is actually given to whoever is using that watcher encapsulated within a std::unique_ptr. C++11's unique pointers are designed such as you can pass a custom deleter, or a delete callback so to speak. Hence, we can create a method that get a Watcher for a Slot, and register as the deleter of that Watcher a lambda function designed to call unregisterWatcher. Note that the slot MUST always lives longer than the unique pointer and its associated watcher (it should not be a problem in most cases). Let's finish that Slot class forever and ever:

template <class Type, class Key>
class Slot
{
public:
    using ThisType = Slot<Type, Key>;
    using WatcherType = Watcher<Type, Key>;

    // We use unique_ptr for a strong ownership policy.
    // We use std::function to declare the type of our deleter.
    using WatcherTypePtr = std::unique_ptr<WatcherType, std::function<void(WatcherType*)>> ;

...

public:
    WatcherTypePtr doGetWatcher()
    {
        // Create a unique_ptr and pass a lambda as a deleter.
        // The lambda capture "this" and will call unregisterWatcher.
        WatcherTypePtr watcher(new WatcherType(*this), [this](WatcherType* toBeDelete) {
            this->unregisterWatcher(toBeDelete);});

        registerWatcher(watcher.get());

        return watcher;
    }
...
};

Are we done? Hell no, but we will be really soon. All we need is to expose the possibility to acquire a watcher from the repository itself. In the same manner as set and get, we simply dispatch using the type and the key on one of our slot:

template <class Type, class Key = DefaultSlotKey>
typename Slot<Type, Key>::WatcherTypePtr getWatcher() // typename is used for disambiguate
{
    return Slot<Type, Key>::doGetWatcher();
}

WAIT, don't close that page too fast. If you want to be able to snub everyone, you can replace this ugly typename Slot::WatcherTypePtr with auto and claim that your repository class is C++14 only! Grab the full code of what we build together on gist and enjoy!

Conclusion:

Once again, I hope you enjoyed this post about one of my favourite subject: C++. I might not be the best teacher nor the best author but I wish that you learnt something today! Please, if you any suggestions or questions, feel free to post anything in the commentaries. My broken English being what it is, I kindly accept any help for my written mistakes.

Many thanks to my colleagues that greatly helped me by reviewing my code and for the time together.


An introduction to C++'s SFINAE concept: compile-time introspection of a class member

Posted on Sat 31 October 2015 in C++ • Tagged with C++11, C++14, TMP, meta programmingLeave a comment

Trivia:

As a C++ enthusiast, I usually follow the annual C++ conference cppconf or at least try to keep myself up-to-date with the major events that happen there. One way to catch up, if you can't afford a plane ticket or the ticket, is to follow the youtube channel dedicated to this conference. This year, I was impressed by Louis Dionne talk entitled "C++ Metaprogramming: A Paradigm Shift". One feature called is_valid that can be found in Louis's Boost.Hana library particulary caught my attention. This genious is_valid function heavily rely on an even more "magic" C++ programming technique coined with the term SFINAE discovered at the end of the previous century. If this acronym doesn't speak to you, don't be scared, we are going to dive straight in the subject.

Note: for the sake of your sanity and the fact that errare humanum est, this article might not be 100% accurate!

Introspection in C++?

Before explaining what is SFINAE, let's explore one of its main usage: introspection. As you might be aware, C++ doesn't excel when it comes to examine the type or properties of an object at runtime. The best ability provided by default would be RTTI. Not only RTTI isn't always available, but it also gives you barely more than the current type of the manipulated object. Dynamic languages or those having reflection on the other hand are really convenient in some situations like serialization.

For instance, in Python, using reflection, one can do the following:

class A(object):
    # Simply overrides the 'object.__str__' method.
    def __str__(self):
        return "I am a A"

class B(object):
    # A custom method for my custom objects that I want to serialize.
    def serialize(self):
        return "I am a B"

class C(object):
    def __init__(self):
        # Oups! 'serialize' is not a method. 
        self.serialize = 0

    def __str__(self):
        return "I am a C"

def serialize(obj):
    # Let's check if obj has an attribute called 'serialize'.
    if hasattr(obj, "serialize"):
        # Let's check if this 'serialize' attribute is a method.
        if hasattr(obj.serialize, "__call__"):
            return obj.serialize()

    # Else we call the __str__ method.
    return str(obj)

a = A()
b = B()
c = C()

print(serialize(a)) # output: I am a A.
print(serialize(b)) # output: I am a B.
print(serialize(c)) # output: I am a C.

As you can see, during serialization, it comes pretty handy to be able to check if an object has an attribute and to query the type of this attribute. In our case, it permits us to use the serialize method if available and fall back to the more generic method str otherwise. Powerful, isn't it? Well, we can do it in plain C++!

Here is the C++14 solution mentionned in Boost.Hana documentation, using is_valid:

#include <boost/hana.hpp>
#include <iostream>
#include <string>

using namespace std;
namespace hana = boost::hana;

// Check if a type has a serialize method.
auto hasSerialize = hana::is_valid([](auto&& x) -> decltype(x.serialize()) { });

// Serialize any kind of objects.
template <typename T>
std::string serialize(T const& obj) {
    return hana::if_(hasSerialize(obj), // Serialize is selected if available!
                     [](auto& x) { return x.serialize(); },
                     [](auto& x) { return to_string(x); }
    )(obj);
}

// Type A with only a to_string overload.
struct A {};

std::string to_string(const A&)
{
    return "I am a A!";
}

// Type B with a serialize method.
struct B
{
    std::string serialize() const
    {
        return "I am a B!";
    }
};

// Type C with a "wrong" serialize member (not a method) and a to_string overload.
struct C
{
    std::string serialize;
};

std::string to_string(const C&)
{
    return "I am a C!";
}

int main() {
    A a;
    B b;
    C c;

    std::cout << serialize(a) << std::endl;
    std::cout << serialize(b) << std::endl;
    std::cout << serialize(c) << std::endl;
}

As you can see, it only requires a bit more of boilerplate than Python, but not as much as you would expect from a language as complexe as C++. How does it work? Well if you are too lazy to read the rest, here is the simplest answer I can give you: unlike dynamically typed languages, your compiler has access a lot of static type information once fired. It makes sense that we can constraint your compiler to do a bit of work on these types! The next question that comes to your mind is "How to?". Well, right below we are going to explore the various options we have to enslave our favorite compiler for fun and profit! And we will eventually recreate our own is_valid.

The old-fashioned C++98-way:

Whether your compiler is a dinosaur, your boss refuses to pay for the latest Visual Studio license or you simply love archeology, this chapter will interest you. It's also interesting for the people stuck between C++11 and C++14. The solution in C++98 relies on 3 key concepts: overload resolution, SFINAE and the static behavior of sizeof.

Overload resolution:

A simple function call like "f(obj);"" in C++ activates a mechanism to figure out which f function shoud be called according to the argument obj. If a set of f functions could accept obj as an argument, the compiler must choose the most appropriate function, or in other words resolve the best overload! Here is a good cppreference page explaining the full process: Overload resolution. The rule of thumb in this case is the compiler picks the candidate function whose parameters match the arguments most closely is the one that is called. Nothing is better than a good example:

void f(std::string s); // int can't be convert into a string.
void f(double d); // int can be implicitly convert into a double, so this version could be selected, but...
void f(int i); // ... this version using the type int directly is even more close!

f(1); // Call f(int i);

In C++ you also have some sink-hole functions that accept everything. First, function templates accept any kind of parameter (let's say T). But the true black-hole of your compiler, the devil variable vacuum, the oblivion of the forgotten types are the variadic functions. Yes, exactly like the horrible C printf.

std::string f(...); // Variadic functions are so "untyped" that...
template <typename T> std::string f(const T& t); // ...this templated function got the precedence!

f(1); // Call the templated function version of f.

The fact that function templates are less generic than variadic functions is the first point you must remember!

Note: A templated function can actually be more precise than a normal function. However, in case of a draw, the normal function will have the precedence.

SFINAE:

I am already teasing you with the power for already few paragraphs and here finally comes the explanation of this not so complex acronym. SFINAE stands for Substitution Failure Is Not An Error. In rough terms, a substitution is the mechanism that tries to replace the template parameters with the provided types or values. In some cases, if the substitution leads to an invalid code, the compiler shouldn't throw a massive amount of errors but simply continue to try the other available overloads. The SFINAE concept simply guaranties such a "sane" behavior for a "sane" compiler. For instance:

/*
 The compiler will try this overload since it's less generic than the variadic.
 T will be replace by int which gives us void f(const int& t, int::iterator* b = nullptr);
 int doesn't have an iterator sub-type, but the compiler doesn't throw a bunch of errors.
 It simply tries the next overload. 
*/
template <typename T> void f(const T& t, typename T::iterator* it = nullptr) { }

// The sink-hole.
void f(...) { }

f(1); // Calls void f(...) { }

All the expressions won't lead to a SFINAE. A broad rule would be to say that all the substitutions out of the function/methods body are "safes". For a better list, please take a look at this wiki page. For instance, a wrong substitution within a function body will lead to a horrible C++ template error:

// The compiler will be really unhappy when it will later discover the call to hahahaICrash. 
template <typename T> void f(T t) { t.hahahaICrash(); }
void f(...) { } // The sink-hole wasn't even considered.

f(1);

The operator sizeof:

The sizeof operator is really a nice tool! It permits us to returns the size in bytes of a type or an expression at compilation time. sizeof is really interesting as it accurately evaluates an expression as precisely as if it were compiled. One can for instance do:

typedef char type_test[42];
type_test& f();

// In the following lines f won't even be truly called but we can still access to the size of its return type.
// Thanks to the "fake evaluation" of the sizeof operator.
char arrayTest[sizeof(f())];
std::cout << sizeof(f()) << std::endl; // Output 42.

But wait! If we can manipulate some compile-time integers, couldn't we do some compile-time comparison? The answer is: absolutely yes, my dear reader! Here we are:

typedef char yes; // Size: 1 byte.
typedef yes no[2]; // Size: 2 bytes.

// Two functions using our type with different size.
yes& f1();
no& f2();

std::cout << (sizeof(f1()) == sizeof(f2())) << std::endl; // Output 0.
std::cout << (sizeof(f1()) == sizeof(f1())) << std::endl; // Output 1.

Combining everything:

Now we have all the tools to create a solution to check the existence of a method within a type at compile time. You might even have already figured it out most of it by yourself. So let's create it:

template <class T> struct hasSerialize
{
    // For the compile time comparison.
    typedef char yes[1];
    typedef yes no[2];

    // This helper struct permits us to check that serialize is truly a method.
    // The second argument must be of the type of the first.
    // For instance reallyHas<int, 10> would be substituted by reallyHas<int, int 10> and works!
    // reallyHas<int, &C::serialize> would be substituted by reallyHas<int, int &C::serialize> and fail!
    // Note: It only works with integral constants and pointers (so function pointers work).
    // In our case we check that &C::serialize has the same signature as the first argument!
    // reallyHas<std::string (C::*)(), &C::serialize> should be substituted by 
    // reallyHas<std::string (C::*)(), std::string (C::*)() &C::serialize> and work!
    template <typename U, U u> struct reallyHas;

    // Two overloads for yes: one for the signature of a normal method, one is for the signature of a const method.
    // We accept a pointer to our helper struct, in order to avoid to instantiate a real instance of this type.
    // std::string (C::*)() is function pointer declaration.
    template <typename C> static yes& test(reallyHas<std::string (C::*)(), &C::serialize>* /*unused*/) { }
    template <typename C> static yes& test(reallyHas<std::string (C::*)() const, &C::serialize>* /*unused*/) { }

    // The famous C++ sink-hole.
    // Note that sink-hole must be templated too as we are testing test<T>(0).
    // If the method serialize isn't available, we will end up in this method.
    template <typename> static no& test(...) { /* dark matter */ }

    // The constant used as a return value for the test.
    // The test is actually done here, thanks to the sizeof compile-time evaluation.
    static const bool value = sizeof(test<T>(0)) == sizeof(yes);
};

// Using the struct A, B, C defined in the previous hasSerialize example.
std::cout << hasSerialize<A>::value << std::endl;
std::cout << hasSerialize<B>::value << std::endl;
std::cout << hasSerialize<C>::value << std::endl;

The reallyHas struct is kinda tricky but necessary to ensure that serialize is a method and not a simple member of the type. You can do a lot of test on a type using variants of this solution (test a member, a sub-type...) and I suggest you to google a bit more about SFINAE tricks. Note: if you truly want a pure compile-time constant and avoid some errors on old compilers, you can replace the last value evaluation by: "enum { value = sizeof(test(0)) == sizeof(yes) };".

You might also wonder why it doesn't work with inheritence. Inheritence in C++ and dynamic polymorphism is a concept available at runtime, or in other words, a data that the compiler won't have and can't guess! However, compile time type inspection is much more efficient (0 impact at runtime) and almost as powerful as if it were at runtime. For instance:

// Using the previous A struct and hasSerialize helper.

struct D : A
{
    std::string serialize() const
    {
        return "I am a D!";
    }
};

template <class T> bool testHasSerialize(const T& /*t*/) { return hasSerialize<T>::value; }

D d;
A& a = d; // Here we lost the type of d at compile time.
std::cout << testHasSerialize(d) << std::endl; // Output 1.
std::cout << testHasSerialize(a) << std::endl; // Output 0.

Last but no least, our test cover the main cases but not the tricky ones like a Functor:

struct E
{
    struct Functor
    {
        std::string operator()()
        {
            return "I am a E!";
        }
    };

    Functor serialize;
};

E e;
std::cout << e.serialize() << std::endl; // Succefully call the functor.
std::cout << testHasSerialize(e) << std::endl; // Output 0.

The trade-off for a full coverage would be the readability. As you will see, C++11 shines in that domain!

Time to use our genius idea:

Now you would think that it will be super easy to use our hasSerialize to create a serialize function! Okay let's try it:

template <class T> std::string serialize(const T& obj)
{
    if (hasSerialize<T>::value) {
        return obj.serialize(); // error: no member named 'serialize' in 'A'.
    } else {
        return to_string(obj);
    }
}

A a;
serialize(a);

It might be hard to accept, but the error raised by your compiler is absolutely normal! If you consider the code that you will obtain after substitution and compile-time evaluation:

std::string serialize(const A& obj)
{
    if (0) { // Dead branching, but the compiler will still consider it!
        return obj.serialize(); // error: no member named 'serialize' in 'A'.
    } else {
        return to_string(obj);
    }
}

Your compiler is really a good guy and won't drop any dead-branch, and obj must therefore have both a serialize method and a to_string overload in this case. The solution consists in spliting the serialize function into two different functions: one where we solely use obj.serialize() and one where we use to_string according to obj's type. We come back to an earlier problem that we already solved, how to split according to a type? SFINAE, for sure! At that point we could re-work our hasSerialize function into a serialize function and make it return a std::string instead of compile time boolean. But we won't do it that way! It's cleaner to separate the hasSerialize test from its usage serialize.

We need to find a clever SFINAE solution on the signature of "template <class T> std::string serialize(const T& obj)". I bring you the last piece of the puzzle called enable_if.

template<bool B, class T = void> // Default template version.
struct enable_if {}; // This struct doesn't define "type" and the substitution will fail if you try to access it.

template<class T> // A specialisation used if the expression is true. 
struct enable_if<true, T> { typedef T type; }; // This struct do have a "type" and won't fail on access.

// Usage:
enable_if<true, int>::type t1; // Compiler happy. t's type is int.
enable_if<hasSerialize<B>::value, int>::type t2; // Compiler happy. t's type is int.

enable_if<false, int>::type t3; // Compiler unhappy. no type named 'type' in 'enable_if<false, int>';
enable_if<hasSerialize<A>::value, int>::type t4; // no type named 'type' in 'enable_if<false, int>';

As you can see, we can trigger a substitution failure according to a compile time expression with enable_if. Now we can use this failure on the "template <class T> std::string serialize(const T& obj)" signature to dispatch to the right version. Finally, we have the true solution of our problem:

template <class T> typename enable_if<hasSerialize<T>::value, std::string>::type serialize(const T& obj)
{
    return obj.serialize();
}

template <class T> typename enable_if<!hasSerialize<T>::value, std::string>::type serialize(const T& obj)
{
    return to_string(obj);
}

A a;
B b;
C c;

// The following lines work like a charm!
std::cout << serialize(a) << std::endl;
std::cout << serialize(b) << std::endl;
std::cout << serialize(c) << std::endl;

Two details worth being noted! Firstly we use enable_if on the return type, in order to keep the paramater deduction, otherwise we would have to specify the type explicitely "serialize<A>(a)". Second, even the version using to_string must use the enable_if, otherwise serialize(b) would have two potential overloads available and raise an ambiguity. If you want to check the full code of this C++98 version, here is a gist. Life is much easier in C++11, so let's see the beauty of this new standard!

Note: it's also important to know that this code creates a SFINAE on an expression ("&C::serialize"). Whilst this feature wasn't required by the C++98 standard, it was already in use depending on your compiler. It trully became a safe choice in C++11.

When C++11 came to our help:

After the great century leap year in 2000, people were fairly optimistic about the coming years. Some even decided to design a new standard for the next generation of C++ coders like me! Not only this standard would ease TMP headaches (Template Meta Programming side-effects), but it would be available in the first decade, hence its code-name C++0x. Well, the standard sadly came the next decade (2011 ==> C++11), but it brought a lot of features interesting for the purpose of this article. Let's review them!

decltype, declval, auto & co:

Do you remember that the sizeof operator does a "fake evaluation" of the expression that you pass to it, and return gives you the size of the type of the expression? Well C++11 adds a new operator called decltype. decltype gives you the type of the of the expression it will evaluate. As I am kind, I won't let you google an example and give it to you directly:

B b;
decltype(b.serialize()) test = "test"; // Evaluate b.serialize(), which is typed as std::string.
// Equivalent to std::string test = "test";

declval is an utility that gives you a "fake reference" to an object of a type that couldn't be easily construct. declval is really handy for our SFINAE constructions. cppreference example is really straightforward, so here is a copy:

struct Default {
    int foo() const {return 1;}
};

struct NonDefault {
    NonDefault(const NonDefault&) {}
    int foo() const {return 1;}
};

int main()
{
    decltype(Default().foo()) n1 = 1; // int n1
//  decltype(NonDefault().foo()) n2 = n1; // error: no default constructor
    decltype(std::declval<NonDefault>().foo()) n2 = n1; // int n2
    std::cout << "n2 = " << n2 << '\n';
}

The auto specifier specifies that the type of the variable that is being declared will be automatically deduced. auto is equivalent of var in C#. auto in C++11 has also a less famous but nonetheless usage for function declaration. Here is a good example:

bool f();
auto test = f(); // Famous usage, auto deduced that test is a boolean, hurray!



//                             vvv t wasn't declare at that point, it will be after as a parameter!
template <typename T> decltype(t.serialize()) g(const T& t) {   } // Compilation error

// Less famous usage:
//                    vvv auto delayed the return type specification!
//                    vvv                vvv the return type is specified here and use t!
template <typename T> auto g(const T& t) -> decltype(t.serialize()) {   } // No compilation error.

As you can see, auto permits to use the trailing return type syntax and use decltype coupled with an expression involving one of the function argument. Does it means that we can use it to test the existence of serialize with a SFINAE? Yes Dr. Watson! decltype will shine really soon, you will have to wait for the C++14 for this tricky auto usage (but since it's a C++11 feature, it ends up here).

constexpr:

C++11 also came with a new way to do compile-time computations! The new keyword constexpr is a hint for your compiler, meaning that this expression is constant and could be evaluate directly at compile time. In C++11, constexpr has a lot of rules and only a small subset of VIEs (Very Important Expression) expressions can be used (no loops...)! We still have enough for creating a compile-time factorial function:

constexpr int factorial(int n)
{
    return n <= 1? 1 : (n * factorial(n - 1));
}

int i = factorial(5); // Call to a constexpr function.
// Will be replace by a good compiler by:
// int i = 120;

constexpr increased the usage of std::true_type & std::false_type from the STL. As their name suggest, these types encapsulate a constexpr boolean "true" and a constrexpr boolean "false". Their most important property is that a class or a struct can inherit from them. For instance:

struct testStruct : std::true_type { }; // Inherit from the true type.

constexpr bool testVar = testStruct(); // Generate a compile-time testStruct.
bool test = testStruct::value; // Equivalent to: test = true;
test = testVar; // true_type has a constexpr converter operator, equivalent to: test = true;

Blending time:

First solution:

In cooking, a good recipe requires to mix all the best ingredients in the right proportions. If you don't want to have a spaghetti code dating from 1998 for dinner, let's revisit our C++98 hasSerialize and serialize functions with "fresh" ingredients from 2011. Let's start by removing the rotting reallyHas trick with a tasty decltype and bake a bit of constexpr instead of sizeof. After 15min in the oven (or fighting with a new headache), you will obtain:

template <class T> struct hasSerialize
{
    // We test if the type has serialize using decltype and declval.
    template <typename C> static constexpr decltype(std::declval<C>().serialize(), bool()) test(int /* unused */)
    {
        // We can return values, thanks to constexpr instead of playing with sizeof.
        return true;
    }

    template <typename C> static constexpr bool test(...)
    {
        return false;
    }

    // int is used to give the precedence!
    static constexpr bool value = test<T>(int());
};

You might be a bit puzzled by my usage of decltype. The C++ comma operator "," can create a chain of multiple expressions. In decltype, all the expressions will be evaluated, but only the last expression will be considered for the type. The serialize doesn't need any changes, minus the fact that the enable_if function is now provided in the STL. For your tests, here is a gist.

Second solution:

Another C++11 solution described in Boost.Hanna documentation and using std::true_type and std::false_type, would be this one:

// Primary template, inherit from std::false_type.
// ::value will return false. 
// Note: the second unused template parameter is set to default as std::string!!!
template <typename T, typename = std::string>
struct hasSerialize
        : std::false_type
{

};

// Partial template specialisation, inherit from std::true_type.
// ::value will return true. 
template <typename T>
struct hasSerialize<T, decltype(std::declval<T>().serialize())>
        : std::true_type
{

};

This solution is, in my own opinion, more sneaky! It relies on a not-so-famous-property of default template parameters. But if your soul is already (stack-)corrupted, you may be aware that the default parameters are propagated in the specialisations. So when we use hasSerialize<OurType>::value, the default parameter comes into play and we are actually looking for hasSerialize<OurType, std::string>::value both on the primary template and the specialisation. In the meantime, the substitution and the evaluation of decltype are processed and our specialisation has the signature hasSerialize<OurType, std::string> if OurType has a serialize method that returns a std::string, otherwise the substitution fails. The specialisation has therefore the precedence in the good cases. One will be able to use the std::void_t C++17 helper in these cases. Anyway, here is a gist you can play with!

I told you that this second solution hides a lot of complexity, and we still have a lot of C++11 features unexploited like nullptr, lambda, r-values. No worries, we are going to use some of them in C++14!

The supremacy of C++14:

According to the Gregorian calendar in the upper-right corner of my XFCE environment, we are in 2015! I can turn on the C++14 compilation flag on my favorite compiler safely, isn't it? Well, I can with clang (is MSVC using a maya calendar?). Once again, let's explore the new features, and use them to build something wonderful! We will even recreate an is_valid, like I promised at the beggining of this article.

auto & lambdas:

Return type inference:

Some cool features in C++14 come from the relaxed usage of the auto keyword (the one used for type inference).

Now, auto can be used on the return type of a function or a method. For instance:

auto myFunction() // Automagically figures out that myFunction returns ints.
{
    return int();
}

It works as long as the type is easily "guessable" by the compiler. We are coding in C++ after all, not OCaml!

A feature for functional lovers:

C++11 introduced lambdas. A lambda has the following syntax:

[capture-list](params) -> non-mandatory-return-type { ...body... }

A useful example in our case would be:

auto l1 = [](B& b) { return b.serialize(); }; // Return type figured-out by the return statement.
auto l3 = [](B& b) -> std::string { return b.serialize(); }; // Fixed return type.
auto l2 = [](B& b) -> decltype(b.serialize()) { return b.serialize(); }; // Return type dependant to the B type.

std::cout << l1(b) << std::endl; // Output: I am a B!
std::cout << l2(b) << std::endl; // Output: I am a B!
std::cout << l3(b) << std::endl; // Output: I am a B!

C++14 brings a small change to the lambdas but with a big impact! Lambdas accept auto parameters: the parameter type is deduced according the argument. Lambdas are implemented as an object having an newly created unnamed type, also called closure type. If a lambda has some auto parameters, its "Functor operator" operator() will be simply templated. Let's take a look:

// ***** Simple lambda unamed type *****
auto l4 = [](int a, int b) { return a + b; };
std::cout << l4(4, 5) << std::endl; // Output 9.

// Equivalent to:
struct l4UnamedType
{
    int operator()(int a, int b) const
    {
        return a + b;
    }
};

l4UnamedType l4Equivalent = l4UnamedType();
std::cout << l4Equivalent(4, 5) << std::endl; // Output 9 too.



// ***** auto parameters lambda unnamed type *****

// b's type is automagically deduced!
auto l5 = [](auto& t) -> decltype(t.serialize()) { return t.serialize(); };

std::cout << l5(b) << std::endl; // Output: I am a B!
std::cout << l5(a) << std::endl; // Error: no member named 'serialize' in 'A'.

// Equivalent to:
struct l5UnamedType
{
    template <typename T> auto operator()(T& t) const -> decltype(t.serialize()) // /!\ This signature is nice for a SFINAE!
    {
        return t.serialize();
    }
};

l5UnamedType l5Equivalent = l5UnamedType();

std::cout << l5Equivalent(b) << std::endl; // Output: I am a B!
std::cout << l5Equivalent(a) << std::endl; // Error: no member named 'serialize' in 'A'.

More than the lambda itself, we are interested by the generated unnamed type: its lambda operator() can be used as a SFINAE! And as you can see, writing a lambda is less cumbersome than writing the equivalent type. It should remind you the beggining of my initial solution:

// Check if a type has a serialize method.
auto hasSerialize = hana::is_valid([](auto&& x) -> decltype(x.serialize()) { });

And the good new is that we have everything to recreate is_valid, right now!

The making-of a valid is_valid:

Now that we have a really stylish manner to generate a unnamed types with potential SFINAE properties using lambdas, we need to figure out how to use them! As you can see, hana::is_valid is a function that takes our lambda as a parameter and return a type. We will call the type returned by is_valid the container. The container will be in charge to keep the lambda's unnamed type for a later usage. Let's start by writing the is_valid function and its the containter:

template <typename UnnamedType> struct container
{
    // Remembers UnnamedType.
};

template <typename UnnamedType> constexpr auto is_valid(const UnnamedType& t) 
{
    // We used auto for the return type: it will be deduced here.
    return container<UnnamedType>();
}

auto test = is_valid([](const auto& t) -> decltype(t.serialize()) {})
// Now 'test' remembers the type of the lambda and the signature of its operator()!

The next step consists at extending container with the operator operator() such as we can call it with an argument. This argument type will be tested against the UnnamedType! In order to do a test on the argument type, we can use once again a SFINAE on a reacreated 'UnnamedType' object! It gives us this solution:

template <typename UnnamedType> struct container
{
// Let's put the test in private.
private:
    // We use std::declval to 'recreate' an object of 'UnnamedType'.
    // We use std::declval to also 'recreate' an object of type 'Param'.
    // We can use both of these recreated objects to test the validity!
    template <typename Param> constexpr auto testValidity(int /* unused */)
    -> decltype(std::declval<UnnamedType>()(std::declval<Param>()), std::true_type())
    {
        // If substitution didn't fail, we can return a true_type.
        return std::true_type();
    }

    template <typename Param> constexpr std::false_type testValidity(...)
    {
        // Our sink-hole returns a false_type.
        return std::false_type();
    }

public:
    // A public operator() that accept the argument we wish to test onto the UnnamedType.
    // Notice that the return type is automatic!
    template <typename Param> constexpr auto operator()(const Param& p)
    {
        // The argument is forwarded to one of the two overloads.
        // The SFINAE on the 'true_type' will come into play to dispatch.
        // Once again, we use the int for the precedence.
        return testValidity<Param>(int());
    }
};

template <typename UnnamedType> constexpr auto is_valid(const UnnamedType& t) 
{
    // We used auto for the return type: it will be deduced here.
    return container<UnnamedType>();
}

// Check if a type has a serialize method.
auto hasSerialize = is_valid([](auto&& x) -> decltype(x.serialize()) { });

If you are a bit lost at that point, I suggest you take your time and re-read all the previous example. You have all the weapons you need, now fight C++!

Our hasSerialize now takes an argument, we therefore need some changes for our serialize function. We can simply post-pone the return type using auto and use the argument in a decltype as we learn. Which gives us:

// Notice how I simply swapped the return type on the right?
template <class T> auto serialize(T& obj) 
-> typename std::enable_if<decltype(hasSerialize(obj))::value, std::string>::type
{
    return obj.serialize();
}

template <class T> auto serialize(T& obj) 
-> typename std::enable_if<!decltype(hasSerialize(obj))::value, std::string>::type
{
    return to_string(obj);
}

FINALLY!!! We do have a working is_valid and we could use it for serialization! If I were as vicious as my SFINAE tricks, I would let you copy each code pieces to recreate a fully working solution. But today, Halloween's spirit is with me and here is gist. Hey, hey! Don't close this article so fast! If you are true a warrior, you can read the last part!

For the fun:

There are few things I didn't tell you, on purpose. This article would otherwise be twice longer, I fear. I highly suggest you to google a bit more about what I am going to speak about.

  • Firstly, if you wish to have a solution that works with the Boost.Hana static if_, you need to change the return type of our testValidity methods by Hana's equivalents, like the following:

    template <typename Param> constexpr auto test_validity(int /* unused */)
    -> decltype(std::declval<UnnamedType>()(std::declval<Param>()), boost::hana::true_c)
    {
        // If substitution didn't fail, we can return a true_type.
        return boost::hana::true_c;
    }
    
    template <typename Param> constexpr decltype(boost::hana::false_c) test_validity(...)
    {
        // Our sink-hole returns a false_type.
        return boost::hana::false_c;
    }
    

    The static if_ implementation is really interesting, but at least as hard as our is_valid problem solved in this article. I might dedicate another article about it, one day!

  • Did you noticed that we only check one argument at a time? Couldn't we do something like:

    auto test = is_valid([](auto&& a, auto&& b) -> decltype(a.serialize(), b.serialize()) { });
    A a;
    B b;
    
    std::cout << test(a, b) << std::endl;
    

    Actually we can, using some parameter packs. Here is the solution:

    template <typename UnnamedType> struct container
    {
    // Let's put the test in private.
    private:
        // We use std::declval to 'recreate' an object of 'UnnamedType'.
        // We use std::declval to also 'recreate' an object of type 'Param'.
        // We can use both of these recreated objects to test the validity!
        template <typename... Params> constexpr auto test_validity(int /* unused */)
        -> decltype(std::declval<UnnamedType>()(std::declval<Params>()...), std::true_type())
        {
            // If substitution didn't fail, we can return a true_type.
            return std::true_type();
        }
    
        template <typename... Params> constexpr std::false_type test_validity(...)
        {
            // Our sink-hole returns a false_type.
            return std::false_type();
        }
    
    public:
        // A public operator() that accept the argument we wish to test onto the UnnamedType.
        // Notice that the return type is automatic!
        template <typename... Params> constexpr auto operator()(Params&& ...)
        {
            // The argument is forwarded to one of the two overloads.
            // The SFINAE on the 'true_type' will come into play to dispatch.
            return test_validity<Params...>(int());
        }
    };
    
    template <typename UnnamedType> constexpr auto is_valid(UnnamedType&& t) 
    {
        // We used auto for the return type: it will be deduced here.
        return container<UnnamedType>();
    }
    
  • This code is working even if my types are incomplete, for instance a forward declaration, or a normal declaration but with a missing definition. What can I do? Well, you can insert a check on the size of your type either in the SFINAE construction or before calling it: "static_assert( sizeof( T ), "type is incomplete." );".

  • Finally, why are using the notation "&&" for the lambdas parameters? Well, these are called forwarding references. It's a really complex topic, and if you are interested, here is good article about it. You need to use "auto&&" due to the way declval is working in our is_valid implementation!

Notes:

This is my first serious article about C++ on the web and I hope you enjoyed it! I would be glad if you have any suggestions or questions and that you wish to share with me in the commentaries.

Anyway, thanks to Naav and Superboum for rereading this article and theirs suggestions. Few suggestions were also provided by the reddit community or in the commentaries of this post, thanks a lot guys!


Why starting this blog?

Posted on Sat 17 October 2015 in News • Tagged with blogLeave a comment

Whilst I never had an urge to daily browse the various social networks, I have always liked to read technical articles and to follow few blogs. I wanted, since few years, to write my thoughts somewhere in order to clarify them and be able to easily fetch them later on. I procrastinated the idea of running a tech blog for few years, my aversion for the over-engineered web technologies and a lack of motivation restrained me. However, I have more spare time these days and I stumbled upon a simple and elegant Python static blog generator called Pelican. With Pelican I don't have to pull a thousand of npm packages for a sanatizing function neither to use the latest language for hipsters with an eccentric syntax. Static pages also greatly simplify the maintenance and decrease the security burdens of dynamic websites.

If this blog is mostly for my own purpose, I also hope that I could get an external opinion on my posts, my methods or even my skills in the beautiful English language.