Trip report - Meeting C++ 2017

Posted on Thu 16 November 2017 in C++ • Tagged with C++, event, meetingcppLeave a comment

Finally, after years of watching youtube videos on that topic, I made it to my first C++ international conference! Thanks to my current employer King, I went last week to Meeting C++ in Berlin, which is, as far as I know, the biggest C++ event in Europe. I really enjoyed my time there with few hundred other fellow C++ enthusiasts. In this post, I will try to relate how I experienced the event and dress a list of the must-watch talks.

About meeting C++:

The concept:

Held in the magnificient Andels Hotel in Berlin, Meeting C++ offers the possibility to attend keynotes, talks and lightning talks (respectively lasting 2 hours, 50min and 5min) about our favourite language C++ for 3 days (one extra day added for the 6th edition of the event). C++ being multi-paradigm and a general-purpose programming language, the variety of the topics being discussed is pretty wide. It ranges from subject on "(Template) Meta-programming" to a deep dive on "How a C++ debugger work", from begginner friendly talks to hairy discussions on the yet-to-be-standardised white paper on std::expected<T, E>.

As some talks happen simulteanously in different rooms, you cannot physically attend all the talks. Instead, you would usually prepare your own schedule by trying to guess the content of the talks from their summary. It is always a dilemna to choose between a topic that you like with the risk to have nothing new to learn, and a brand-new topic for you, where sleepness may kick-in midway to the presentation. If you are curious and daring, the lightning talks on the last day permit you to randomly discover antyhing about C++ in succint presentations. In any case, you can always catch-up the missed talks by checking the Youtube channel.

Generally, I was not disapointed by the quality of the slides and the speakers. I picked up quite a few new concepts, prepared myself for the future of C++ (C++20) and refreshed myself on some fields I did not touch for a while.

More than just talks:

Where Meeting C++ really shines is in its capacity to gather roughly 600 passionated developers from various backgrounds (university, gaming industry, finance, Sillicon Valley's giants...) in one building and share. Just share anything about C++, about reverse-engineering, about your job, about your country, about the german food in your plate! The C++ community represented at this event is very active, open-minded and willing to help. The catering between the sessions and the dinner parties permit you to meet anyone easily. The even team also oganises some really fun events

In a room full of "world-class developers", it is easy to be intimidated, but you should not hesitate to reach them. They will not nitpick your words nor snob you. For some people, it is a dream to meet an Hollywood Star in the street, for me it was really delightful to have casual conversations with these "legendary" coders from the web.

The chief's suggestions of the day:

Here is a menu of most of the talks I attended. The legend is pretty simple:

  • πŸ’€ : The difficulty of the talk (πŸ’€: Begginer friendly, πŸ’€πŸ’€: Intermediate, πŸ’€πŸ’€πŸ’€: High exposure to C++'s dark corners)
  • β˜… : My interest for the talk (β˜…: Good talk, β˜…β˜…: Tasty talk, β˜…β˜…β˜…: Legendary talk)

I will not spoil all the talks, but simply try to give an overview of what you can expect within them. Note that all the talks are generally of high quality and my appreciation very subjective. I have seen people with very different "favorite talk".

[Keynote] Better Code: Human interface - By Sean Parent - πŸ’€ β˜…β˜…

Sean Parent is a Principal Scientist at Adobe Systems and has been working on the famous software Photoshop for more than 15 years. Sean Parent is a regular and prominent speaker at C++ conferences, one of his recently most famous talk being Better Code: Runtime Polyphormism from the same series of talks (Better Code) as the one he gave during Meeting C++.

Thorough his keynote, Sean was conveing the message that in order to have ergonomic human interfaces, you must design your code to reflects its usage through UI. By following such a principle, one can easily come-up with good namings, semantics and grouping of your UI components.

For instance, Sean was explaining that most of the menu actions in Photoshop somehow mapped some object, their methods, properties and most importantly their relations:

  • The menu action Create New Layer will somehow call the constructor of a class called something like Layer.
  • Likewise the action Delete A Layer would call its destructor.
  • A selection in Photoshop will most likely translate in a container of objects.

As a counter-example he explained that the old version of Gmail used to have a confusing flow when it comes to the most trivial usage of a mail service: creating a mail. A link, which implies navigation, was used instead of button for the "compose message" action.

Gmail failed compose button

Sean put a strong emphasis that relationships are the most difficult part of an architecture to represent. He came up with few examples on how std::stable_partition can be used to solve in an elegant way the gathering and display of items

Overall a very nice talk, but on a very abstract topic, since not much has been explored on that subject yet! This is worth paying attention in game-programming where a good UI is a key-part of the success of a game.

[Talk] Threads and Locks must Go - Rainer Grimm - πŸ’€πŸ’€ β˜…

In this talk Rainer Grimm, a German author of multiple C++ books, brought under the spotlight the concurrency features introduced by the new C++ standard C++17 and the coming one C++20. Here is a short summary of my favourite features:

For C++17 (and concurrency TS):

std::vector<int> v = {...}; // A bit vector... 
std::sort(std::execution::par{}, v.begin(), v.end());
// Due to "par", this **might** execute the sort in parallel using a thread pool of some sort.
std::future<int> foo();

auto f = foo();
f.then([](std::future<int>& f) {
    // Will be called when f is done.
    std::cout << f.get(); // Will therefore not block.
});

Hopefully for C++20:

  • The stackless coroutines as implemented by MSVC and clang. This introduce two keywords co_await and co_yield. Considering the previous example using std::future, it could be rewritten in the following way:
std::future<int> foo();

int x = co_await foo(); // The continuation is "generated" by the compiler using the keyword co_await.
std::cout << x; // Everything after co_await is implicitely part of the continuation.
int x; y; // Global vars

void foo() { // Foo can be called by multiple thread simuteanously.
    // Reads-writes on x and y are now thread-safe and synchronized.
    synchronized {
        ++x; 
        ++y;
    }
}

As explained by Rainer Grimm, we will have the possibility to easily bring concurrency to our C++ codebases without ressorting to the low-level, and tricky to get right, features like thread and locks. While I appreciated the talk, it lacked a bit of novelty as I was already aware of most of the features.

[Talk] Strong types for strong interfaces - Johnathan Boccora - πŸ’€ β˜…β˜…β˜…

A must watch! Even when facing some technical issues, Johnathan is very good speaker and I was quickly captivated by the topic of strong types. Jonathan is also a talented writer with his famous blog fluentcpp (I would really suggest to have a look at it once in a while).

As C++ developers, we heavily rely on the language's type system to express our intentions to other developers, to optimise our code and avoid shooting ourselves into our feet. Yet, we always reuse some types to express very different properties of our system. For instance to describe a person, you would use an int for her/his age and his/her weight. Did it ever come to you that the unit year (for age) should be a very different type than kg (for weight)? The concept of strong type would solve this problem by introducing new int-compatible types:

 // We need to have these empty tag types to create entirely new types and not just weakly-typed aliases.
using kg = strong_type<int, struct KgTag>;
using years = strong_type<int, struct YearsTag>;

void unsafe_create_person(int age, int weight);
void create_person(years age, kg weight); // Explicit interface.

int age = 42;
int weight = 1337;

unsafe_create_person(weight, age); // Oops I inversed the arguments but no compiler error.

create_person(years(age), kg(weight))); // Much less error-prone.

As a bonus, strong types can actually affect positively the performances of your codebase as the compiler can agressively optimise without violating strict-aliasing rules, since the types are now strictly unrelated.

This concept is not new and is already used in std::chrono or Boost.Unit, but it was really refreshing to have an explanation with simple words and good examples! I am now very keen to use this in my personal projects and at work too.

[Talk] How C++ Debuggers Work - Simon Brand (4/5) - πŸ’€πŸ’€ β˜…β˜…

Simon Brand, also known as TartanLlama (a really fancy fictious name for a Scott), presented us how a mixture of calls to ptrace, injection of the int3 opcode, parsing of the DWARF format and perseverance is the base to create a debugger on a x86(_64) architecture with a Unix platform (or Linux platform only if you OS specific calls like process_vm_readv, process_vm_writev).

Unlike some of the other talks, it would be hard to give succinct code examples, but I trully appreciated his presentation! When it comes to low-level APIs for debbuging and reverse-engineering, I have a better understanding of the Windows platform. I think that Simon did an excellent job to help me transfer my knowledge to the Unix world.

If one day I have to tackle the creation of a debugger on Unix, I would certainely come back to this talk or follow his series of blog posts on the same subject. I also think that as programmer, it is always beneficial to have some knowledge on the underlying mechanims of the tools you use (gdb, or lldb in that case). I would, therefore suggest to watch that talk to any C++ enthusiast willing to progress in their art of programming.

[Talk] The Three Little Dots and the Big Bad Lambdas - Joel Falcou - πŸ’€πŸ’€πŸ’€ β˜…β˜…β˜…

I am always excited by watching a talk from Joel Falcou: he is a venerable (template) metra-programmer wizzard with a very didactic approach to explain things (and also we share the same nationality \O/). Once again, I was not disapointed by his session.

With a lot of humour, Joel introduced a new facet to meta-programming in C++. We used to have template meta-programming to manipulate types at compile-time (and with difficulties values), then came constexpr to ease value computation, and recently a Louis Dionne came-up with a powerful combo of these two cadrants with Boost.Hana. Joel statement was that lambdas expressions combined with auto and parameter packs are powerful enough to replace some cases where we would have resort to use template meta-programming or the uglier infamous macros! Joel came to that conclusion after being inspired by the language MetaOCaml.

Let's say that you want to fill a vector with push-back instruction generated at compile-time:

#include <tuple>
#include <iostream>
#include <array>
#include <utility>
#include <vector>

template <class F, std::size_t... I>
void apply_imp(F&& f, std::index_sequence<I...>) {
    (f(I), ...); // C++17 fold expression.
}

template <int N, class F>
void apply(F&& f) {
    apply_imp(f, std::make_index_sequence<N>{});
}

std::vector<int> bob;

auto bind = [](auto& v, auto f) { return [&v, f](auto x){ f(v, x); }; };
auto push_back = bind(bob, [](auto& v, int x) { v.push_back(x * 3); });

apply<3>(push_back);
// Will generate at compile time:
// bob.push_back(0);
// bob.push_back(3);
// bob.push_back(6);

This example is fairly trivial and there would be a high chance that you would reach the same assembly output using a simple for loop. But it is very interesting to notice that lambdas are reusable type-safe units of code that you transport, combine and "instantiate" at any time. Performance-wise, lambdas are pretty incredible according to Joel's measurement on his linear-algebra project. C++17 constexpr lambdas could also help on that topic. One drawback might be the debugging complexity when navigating in nested lambdas. I still need to wrap my head around this new concept and I am eager to rewatch Joel's talk to explore it more!

[Keynote] Its complicated! - Kate Gregory - πŸ’€ β˜…β˜…β˜…

While excellent, Kate's keynote would be very hard to summarise correctly within few paragraphs. It makes you reflect on the difficulties to introduce C++ to newcomers. You would hope that there is a subset of the language that could be easily assimilate by anyone, Kate argues that the reality is sadly more complicated than that. Just have a look at how long are the C++ core guidelines on passing parameters to function calls. One day or another, a begginer must learn on how to pass parameters with good semantics and in an optimised fashio. Well, good luck to her/him! On the other hand, it does not mean that the language could have been designed in a simpler way. What we should strive for instead might be better naming of these concepts: the acronym RAII (Resource Acquisition Is Initialization) is obviously not as straightforward as COW (Copy-on-write). Whether you are a "newbie" or the best "lead over-engineer" of your company, this talk is really worth a look!

[Talk] There Is A New Future - Felix Petriconi - πŸ’€πŸ’€ β˜…β˜…

Felix Petriconi and Sean Parent have been working on a the stlab library for quite some time. stlab takes the best of the various future implementations std::future (C++11), std::future (C++14) or boost::future, and adds a bit of it owns features on top of it. For instance, stlabs supports passing explicit executors to control where async will execute the tasks, and where the continuation associated with then will be executed too. Executors are akin to event-loops (or message-pumps in the .Net world) that will process the tasks.

// This task will be executed on ``an_executor``
auto f = stlab::async(an_executor, [] { return 42; });

// The continuation on another executor.
f.then(another_executor, [](int x) { std::cout << x; });

While executors are present in Boost.Thread, stlabs's channels are unique to this future library. Channels are one of the Go language's favorite toy. It is a neat way to create communication between a sender and a receiver on different executors:

auto [sender, receiver] = channel<int>(receive_executor); // Create the link.

receiver | [](int x) { std::cout << x; }; // Define what should happen at the reception.

// Establish the connections.
receiver.set_ready();

sender(42); // Sent 42 through the channel.
// Receiver will print 42 when executing the task.

I really like some of the features in stlabs, hopefully this could be incorporated into the C++ standard (the executors are down in the pipe of the standardisation process).

[Talk] Introduction to proposed std::expected - Niall Douglas- πŸ’€πŸ’€πŸ’€ β˜…

Are you the kind of person that would rather have errors on the return values rather than using exceptions. Niall has a solution for you: std::expected<T, E>. You can see std::expected<T, E> either as a std::optional<T> with a empty state containing an error for being empty, or as a std::variant<T, E> where you agree that the first alternative is the return value and the second alternative is the potential error. Example:

std::expected<int, std::string> foo(int x) {
    if (x < 0) return std::make_unexpected("x < 0");    
    return 42;
}

auto result = foo(-1);

if (!result) std::cout << result.error().value(); // Prints " x < 0".

std::expected starts to be cumbersome to use when combining or propogating returned results. To palliate this problem, std::expected exposes a Monad interface, with the bind member function coming directly to your mind. If you are a Haskell user, std::expected should remind you of the Maybe Monad. Using bind is still verbose and hopefully we obtain a dedicated keyword try to ease our pain.

[Talk] The most valuable values - Juan Pedro BolΓ­var Puente - πŸ’€πŸ’€ β˜…β˜…

During his presentation, Juan actively promoted value semantic over reference semantic and did so with some analogies from our physical world (from our dear philosopher Platos) and code examples. The talk quickly moved onto immutability and functionnal programming applied to user interfaces. There is a trend in the web sphere to follow a software architectural pattern called flux with a main implementation the redux framework. Arguably, flux is a glorified good old MVC (Model View Controller) architecture with a strong emphasis on the immutability of the model and strict flow on the interactions between the components of MVC. Model, View and Controller also get respectively renamed to Store, View and Dispatcher. An action submitted to the Dispatcher will update the Store with a new Store in a determinstic way, which will imply a redraw of the View.

Juan succeeded to mimic redux in the C++ world using his library immer. To demonstrate the capabilities of his library, Juan recreated an emacs-like editor. The beauty of having an immutable Store is truly expressed in the time-traveling machine that you can create from it: by saving all the states of the Store, you can easily come back to a previous state of your application (similar to undo / redo). You should absolutely watch the video to understand how easy it seems to implement this. On top of that you will have the chance to watch what might be the most audacious ending of a C++ talk I have ever seen.

[Talk] Reactive Equations - AndrΓ© Bergner - πŸ’€πŸ’€πŸ’€ β˜…β˜…β˜…

As a meta-programmer aficionado, this was the most "devilish" talk, and therefore highly thrilling, I attended during the event. It was not the first time I heard from AndrΓ© Bergner, he attended the cppcon in 2015 and I remembered that he presented a nice way to have currying on your function. This time, AndrΓ© focused on reactive equations. If this sounds foreign to you, you might be more familiar with data binding in Qt QML using Javascript expression. AndrΓ©'s reactive equations are similar but with simpler expressions:

// ***** Using a pseudo-language, let's define the equations *****
x : float
y : float
z : float

y = x * 42 // When x is updated, y will be automatically updated.
z = y + x // When y or x is updated, z will be automatically updated.

You may notice that I didn't write any C++ code. By default C++ would not permit to have expressions that update themselves by "magic" if one variable changes. You could write the update logic manually, but with a lot of long equations, this becomes very error prone. Instead AndrΓ© created a DSL (Domain Specific Language), which is equivalent to create a language within C++ itself. To define his DSL, AndrΓ© used expression templates. Expression templates are tricky creatures, which roughly consist in encapsulating C++ expressions into a type at compile-time. This type will retain all the operators / functions (let's call them operations) that you applied in your expression. These operations can be queried at compile-time to generate other expression that you will execute at runtime. In AndrΓ©'s case, the encapsulated operations from his reactive equations would be used to automagically generate the update logic. To facilitate his task, AndrΓ© heavily relied on Boost.Proto. If you are versed in the art of meta-programming, this will certainely be entertaining to you!

[Talk] Free your functions - Klaus Iglberger - πŸ’€ β˜…β˜…β˜…

This was a glorious hymn to our beloved free functions by Klaus Iglberger. Programmers often resort to use member functions and inheritance to provide polyphormism in C++, often overlooking that free functions and overloading would be a smarter choice.

Let's take a situation where you would need to implement a serialise function for a bunch of unrelated types. Would rather use the implementation 1?

struct serialisable {
    virtual ~serialisable();
    virtual std::string serialise();
};

struct A : serialisable {
    std::string serialise() override { /* return something... */ };
};

struct B : serialisable {
    std::string serialise() override { /* return something... */ };
};

Or the solution 2?

struct A {};
struct B {};

std::string serialise(const A& a) { /* return something... */ }
std::string serialise(const B& b) { /* return something... */ }

As Kate explained, it is complicated! If you are looking for runtime polyphormism, then you will certainely use the solution 1. If not, the solution 2 is actually preferable. It has a lot of advantages that Klaus explained for one full hour. My favorite one being that you can extend your polyphormism to types that you do not own. Let's say that you want to serialise std::vector, you can simply write an overload for it:

template <class T>
std::string serialise(const std::vector<T>& v) { /* return something... */ }

In practice, nothing prevent you from mixing a both solutions to your needs. One counter-argument being that free functions have an ugly syntax: v.serialise(); feels more natural than serialize(v);. That issue could have been solve with the unified call syntax proposal by Bjarne Stroustrup and Herb Sutter. Sadly, it was rejected by the C++ committee.

[Talk] Reader-Write Lock versus Mutex - Understanding a Lost Bet - Jeffrey Mendelsohn - πŸ’€πŸ’€πŸ’€ β˜…β˜…

Jeffrey Mendelson from Bloomberg had a bet with a colleague on whether a readers-writer lock would be faster than a mutex to protect the access on a resource that could be written by a single writer, but could have multiple readers simultaneously. The readers-writer lock would follow exactly that behaviour (multiple readers, single writer). The mutex would keep the exclusivity to one writer or one reader only! Jeffrey lost the bet, but that did not hindered him from exploring the reasons behind his lost. It was challenging for me to grasp all the implications on this topic, but here is what I understood:

  • Jeffrey's reader-writer lock was made of atomic variables to keep track of the amount of readers and writers. If the resource was currently written onto, the readers and other writers would wait onto a semaphor to be waken-up later on.
  • If the amount of time spent by the readers or the writers on the resource is fairly long, the readers-writer lock will actually perform better than the mutex as multiple reader can process simultanesouly.
  • On the other hand, if the writing and reading operations are very fast, the atomic operations on the counters will start to be costly comparatively. Atomics tend to have nonnegligible effects on the cache lines of your CPU(s). In this case, loosing the ability to have multiple readers is actually not as dramatic as you would think in comparison of stressing your cache.
  • Jeffrey came up with an hybrid solution that combines both a readers-writer lock and a fallback to a mutex that outperformed the previous solutions.

Once the video of this talk is uploaded, I must have a complete rewatch of it. It always amusing that our intuitions can be entirely wrong when it comes to concurrency and programming.

[Other] The not-so-secret lightning talks

Videos: coming-soon

Before the last Keynote, we had the pleasure to listen to some not-so-unexpected lightning talks. I will not spoil too much of it!

I just want to express my gratitude to Guy Davidson and Sean Parent for bringing diversity in the C++ community under the spotlight. It was more than welcome and I am glad of these initiatives.

Conclusion:

Once again, I was amazed by the C++ community and how a group of dedicated persons can build such nice event. I am already eager to fly to one of the big conference next year: Meeting C++ or cppcon. I would also encourage anyone with a bit of passion for this language or programming in general to give a try to conferences or local groups, you will discover more than you would expect!


C++ Stockholm 0x02

Posted on Mon 20 February 2017 in News • Tagged with blog C++17 C++ StockholmLeave a comment

I had the chance to participate last week in a C++ meetup. This Stockholm meetup group is fairly recent but has a lot of potential! I am always enjoying my time during these sessions, surrounded by the smart and passionated people. In the unlikely chance that you are living around Stockholm, or decided to visit the capital of Scandinavia (and start missing your keyboard), I would highly recommend you to join this group.

For the second edition of our conference C++ Stockholm 0x02, I volunteered for talk about a subject I already adressed in the blog: SFINAE and compile-time introspection. Overall, it was a good exercise for a first talk but I definitely need more training :). I discovered that it is much harder than I expected to explain code in live. Thanks to the meetup team, I have a video of this talk with a very nice editing job done on it. It is always a weird experience to watch yourself on a video and was not so keen to put it here in the first place. But since this video has already leaked among my friends (or foes in that case) and colleagues, I guess I might place it for the posterity! So, here it is:

You can find a pdf version of the slides right here.


An introduction to C++'s variadic templates: a thread-safe multi-type map

Posted on Mon 01 February 2016 in C++ • Tagged with C++11, C++14, variadic templates, meta programmingLeave a comment

Trivia:

One of our favorite motto in our C++ team at work is: you shall use dependency injections instead of singletons! It actually comes with our unit-testing strategy. If the various components of your architecture are too tightly coupled, it becomes a tremendous effort to deeply test small critical chunks of your code. Singletons are that kind of beast that revives itself without your permission and comes from hell to haunt your lovely unit-tests. Our main project being multi-threaded (hence highly bug-prone) and vital for the company, "singleton" became a forbidden word. Yet, our team recently started going down the dark path. Thanks to C++11 and its variadic templates, I carefully crafted a thread-safe multi-type map container that simplified our configuration reloading system and saved us from the dark side of the coder force. If you always wondered what are variadic templates, how C++11's tuples can be implemented, I am going to present these concepts in this post using my container as a cobaye.

Note: for the sake of your sanity and the fact that errare humanum est, this article might not be 100% accurate!

Why would I use a thread-safe multi-type map?

Let me explain our odyssey: we are working on a highly modular and multi-threaded application. One of its core feature is the ability to reload various configuration files or assets used by some components spread accross many threads and a giant hierarchy of objects. The reloading process is automic using Linux's inotify monitoring filesystem events. One thread is dedicated to the reception of filesystem events and must react accordingly by parsing any changes and pushing them to other threads. At first, we used, to pass-by any newly parsed asset, some thread-safe queues or something analog to go channels. Since we did not want to use singletons, we had to pass references to our queues all along our object hierarchy. Sadly, our queue implementation is one to one and supports only one type, none of our config/asset types share the same base-type. For each asset type and each component using this asset, we had to create a new queue and pass-it all along our hierarchy. That is certainely not convenient! What we really wanted was a hybrid class between a std::map and a std::tuple.

We could have used a std::map with Boost.Variant to store our items, using a type like the following "std::map< std::string, std::shared_ptr< Boost.Variant < ConfigType1, ConfigType2>>>". Boost.Variant permits to encapsulate a heterogeneous set of types without common base-type or base-class, which solves one of our point. Another solution would be to encapsulate manually all our configuration classes in the same familly of classes, that is pretty cumbersome. But anyway, std::map does not guaranty any safety if you are writing and reading at the same time on a map slot. Secondly, std::shared_ptr does guaranty a thread-safe destruction of the pointee object (i.e: the reference counter is thread-safe) but nothing for the std::shared_ptr object itself. It means that copying a std::shared_ptr that could potentially be modified from another thread, might lead to an undefined behaviour. Even if we were to encapsulate all these unsafe accesses with mutexes, we are still lacking a nice mechanism to get update notifications for our objects. We do not want to constantly poll the latest version and propagate it through our code. And finally, if that solution were elegant enough, why would I currently write this blog post?

C++11 brings another collection type called std::tuple. It permits to store a set of elements of heterogeneous types. Take a look at this short example:

auto myTuple = std::make_tuple("Foo", 1337, 42);

std::cout << std::get<0>(myTuple) << std::endl; // Access element by index: "Foo"
std::cout << std::get<1>(myTuple) << std::endl; // Access element by index: 1337
std::cout << std::get<2>(myTuple) << std::endl; // Access element by index: 42
std::cout << std::get<const char*>(myTuple) << std::endl; // Access element by type: "Foo"

// compilation error: static_assert failed "tuple_element index out of range"
std::cout << std::get<3>(myTuple) << std::endl;

// compilation error: static_assert failed "type can only occur once in type list"
std::cout << std::get<int>(myTuple) << std::endl;

Tuples are that kind of C++11 jewelry that should decide your old-fashioned boss to upgrade your team's compiler (and his ugly tie). Not only I could store a const char* and two ints without any compiling error, but I could also access them using compile-time mechanisms. In some way, you can see tuples as a compile-time map using indexes or types as keys to reach its elements. You cannot use an index out of bands, it will be catched at compile-time anyway! Sadly, using a type as a key to retrieve an element is only possible if the type is unique in the tuple. At my work, we do have few config objects sharing the same class. Anyway, tuples weren't fitting our needs regarding thread safety and update events. Let's see what we could create using tasty tuples as an inspiration.

Note that some tuples implementations were already available before C++11, notably in boost. C++11 variadic templates are just very handy, as you will see, to construct such a class.

A teaser for my repository class:

To keep your attention for the rest of this post, here is my thread-safe multi-type map in action:

#include <iostream>
#include <memory>
#include <string>

#include "repository.hpp"

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

// Create a type for our repository.
using MyRepository = Repository
    <
        Slot<std::string>, // One slot for std::string.
        Slot<int, Key1>, // Two slots for int.
        Slot<int, Key2> // Must be differentiate using "type keys" (Key1, Key2).
    >;

int main()
{
    MyRepository myRepository;

    myRepository.emplace<std::string>("test"); // Construct the shared_ptr within the repository.
    myRepository.emplace<int, Key1>(1337);
    myRepository.set<int, Key2>(std::make_shared<int>(42)); // Set the shared_ptr manually.

    // Note: I use '*' as get returns a shared_ptr.
    std::cout << *myRepository.get<std::string>() << std::endl; // Print "test".
    std::cout << *myRepository.get<int, Key1>() << std::endl; // Print 1337.
    std::cout << *myRepository.get<int, Key2>() << std::endl; // Print 42.

    std::cout << *myRepository.get<int>() << std::endl;
    //             ^^^ Compilation error: which int shall be selected? Key1 or Key2?

    auto watcher = myRepository.getWatcher<std::string>(); // Create a watcher object to observe changes on std::string.
    std::cout << watcher->hasBeenChanged() << std::endl; // 0: no changes since the watcher creation.

    myRepository.emplace<std::string>("yo"); // Emplace a new value into the std::string slot.
    std::cout << watcher->hasBeenChanged() << std::endl; // 1: the std::string slot has been changed.

    std::cout << *watcher->get() << std::endl; // Poll the value and print "yo".
    std::cout << watcher->hasBeenChanged() << std::endl; // 0: no changes since the last polling.

    return EXIT_SUCCESS;
}

First and foremost, its name repository might not be well-suited for its responsibility. If your native language is the same as shakespeare and come-up with a better term, please feel free to submit it. In our internal usage, config repository sounded great!

I start by describing the slots necessary for my application by creating a new type MyRepository using a type alias. As you can see, I use the type of the slots as a key for accessing elements. But in case of contention, I must use a second key: an "empty type" ; like Key1 and Key2 in this example. If using types as keys seems odd for you, fear not! Here is the most rational explanation I can share with you: we are trying to benefit from our "know-it-all compiler". Your compiler is mainly manipulating types, one can change its flow using these types during the compilation process. Note that these structs are not even complete (no definition), it has no impact for the runtime memory or runtime execution and that's the amazing part of meta-programming. The dispatch of an expression such as "myRepository.get< int, Key1>()" is done during your build-time.

You may also notice that every slot is actually a std::shared_ptr. It enforces a clean ressource management: in a multithreaded application, one must be really careful of the lifetime of heap objects. std::shared_ptr in this case permits me to ensure that even if someone replaces a value in a slot, other components on other threads manipulating the old value won't end up with a dangling pointer/reference bomb in their hands. Another solution would be to use plain value objects, but not only it would require copying big objects in every other components but it would also remove polymorphism.

As for the updates signalisation, you first create a watcher object that establishes a contract between a desired slot to watch and your context. You can thereafter query in thread-safe way weither an update has been made and, if so, poll the latest changes. The watcher object is actually a std::unique_ptr for a special class, it cannot be moved nor copied without your permission and will automagically disable the signalisation contract between the slot and your context, once destroyed. We will dive deeper in this topic in the comming sections.

Within our application, the repository object is encapsulated into a RuntimeContext object. This RuntimeContext object is created explicitely within our main entry point and passed as a reference to a great part of our components. We therefore keep the possibility to test our code easily by setting this RuntimeContext with different implementations. Here is a simplified version of our usage:

// runtimecontext.hpp
#include "repository.hpp"

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

class ConfigType1; // Defined in another file.
class ConfigType2; // Defined in another file.

// Create a type for our repository.
using ConfigRepository = Repository
    <
        Slot<ConfigType1>,
        Slot<ConfigType2, Key1>,
        Slot<ConfigType2, Key2>
    >;

struct RuntimeContext
{
    ILogger* logger;
    // ...
    ConfigRepository configRepository;
};

// Main.cpp

#include "runtimecontext.hpp"

int main()
{
    RuntimeContext runtimeContext;
    // Setup:
    runtimeContext.logger = new StdOutLogger();
    // ...

    // Let's take a reference to the context and change the configuration repository when necessary. 
    startConfigurationMonitorThread(runtimeContext);

    // Let's take a reference and pass it down to all our components in various threads.
    startOurApplicationLogic(runtimeContext);

    return EXIT_SUCCESS;
}

Time for a C++11 implementation:

We can decompose the solution in 3 steps: at first we need to implement a map that accepts multiple types, we then need to work on the thread safety and finish by the watcher mechanism. Let's first fulfill the mission of this post: introducing you to variadic templates to solve the multiple-type problem.

Variadic templates:

You may not have heard of variadic templates in C++11 but I bet that you already used variadic functions like printf in C (maybe in a previous unsafe life). As wikipedia kindly explains "a variadic function is a function of indefinite which accepts a variable number of arguments". In other words, a variadic function has potentially an infinite number of parameters. Likewise, a variadic template has potentially an infinite number of parameters. Let's see how to use them!

Usage for variadic function templates:

Let's say that you wish to create a template that accept an infinite number of class as arguments. You will use the following notation:

template <class... T>

You specify a group of template parameters using the ellipsis notation named T. Note that this ellipsis notation is consistent with C's variadic function notation. This group of parameters, called a parameter-pack, can then be used in your function template or your class template by expanding them. One must use the ellipsis notation again (this time after T) to expand the parameter pack T:

template <class... T> void f(T...)
//              ^ pack T       ^expansion
{
    // Your function content.
}

Now that we have expanded T, what can we do Sir? Well, first you give to your expanded parameter types, a fancy name like t.

template <class... T> void f(T... t)
//                                ^ your fancy t.
{
    // Your function content.
}

If T = T1, T2, then T... t = T1 t1, T2 t2 and t = t1, t2. Brilliant, but is that all? Sure no! You can then expand again t using an "suffix-ellipsis" again:

template <class... T> void f(T... t)
{
    anotherFunction(t...);
    //                ^ t is expanded here! 
}

Finally, you can call this function f as you would with a normal function template:

template <class... T> void f(T... t)
{
    anotherFunction(t...);
}

f(1, "foo", "bar"); // Note: the argument deduction avoids us to use f<int, const char*, const char*>
// f(1, "foo", "bar") calls a generated f(int t1, const char* t2, const char* t3)
// with T1 = int, T2 = const char* and T3 = const char*,
// that itself calls anotherFunction(t1, t2, t3) equivalent to call anotherFunction(1, "foo", "bar");

Actually, the expansion mechanism is creating comma-separated replication of the pattern you apply the ellipsis onto. If you think I am tripping out with template-related wording, here is a much more concret example:

template <class... T> void g(T... t)
{
    anotherFunction(t...);
}

template <class... T> void f(T*... t)
{
    g(static_cast<double>(*t)...);
}

int main()
{
    int a = 2;
    int b = 3;

    f(&a, &b); // Call f(int* t1, int* t2).
    // Do a subcall to g(static_cast<double>(*t1), static_cast<double>(*t2)).

    return EXIT_SUCCESS;
}

I could use the pattern '*' for f parameters and therefore take them as a pointer! In the same manner, I applied the pattern 'static_cast< double>(*) to get the value of each arguments and cast them as doubles before forwarding them to g.

One last example before moving to variadic class templates. One can combine "normal" template parameters with parameter packs and initiate a compile recursion on function templates. Let's take a look at this printing function:

#include <iostream>

template <class HEAD> void print(HEAD head)
{
    std::cout << "Stop: " << head << std::endl;
}

template <class HEAD, class... TAIL> void print(HEAD head, TAIL... tail)
{
    std::cout << "Recurse: " << head << std::endl;
    print(tail...);
}

int main()
{
    print(42, 1337, "foo");

    // Print:
    // Recurse: 42
    // Recurse: 1337
    // Stop: foo

    // Call print<int, int, const char*> (second version of print).
    // The first int (head) is printed and we call print<int, const char*> (second version of print).
    // The second int (head again) is printed and we call print<const char*> (first version of print).
    // We reach recursion stopping condition, only one element left.

    return EXIT_SUCCESS;
}

Variadic templates are very interesting and I wouldn't be able to cover all their features within this post. It roughly feels like functional programming using your compiler, and even some Haskellers might listen to you if you bring that topic during a dinner. For those interested, I would challenge them to write a type-safe version of printf using variadic templates with the help of this reference. After that, you will run and scream of fear at the precense of C's vargs.

"Variadic" inheritance:

Sometimes during my programming sessions, I have a very awkward sensation that my crazy code will never compile and, yet, I finally see "build finished" in my terminal. I am talking about that kind of Frankenstein constructions:

struct A { };

struct B { };

template <class... T>
struct C: public T... // Variadic inheritance
{

};

C<A, B> c;

Yes, we can now create a class inheriting of an infinite number of bases. If you remember my explanation about pattern replications separated by commas, you can imaginge that struct C: public T... will be "transformed" in struct C: public A, public B, public T being the pattern. We start to be able to combine multiple types, each exposing a small amount of methods, to create a flexible concret type. That's one step closer to our multi-type map, and if you are interested in this concept, take a look at mixins.

Instead of inheriting directly from multiple types, couldn't we inherit from some types that encapsulate our types? Absolutely! A traditional map has some slots accessible using keys and these slots contain a value. If you give me base-class you are looking for, I can give you access to the value it contains:

#include <iostream>

struct SlotA
{
    int value;
};

struct SlotB
{
    std::string value;
};

// Note: private inheritance, no one can access directly to the slots other than C itself.
struct Repository: private SlotA, private SlotB
{

    void setSlotA(const int& value)
    {
        // I access the base-class's value
        // Since we have multiple base with a value field, we need to "force" the access to SlotA.
        SlotA::value = value;
    }

    int getSlotA()
    {
        return SlotA::value;
    }

    void setSlotB(const std::string& b)
    {
        SlotB::value = b;
    }

    std::string getSlotB()
    {
        return SlotB::value;
    }
};


int main()
{
    Repository r;

    r.setSlotA(42);
    std::cout << r.getSlotA() << std::endl; // Print: 42.

    r.setSlotB(std::string("toto"));
    std::cout << r.getSlotB() << std::endl; // Print: "toto".

    return EXIT_SUCCESS;
}

This code is not generic at all! We know how to create a generic Slot using a simple template, and we acquired the magic "create varidiac inheritance" skill. If my Repository class inherit from Slot< TypeA> and you call a method template with TypeA as a template argument, I can call the doGet method of the Slot< TypeA> base-class and give you back the value of TypeA in that repository. Let's fix the previous ugly copy-paste code:

#include <iostream>
#include <string>

template <class Type>
class Slot
{
protected:
    Type& doGet() // A nice encapsulation, that will be usefull later on.
    {
        return value_;
    }

    void doSet(const Type& value) // Same encapsulation.
    {
        value_ = value;
    }
private:
    Type value_;
};

template <class... Slots>
class Repository : private Slots... // inherit from our slots...
{
public:
    template <class Type> // Give me a type and,
    Type& get()
    {
        return Slot<Type>::doGet(); // I can select the Base class.
    }

    template <class Type>
    void set(const Type& value)
    {
        Slot<Type>::doSet(value);
    }
};

// Incomplete types used as compile-time keys.
struct Key1;
struct Key2;

// Create a type for our repository.
using MyRepository = Repository
        <
                Slot<int>,       // Let's pick the type of our slots.
                Slot<std::string>
        >;

int main()
{
    MyRepository myRepository;

    myRepository.set<std::string>("toto");
    myRepository.set(42); // Notice the type deduction: we pass an int, so it writes in the int slot.

    std::cout << myRepository.get<int>() << std::endl; // Print: "toto".
    std::cout << myRepository.get<std::string>() << std::endl; // Print: 42.

    return EXIT_SUCCESS;
}

This repository starts to take shape, but we are not yet done! If you try to have two int slots, you will raise a compilation error: "base class 'Slot' specified more than once as a direct base class". We need to add another key-type to our slot class with a default value and we need to modify our repository methods to handle it:

struct DefaultSlotKey; // No needs for a definition

template <class T, class Key = DefaultSlotKey> // The Key type will never be trully used. 
class Slot
{
    // ...
};

template <class... Slots>
class Repository : private Slots...
{
public:
    template <class Type, class Key = DefaultSlotKey> // The default key must be here too.
    Type& get()
    {
        return Slot<Type, Key>::doGet();
    }

    template <class Type, class Key = DefaultSlotKey>
    void set(const Type& value)
    {
        Slot<Type, Key>::doSet(value);
    }
};

struct Key1; // No need for definition.
struct Key2;

// Now you can do:
using MyRepository = Repository
    <
            Slot<int>,       // Let's pick the type of our slots.
            Slot<std::string, Key1>,
            Slot<std::string, Key2>
    >;

Here is a UML representation of this Repository using distinct Keys for the type std::string: A nice UML diagram of my classes

Our repository class is missing an emplace method, right? emplace is taking a variable number of arguments with different types and forward them to create an object within one of our slots. A variable number of arguments and types must remind you something... variadic templates! Let's create this variadic emplace method as well as its equivalent in the Slot class:

// In class Slot:

template <class... Args>
void doEmplace(const Args&... args) // Here the pattern is const  &.
{
    value_ = Type(args...); // copy-operator (might use move semantics).
}

// In class Repository:
template <class Type, class Key = DefaultSlotKey, class... Args>
void emplace(const Args&... args) // Here the pattern is const  &.
{
    Slot<Type, Key>::doEmplace(args...);
}

// Usage:
myRepository.emplace<std::string>(4, 'a'); // Create a std::string "aaaa".

One last improvement for the future users of your repositories! If one morning, badly awake, a coworker of yours is trying to get a type or key that doesn't exist (like myRepository.get< double>();), he might be welcomed by such a message:

/home/jguegant/Coding/ConfigsRepo/main.cpp:36:33: error: call to non-static member function without an object argument
    return Slot<Type, Key>::doGet();
           ~~~~~~~~~~~~~~~~~^~~~~
/home/jguegant/Coding/ConfigsRepo/main.cpp:67:18: note: in instantiation of function template specialization 'Repository<Slot<int, DefaultSlotKey>, Slot<std::__1::basic_string<char>, DefaultSlotKey> >::get<double, DefaultSlotKey>' requested here
    myRepository.get<double>();
                 ^
/home/jguegant/Coding/ConfigsRepo/main.cpp:36:33: error: 'doGet' is a protected member of 'Slot<double, DefaultSlotKey>'
        return Slot<Type, Key>::doGet();
                                ^
/home/jguegant/Coding/ConfigsRepo/main.cpp:10:11: note: declared protected here
    Type& doGet()
          ^
2 errors generated.

This message is very confusing, our class does not inherit from Slot< double, DefaultSlotKey>! And we are talking about a clang output, I wonder what gcc or MSVC could produce... If you do not want to be assinated from your moody colleague with a spoon, here is a nice solution using C++11's static_asserts. Static asserts give you the possibility to generate your own compiler error messages in the same fashion as normal asserts but at compile-time. Using a the trait like std::is_base_of, you can suggest the user of your repository to check twice his type. Let's put this static_assert at the beggining of all the methods of Repository:

static_assert(std::is_base_of<Slot<Type, Key>, Repository<Slots...>>::value, 
          "Please ensure that this type or this key exists in this repository");

We are done for this part (finally...), time to think about multi-threading! If you want to know more about the magic behind std::is_base_of, I would suggest you to read my previous post on SFINAE, it might give you few hints. Here is a gist of what we achieved so far. Did you notice the change on emplace? If you do not understand it, have a look at this explanation on perfect forwarding. Sadly, it would be a way too long topic for this post (trust me on that point!) and has a minor impact on our repository right now.

Let's play safe:

The repository we just succeeded to craft can now be used in a single-thread environment without further investigation. But the initial decision was to make this class manipulable from multiple-threads without any worries considering the safety of our operations. As explained in the beginning of this post, we will not use direct values as we currently do, but instead allocate our objects on the heap and use some shared pointers to strictly control their lifetime. No matter which version (recent or deprecated) of the object a thread is manipulating, it's lifetime will be extended until the last thread using it definitely release it. It also implies that the objects themselves are thread-safe. In the case of read-only objects like configs or assets, it shouldn't be too much a burden. In this gist, you will find a repository version using std::shared_ptrs.

std::shared_ptr is an amazing feature of C++11 when dealing with multi-threading, but has its weakness. Within my code (in the previous gist link) a race condition can occur:

// What if I try to copy value_ at the return point...
std::shared_ptr<Type> doGet() const
{
    return value_;
}

// ... meanwhile another thread is changing value_ to value?
void doSet(const std::shared_ptr<Type> &value)
{
    value_ = value;
}

As specified: "If multiple threads of execution access the same std::shared_ptr object without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur". Note that we are talking about the same shared pointer. Multiple shared pointer copies pointing to the same object are fine, as long as these copies originated from the same shared pointer in first place. Copies are sharing the same control block, where the reference counters (one for shared_ptr and one for weak_ptr) are located, and the specification says "the control block of a shared_ptr is thread-safe: different std::shared_ptr objects can be accessed using mutable operations, such as operator= or reset, simultaneously by multiple threads, even when these instances are copies, and share the same control block internally.".

Depending on the age of your compiler and its standard library, I suggest two solutions:

1) A global mutex:

A straightforward solution relies on a std::mutex that we lock during doGet and doSet execution:

...
    std::shared_ptr<Type> doGet()
    {
        // The lock is enabled until value_ has been copied!
        std::lock_guard<std::mutex> lock(mutex_);
        return value_;
    }

    void doSet(const std::shared_ptr<Type> &value)
    {
        // The lock is enabled until value has been copied into value!
        std::lock_guard<std::mutex> lock(mutex_);
        value_ = value;
    }

private:
    std::mutex mutex_;
...

This solution is ideal if you have a Linux distribution that only ships gcc 4.8.x like mine. While not particularly elegant, it doesn't have a great impact on performances compared to the next solution.

2) Atomic access functions:

Starting from gcc 4.9, one can use atomic access functions to manipulate shared pointers. I dream of a day where a specialisation for std::atomic< std::shared_ptr> exists, but from now, we will resort to use std::atomic_load and std::atomic_exchange:

...
    std::shared_ptr<Type> doGet() const
    {
        return std::atomic_load(&value_);
    }

    void doSet(const std::shared_ptr<Type> &value)
    {
        std::atomic_exchange(&value_, value);
    }

private:
    std::shared_ptr<Type> value_;
...

Atomics are elegants and can often bring a great increase of performances if using lock-free instructions internally. Sadly, in the case of shared_ptrs, atomic_is_lock_free will return you false. By digging in libstdc++ and libc++, you will find some mutexes. gcc seems to use a fixed size "pool" of mutexes attributed to a shared_ptr according to a hash of its pointee address, when dealing with atomic operations. In other words, no rocket-science for atomic shared pointers until now.

Our own watchers:

"...I shall live and die at my post. I am the sword in the darkness. I am the watcher on the walls. I am the shield that guards the realms of men..." -- The Night's Watch oath

We want to be able to seal a bond between one of the slot and a context. By context, I mean the lifetime of an object in a thread, a function or a method. If an update has been made on that slot, we must be signaled in that context and to retrieve the new update. The bond must be destroyed if the context does not exist anymore. It should reminds you the Night's Watch oath ... as well as the RAII idiom: "holding a resource is tied to object lifetime: resource acquisition is done during object creation, by the constructor, while resource deallocation is done during object destruction, by the destructor. If objects are destroyed properly, resource leaks do not occur.". A strong ownership policy can be obtained with the help of a std::unique_ptr and the signalisation can be done using a boolean flag.

We will, therefore, encapsulate a std::atomic_bool into a class Watcher automagically registered to a slot once created, and unregistered once destructed. This Watcher class also takes as a reference the slot in order to query its value as you can see:

template <class Type, class Key>
class Watcher
{
public:
    Watcher(Slot<Type, Key>& slot):
            slot_(slot),
            hasBeenChanged_(false)
    {
    }

    Watcher(const Watcher&) = delete;               // Impossible to copy that class.

    Watcher & operator=(const Watcher&) = delete;   // Impossible to copy that class.

    bool hasBeenChanged() const
    {
        return hasBeenChanged_;
    }

    void triggerChanges()
    {
        hasBeenChanged_ = true;
    }

    auto get() -> decltype(std::declval<Slot<Type, Key>>().doGet())
    {
        hasBeenChanged_ = false; // Note: even if there is an update of the value between this line and the getValue one,
        // we will still have the latest version.
        // Note 2: atomic_bool automatically use a barrier and the two operations can't be inversed.
        return slot_.doGet();
    }

private:
    Slot<Type, Key>& slot_;
    std::atomic_bool hasBeenChanged_;
};

As for the automatic registration, we will add two private methods registerWatcher and unregisterWatcher to our Slot class that add or remove a watcher from an internal list. The list is always protected, when accessed, with a std::mutex and tracks all the current watchers that must be signaled when set is called on that slot.

template <class Type, class Key>
class Slot
{
public:
    using ThisType = Slot<Type, Key>;
    using WatcherType = Watcher<Type, Key>;

...
private:
    void registerWatcher(WatcherType* newWatcher)
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        watchers_.push_back(newWatcher);
    }

    void unregisterWatcher(WatcherType *toBeDelete)
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        watchers_.erase(std::remove(watchers_.begin(), watchers_.end(), toBeDelete), watchers_.end());

        delete toBeDelete; // Now that we removed the watcher from the list, we can proceed to delete it.
    }

    void signal()
    {
        std::lock_guard<std::mutex> l(watchers_mutex_);
        for (auto watcher : watchers_) {
            watcher->triggerChanges(); // Let's raise the hasBeenChanged_ atomic boolean flag. 
        }
    }

private:
    std::vector<WatcherType*> watchers_; // All the registered watchers are in that list.

...
};

You may have notice that we are passing a bare WatcherType pointers. The ownership is actually given to whoever is using that watcher encapsulated within a std::unique_ptr. C++11's unique pointers are designed such as you can pass a custom deleter, or a delete callback so to speak. Hence, we can create a method that get a Watcher for a Slot, and register as the deleter of that Watcher a lambda function designed to call unregisterWatcher. Note that the slot MUST always lives longer than the unique pointer and its associated watcher (it should not be a problem in most cases). Let's finish that Slot class forever and ever:

template <class Type, class Key>
class Slot
{
public:
    using ThisType = Slot<Type, Key>;
    using WatcherType = Watcher<Type, Key>;

    // We use unique_ptr for a strong ownership policy.
    // We use std::function to declare the type of our deleter.
    using WatcherTypePtr = std::unique_ptr<WatcherType, std::function<void(WatcherType*)>> ;

...

public:
    WatcherTypePtr doGetWatcher()
    {
        // Create a unique_ptr and pass a lambda as a deleter.
        // The lambda capture "this" and will call unregisterWatcher.
        WatcherTypePtr watcher(new WatcherType(*this), [this](WatcherType* toBeDelete) {
            this->unregisterWatcher(toBeDelete);});

        registerWatcher(watcher.get());

        return watcher;
    }
...
};

Are we done? Hell no, but we will be really soon. All we need is to expose the possibility to acquire a watcher from the repository itself. In the same manner as set and get, we simply dispatch using the type and the key on one of our slot:

template <class Type, class Key = DefaultSlotKey>
typename Slot<Type, Key>::WatcherTypePtr getWatcher() // typename is used for disambiguate
{
    return Slot<Type, Key>::doGetWatcher();
}

WAIT, don't close that page too fast. If you want to be able to snub everyone, you can replace this ugly typename Slot::WatcherTypePtr with auto and claim that your repository class is C++14 only! Grab the full code of what we build together on gist and enjoy!

Conclusion:

Once again, I hope you enjoyed this post about one of my favourite subject: C++. I might not be the best teacher nor the best author but I wish that you learnt something today! Please, if you any suggestions or questions, feel free to post anything in the commentaries. My broken English being what it is, I kindly accept any help for my written mistakes.

Many thanks to my colleagues that greatly helped me by reviewing my code and for the time together.


An introduction to C++'s SFINAE concept: compile-time introspection of a class member

Posted on Sat 31 October 2015 in C++ • Tagged with C++11, C++14, TMP, meta programmingLeave a comment

Trivia:

As a C++ enthusiast, I usually follow the annual C++ conference cppconf or at least try to keep myself up-to-date with the major events that happen there. One way to catch up, if you can't afford a plane ticket or the ticket, is to follow the youtube channel dedicated to this conference. This year, I was impressed by Louis Dionne talk entitled "C++ Metaprogramming: A Paradigm Shift". One feature called is_valid that can be found in Louis's Boost.Hana library particulary caught my attention. This genious is_valid function heavily rely on an even more "magic" C++ programming technique coined with the term SFINAE discovered at the end of the previous century. If this acronym doesn't speak to you, don't be scared, we are going to dive straight in the subject.

Note: for the sake of your sanity and the fact that errare humanum est, this article might not be 100% accurate!

Introspection in C++?

Before explaining what is SFINAE, let's explore one of its main usage: introspection. As you might be aware, C++ doesn't excel when it comes to examine the type or properties of an object at runtime. The best ability provided by default would be RTTI. Not only RTTI isn't always available, but it also gives you barely more than the current type of the manipulated object. Dynamic languages or those having reflection on the other hand are really convenient in some situations like serialization.

For instance, in Python, using reflection, one can do the following:

class A(object):
    # Simply overrides the 'object.__str__' method.
    def __str__(self):
        return "I am a A"

class B(object):
    # A custom method for my custom objects that I want to serialize.
    def serialize(self):
        return "I am a B"

class C(object):
    def __init__(self):
        # Oups! 'serialize' is not a method. 
        self.serialize = 0

    def __str__(self):
        return "I am a C"

def serialize(obj):
    # Let's check if obj has an attribute called 'serialize'.
    if hasattr(obj, "serialize"):
        # Let's check if this 'serialize' attribute is a method.
        if hasattr(obj.serialize, "__call__"):
            return obj.serialize()

    # Else we call the __str__ method.
    return str(obj)

a = A()
b = B()
c = C()

print(serialize(a)) # output: I am a A.
print(serialize(b)) # output: I am a B.
print(serialize(c)) # output: I am a C.

As you can see, during serialization, it comes pretty handy to be able to check if an object has an attribute and to query the type of this attribute. In our case, it permits us to use the serialize method if available and fall back to the more generic method str otherwise. Powerful, isn't it? Well, we can do it in plain C++!

Here is the C++14 solution mentionned in Boost.Hana documentation, using is_valid:

#include <boost/hana.hpp>
#include <iostream>
#include <string>

using namespace std;
namespace hana = boost::hana;

// Check if a type has a serialize method.
auto hasSerialize = hana::is_valid([](auto&& x) -> decltype(x.serialize()) { });

// Serialize any kind of objects.
template <typename T>
std::string serialize(T const& obj) {
    return hana::if_(hasSerialize(obj), // Serialize is selected if available!
                     [](auto& x) { return x.serialize(); },
                     [](auto& x) { return to_string(x); }
    )(obj);
}

// Type A with only a to_string overload.
struct A {};

std::string to_string(const A&)
{
    return "I am a A!";
}

// Type B with a serialize method.
struct B
{
    std::string serialize() const
    {
        return "I am a B!";
    }
};

// Type C with a "wrong" serialize member (not a method) and a to_string overload.
struct C
{
    std::string serialize;
};

std::string to_string(const C&)
{
    return "I am a C!";
}

int main() {
    A a;
    B b;
    C c;

    std::cout << serialize(a) << std::endl;
    std::cout << serialize(b) << std::endl;
    std::cout << serialize(c) << std::endl;
}

As you can see, it only requires a bit more of boilerplate than Python, but not as much as you would expect from a language as complexe as C++. How does it work? Well if you are too lazy to read the rest, here is the simplest answer I can give you: unlike dynamically typed languages, your compiler has access a lot of static type information once fired. It makes sense that we can constraint your compiler to do a bit of work on these types! The next question that comes to your mind is "How to?". Well, right below we are going to explore the various options we have to enslave our favorite compiler for fun and profit! And we will eventually recreate our own is_valid.

The old-fashioned C++98-way:

Whether your compiler is a dinosaur, your boss refuses to pay for the latest Visual Studio license or you simply love archeology, this chapter will interest you. It's also interesting for the people stuck between C++11 and C++14. The solution in C++98 relies on 3 key concepts: overload resolution, SFINAE and the static behavior of sizeof.

Overload resolution:

A simple function call like "f(obj);"" in C++ activates a mechanism to figure out which f function shoud be called according to the argument obj. If a set of f functions could accept obj as an argument, the compiler must choose the most appropriate function, or in other words resolve the best overload! Here is a good cppreference page explaining the full process: Overload resolution. The rule of thumb in this case is the compiler picks the candidate function whose parameters match the arguments most closely is the one that is called. Nothing is better than a good example:

void f(std::string s); // int can't be convert into a string.
void f(double d); // int can be implicitly convert into a double, so this version could be selected, but...
void f(int i); // ... this version using the type int directly is even more close!

f(1); // Call f(int i);

In C++ you also have some sink-hole functions that accept everything. First, function templates accept any kind of parameter (let's say T). But the true black-hole of your compiler, the devil variable vacuum, the oblivion of the forgotten types are the variadic functions. Yes, exactly like the horrible C printf.

std::string f(...); // Variadic functions are so "untyped" that...
template <typename T> std::string f(const T& t); // ...this templated function got the precedence!

f(1); // Call the templated function version of f.

The fact that function templates are less generic than variadic functions is the first point you must remember!

Note: A templated function can actually be more precise than a normal function. However, in case of a draw, the normal function will have the precedence.

SFINAE:

I am already teasing you with the power for already few paragraphs and here finally comes the explanation of this not so complex acronym. SFINAE stands for Substitution Failure Is Not An Error. In rough terms, a substitution is the mechanism that tries to replace the template parameters with the provided types or values. In some cases, if the substitution leads to an invalid code, the compiler shouldn't throw a massive amount of errors but simply continue to try the other available overloads. The SFINAE concept simply guaranties such a "sane" behavior for a "sane" compiler. For instance:

/*
 The compiler will try this overload since it's less generic than the variadic.
 T will be replace by int which gives us void f(const int& t, int::iterator* b = nullptr);
 int doesn't have an iterator sub-type, but the compiler doesn't throw a bunch of errors.
 It simply tries the next overload. 
*/
template <typename T> void f(const T& t, typename T::iterator* it = nullptr) { }

// The sink-hole.
void f(...) { }

f(1); // Calls void f(...) { }

All the expressions won't lead to a SFINAE. A broad rule would be to say that all the substitutions out of the function/methods body are "safes". For a better list, please take a look at this wiki page. For instance, a wrong substitution within a function body will lead to a horrible C++ template error:

// The compiler will be really unhappy when it will later discover the call to hahahaICrash. 
template <typename T> void f(T t) { t.hahahaICrash(); }
void f(...) { } // The sink-hole wasn't even considered.

f(1);

The operator sizeof:

The sizeof operator is really a nice tool! It permits us to returns the size in bytes of a type or an expression at compilation time. sizeof is really interesting as it accurately evaluates an expression as precisely as if it were compiled. One can for instance do:

typedef char type_test[42];
type_test& f();

// In the following lines f won't even be truly called but we can still access to the size of its return type.
// Thanks to the "fake evaluation" of the sizeof operator.
char arrayTest[sizeof(f())];
std::cout << sizeof(f()) << std::endl; // Output 42.

But wait! If we can manipulate some compile-time integers, couldn't we do some compile-time comparison? The answer is: absolutely yes, my dear reader! Here we are:

typedef char yes; // Size: 1 byte.
typedef yes no[2]; // Size: 2 bytes.

// Two functions using our type with different size.
yes& f1();
no& f2();

std::cout << (sizeof(f1()) == sizeof(f2())) << std::endl; // Output 0.
std::cout << (sizeof(f1()) == sizeof(f1())) << std::endl; // Output 1.

Combining everything:

Now we have all the tools to create a solution to check the existence of a method within a type at compile time. You might even have already figured it out most of it by yourself. So let's create it:

template <class T> struct hasSerialize
{
    // For the compile time comparison.
    typedef char yes[1];
    typedef yes no[2];

    // This helper struct permits us to check that serialize is truly a method.
    // The second argument must be of the type of the first.
    // For instance reallyHas<int, 10> would be substituted by reallyHas<int, int 10> and works!
    // reallyHas<int, &C::serialize> would be substituted by reallyHas<int, int &C::serialize> and fail!
    // Note: It only works with integral constants and pointers (so function pointers work).
    // In our case we check that &C::serialize has the same signature as the first argument!
    // reallyHas<std::string (C::*)(), &C::serialize> should be substituted by 
    // reallyHas<std::string (C::*)(), std::string (C::*)() &C::serialize> and work!
    template <typename U, U u> struct reallyHas;

    // Two overloads for yes: one for the signature of a normal method, one is for the signature of a const method.
    // We accept a pointer to our helper struct, in order to avoid to instantiate a real instance of this type.
    // std::string (C::*)() is function pointer declaration.
    template <typename C> static yes& test(reallyHas<std::string (C::*)(), &C::serialize>* /*unused*/) { }
    template <typename C> static yes& test(reallyHas<std::string (C::*)() const, &C::serialize>* /*unused*/) { }

    // The famous C++ sink-hole.
    // Note that sink-hole must be templated too as we are testing test<T>(0).
    // If the method serialize isn't available, we will end up in this method.
    template <typename> static no& test(...) { /* dark matter */ }

    // The constant used as a return value for the test.
    // The test is actually done here, thanks to the sizeof compile-time evaluation.
    static const bool value = sizeof(test<T>(0)) == sizeof(yes);
};

// Using the struct A, B, C defined in the previous hasSerialize example.
std::cout << hasSerialize<A>::value << std::endl;
std::cout << hasSerialize<B>::value << std::endl;
std::cout << hasSerialize<C>::value << std::endl;

The reallyHas struct is kinda tricky but necessary to ensure that serialize is a method and not a simple member of the type. You can do a lot of test on a type using variants of this solution (test a member, a sub-type...) and I suggest you to google a bit more about SFINAE tricks. Note: if you truly want a pure compile-time constant and avoid some errors on old compilers, you can replace the last value evaluation by: "enum { value = sizeof(test(0)) == sizeof(yes) };".

You might also wonder why it doesn't work with inheritence. Inheritence in C++ and dynamic polymorphism is a concept available at runtime, or in other words, a data that the compiler won't have and can't guess! However, compile time type inspection is much more efficient (0 impact at runtime) and almost as powerful as if it were at runtime. For instance:

// Using the previous A struct and hasSerialize helper.

struct D : A
{
    std::string serialize() const
    {
        return "I am a D!";
    }
};

template <class T> bool testHasSerialize(const T& /*t*/) { return hasSerialize<T>::value; }

D d;
A& a = d; // Here we lost the type of d at compile time.
std::cout << testHasSerialize(d) << std::endl; // Output 1.
std::cout << testHasSerialize(a) << std::endl; // Output 0.

Last but no least, our test cover the main cases but not the tricky ones like a Functor:

struct E
{
    struct Functor
    {
        std::string operator()()
        {
            return "I am a E!";
        }
    };

    Functor serialize;
};

E e;
std::cout << e.serialize() << std::endl; // Succefully call the functor.
std::cout << testHasSerialize(e) << std::endl; // Output 0.

The trade-off for a full coverage would be the readability. As you will see, C++11 shines in that domain!

Time to use our genius idea:

Now you would think that it will be super easy to use our hasSerialize to create a serialize function! Okay let's try it:

template <class T> std::string serialize(const T& obj)
{
    if (hasSerialize<T>::value) {
        return obj.serialize(); // error: no member named 'serialize' in 'A'.
    } else {
        return to_string(obj);
    }
}

A a;
serialize(a);

It might be hard to accept, but the error raised by your compiler is absolutely normal! If you consider the code that you will obtain after substitution and compile-time evaluation:

std::string serialize(const A& obj)
{
    if (0) { // Dead branching, but the compiler will still consider it!
        return obj.serialize(); // error: no member named 'serialize' in 'A'.
    } else {
        return to_string(obj);
    }
}

Your compiler is really a good guy and won't drop any dead-branch, and obj must therefore have both a serialize method and a to_string overload in this case. The solution consists in spliting the serialize function into two different functions: one where we solely use obj.serialize() and one where we use to_string according to obj's type. We come back to an earlier problem that we already solved, how to split according to a type? SFINAE, for sure! At that point we could re-work our hasSerialize function into a serialize function and make it return a std::string instead of compile time boolean. But we won't do it that way! It's cleaner to separate the hasSerialize test from its usage serialize.

We need to find a clever SFINAE solution on the signature of "template <class T> std::string serialize(const T& obj)". I bring you the last piece of the puzzle called enable_if.

template<bool B, class T = void> // Default template version.
struct enable_if {}; // This struct doesn't define "type" and the substitution will fail if you try to access it.

template<class T> // A specialisation used if the expression is true. 
struct enable_if<true, T> { typedef T type; }; // This struct do have a "type" and won't fail on access.

// Usage:
enable_if<true, int>::type t1; // Compiler happy. t's type is int.
enable_if<hasSerialize<B>::value, int>::type t2; // Compiler happy. t's type is int.

enable_if<false, int>::type t3; // Compiler unhappy. no type named 'type' in 'enable_if<false, int>';
enable_if<hasSerialize<A>::value, int>::type t4; // no type named 'type' in 'enable_if<false, int>';

As you can see, we can trigger a substitution failure according to a compile time expression with enable_if. Now we can use this failure on the "template <class T> std::string serialize(const T& obj)" signature to dispatch to the right version. Finally, we have the true solution of our problem:

template <class T> typename enable_if<hasSerialize<T>::value, std::string>::type serialize(const T& obj)
{
    return obj.serialize();
}

template <class T> typename enable_if<!hasSerialize<T>::value, std::string>::type serialize(const T& obj)
{
    return to_string(obj);
}

A a;
B b;
C c;

// The following lines work like a charm!
std::cout << serialize(a) << std::endl;
std::cout << serialize(b) << std::endl;
std::cout << serialize(c) << std::endl;

Two details worth being noted! Firstly we use enable_if on the return type, in order to keep the paramater deduction, otherwise we would have to specify the type explicitely "serialize<A>(a)". Second, even the version using to_string must use the enable_if, otherwise serialize(b) would have two potential overloads available and raise an ambiguity. If you want to check the full code of this C++98 version, here is a gist. Life is much easier in C++11, so let's see the beauty of this new standard!

Note: it's also important to know that this code creates a SFINAE on an expression ("&C::serialize"). Whilst this feature wasn't required by the C++98 standard, it was already in use depending on your compiler. It trully became a safe choice in C++11.

When C++11 came to our help:

After the great century leap year in 2000, people were fairly optimistic about the coming years. Some even decided to design a new standard for the next generation of C++ coders like me! Not only this standard would ease TMP headaches (Template Meta Programming side-effects), but it would be available in the first decade, hence its code-name C++0x. Well, the standard sadly came the next decade (2011 ==> C++11), but it brought a lot of features interesting for the purpose of this article. Let's review them!

decltype, declval, auto & co:

Do you remember that the sizeof operator does a "fake evaluation" of the expression that you pass to it, and return gives you the size of the type of the expression? Well C++11 adds a new operator called decltype. decltype gives you the type of the of the expression it will evaluate. As I am kind, I won't let you google an example and give it to you directly:

B b;
decltype(b.serialize()) test = "test"; // Evaluate b.serialize(), which is typed as std::string.
// Equivalent to std::string test = "test";

declval is an utility that gives you a "fake reference" to an object of a type that couldn't be easily construct. declval is really handy for our SFINAE constructions. cppreference example is really straightforward, so here is a copy:

struct Default {
    int foo() const {return 1;}
};

struct NonDefault {
    NonDefault(const NonDefault&) {}
    int foo() const {return 1;}
};

int main()
{
    decltype(Default().foo()) n1 = 1; // int n1
//  decltype(NonDefault().foo()) n2 = n1; // error: no default constructor
    decltype(std::declval<NonDefault>().foo()) n2 = n1; // int n2
    std::cout << "n2 = " << n2 << '\n';
}

The auto specifier specifies that the type of the variable that is being declared will be automatically deduced. auto is equivalent of var in C#. auto in C++11 has also a less famous but nonetheless usage for function declaration. Here is a good example:

bool f();
auto test = f(); // Famous usage, auto deduced that test is a boolean, hurray!



//                             vvv t wasn't declare at that point, it will be after as a parameter!
template <typename T> decltype(t.serialize()) g(const T& t) {   } // Compilation error

// Less famous usage:
//                    vvv auto delayed the return type specification!
//                    vvv                vvv the return type is specified here and use t!
template <typename T> auto g(const T& t) -> decltype(t.serialize()) {   } // No compilation error.

As you can see, auto permits to use the trailing return type syntax and use decltype coupled with an expression involving one of the function argument. Does it means that we can use it to test the existence of serialize with a SFINAE? Yes Dr. Watson! decltype will shine really soon, you will have to wait for the C++14 for this tricky auto usage (but since it's a C++11 feature, it ends up here).

constexpr:

C++11 also came with a new way to do compile-time computations! The new keyword constexpr is a hint for your compiler, meaning that this expression is constant and could be evaluate directly at compile time. In C++11, constexpr has a lot of rules and only a small subset of VIEs (Very Important Expression) expressions can be used (no loops...)! We still have enough for creating a compile-time factorial function:

constexpr int factorial(int n)
{
    return n <= 1? 1 : (n * factorial(n - 1));
}

int i = factorial(5); // Call to a constexpr function.
// Will be replace by a good compiler by:
// int i = 120;

constexpr increased the usage of std::true_type & std::false_type from the STL. As their name suggest, these types encapsulate a constexpr boolean "true" and a constrexpr boolean "false". Their most important property is that a class or a struct can inherit from them. For instance:

struct testStruct : std::true_type { }; // Inherit from the true type.

constexpr bool testVar = testStruct(); // Generate a compile-time testStruct.
bool test = testStruct::value; // Equivalent to: test = true;
test = testVar; // true_type has a constexpr converter operator, equivalent to: test = true;

Blending time:

First solution:

In cooking, a good recipe requires to mix all the best ingredients in the right proportions. If you don't want to have a spaghetti code dating from 1998 for dinner, let's revisit our C++98 hasSerialize and serialize functions with "fresh" ingredients from 2011. Let's start by removing the rotting reallyHas trick with a tasty decltype and bake a bit of constexpr instead of sizeof. After 15min in the oven (or fighting with a new headache), you will obtain:

template <class T> struct hasSerialize
{
    // We test if the type has serialize using decltype and declval.
    template <typename C> static constexpr decltype(std::declval<C>().serialize(), bool()) test(int /* unused */)
    {
        // We can return values, thanks to constexpr instead of playing with sizeof.
        return true;
    }

    template <typename C> static constexpr bool test(...)
    {
        return false;
    }

    // int is used to give the precedence!
    static constexpr bool value = test<T>(int());
};

You might be a bit puzzled by my usage of decltype. The C++ comma operator "," can create a chain of multiple expressions. In decltype, all the expressions will be evaluated, but only the last expression will be considered for the type. The serialize doesn't need any changes, minus the fact that the enable_if function is now provided in the STL. For your tests, here is a gist.

Second solution:

Another C++11 solution described in Boost.Hanna documentation and using std::true_type and std::false_type, would be this one:

// Primary template, inherit from std::false_type.
// ::value will return false. 
// Note: the second unused template parameter is set to default as std::string!!!
template <typename T, typename = std::string>
struct hasSerialize
        : std::false_type
{

};

// Partial template specialisation, inherit from std::true_type.
// ::value will return true. 
template <typename T>
struct hasSerialize<T, decltype(std::declval<T>().serialize())>
        : std::true_type
{

};

This solution is, in my own opinion, more sneaky! It relies on a not-so-famous-property of default template parameters. But if your soul is already (stack-)corrupted, you may be aware that the default parameters are propagated in the specialisations. So when we use hasSerialize<OurType>::value, the default parameter comes into play and we are actually looking for hasSerialize<OurType, std::string>::value both on the primary template and the specialisation. In the meantime, the substitution and the evaluation of decltype are processed and our specialisation has the signature hasSerialize<OurType, std::string> if OurType has a serialize method that returns a std::string, otherwise the substitution fails. The specialisation has therefore the precedence in the good cases. One will be able to use the std::void_t C++17 helper in these cases. Anyway, here is a gist you can play with!

I told you that this second solution hides a lot of complexity, and we still have a lot of C++11 features unexploited like nullptr, lambda, r-values. No worries, we are going to use some of them in C++14!

The supremacy of C++14:

According to the Gregorian calendar in the upper-right corner of my XFCE environment, we are in 2015! I can turn on the C++14 compilation flag on my favorite compiler safely, isn't it? Well, I can with clang (is MSVC using a maya calendar?). Once again, let's explore the new features, and use them to build something wonderful! We will even recreate an is_valid, like I promised at the beggining of this article.

auto & lambdas:

Return type inference:

Some cool features in C++14 come from the relaxed usage of the auto keyword (the one used for type inference).

Now, auto can be used on the return type of a function or a method. For instance:

auto myFunction() // Automagically figures out that myFunction returns ints.
{
    return int();
}

It works as long as the type is easily "guessable" by the compiler. We are coding in C++ after all, not OCaml!

A feature for functional lovers:

C++11 introduced lambdas. A lambda has the following syntax:

[capture-list](params) -> non-mandatory-return-type { ...body... }

A useful example in our case would be:

auto l1 = [](B& b) { return b.serialize(); }; // Return type figured-out by the return statement.
auto l3 = [](B& b) -> std::string { return b.serialize(); }; // Fixed return type.
auto l2 = [](B& b) -> decltype(b.serialize()) { return b.serialize(); }; // Return type dependant to the B type.

std::cout << l1(b) << std::endl; // Output: I am a B!
std::cout << l2(b) << std::endl; // Output: I am a B!
std::cout << l3(b) << std::endl; // Output: I am a B!

C++14 brings a small change to the lambdas but with a big impact! Lambdas accept auto parameters: the parameter type is deduced according the argument. Lambdas are implemented as an object having an newly created unnamed type, also called closure type. If a lambda has some auto parameters, its "Functor operator" operator() will be simply templated. Let's take a look:

// ***** Simple lambda unamed type *****
auto l4 = [](int a, int b) { return a + b; };
std::cout << l4(4, 5) << std::endl; // Output 9.

// Equivalent to:
struct l4UnamedType
{
    int operator()(int a, int b) const
    {
        return a + b;
    }
};

l4UnamedType l4Equivalent = l4UnamedType();
std::cout << l4Equivalent(4, 5) << std::endl; // Output 9 too.



// ***** auto parameters lambda unnamed type *****

// b's type is automagically deduced!
auto l5 = [](auto& t) -> decltype(t.serialize()) { return t.serialize(); };

std::cout << l5(b) << std::endl; // Output: I am a B!
std::cout << l5(a) << std::endl; // Error: no member named 'serialize' in 'A'.

// Equivalent to:
struct l5UnamedType
{
    template <typename T> auto operator()(T& t) const -> decltype(t.serialize()) // /!\ This signature is nice for a SFINAE!
    {
        return t.serialize();
    }
};

l5UnamedType l5Equivalent = l5UnamedType();

std::cout << l5Equivalent(b) << std::endl; // Output: I am a B!
std::cout << l5Equivalent(a) << std::endl; // Error: no member named 'serialize' in 'A'.

More than the lambda itself, we are interested by the generated unnamed type: its lambda operator() can be used as a SFINAE! And as you can see, writing a lambda is less cumbersome than writing the equivalent type. It should remind you the beggining of my initial solution:

// Check if a type has a serialize method.
auto hasSerialize = hana::is_valid([](auto&& x) -> decltype(x.serialize()) { });

And the good new is that we have everything to recreate is_valid, right now!

The making-of a valid is_valid:

Now that we have a really stylish manner to generate a unnamed types with potential SFINAE properties using lambdas, we need to figure out how to use them! As you can see, hana::is_valid is a function that takes our lambda as a parameter and return a type. We will call the type returned by is_valid the container. The container will be in charge to keep the lambda's unnamed type for a later usage. Let's start by writing the is_valid function and its the containter:

template <typename UnnamedType> struct container
{
    // Remembers UnnamedType.
};

template <typename UnnamedType> constexpr auto is_valid(const UnnamedType& t) 
{
    // We used auto for the return type: it will be deduced here.
    return container<UnnamedType>();
}

auto test = is_valid([](const auto& t) -> decltype(t.serialize()) {})
// Now 'test' remembers the type of the lambda and the signature of its operator()!

The next step consists at extending container with the operator operator() such as we can call it with an argument. This argument type will be tested against the UnnamedType! In order to do a test on the argument type, we can use once again a SFINAE on a reacreated 'UnnamedType' object! It gives us this solution:

template <typename UnnamedType> struct container
{
// Let's put the test in private.
private:
    // We use std::declval to 'recreate' an object of 'UnnamedType'.
    // We use std::declval to also 'recreate' an object of type 'Param'.
    // We can use both of these recreated objects to test the validity!
    template <typename Param> constexpr auto testValidity(int /* unused */)
    -> decltype(std::declval<UnnamedType>()(std::declval<Param>()), std::true_type())
    {
        // If substitution didn't fail, we can return a true_type.
        return std::true_type();
    }

    template <typename Param> constexpr std::false_type testValidity(...)
    {
        // Our sink-hole returns a false_type.
        return std::false_type();
    }

public:
    // A public operator() that accept the argument we wish to test onto the UnnamedType.
    // Notice that the return type is automatic!
    template <typename Param> constexpr auto operator()(const Param& p)
    {
        // The argument is forwarded to one of the two overloads.
        // The SFINAE on the 'true_type' will come into play to dispatch.
        // Once again, we use the int for the precedence.
        return testValidity<Param>(int());
    }
};

template <typename UnnamedType> constexpr auto is_valid(const UnnamedType& t) 
{
    // We used auto for the return type: it will be deduced here.
    return container<UnnamedType>();
}

// Check if a type has a serialize method.
auto hasSerialize = is_valid([](auto&& x) -> decltype(x.serialize()) { });

If you are a bit lost at that point, I suggest you take your time and re-read all the previous example. You have all the weapons you need, now fight C++!

Our hasSerialize now takes an argument, we therefore need some changes for our serialize function. We can simply post-pone the return type using auto and use the argument in a decltype as we learn. Which gives us:

// Notice how I simply swapped the return type on the right?
template <class T> auto serialize(T& obj) 
-> typename std::enable_if<decltype(hasSerialize(obj))::value, std::string>::type
{
    return obj.serialize();
}

template <class T> auto serialize(T& obj) 
-> typename std::enable_if<!decltype(hasSerialize(obj))::value, std::string>::type
{
    return to_string(obj);
}

FINALLY!!! We do have a working is_valid and we could use it for serialization! If I were as vicious as my SFINAE tricks, I would let you copy each code pieces to recreate a fully working solution. But today, Halloween's spirit is with me and here is gist. Hey, hey! Don't close this article so fast! If you are true a warrior, you can read the last part!

For the fun:

There are few things I didn't tell you, on purpose. This article would otherwise be twice longer, I fear. I highly suggest you to google a bit more about what I am going to speak about.

  • Firstly, if you wish to have a solution that works with the Boost.Hana static if_, you need to change the return type of our testValidity methods by Hana's equivalents, like the following:

    template <typename Param> constexpr auto test_validity(int /* unused */)
    -> decltype(std::declval<UnnamedType>()(std::declval<Param>()), boost::hana::true_c)
    {
        // If substitution didn't fail, we can return a true_type.
        return boost::hana::true_c;
    }
    
    template <typename Param> constexpr decltype(boost::hana::false_c) test_validity(...)
    {
        // Our sink-hole returns a false_type.
        return boost::hana::false_c;
    }
    

    The static if_ implementation is really interesting, but at least as hard as our is_valid problem solved in this article. I might dedicate another article about it, one day!

  • Did you noticed that we only check one argument at a time? Couldn't we do something like:

    auto test = is_valid([](auto&& a, auto&& b) -> decltype(a.serialize(), b.serialize()) { });
    A a;
    B b;
    
    std::cout << test(a, b) << std::endl;
    

    Actually we can, using some parameter packs. Here is the solution:

    template <typename UnnamedType> struct container
    {
    // Let's put the test in private.
    private:
        // We use std::declval to 'recreate' an object of 'UnnamedType'.
        // We use std::declval to also 'recreate' an object of type 'Param'.
        // We can use both of these recreated objects to test the validity!
        template <typename... Params> constexpr auto test_validity(int /* unused */)
        -> decltype(std::declval<UnnamedType>()(std::declval<Params>()...), std::true_type())
        {
            // If substitution didn't fail, we can return a true_type.
            return std::true_type();
        }
    
        template <typename... Params> constexpr std::false_type test_validity(...)
        {
            // Our sink-hole returns a false_type.
            return std::false_type();
        }
    
    public:
        // A public operator() that accept the argument we wish to test onto the UnnamedType.
        // Notice that the return type is automatic!
        template <typename... Params> constexpr auto operator()(Params&& ...)
        {
            // The argument is forwarded to one of the two overloads.
            // The SFINAE on the 'true_type' will come into play to dispatch.
            return test_validity<Params...>(int());
        }
    };
    
    template <typename UnnamedType> constexpr auto is_valid(UnnamedType&& t) 
    {
        // We used auto for the return type: it will be deduced here.
        return container<UnnamedType>();
    }
    
  • This code is working even if my types are incomplete, for instance a forward declaration, or a normal declaration but with a missing definition. What can I do? Well, you can insert a check on the size of your type either in the SFINAE construction or before calling it: "static_assert( sizeof( T ), "type is incomplete." );".

  • Finally, why are using the notation "&&" for the lambdas parameters? Well, these are called forwarding references. It's a really complex topic, and if you are interested, here is good article about it. You need to use "auto&&" due to the way declval is working in our is_valid implementation!

Notes:

This is my first serious article about C++ on the web and I hope you enjoyed it! I would be glad if you have any suggestions or questions and that you wish to share with me in the commentaries.

Anyway, thanks to Naav and Superboum for rereading this article and theirs suggestions. Few suggestions were also provided by the reddit community or in the commentaries of this post, thanks a lot guys!


Why starting this blog?

Posted on Sat 17 October 2015 in News • Tagged with blogLeave a comment

Whilst I never had an urge to daily browse the various social networks, I have always liked to read technical articles and to follow few blogs. I wanted, since few years, to write my thoughts somewhere in order to clarify them and be able to easily fetch them later on. I procrastinated the idea of running a tech blog for few years, my aversion for the over-engineered web technologies and a lack of motivation restrained me. However, I have more spare time these days and I stumbled upon a simple and elegant Python static blog generator called Pelican. With Pelican I don't have to pull a thousand of npm packages for a sanatizing function neither to use the latest language for hipsters with an eccentric syntax. Static pages also greatly simplify the maintenance and decrease the security burdens of dynamic websites.

If this blog is mostly for my own purpose, I also hope that I could get an external opinion on my posts, my methods or even my skills in the beautiful English language.