Jean Guegant's Bloghttps://jguegant.github.io/blogs/tech/2020-05-17T17:40:00+02:00Software Engineer - C++, security, game development and random thoughts.Let's unravel the secrets behind C++17's structured bindings2020-05-17T17:40:00+02:002020-05-17T17:40:00+02:00Jean Gueganttag:jguegant.github.io,2020-05-17:/blogs/tech/structured-bindings.html<h2>Trivia:</h2>
<p>I had a good intuition on how structured bindings worked when C++17 came out. The feature is quite intuitive to use and provides great help.
But it is not until recently that I actually read <a href="http://eel.is/c++draft/dcl.struct.bind">the part of the standard</a> that describes how this truly works under the …</p><h2>Trivia:</h2>
<p>I had a good intuition on how structured bindings worked when C++17 came out. The feature is quite intuitive to use and provides great help.
But it is not until recently that I actually read <a href="http://eel.is/c++draft/dcl.struct.bind">the part of the standard</a> that describes how this truly works under the hood. As always, the standard is rather cryptic with all its language lawyer terms and for once <a href="https://en.cppreference.com/w/cpp/language/structured_binding">cppreference</a> is not so succinct either.</p>
<p>For future me, I decided to write a dumbed-down version of what happens here!</p>
<p>Disclaimer: this post reflects my humble understanding of the standard, feel free to reach me if I got anything wrong in there.</p>
<h2>Destructuring the structured bindings:</h2>
<p>Similar to the <a href="https://en.cppreference.com/w/cpp/language/range-for">range-based for loops</a>, this new "structured binding" feature can be seen a syntax-sugar. This means that we can produce a code equivalent to what a structured binding declaration would do.</p>
<p>So let's start with a simple case:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
</code></pre></div>
<p>The way you should interpret this code is the following:</p>
<ul>
<li>I am declaring a new <strong>anonymous</strong> variable that encompass <code>[x, y]</code>.</li>
<li>This new variable will have its type deduced and be a copy of what foo returns since we have <code>auto</code>. </li>
</ul>
<p>In other words, your compiler starts by producing a code very similar to this:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
</code></pre></div>
<p>What this implies is the <code>auto</code> keyword is <strong>applied to the anonymous variable</strong> and <strong>NOT</strong> the identifier <code>x</code> and <code>y</code> as some may think.
The anonymous variable itself is not visible to anyone except the compiler, it is not possible to actually check its type, but you have to trust me here! </p>
<p>Now let's assume that <code>foo</code> is returning a type by <strong>value</strong> - <code>T foo()</code> - and therefore produce a <strong>temporary</strong> variable. This means that we can only use <code>auto</code>, <code>const auto&</code> or <code>auto&&</code> but NOT <code>auto&</code>. <code>auto</code> will do a copy of that temporary, <code>const auto&</code> and <code>auto&&</code> will bind to that temporary and <a href="https://en.cppreference.com/w/cpp/language/reference_initialization#Lifetime_of_a_temporary">prolong its life until the end of the scope</a>. <code>auto&</code> would not work here as l-value references cannot bind to temporary values. <code>auto&</code> will be useful only if your (member) function <code>foo</code> returns a l-value reference - <code>T& foo()</code> - and you want to avoid a copy while having non-const access.</p>
<p>Note that the expression on the right part of the equal operator can be more complicated than just a call to a function like <code>foo</code>.
Anything that could assign <code>a_secret_variable</code> could belong there, like <code>auto&& a_secret_variable = std::pair(2, "bob");</code>.</p>
<p>Now that the compiler produced the anonymous variable, how do we obtain <code>x</code> and <code>y</code> from it? This will depend on the type of expression on the right.
There are three cases: one for array types, one for simple types and one for types that act like std::tuple.
The two first ones are pretty straightforward, while the later one requires a bit more explanations.</p>
<h3>Arrays:</h3>
<p>Let's try to bind to an array:</p>
<div class="highlight"><pre><span></span><code><span class="kt">int</span> <span class="n">an_array</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span><span class="mi">1337</span><span class="p">,</span> <span class="mi">1337</span><span class="p">};</span>
<span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">an_array</span><span class="p">;</span>
<span class="n">x</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="n">y</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">an_array</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o"><<</span> <span class="s">","</span> <span class="o"><<</span> <span class="n">an_array</span><span class="p">[</span><span class="mi">1</span><span class="p">];</span> <span class="c1">// What happen here?</span>
</code></pre></div>
<p>After substitution of <code>[x, y]</code> with our anonymous variable we obtain:</p>
<div class="highlight"><pre><span></span><code><span class="kt">int</span> <span class="n">an_array</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span><span class="mi">1337</span><span class="p">,</span> <span class="mi">1337</span><span class="p">};</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">an_array</span><span class="p">;</span>
<span class="n">a_secret_variable</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="n">a_secret_variable</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">an_array</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o"><<</span> <span class="s">","</span> <span class="o"><<</span> <span class="n">an_array</span><span class="p">[</span><span class="mi">1</span><span class="p">];</span>
</code></pre></div>
<p>As you can see, <code>x</code> and <code>y</code> do not really exist as variables. They don't even have their own storage! Rather, they should be seen as special identifiers ; what the standard call "the names of lvalues that refer to some elements". In this scenario, <code>x</code> is a special identifier that refers to <code>a_secret_variable[0]</code>. And <code>y</code> is a special identifier that refers to <code>a_secret_variable[1]</code>.
Since <code>a_secret_variable</code> is a l-value reference to <code>an_array</code>, mutating <code>a_secret_variable</code> is equivalent to mutating <code>an_array</code>. Therefore, this snippet will print <code>42,42</code>.</p>
<p>Note that the number of special identifiers must match the size of your array! Your compiler will yield an error otherwise.</p>
<h3>Simple types:</h3>
<p>What I call simple types are types whose non-static members are publicly available. Typically a plain old C-like <code>struct</code> would fit well into that category.
In the standard, this case happens only if the type is not already in the category "arrays" or "tuple-like types".</p>
<p>If your type falls into that "simple types" category, the end-result will be very similar to arrays. The difference is that the special identifiers will refer to the non-static members of your type. The special identifiers are referring each members of your type from up to down.</p>
<p>So if we have such code:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">a</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b</span><span class="p">;</span>
<span class="p">};</span>
<span class="n">A</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span><span class="mi">1337</span><span class="p">,</span> <span class="mi">1337</span><span class="p">};</span>
<span class="p">}</span>
<span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span> <span class="o"><<</span> <span class="s">","</span> <span class="o"><<</span> <span class="n">y</span><span class="p">;</span> <span class="c1">// Prints 1337, 1337</span>
</code></pre></div>
<p>Let's observe what such a simple example becomes:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">a</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b</span><span class="p">;</span>
<span class="p">};</span>
<span class="n">A</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span><span class="mi">1337</span><span class="p">,</span> <span class="mi">1337</span><span class="p">};</span>
<span class="p">}</span>
<span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">a_secret_variable</span><span class="p">.</span><span class="n">a</span> <span class="o"><<</span> <span class="s">","</span> <span class="o"><<</span> <span class="n">a_secret_variable</span><span class="p">.</span><span class="n">b</span><span class="p">;</span>
</code></pre></div>
<p>As you can see, <code>a</code> is the first member of <code>A</code>. <code>x</code> being also first in the identifier list will refer to <code>a</code>.
Note that the number of special identifiers must also match the number of members available in your type!</p>
<p>There is a subtle catch here though. What would <code>decltype(x)</code> yield as type (what the standard calls referenced type)? One would assume that this would be strictly the same as <code>decltype(a_secret_variable.a)</code> which is equivalent to <code>int</code>. The standard made a nice plot-twist here and <code>decltype(x)</code> will, in rough terms, have a type equivalent to <code>decltype(a_secret_variable.a)</code> <strong>plus</strong> the <code>const</code> or <code>volatile</code> qualifiers attached to <code>a_secret_variable</code>. In our case <code>a_secret_variable</code> is <code>const</code>, which result in <code>decltype(x)</code> being equivalent to <code>const decltype(a_secret_variable.a)</code> or in a simpler form <code>const int</code>.
C++ is like a box of chocolates... you never know what you're gonna get!</p>
<p>So why did the standard committee went into creating such "special identifiers" and not simply have proper reference variables?
If you do so, you have multiple issues, like:</p>
<ul>
<li>How would you handle bit fields? There is not such a thing as reference to bit fields.</li>
<li>Imagine that you were to capture those variables in a lambda <code>[=](){}</code>. What should happen here? Shall this copy the entire <code>a_secret_variable</code> or just the members of it?</li>
<li>...</li>
</ul>
<p>For these reasons, the standard had to treat those identifiers in a special way.</p>
<h3>Tuple-like types:</h3>
<p>The last kind of types you can bind are tuple-like types. In layman's terms, these types are similar to <a href="https://en.cppreference.com/w/cpp/utility/tuple">std::tuple</a> in the sense that you can access their members/values using <code>std::get</code> or a member function <code>get</code> on them. These types do not give a direct access to their members/values but still want to be part of the cool club of structured bindings. Ultimately, this led the standard to use type traits to describe how to access these members. </p>
<h4>Example with std::tuple:</h4>
<p>Let's start with a simple case that we will decompose in few steps:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">tuple</span> <span class="nf">my_fancy_tuple</span><span class="p">(</span><span class="mi">43</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"fiction"</span><span class="p">));</span>
<span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">my_fancy_tuple</span><span class="p">;</span>
<span class="n">y</span> <span class="o">=</span> <span class="s">"factory"</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span> <span class="o"><<</span> <span class="s">","</span> <span class="o"><<</span> <span class="n">y</span><span class="p">;</span> <span class="c1">// Prints 43,factory.</span>
</code></pre></div>
<p>As always, your compiler will first introduce an anonymous variable:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">tuple</span> <span class="nf">my_fancy_tuple</span><span class="p">(</span><span class="mi">43</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"fiction"</span><span class="p">));</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">my_fancy_tuple</span><span class="p">;</span>
</code></pre></div>
<p>As a sanity check, your compiler will check that your special identifiers like <code>x</code> or <code>y</code> are enough to cover all the values that your tuple-like type holds.
To do so, it will do a "compile-time call" equivalent to <code>std::tuple_size_v<std::remove_reference_t<decltype(a_secret_variable)>></code>. <a href="https://en.cppreference.com/w/cpp/utility/tuple/tuple_size">std::tuple_size</a> is already defined for <code>std::tuple</code> and will return <code>2</code> in our example. Since we have two special identifiers and the size of the tuple is two, no members/values are left behind and the compiler allows us to move to the next step.</p>
<p>To be able to use special identifiers referring to values/members of <code>a_secret_variable</code>, the compiler need to extract those with some calls to <code>std::get</code> or a member function of your type named <code>get</code>. It would be quite slow to repeat this extraction repeatedly, therefore the compiler will introduce a <strong>new set of anonymous variables</strong> to store the result of those calls.
The standard also provides a second customisation point by letting the creator of the tuple-like class decided which type these variables will have.
This customisation point is called <a href="https://en.cppreference.com/w/cpp/utility/tuple/tuple_element">std::tuple_element</a>.
Here is what this will look like in code:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">tuple</span> <span class="nf">my_fancy_tuple</span><span class="p">(</span><span class="mi">43</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"fiction"</span><span class="p">));</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">my_fancy_tuple</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">tuple_element_t</span><span class="o"><</span><span class="mi">0</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">remove_reference_t</span><span class="o"><</span><span class="k">decltype</span><span class="p">(</span><span class="n">a_secret_variable</span><span class="p">)</span><span class="o">>>&</span> <span class="n">secret_x</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">0</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
<span class="c1">// ^ The type of the member with index 0 ^ anonymous ^ Gets the value of member 0.</span>
<span class="n">std</span><span class="o">::</span><span class="n">tuple_element_t</span><span class="o"><</span><span class="mi">1</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">remove_reference_t</span><span class="o"><</span><span class="k">decltype</span><span class="p">(</span><span class="n">a_secret_variable</span><span class="p">)</span><span class="o">>>&</span> <span class="n">secret_y</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">1</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
</code></pre></div>
<p>Given <a href="https://en.cppreference.com/w/cpp/utility/tuple/tuple_element">std::tuple_element</a> specialisation for <code>std::tuple</code>, this results in:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">tuple</span> <span class="nf">my_fancy_tuple</span><span class="p">(</span><span class="mi">43</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"fiction"</span><span class="p">));</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">my_fancy_tuple</span><span class="p">;</span>
<span class="kt">int</span><span class="o">&</span> <span class="n">secret_x</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">0</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&</span> <span class="n">secret_y</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">1</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
</code></pre></div>
<p>Note that the variables are defacto references ; more precisely l-value references in this case. This is to avoid any unnecessary copies of your data.
So what if <code>get</code> had returned anything else than l-value reference? The compiler would adapt itself and generate variables which are r-value references "<code>&&</code>" with the all the benefit you have from those: it extends the lifetime of temporaries and permits mutability of those.</p>
<p>Afterwards, the compiler proceeds by associating to these new variables our special identifiers <code>x</code> and <code>y</code>. Once again, these special identifiers simply refer to <code>secret_x</code> and <code>secret_y</code> but are not proper variables. This leads us to this weird code as an output where <code>y</code> would spawn out of nowhere but pretends to be <code>secret_y</code>:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">tuple</span> <span class="nf">my_fancy_tuple</span><span class="p">(</span><span class="mi">43</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"fiction"</span><span class="p">));</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">a_secret_variable</span> <span class="o">=</span> <span class="n">my_fancy_tuple</span><span class="p">;</span>
<span class="kt">int</span><span class="o">&</span> <span class="n">secret_x</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">0</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&</span> <span class="n">secret_y</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">1</span><span class="o">></span><span class="p">(</span><span class="n">my_fancy_tuple</span><span class="p">);</span>
<span class="n">y</span> <span class="o">=</span> <span class="s">"factory"</span><span class="p">;</span> <span class="c1">// y refers to secret_y and therefore mutate the variable that secret_y is bound to.</span>
</code></pre></div>
<p>To further add to this complicated situation, the reference type of <code>y</code> (the result of <code>decltype(y)</code>) is also receiving its own treatment.
You would expect it to be same as <code>secret_y</code> which is <code>std::string&</code>, but the compiler is assigning the type resulting of <code>std::tuple_element</code> instead.
In our case, <code>std::tuple_element</code> gave for the member at index 1 the type <code>std::string</code>, so <code>decltype(y)</code> will return <code>std::string</code>.
While slightly awkward, this gave us some types for <code>x</code> and <code>y</code> that would be similar to if we were accessing to the real members of the tuple directly! </p>
<h4>Playing with a custom type:</h4>
<p>As you observed, if a type has a specialisation for <code>std::tuple_element</code> and <code>std::tuple_size</code> and has a <code>std::get</code> overload or a <code>get</code> member function, then it can be morphed into some structured bindings. So let's write a very dumb type with two members we want to expose this way:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="n">A</span><span class="p">(</span><span class="kt">int</span> <span class="n">x</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">y</span><span class="p">)</span> <span class="o">:</span> <span class="n">x_</span><span class="p">(</span><span class="n">x</span><span class="p">),</span> <span class="n">y_</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">y</span><span class="p">))</span> <span class="p">{}</span>
<span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">I</span><span class="o">></span>
<span class="c1">// ^ The index of the member you want to get.</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">get</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Dispatch to the right member using if constexpr.</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">I</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">x_</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">y_</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">I</span><span class="o">></span>
<span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="n">get</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span>
<span class="c1">// ^ A const overload.</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">I</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">x_</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">y_</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="kt">int</span> <span class="n">x_</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">y_</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">namespace</span> <span class="nn">std</span> <span class="p">{</span>
<span class="k">template</span><span class="o"><></span>
<span class="k">struct</span> <span class="nc">tuple_size</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">integral_constant</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span><span class="p">,</span> <span class="mi">2</span><span class="o">></span> <span class="p">{};</span>
<span class="c1">// ^ A always has 2 members ^</span>
<span class="k">template</span><span class="o"><></span>
<span class="k">struct</span> <span class="nc">tuple_element</span><span class="o"><</span><span class="mi">0</span><span class="p">,</span> <span class="n">A</span><span class="o">></span> <span class="p">{</span>
<span class="c1">// ^ The member at index 0 has type int.</span>
<span class="k">using</span> <span class="n">type</span> <span class="o">=</span> <span class="kt">int</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">template</span><span class="o"><></span>
<span class="k">struct</span> <span class="nc">tuple_element</span><span class="o"><</span><span class="mi">1</span><span class="p">,</span> <span class="n">A</span><span class="o">></span> <span class="p">{</span>
<span class="c1">// The other one has type std::string.</span>
<span class="k">using</span> <span class="n">type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div>
<p>As you can see, you don't need a diploma in rocket-science to have your types support structured bindings.
A couple of lines gives enough clues to your compiler to be able to generate the needed boilerplate behind the scene.
In that respect, the standard was quite well designed!</p>
<p>There are still two things I would like to focus your attention on.</p>
<h5>Handling const:</h5>
<p>What happen if we create a structured binding with <code>const auto&</code> on our type? Such as:</p>
<div class="highlight"><pre><span></span><code><span class="n">A</span> <span class="n">a</span><span class="p">{</span><span class="mi">1</span><span class="p">,</span> <span class="s">"yo"</span><span class="p">};</span>
<span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">a</span><span class="p">;</span>
</code></pre></div>
<p>If you followed carefully the steps on <code>std::tuple</code>, you would notice that this will call <code>std::tuple_element</code> and <code>std::tuple_size</code> with <code>const A</code> as a parameter.
As you can see in my code, I did not specialise those two type-traits for <code>const A</code>. Yet, this will work as it should!
It turns out that the committee was friendly enough to supply a partial specialisation for any <code>const T</code> out there.
By default, these standard provided specialisations will recursively call your own specialisation with <code>T</code> while adding <code>const</code> where they should.
That's neat!</p>
<h5>A very odd get:</h5>
<p>The last interesting tidbit is the following: what if we decided that our <code>get</code> return <code>x_</code> and <code>y_</code> by value? How would that work?</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">I</span><span class="o">></span>
<span class="k">auto</span> <span class="n">get</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span>
<span class="c1">// ^ by value</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">I</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">x_</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">y_</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">};</span>
<span class="n">A</span> <span class="n">a</span><span class="p">{</span><span class="mi">1</span><span class="p">,</span> <span class="s">"yo"</span><span class="p">};</span>
<span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">a</span><span class="p">;</span>
</code></pre></div>
<p>As I explained earlier, the standard is actually ready for that extremely hairy situation.
Here the results from <code>get</code> would be captured by some r-value references such as:</p>
<div class="highlight"><pre><span></span><code><span class="kt">int</span><span class="o">&&</span> <span class="n">secret_x</span> <span class="o">=</span> <span class="n">secret_a</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="mi">0</span><span class="o">></span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&&</span> <span class="n">secret_y</span> <span class="o">=</span> <span class="n">secret_a</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="mi">1</span><span class="o">></span><span class="p">();</span>
</code></pre></div>
<p>The temporaries resulting from those hidden calls to <code>get</code> would live until the end of their scope.
I am not quite sure if this a genius move from the committee or if this came as by-product of C++'s complexity as I see no point in allowing such weird construction...
With these horrifying thoughts, I let you reach the conclusion of this post!</p>
<h2>Conclusion:</h2>
<p>I promised a dumbed-down explanation of structured bindings but ended up writing substantially more than what the standard says.
If you have to remember anything from this post, here are the key-points you should focus on:</p>
<ul>
<li>The qualifiers (const, auto, &...) are applied to the entire variable behind your structured bindings and not the identifiers.</li>
<li>The rest of the boilerplate behind the scene does mostly what you would expect. The only caveat comes for <code>decltype(an_identifier)</code> where it gets hairy. </li>
<li>You can bring the power of structured bindings to your types quite easily with few type traits. </li>
</ul>Making a STL-compatible hash map from scratch - Part 3 - The wonderful world of iterators and allocators2020-05-01T17:40:00+02:002020-05-01T17:43:00+02:00Jean Gueganttag:jguegant.github.io,2020-05-01:/blogs/tech/dense-hash-map3.html<p>This post is part of a series of posts:</p>
<ul>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">Part 1 - Beating std::unordered_map</a></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">Part 2 - Growth Policies & The Schrodinger std::pair</a></li>
<li><strong>Part 3 - The wonderful world of iterators and allocators (Current)</strong></li>
<li>Part 4 - ... (Coming Soon)</li>
</ul>
<p>In the <a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">previous post</a>, we prepared our data-structure to be able to store our …</p><p>This post is part of a series of posts:</p>
<ul>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">Part 1 - Beating std::unordered_map</a></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">Part 2 - Growth Policies & The Schrodinger std::pair</a></li>
<li><strong>Part 3 - The wonderful world of iterators and allocators (Current)</strong></li>
<li>Part 4 - ... (Coming Soon)</li>
</ul>
<p>In the <a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">previous post</a>, we prepared our data-structure to be able to store our key/value pairs in a performant way: our design permits us to expose, for safety, immutable keys to our users while internally having mutable access for speed. We also had fun with more bit magic to create a special growth policy for our bucket container. You can find a reference implementation <a href="https://github.com/Jiwan/dense_hash_map">right here</a>.</p>
<p>At this point, you are probably eager to start working on the algorithms of our <code>dense_hash_map</code>. Alas, we are not there yet! Two new kind of challengers entered the arena: <strong>allocators</strong> and <strong>iterators</strong>. Like any word finishing by "or" (Terminator, Alligator, Abductor, Debtor, Elevator, Moderator, Emperor...) there is a certain violence or authority coming out of it of these twos. And rightfully so, having those in your project is a good indicator that you will spend nights banging your head against a brick wall.</p>
<p>Let me be your mentor and I will guide you through these tough areas!</p>
<p><center><img width=40% height=40% src="https://jguegant.github.io/blogs/tech/images/terminator.jpg" alt="Terminator"/></center></p>
<h1>Part 3 - The wonderful world of iterators and allocators:</h1>
<h2>Two iterators with two different endeavors:</h2>
<p>If we base ourselves on the <a href="https://en.cppreference.com/w/cpp/container/unordered_map">unordered_map's interface</a>, we should have at least four different sorts of iterator:</p>
<table>
<thead>
<tr>
<th><em>Name</em></th>
<th><em>Description</em></th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>iterator</strong></td>
<td>Iterate through all key/value pairs of the map</td>
</tr>
<tr>
<td><strong>const_iterator</strong></td>
<td>Similar to iterator but yields const references to the pairs</td>
</tr>
<tr>
<td><strong>local_iterator</strong></td>
<td>Iterator through all the key/value pairs in one bucket</td>
</tr>
<tr>
<td><strong>const_local_iterator</strong></td>
<td>Similar to local_iterator but yields const references to the pairs</td>
</tr>
</tbody>
</table>
<p>It becomes clear that we will actually need to create only two types of iterators: <strong>iterator</strong> and <strong>local_iterator</strong>. The other two can be easily derived from the first ones. We will just sprinkle some <code>const</code> where we should.</p>
<h3>The interior of iterator:</h3>
<p>If you wonder which kind of iterator to tackle first, <code>iterator</code> and its "little bro" <code>const_iterator</code> are probably the more interesting ones for our users.
Quite often you will want to iterate through all key/value pairs to perform some operations and this is what <code>iterator</code> is dedicated for.
More precisely, <code>iterator</code> is the type returned by <a href="https://en.cppreference.com/w/cpp/container/unordered_map/begin">begin</a>, <a href="https://en.cppreference.com/w/cpp/container/unordered_map/end">end</a> & Co. which allows you to create a range-based for loop such as:</p>
<div class="highlight"><pre><span></span><code><span class="k">for</span> <span class="p">(</span><span class="k">auto</span><span class="o">&</span> <span class="p">[</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">]</span> <span class="o">:</span> <span class="n">my_map</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">key</span> <span class="o"><<</span> <span class="s">", "</span> <span class="o"><<</span> <span class="n">value</span> <span class="o"><<</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>The modern picasso in me decided to show you what this range-loop would do internally:</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/dense-hash-map-iterator.webp" alt="Iterator"/></center></p>
<p>Yes, it should be as simple as iterating in our <code>nodes_</code> container. No, it won't be as easy as you may think.</p>
<p>To iterate over the <code>nodes_</code> container, we can simply use its own... iterators. Conveniently, <code>nodes_</code>'s iterator type is also following the concept
<a href="https://en.cppreference.com/w/cpp/named_req/ForwardIterator">LegacyForwardIterator</a> which is also needed for our <code>dense_hash_map::iterator</code> type.
Even better, it actually follows the <a href="https://en.cppreference.com/w/cpp/named_req/RandomAccessIterator">LegacyRandomAccessIterator concept</a> which is a powerful subset of the <strong>LegacyForwardIterator</strong> concept.</p>
<p>Sadly, <code>nodes_</code>'s iterator type has <code>node<Key, T>&</code> as its <code>reference</code> type when we need <code>std::pair<const Key, T>&</code> for <code>dense_hash_map::iterator</code>.
What we need is a projection onto the member <code>pair</code> of <code>node<Key, T></code> while iterating over <code>nodes_</code>.</p>
<h4>C++20 in all its splendor:</h4>
<p>In an ideal world, we would have a <strong>C++20</strong> compiler shipped with a fully <strong>C++20</strong> compliant standard library. Within it, we would have the <a href="https://en.cppreference.com/w/cpp/ranges">holly ranges library</a>, which would permit to lazily transform our <code>nodes_</code> into another one:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// ...</span>
<span class="k">class</span> <span class="nc">dense_hash_map</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">auto</span> <span class="n">begin</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">projected_range</span><span class="p">().</span><span class="n">begin</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">end</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">projected_range</span><span class="p">().</span><span class="n">end</span><span class="p">();</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="k">private</span><span class="o">:</span>
<span class="k">auto</span> <span class="n">projected_range</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nodes_</span> <span class="o">|</span> <span class="n">std</span><span class="o">::</span><span class="n">views</span><span class="o">::</span><span class="n">transform</span><span class="p">([](</span><span class="k">auto</span><span class="o">&</span> <span class="n">node</span><span class="p">){</span> <span class="k">return</span> <span class="n">node</span><span class="p">.</span><span class="n">pair</span><span class="p">.</span><span class="n">pair</span><span class="p">();</span> <span class="p">});</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p>The ranges library has its quirks and limitations as some fervent members of the lost C++ society will point out.
But for this kind of scenario, it is of great help. It is vastly superior to the C++17 solution as we will see.
You can easily implement <code>cbegin</code> and <code>cend</code> in a similar way: just make your <code>projected_range</code> function const.
<code>std::views::transform</code> even retains the concept of the ranges/iterators it is applied to, meaning that iterators it output in our case are still <code>LegacyRandomAccessIterator</code>.</p>
<p>In <strong>C++20</strong>, you can easily ensure that your iterator will be compliant using the newly adopted <a href="https://en.cppreference.com/w/cpp/language/constraints">constraints and concepts features</a>. Somewhere in your library, you could forge a <code>static_assert</code> such as:</p>
<div class="highlight"><pre><span></span><code><span class="k">static_assert</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">random_access_iterator</span><span class="o"><</span><span class="n">dense_hash_map_iterator</span><span class="o"><</span><span class="p">...</span><span class="o">></span><span class="p">);</span>
</code></pre></div>
<p>Let's assume that <code>std::random_access_iterator</code> had a <strong>compound requirement</strong> that ensure your iterator has a prefix increment operator as follow:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">It</span><span class="o">></span>
<span class="k">concept</span> <span class="nc">random_access_iterator</span> <span class="o">=</span> <span class="k">requires</span><span class="p">(</span><span class="n">It</span> <span class="n">it</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">{</span> <span class="o">++</span><span class="n">it</span> <span class="p">}</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="n">same_as</span><span class="o"><</span><span class="n">It</span><span class="o">&></span><span class="p">;</span> <span class="c1">// Ensure that we can apply `++` to it and that it would return a reference.</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p>If, in a moment of inadvertence, you were to remove that operator, your compiler would gently remind you about it. Here is how <strong>GCC</strong> puts you in the right track:</p>
<div class="highlight"><pre><span></span><code><span class="n">note</span><span class="o">:</span> <span class="n">constraints</span> <span class="n">not</span> <span class="n">satisfied</span>
<span class="n">required</span> <span class="n">by</span> <span class="n">the</span> <span class="n">constraints</span> <span class="n">of</span> <span class="s1">'template<class It> concept random_access_iterator'</span> <span class="k">in</span> <span class="n">requirements</span> <span class="k">with</span> <span class="s1">'dense_hash_map_iterator it'</span>
<span class="n">note</span><span class="o">:</span> <span class="n">the</span> <span class="n">required</span> <span class="n">expression</span> <span class="s1">'++ it'</span> <span class="k">is</span> <span class="n">invalid</span>
<span class="o">{</span> <span class="o">++</span><span class="n">it</span> <span class="o">}</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="n">same_as</span><span class="o"><</span><span class="n">It</span><span class="o">&>;</span>
</code></pre></div>
<p>C++ concepts are not just for <strong>meta-programming</strong>. It is also an elegant way to test your code with the help of your compiler.
One could also hope that IDEs will in the future provide a convenient way to generate stubs from a concept.
Of course, concepts have their limits when it comes to asserting actual runtime behaviour. Unit-tests are still your best ally for that!</p>
<h4>The inferior C++17 solution:</h4>
<p>As the Lieutenant-Colonel Bear Grylls would say: "the rules of survival never change, whether you're in a desert or in an old C++ project.".
We are left on our own without any ranges at our disposal. We must forge our own iterator type by hand quickly!</p>
<p>While iterators are quite simple to use, writing them can be tedious. One has to scrupulously respect the concept your iterator supports.
So in our case, we must implement all the contrainsts a <a href="https://en.cppreference.com/w/cpp/named_req/RandomAccessIterator">LegacyRandomAccessIterator</a> has.
Which in turn, means implementing all the constraints a <a href="https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator">LegacyBidirectionalIterator</a> has.
Which in turn, means implementing all the constraints a <a href="https://en.cppreference.com/w/cpp/named_req/ForwardIterator">LegacyForwardIterator</a> has.
Which in turn, means implementing... okkkk...ay... you get it. It's a list of constraints that no sane person would remember under normal circumstances.</p>
<p>Another old-school solution to avoid introducing any mistake in your iterator class is to cross-check all the members and free functions related to your class against an iterator of the same concept from a venerable library out there. In our case, we are writing an adaptor to <code>std::vector</code>'s iterator. A good candidate for cross-checks could be <a href="https://github.com/llvm/llvm-project/blob/2f3e86b31818222a0ab87c4114215e86b89c9dfc/libcxx/include/vector#L486">libc++'s std::vector</a> <a href="https://github.com/llvm/llvm-project/blob/f82dba019253ced73ceadfde10e5f150bdb182f3/libcxx/include/iterator">iterator</a>. To do so, you would write unit-tests for all the members function for that iterator then try to apply them onto your own iterator.</p>
<p>/!\ Important note - I would strongly advise NOT TO COPY-PASTE from a library for various reasons:</p>
<ol>
<li>You could easily fall into plagiarism and all the legal issues around it.</li>
<li>Your library probably does not have all the constraints a standard library has (naming, compatibilities...).</li>
<li>You will not learn much out of it.</li>
</ol>
<p>Given that <strong>C++20</strong> is not fully mature in all major compilers, I went for the tedious <strong>C++17</strong> solution. Thus was born <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/dense_hash_map_iterator.hpp#L13">dense_hash_map_iterator</a>.</p>
<p>My iterator class takes <strong>five</strong> templates parameters:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Container</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">isConst</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">projectToConstKey</span><span class="o">></span>
<span class="k">class</span> <span class="nc">dense_hash_map_iterator</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p>The three first template parameters are rather obvious, it handles which <code>Key</code> / <code>T</code> pairs we will deal with and which <code>Container</code> type stores them.
The last two template parameters are here to kill multiple birds with one stone. Our iterator class will both represent <code>iterator</code> and <code>const_iterator</code> by setting the first parameter <code>isConst</code>. It also gives you the choice on which version of <a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">the Schrodinger std::pair</a> we want to project onto with the <code>projectToConstKey</code> parameter.</p>
<p>Afterwards, we can start to define some important <strong>usings</strong> we can re-use within our class: </p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">dense_hash_map_iterator</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">using</span> <span class="n">projected_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span>
<span class="n">std</span><span class="o">::</span><span class="n">conditional_t</span><span class="o"><</span> <span class="c1">// Which version of the Schrodinger pair we want.</span>
<span class="n">projectToConstKey</span><span class="p">,</span>
<span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="c1">// We want the one with an immutable key.</span>
<span class="n">Key</span> <span class="c1">// We want the one with an mutable key that can be move around.</span>
<span class="o">></span><span class="p">,</span>
<span class="n">T</span>
<span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">sub_iterator_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">conditional_t</span><span class="o"><</span> <span class="c1">// Choose the underlying iterator type we want to work on: const or non-const.</span>
<span class="n">isConst</span><span class="p">,</span>
<span class="k">typename</span> <span class="nc">Container</span><span class="o">::</span><span class="n">const_iterator</span><span class="p">,</span>
<span class="k">typename</span> <span class="nc">Container</span><span class="o">::</span><span class="n">iterator</span>
<span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">value_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">conditional_t</span><span class="o"><</span> <span class="c1">// The value type our iterator will return depends on:</span>
<span class="n">isConst</span><span class="p">,</span> <span class="c1">// <== The constness of our iterator `isConst`.</span>
<span class="k">const</span> <span class="n">projected_type</span><span class="p">,</span> <span class="c1">// <== The version of the Schrodinger pair we choose `projected_type`.</span>
<span class="n">projected_type</span>
<span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">reference</span> <span class="o">=</span> <span class="n">value_type</span><span class="o">&</span><span class="p">;</span>
<span class="k">using</span> <span class="n">pointer</span> <span class="o">=</span> <span class="n">value_type</span><span class="o">*</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p>Our usings form, in some way, a matrix of all iterator types we can get from the class template <code>dense_hash_map_iterator</code>. As the output of the matrix is the <code>value_type</code> type we will return and sub-iterator type <code>sub_iterator_type</code> we will work on. Writing the rest of the <code>dense_hash_map_iterator</code> becomes a rather boring task where almost every single call gets forwarded to a <code>sub_iterator_</code> member. Here is a very mundane implementation of the prefix increment operator:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">dense_hash_map_iterator</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="n">dense_hash_map_iterator</span><span class="p">()</span> <span class="k">noexcept</span> <span class="c1">// Our main constructor that takes the sub-iterator we will project from.</span>
<span class="o">:</span> <span class="n">sub_iterator_</span><span class="p">(</span><span class="n">sub_iterator_type</span><span class="p">{})</span> <span class="p">{}</span>
<span class="k">auto</span> <span class="k">operator</span><span class="o">++</span><span class="p">()</span> <span class="k">noexcept</span> <span class="o">-></span> <span class="n">dense_hash_map_iterator</span><span class="o">&</span>
<span class="p">{</span>
<span class="o">++</span><span class="n">sub_iterator_</span><span class="p">;</span> <span class="c1">// Increment our sub-i... zZZzz zzZZZzzzzz</span>
<span class="k">return</span> <span class="o">*</span><span class="k">this</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">sub_iterator_type</span> <span class="n">sub_iterator_</span><span class="p">;</span> <span class="c1">// Our sub iterator member.</span>
<span class="p">};</span>
</code></pre></div>
<p>I can predict that you are already virtually yawning at the idea of implementing the rest of this class. So instead of doing a long and monotonous listing of all these member functions, here are the "highlights" you should look for.</p>
<p><strong> <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/dense_hash_map_iterator.hpp#L52">Some conditional operators</a> </strong></p>
<p>Given that <code>value_type</code>, <code>reference</code> and <code>pointer</code> depend on <code>projectToConstKey</code>, all the members functions (operator*, operator[], operator->) returning one of these types need to adapt their body to <code>projectToConstKey</code>. Our beloved <code>if constexpr</code> is back at it:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="k">operator</span><span class="o">*</span><span class="p">()</span> <span class="k">const</span> <span class="k">noexcept</span> <span class="o">-></span> <span class="n">reference</span> <span class="c1">// As soon as we observe the Schrodinger pair...</span>
<span class="p">{</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">projectToConstKey</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// ... its state quantum state gets resolved. </span>
<span class="k">return</span> <span class="n">sub_iterator_</span><span class="o">-></span><span class="n">pair</span><span class="p">.</span><span class="n">const_key_pair</span><span class="p">();</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">sub_iterator_</span><span class="o">-></span><span class="n">pair</span><span class="p">.</span><span class="n">pair</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>This will correctly dispatch to the correct version of <a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">the Schrodinger std::pair</a> at compile time.
I am really glad that we do not rely on <a href="https://jguegant.github.io/blogs/tech/sfinae-introduction.html">SFINAE</a> for these constructions.</p>
<p><strong> <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/dense_hash_map_iterator.hpp#L47">A conversion constructor</a> </strong></p>
<p>It is very handy to be able to assign an <code>iterator</code> to a <code>const_iterator</code> but not the other way around. The magic recipe behind such mechanisms consists in writing a rather awkward constructor:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="kt">bool</span> <span class="n">DepIsConst</span> <span class="o">=</span> <span class="n">isConst</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o"><</span><span class="n">DepIsConst</span><span class="p">,</span> <span class="kt">int</span><span class="o">></span> <span class="o">=</span> <span class="mi">0</span><span class="o">></span>
<span class="c1">// ^^^ Only if isConst is true....</span>
<span class="n">dense_hash_map_iterator</span><span class="p">(</span><span class="k">const</span> <span class="n">dense_hash_map_iterator</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">Container</span><span class="p">,</span> <span class="nb">false</span><span class="p">,</span> <span class="n">projectToConstKey</span><span class="o">>&</span> <span class="n">other</span><span class="p">)</span> <span class="k">noexcept</span>
<span class="o">:</span> <span class="n">sub_iterator_</span><span class="p">(</span><span class="n">other</span><span class="p">.</span><span class="n">sub_iterator_</span><span class="p">)</span> <span class="c1">// ^^ ... we have a constructor that take non-const iterator.</span>
<span class="p">{}</span>
</code></pre></div>
<p>Once again, the absence of <strong>C++20</strong> can be felt here. We want this constructor to be available only when <code>isConst</code> is <code>true</code>: in other words only a <code>const_iterator</code> has this extra constructor. In <strong>C++20</strong>, a well-placed <a href="https://en.cppreference.com/w/cpp/language/constraints#Requires_clauses">requires clause</a> would conditionally enable that constructor. But in <strong>C++17</strong> we have to resort to an disgusting <strong>SFINAE</strong> trick using <a href="https://en.cppreference.com/w/cpp/types/enable_if">std::enable_if_t</a>. To make the matter uglier, <a href="https://stackoverflow.com/questions/14603163/how-to-use-sfinae-for-selecting-constructors">the complicated rules of template substitution</a> forces us to have the somewhat useless default argument <code>DepIsConst</code> instead of using <code>isConst</code> directly.</p>
<p><strong> <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/dense_hash_map_iterator.hpp#L133">Some external operators</a> </strong></p>
<p>If you want to benefit from your conversion constructor within all your operators of arity 2 (operator==, operator<...), you must be careful on how to craft those. You have different options here: members, non-members, friends, non-friends, template or not template... I find <a href="https://web.mst.edu/~nmjxv3/articles/templates.html">Natasha Jarus explanations on the subject</a> pretty good.</p>
<p>I opted for the option <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/dense_hash_map_iterator.hpp#L126">"give access to a const reference of my sub-iterator to everyone"</a>, including my operators defined as free functions. It avoids a creating a cluster-fudge of forward declarations and <code>friend</code>s at the price of exposing my private parts. No fame, no shame as they say!</p>
<h5>A quick note for some detractors:</h5>
<p>Our beautiful <strong>C++20</strong> solution expressed in few lines, became 207 lines of pure... iterator chaos. Certainly, ranges, concepts or coroutines can do more harm than good <a href="https://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/">under some circumstances</a>. Typically, the next iterator we will work on would not be a good fit for ranges. But entirely discarding their usage due to some limitations is not a smart move either. They do bring a lot of value as clearly shown with our <code>dense_hash_map_iterator</code>!</p>
<h3>local iterator - a forward iterator without glamor:</h3>
<p><code>local iterator</code> is the crippled little cousin of <code>iterator</code>. To start with, its name badly represents what it does: what sort of locality is this about?
It cannot be easily expressed using range views due to its access pattern. And to finish, it is a mere <a href="https://en.cppreference.com/w/cpp/named_req/ForwardIterator">LegacyForwardIterator</a> and can hardly be more than that.</p>
<p>What this ill-named iterator gives you is an access to a specific bucket, i.e jumping through all the pairs that have keys whose hash collide.
Here is what an iteration in a bucket of size two would look like:</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/dense-hash-map-local-iterator.webp" alt="Local iterator"/></center></p>
<p>Here <strong>Key1</strong> and <strong>Key2</strong> hashes collide, so our iterator started on bucket <strong>1</strong> yields both of these pairs.</p>
<p>To reflect the true purpose of this iterator, I named it <code>bucket_iterator</code>.
Internally, our <code>bucket_iterator</code> can be used in conjunction with some of the standard <a href="https://en.cppreference.com/w/cpp/algorithm">algorithms</a>.
For instance, we can apply a <a href="https://en.cppreference.com/w/cpp/algorithm/find">std::find_if</a> to quickly pin-point a pair with a given <strong>key</strong> if we already know this <strong>key</strong> belongs to a specific <strong>bucket</strong>. Externally, I am not quite sure who uses this local/bucket iterator. My wild guess is that sometimes you want, as a user, to fine-tune your hashes or the <a href="https://en.cppreference.com/w/cpp/container/unordered_map/load_factor">load_factor</a> of your hash map. This <strong>local iterator</strong> gives you the ability to debug your hash-map without too much hassle.
Whether this was worth a standardisation or not, I am not exactly sure. You shouldn't go against the sacred standard, so a local iterator in your hash map you should have. </p>
<p>The class <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/bucket_iterator.hpp#L13">bucket_iterator</a> ends-up being very similar to <code>dense_hash_map_iterator</code>. In fact, it takes exactly the same template parameters for the same purpose. It is also a lot smaller since it is only a <strong>LegacyForwardIterator</strong>. It mainly differs in its <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/bucket_iterator.hpp#L47">increment operator</a> and <a href="https://github.com/Jiwan/dense_hash_map/blob/d80d3da01d9981154e78ea85b3135b4a66a150a3/include/jg/details/bucket_iterator.hpp#L35">dereference operator</a> since we are jumping around rather than doing a linear scan:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">bucket_iterator</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">auto</span> <span class="k">operator</span><span class="o">++</span><span class="p">()</span> <span class="k">noexcept</span> <span class="o">-></span> <span class="n">bucket_iterator</span><span class="o">&</span>
<span class="p">{</span>
<span class="n">current_node_index_</span> <span class="o">=</span> <span class="p">(</span><span class="o">*</span><span class="n">nodes_container</span><span class="p">)[</span><span class="n">current_node_index_</span><span class="p">].</span><span class="n">next</span><span class="p">;</span>
<span class="c1">// ^^ ++ == moving to the next node in the linked-list. </span>
<span class="k">return</span> <span class="o">*</span><span class="k">this</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="k">operator</span><span class="o">*</span><span class="p">()</span> <span class="k">const</span> <span class="k">noexcept</span> <span class="o">-></span> <span class="n">reference</span>
<span class="p">{</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">projectToConstKey</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Still using the if constexpr trick to get the right Schrodinger pair.</span>
<span class="k">return</span> <span class="p">(</span><span class="o">*</span><span class="n">nodes_container</span><span class="p">)[</span><span class="n">current_node_index_</span><span class="p">].</span><span class="n">pair</span><span class="p">.</span><span class="n">const_key_pair</span><span class="p">();</span>
<span class="c1">// ^^^ Dereferencing means looking at the node at the current index.</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">(</span><span class="o">*</span><span class="n">nodes_container</span><span class="p">)[</span><span class="n">current_node_index_</span><span class="p">].</span><span class="n">pair</span><span class="p">.</span><span class="n">pair</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">nodes_container_type</span><span class="o">*</span> <span class="n">nodes_container</span><span class="p">;</span> <span class="c1">// The container of all nodes.</span>
<span class="n">node_index_type</span> <span class="n">current_node_index_</span> <span class="o">=</span> <span class="n">node_end_index</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span>
<span class="c1">// ^^ The index of the current node we are on. ^^ By default we are pointing to "nowhere", the end node. </span>
<span class="p">};</span>
</code></pre></div>
<p>As you can see, this is nothing more than a classic iteration over a list. But instead of using a "next pointer", we have a <strong>next index</strong>.
We cannot produce a bidirectional iterator as we would need a <strong>previous index</strong>, neither we can have random access due to the dereference step.</p>
<p>The last part of the puzzle for our iterators is a conversion function. After doing a <code>std::find_if</code> on a <code>bucket_iterator</code> it can be really convenient to send as a result a more useful <code>iterator</code> to our users. In a simplified form of this function looks like this:</p>
<div class="highlight"><pre><span></span><code> <span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Container</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">isConst</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">projectToConstKey</span><span class="o">></span>
<span class="n">uto</span> <span class="n">bucket_iterator_to_iterator</span><span class="p">(</span>
<span class="k">const</span> <span class="n">bucket_iterator</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">Container</span><span class="p">,</span> <span class="n">isConst</span><span class="p">,</span> <span class="n">projectToConstKey</span><span class="o">>&</span> <span class="n">bucket_it</span><span class="p">,</span>
<span class="n">node_container_type</span><span class="o">&</span> <span class="n">nodes</span>
<span class="p">)</span> <span class="o">-></span> <span class="n">dense_hash_map_iterator</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">Container</span><span class="p">,</span> <span class="n">isConst</span><span class="p">,</span> <span class="n">projectToConstKey</span><span class="o">></span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">bucket_it</span><span class="p">.</span><span class="n">current_node_index</span><span class="p">()</span> <span class="o">==</span> <span class="n">node_end_index</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span><span class="n">nodes</span><span class="p">.</span><span class="n">end</span><span class="p">()};</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span><span class="n">std</span><span class="o">::</span><span class="n">next</span><span class="p">(</span><span class="n">nodes</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">bucket_it</span><span class="p">.</span><span class="n">current_node_index</span><span class="p">())};</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>There are two cases:</p>
<ul>
<li>If our <code>bucket_iterator</code> is at the end of the linked-list, it means that it points to nowhere. Therefore, we return a <code>dense_hash_map_iterator</code> also pointing at the end.</li>
<li>Otherwise, we grab the current index our <code>bucket_iterator</code>. We then extract the begin iterator of our container of nodes and moving until that index. We can then craft a <code>dense_hash_map_iterator</code> out of it. Since our container's iterator is <strong>random access</strong> this conversion has very little cost.</li>
</ul>
<p>Enough with <strong>iterators</strong> and let's move onto <strong>allocators</strong>!</p>
<h2>A special constructor for allocators:</h2>
<p>In the C++ lore, we have other "or" contenders when it comes to annoyance: <strong>allocators</strong>.
At this point I am assuming that you all know what an allocator does: it allocates memory for the objects stored inside a container.
But C++ being C++, it becomes a bit more tricky when you have <strong>containers of containers</strong>.</p>
<p><center><img width=40% height=40% src="https://jguegant.github.io/blogs/tech/images/gladiator.jpg" alt="Gladiator"/></center></p>
<h3>Nested containers and their allocator:</h3>
<p>Let's try to play around with a container that would be very similar to our <code>dense_hash_map::nodes_</code> (a vector of pairs), and see how it reacts to custom allocators:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">debug_pmr_resource</span> <span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">memory_resource</span> <span class="c1">// pmr memory resource == allocator on steroids.</span>
<span class="p">{</span>
<span class="k">auto</span> <span class="n">do_allocate</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">bytes</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">alignment</span><span class="p">)</span> <span class="o">-></span> <span class="kt">void</span><span class="o">*</span> <span class="k">override</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Allocated: "</span> <span class="o"><<</span> <span class="n">bytes</span> <span class="o"><<</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">;</span> <span class="c1">// We will print a message everytime something has been allocated.</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">get_default_resource</span><span class="p">()</span><span class="o">-></span><span class="n">allocate</span><span class="p">(</span><span class="n">bytes</span><span class="p">,</span> <span class="n">alignment</span><span class="p">);</span> <span class="c1">// Forward to the default allocator of your app.</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">do_deallocate</span><span class="p">(</span><span class="kt">void</span><span class="o">*</span> <span class="n">p</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">bytes</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">alignment</span><span class="p">)</span> <span class="k">override</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">get_default_resource</span><span class="p">()</span><span class="o">-></span><span class="n">deallocate</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">bytes</span><span class="p">,</span> <span class="n">alignment</span><span class="p">);</span> <span class="c1">// Forward to the default allocator of your app.</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">do_is_equal</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">memory_resource</span><span class="o">&</span><span class="p">)</span> <span class="k">const</span> <span class="k">noexcept</span> <span class="o">-></span> <span class="kt">bool</span> <span class="k">override</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nb">true</span><span class="p">;</span> <span class="c1">// Our resource has no state, all instances are the same.</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="n">debug_pmr_resource</span> <span class="n">my_resource</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">>></span> <span class="n">nodes</span><span class="p">{</span><span class="o">&</span><span class="n">my_resource</span><span class="p">};</span> <span class="c1">// We feed our pmr resource to a pmr vector.</span>
<span class="k">auto</span> <span class="n">some_long_string</span> <span class="o">=</span> <span class="s">"WinnieLoursonEstChauveCommeUneSouris"</span><span class="p">;</span> <span class="c1">// String long enough to disable small string optimization.</span>
<span class="n">test</span><span class="p">.</span><span class="n">emplace_back</span><span class="p">(</span><span class="n">some_long_string</span><span class="p">,</span> <span class="n">some_long_string</span><span class="p">);</span> <span class="c1">// Should construct a pair of two strings.</span>
<span class="c1">// Prints on godbolt: </span>
<span class="c1">// Allocated: 64</span>
</code></pre></div>
<p>Before we dive a bit more into this code snippet, all standard containers with a prefix <code>pmr</code> are just the same usual containers with a predefined <a href="https://en.cppreference.com/w/cpp/memory/polymorphic_allocator">polymorphic allocator</a>. This new allocator saves you from the hassle of writing an allocator the old fashion way. All you need to do is to write a resource with three member functions as shown here. Polymorphic allocators are worth a longer post that I will never write. In the meantime, I suggest you to use your google-fu to find some nice <a href="https://blog.feabhas.com/2019/03/thanks-for-the-memory-allocator/">articles</a> or videos about it.</p>
<p>Back to our snippet... Right here we have an external container <code>std::pmr::vector</code> which takes our resource/allocator and then we construct two strings (some internal containers) in it. How many allocations are we going to see from <code>debug_pmr_resource</code>'s point of view? The answer is <a href="https://godbolt.org/z/zPboQb">one and only one</a>. The vector's buffer will be allocated through <code>debug_pmr_resource</code> but not the buffers of our strings. It is unfortunate to be in such a situation. As a user of some custom allocators, you really want all related objects to be stored in the same pool of memory, even more when this objects are nested structures.</p>
<p>Does this means that you need to make both of these strings "pmr" too and feed them with the <code>debug_pmr_resource</code> at construction? Well, yes and no.
Changing <code>std::string</code> to <code>std::pmr::string</code> is necessary. <code>std::allocator</code> (std::string) and <code>std::polymorphic_allocator</code> (std::pmr::string) are not the same type, there is no C++ world where both of those could be compatible. But the feeding of <code>my_resource</code> is not necessary. There is a mechanism already in place from the standard that mandates that our external container <code>nodes</code> would forward its allocator to its inner containers (the two strings) if the allocator type they use is the same. We can easily <a href="https://godbolt.org/z/Sik7S8">check that</a>:</p>
<div class="highlight"><pre><span></span><code><span class="n">debug_pmr_resource</span> <span class="n">my_resource</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">string</span><span class="o">>></span> <span class="n">nodes</span><span class="p">{</span><span class="o">&</span><span class="n">my_resource</span><span class="p">};</span> <span class="c1">// Note our string are also prefixed by pmr now.</span>
<span class="k">auto</span> <span class="n">some_long_string</span> <span class="o">=</span> <span class="s">"UnPangolinVautMieuxQueRien"</span><span class="p">;</span>
<span class="n">test</span><span class="p">.</span><span class="n">emplace_back</span><span class="p">(</span><span class="n">some_long_string</span><span class="p">,</span> <span class="n">some_long_string</span><span class="p">);</span>
<span class="c1">// Prints on godbolt: </span>
<span class="c1">// Allocated: 80</span>
<span class="c1">// Allocated: 27</span>
<span class="c1">// Allocated: 27</span>
</code></pre></div>
<p>Hurray, we see two more allocations going through the resource! With a size of <code>27</code>, it must really be some buffers storing <code>UnPangolinVautMieuxQueRien</code> plus <code>\0</code>. The allocator forwarding is happening!</p>
<p>The next step for us is to be sure that we can achieve the same success not only with <code>std::pair<std::pmr::string, std::pmr::string></code> but also with our <code>node</code> type we defined in the previous post: the type that store both a Schrodinger <code>std::pair</code> and a <code>next</code> index.</p>
<div class="highlight"><pre><span></span><code><span class="n">debug_pmr_resource</span> <span class="n">my_resource</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">node</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">pmr</span><span class="o">::</span><span class="n">string</span><span class="o">>></span> <span class="n">nodes</span><span class="p">{</span><span class="o">&</span><span class="n">my_resource</span><span class="p">};</span> <span class="c1">// Using our node type.</span>
<span class="k">auto</span> <span class="n">some_long_string</span> <span class="o">=</span> <span class="s">"FreedomFriesAreTooGreasy"</span><span class="p">;</span>
<span class="n">test</span><span class="p">.</span><span class="n">emplace_back</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">some_long_string</span><span class="p">,</span> <span class="n">some_long_string</span><span class="p">);</span>
<span class="c1">// ^ index</span>
<span class="c1">// Would print: </span>
<span class="c1">// Allocated: xx</span>
</code></pre></div>
<p>Bjarne damn it! We have lost the allocator forwarding again! That's unnacceptable for our <code>dense_hash_map</code> internals.
Given that only difference is <code>std::pair</code> and <code>node</code>, should we start to investigate what makes <code>std::pair</code> so special? </p>
<h3>To be a good investigator:</h3>
<p>"If you stare into the C++ standard, the C++ standard stares back at you." - Nietzsche </p>
<p>If you have a look at the pages from the <a href="http://eel.is/c++draft/pairs">standard</a> or <a href="https://en.cppreference.com/w/cpp/utility/pair">cppreference</a> about <code>std::pair</code> you will not find anything useful to us. There are no mentions of allocators in its <a href="https://en.cppreference.com/w/cpp/utility/pair/pair">constructors</a>. How did that even work?</p>
<p>I am not a sadist, so I will help you a bit. The response to your answer is in <a href="https://en.cppreference.com/w/cpp/memory/uses_allocator">std::uses_allocator</a> in <strong>C++17</strong>. This type-trait is used when constructing objects within your allocators (more precisely in <a href="https://en.cppreference.com/w/cpp/memory/make_obj_using_allocator">std::make_obj_using_allocator</a> in <strong>C++20</strong>). It let you check if the object you are creating using your allocator takes an allocator <strong>itself</strong>! Here comes a shortened explanation.</p>
<p>There are two ways std::uses_allocator will detect your object can receive an allocator:</p>
<ol>
<li>If it has a member typedef <code>allocator_type</code>.</li>
<li>If <code>std::uses_allocator</code> is specialised to return true for your object type.</li>
</ol>
<p>Of course, <code>std::pair</code> respects NEITHER of those rules. But here is the caveat:</p>
<blockquote>
<p>As a special case, std::pair is treated as a uses-allocator type even though std::uses_allocator is false for pairs (unlike e.g. std::tuple): see pair-specific overloads of std::polymoprhic_allocator::construct and std::scoped_allocator_adaptor::construct (until C++20)std::uses_allocator_construction_args (since C++20).</p>
</blockquote>
<p>Somewhere, deep in a cave, there is a C++ standard committee troll frenetically enjoying his/her/its joke on us with <code>std::uses_allocator</code> returning false EVEN THOUGH <code>std::pair</code> will be correctly forwarding allocators. Please don't feed it, he/she/it has done enough damage here.</p>
<h3>Harnessing std::uses_allocator's power:</h3>
<p>Unlike the troll, we cannot change the standard to fit our <code>node</code> type. So we need to use <code>std::uses_allocator</code> the proper way.
We will start by adding a specialisation to signal that our type wants to forward allocators:</p>
<div class="highlight"><pre><span></span><code><span class="k">namespace</span> <span class="nn">std</span>
<span class="p">{</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Allocator</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">uses_allocator</span><span class="o"><</span><span class="n">node</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">,</span> <span class="n">Allocator</span><span class="o">></span> <span class="o">:</span> <span class="n">true_type</span>
<span class="p">{</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div>
<p>All these constructors must take a std::allocator_arg_t tag parameter to differentiate them from the others, the non-allocator-forwarding ones. The second parameter is always the instance of the allocator itself alloc and the rest are the parameters you would find in their non-allocator-forwarding equivalents. As I just implied, you must have exactly the same amount of allocator-forwarding constructors as you have normal ones! You must be able to do all operations with or without involving allocators.</p>
<p>As soon as we have an alloc we can send it deep down to the Schrodinger pair. The Schrodinger pair must then construct its mutable std::pair variant taking that allocator in consideration:
This expresses that for any <code>node</code> and any <code>Allocator</code>, an instance of <code>node</code> can receive an instance of <code>allocator</code> to forward it deep down.
By which mean the instance of <code>node</code> will receive that instance <code>allocator</code>? With some special constructors:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">node</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="c1">// ...</span>
<span class="c1">// Constructor that takes arguments to make an index and a pair.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Allocator</span><span class="p">,</span> <span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="n">node</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg_t</span><span class="p">,</span> <span class="k">const</span> <span class="n">Allocator</span><span class="o">&</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">node_index_t</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span> <span class="n">next</span><span class="p">,</span> <span class="n">Args</span><span class="o">&&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span>
<span class="o">:</span> <span class="n">next</span><span class="p">(</span><span class="n">next</span><span class="p">),</span> <span class="n">pair</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg</span><span class="p">,</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o"><</span><span class="n">Args</span><span class="o">></span><span class="p">(</span><span class="n">args</span><span class="p">)...)</span>
<span class="p">{}</span>
<span class="c1">// Copy constructor.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Allocator</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Node</span><span class="o">></span>
<span class="n">node</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg_t</span><span class="p">,</span> <span class="k">const</span> <span class="n">Allocator</span><span class="o">&</span> <span class="n">alloc</span><span class="p">,</span> <span class="k">const</span> <span class="n">Node</span><span class="o">&</span> <span class="n">other</span><span class="p">)</span>
<span class="o">:</span> <span class="n">next</span><span class="p">(</span><span class="n">other</span><span class="p">.</span><span class="n">next</span><span class="p">),</span> <span class="n">pair</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg</span><span class="p">,</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">other</span><span class="p">.</span><span class="n">pair</span><span class="p">.</span><span class="n">pair</span><span class="p">())</span>
<span class="p">{}</span>
<span class="c1">// Move constructor.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Allocator</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Node</span><span class="o">></span>
<span class="n">node</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg_t</span><span class="p">,</span> <span class="k">const</span> <span class="n">Allocator</span><span class="o">&</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">Node</span><span class="o">&&</span> <span class="n">other</span><span class="p">)</span>
<span class="o">:</span> <span class="n">next</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">other</span><span class="p">.</span><span class="n">next</span><span class="p">)),</span> <span class="n">pair</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg</span><span class="p">,</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">other</span><span class="p">.</span><span class="n">pair</span><span class="p">.</span><span class="n">pair</span><span class="p">()))</span>
<span class="p">{}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">nodes_size_type</span> <span class="n">next</span> <span class="o">=</span> <span class="n">node_end_index</span><span class="p">;</span> <span class="c1">// Next index of the node in the linked-list.</span>
<span class="n">key_value_pair_t</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span> <span class="n">pair</span><span class="p">;</span> <span class="c1">// Our glorious Schrodinger pair.</span>
<span class="p">};</span>
</code></pre></div>
<p>All these constructors must take a <code>std::allocator_arg_t</code> tag parameter to differentiate them from the others, the non-allocator-forwarding ones.
The second parameter is always the instance of the allocator itself <code>alloc</code> and the rest are the parameters you would find in their non-allocator-forwarding equivalents. As I just implied, you must have exactly the same amount of allocator-forwarding constructors as you have normal ones! You must be able to do all operations with or without involving allocators.</p>
<p>As soon as we have an <code>alloc</code> we can pass-it deep down to the Schrodinger pair. The Schrodinger pair must then construct its mutable <code>std::pair</code> variant taking that allocator in consideration:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">union</span> <span class="nc">union_key_value_pair</span>
<span class="p">{</span>
<span class="c1">//...</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Allocator</span><span class="p">,</span> <span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="n">union_key_value_pair</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocator_arg_t</span><span class="p">,</span> <span class="k">const</span> <span class="n">Allocator</span><span class="o">&</span> <span class="n">alloc</span><span class="p">,</span> <span class="n">Args</span><span class="o">&&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">auto</span> <span class="n">alloc_copy</span> <span class="o">=</span> <span class="n">alloc</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">allocator_traits</span><span class="o"><</span><span class="n">Allocator</span><span class="o">>::</span><span class="n">construct</span><span class="p">(</span><span class="n">alloc_copy</span><span class="p">,</span> <span class="o">&</span><span class="n">pair_</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o"><</span><span class="n">Args</span><span class="o">></span><span class="p">(</span><span class="n">args</span><span class="p">)...);</span>
<span class="p">}</span>
<span class="c1">//...</span>
<span class="p">};</span>
</code></pre></div>
<p>Once again, <code>union_key_value_pair</code> uses the tag type <code>std::allocator_arg_t</code> to be sure not to collide with other constructors.
We will then construct the <code>pair_</code> in place ; meaning that we will skip the memory allocation part of it since we already have the storage for it. Constructing an object in <strong>C++17</strong> with an allocator requires you a PhD in C++ arcaneries: you need a non-const instance of that allocator coupled to the <a href="https://en.cppreference.com/w/cpp/memory/allocator_traits">allocator_traits</a>. <strong>C++20</strong> can once again save you some time here with <a href="https://en.cppreference.com/w/cpp/memory/make_obj_using_allocator">std::make_obj_using_allocator</a>.</p>
<p>And on this positive note we are done with allocators! Our node class has the same behaviour a std::pair, it will reuse the allocator it was allocated with for its own members.</p>
<h2>Conclusion from the author:</h2>
<p>It was quite a labor to implement the iterators types for our dense_hash_map. We also discovered with stupor that allocators are not working out of the box for custom types. To have or not to have access to C++20 is also a huge factor in how easily you can write such code. C++17 demands a lot more rigor when dealing with "standard" code.</p>
<p>I have been selling these blog posts as us building a hash map together... so I am assuming that you are quite in furor by now since all the prior posts and this one did not contain a single line of algorithms. The rumor is that the next post will be about a maze of insertion algorithms, so stay tuned!</p>Making a STL-compatible hash map from scratch - Part 2 - Growth Policies & The Schrodinger std::pair2020-04-13T17:40:00+02:002020-04-13T17:40:00+02:00Jean Gueganttag:jguegant.github.io,2020-04-13:/blogs/tech/dense-hash-map2.html<p>This post is part of a series of posts:</p>
<ul>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">Part 1 - Beating std::unordered_map</a></li>
<li><strong>Part 2 - Growth Policies & The Schrodinger std::pair (Current)</strong></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map3.html">Part 3 - The wonderful world of iterators and allocators</a></li>
<li>Part 4 - An insertion maze (Coming Soon)</li>
</ul>
<p>In the <a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">previous post</a>, we started a quest that consists in …</p><p>This post is part of a series of posts:</p>
<ul>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">Part 1 - Beating std::unordered_map</a></li>
<li><strong>Part 2 - Growth Policies & The Schrodinger std::pair (Current)</strong></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map3.html">Part 3 - The wonderful world of iterators and allocators</a></li>
<li>Part 4 - An insertion maze (Coming Soon)</li>
</ul>
<p>In the <a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">previous post</a>, we started a quest that consists in implementing an associative container similar to <code>std::unordered_map</code> in the C++ standard library. We saw two approaches that could help us in beating the performance of most <code>std::unordered_map</code> implementations: freeing ourselves from stable-addressing and swapping the modulo operation with some bit-wise operations.
Due to its layout, we called this new associative container a <code>dense_hash_map</code>. You can find a reference implementation <a href="https://github.com/Jiwan/dense_hash_map">right here</a>.</p>
<p>Let's carry-on and start our own implementation right now!</p>
<h1>Part 2 - Structuring our code & Dealing with value_type:</h1>
<h2>Preparing for the adventure:</h2>
<p>Disclaimer: this section starts very gently, if your C++ e-beard or e-hair is already pretty long, you may want to skip to the <strong>Structuring our code</strong> section. </p>
<p>Like any hero starting her/his journey, we need to ensure we have the right items in our inventory.</p>
<p>For C++, here is my list of handy things to have when working on some standard-ish looking code:</p>
<ul>
<li><a href="http://eel.is/c++draft/">eel.is C++ draft</a>, the holly bible of C++ itself (the latest draft of the C++ standard).
eel.is's transformation of the standard latex source files to a clean HTML version is a godsent gift. You can navigate in each of the sections of the standard with ease. In our case, we will mainly refer ourselves to the sections <a href="http://eel.is/c++draft/unord.map">unord.map</a>, <a href="http://eel.is/c++draft/associative.reqmts">associative.reqmts</a> and <a href="http://eel.is/c++draft/containers">container</a>.</li>
<li><a href="https://en.cppreference.com">cppreference</a>, the tuned-down version of the standard. This wiki is of great help when the lawyer language of the standard is too advanced for your taste (sometimes the standard feels downright insulting to your intellectual abilities). On some rare occasions, I have seen <strong>cppreference</strong> have slightly inacurate descriptions of the standard. It always nice to cross-check the holly bible and this wiki. In our case, we will most refer to the <a href="https://en.cppreference.com/w/cpp/container/unordered_map">unordered_map section</a>.</li>
<li><a href="https://godbolt.org/">godbolt</a> to experiment quickly your ideas on different compilers and check how the standard reacts for certain use-cases. With the recently added feature that permits you to run code, <strong>godbolt</strong> is extremely handy. <strong>Goldbot's</strong> grand cousin, <a href="https://cppinsights.io/">cppinsights</a> can help you to explore complicated expressions. </li>
<li><a href="https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines">The C++ core guidelines</a>, a bunch of C++ precepts written or more or less by those behind the holly C++ standard. When given multiples way of doing one task, the core guidelines will direct you towards the better way to achieve that task.</li>
<li>Some linter and formatting tools. I personally like <a href="https://clang.llvm.org/extra/clang-tidy/">clang-tidy</a> and <a href="https://clang.llvm.org/docs/ClangFormat.html">clang-format</a>. </li>
<li>A good unit-test framework like <a href="https://github.com/catchorg/Catch2">catch2</a>. Writing such standard-like code without writing unit-tests is like running naked towards a level 100 creature and expecting to beat it with a simple bamboo stick. Good luck to you to achieve that!</li>
<li>A bunch of warnings applied to your compiler. For the unix-like compilers (GCC or clang), a bare minimum would be <code>-Wall -Wextra -pedantic -Werror</code>.</li>
</ul>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/the-holly-cpp-standard.jpg" alt="The Holly C++ Standard"/></center></p>
<p>And some optional items:</p>
<ul>
<li>A recent compiler that supports a recent version of the language (C++17 in my case). Technically, nothing prevents you to write standard-library-like code with ancient C++, but each version of the standard made it a lot simpler. This is especially true for the meta-programming part.</li>
<li><a href="https://cmake.org/">cmake</a>. Whether you find cmake's syntax an abomination or poetry, it is currently the de-facto standard build tool for any C++ projects.
Just for that sake of consistency in the C++ ecosystem, I would advise to support some sort of <strong>cmake</strong> bindings for your project. </li>
</ul>
<h2>Structuring our code:</h2>
<h3>A header only Library:</h3>
<p>Given that most of the classes, functions or even variables will have template parameters, there will be very little code that could end-up in its own compilation unit. So the <code>dense_hash_map</code> will be header library. The file layout is fairly standard:</p>
<div class="highlight"><pre><span></span><code>├── CMakeLists.txt <span class="c1"># Main CMakeLists.txt file that defines.</span>
├── include
│ └── jg
│ ├── dense_hash_map.hpp <span class="c1"># Main file to be included by users.</span>
│ └── details <span class="c1"># Folder that contains internal structures that users should not depend-on.</span>
│ ├── bucket_iterator.hpp
│ ├── dense_hash_map_iterator.hpp
│ ├── node.hpp
│ ├── power_of_two_growth_policy.hpp
│ └── type_traits.hpp
├── tests <span class="c1"># Unit-tests folder.</span>
│ ├── CMakeLists.txt
│ └── src
│ └── dense_hash_map_tests.cpp
</code></pre></div>
<p>Note: as any C++ egomaniac would do when facing the hard issue of choosing a namespace for his/her project, I went for my initial "jg".</p>
<h3>Our main class in dense_hash_map.hpp:</h3>
<p>To be able to organise our code correctly, let's have look at the "dense" layout we saw in the last <a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">post</a>: </p>
<p><center><img width=60% height=60% src="https://jguegant.github.io/blogs/tech/images/dense_hash_map_layout2.png" alt="The definitive layout"/></center></p>
<p>We will have <strong>two vectors</strong>:</p>
<ul>
<li>One for the <strong>buckets</strong>. More precisely, these are the indices for the first node of the linked-list of each of our buckets.</li>
<li>One for all the <strong>nodes</strong>. These are the nodes in the interleaved linked-lists that hold the key/value pairs.</li>
</ul>
<p>While we are crafting our <code>dense_hash_map</code> from scratch, it doesn't mean that we cannot reuse some of the other containers in the standard library.
Naturally, our <strong>two vectors</strong> will be some good old <code>std::vector</code>. Both of those will be members of our <a href="https://github.com/Jiwan/dense_hash_map/blob/master/include/jg/dense_hash_map.hpp#L892">dense_hash_map class</a>. If we inspire ourselves from <a href="https://en.cppreference.com/w/cpp/container/unordered_map">std::unordered_map's signature</a>, this gives us our first piece of code:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span>
<span class="k">class</span> <span class="nc">Key</span><span class="p">,</span>
<span class="k">class</span> <span class="nc">T</span><span class="p">,</span>
<span class="k">class</span> <span class="nc">Hash</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">hash</span><span class="o"><</span><span class="n">Key</span><span class="o">></span><span class="p">,</span>
<span class="k">class</span> <span class="nc">Pred</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">equal_to</span><span class="o"><</span><span class="n">Key</span><span class="o">></span><span class="p">,</span>
<span class="k">class</span> <span class="nc">Allocator</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">allocator</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="k">class</span> <span class="nc">GrowthPolicy</span> <span class="o">=</span> <span class="n">details</span><span class="o">::</span><span class="n">power_of_two_growth_policy</span>
<span class="o">></span>
<span class="k">class</span> <span class="nc">dense_hash_map</span> <span class="o">:</span> <span class="k">private</span> <span class="n">GrowthPolicy</span>
<span class="p">{</span>
<span class="k">private</span><span class="o">:</span>
<span class="c1">// Define some aliases to avoid further typing later on.</span>
<span class="k">using</span> <span class="n">node_type</span> <span class="o">=</span> <span class="n">details</span><span class="o">::</span><span class="n">node</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span> <span class="c1">// A node type that contains our key/value.</span>
<span class="k">using</span> <span class="n">nodes_container_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">node_type</span><span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">nodes_size_type</span> <span class="o">=</span> <span class="k">typename</span> <span class="nc">nodes_container_type</span><span class="o">::</span><span class="n">size_type</span><span class="p">;</span>
<span class="k">using</span> <span class="n">buckets_container_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">nodes_size_type</span><span class="o">></span><span class="p">;</span>
<span class="k">public</span><span class="o">:</span>
<span class="c1">// ...</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">nodes_container_type</span> <span class="n">nodes_</span><span class="p">;</span>
<span class="n">buckets_container_type</span> <span class="n">buckets_</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>The <code>buckets_container_type</code> is dependent on the <code>size_type</code> of the <code>nodes_container_type</code>. In a <code>std::vector</code>, <code>size_type</code> is the type used for accessing elements using the subscript operator <code>operator[]</code>. It makes sense, since <strong>buckets_</strong> contains indices for <strong>nodes_</strong>. Traditionnaly, and to the <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1428r0.pdf">dismay of plenty</a>, <code>size_type</code> is <code>std::size_t</code> which is an unsigned integer. We will use the value of <code>std::numeric_limits<nodes_size_type>::max()</code> to represent an invalid index into <strong>nodes_</strong> or the end of our linked-lists:</p>
<div class="highlight"><pre><span></span><code><span class="k">static</span> <span class="kr">inline</span> <span class="k">constexpr</span> <span class="n">nodes_size_type</span> <span class="n">node_end_index</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">numeric_limits</span><span class="o"><</span><span class="n">nodes_size_type</span><span class="o">>::</span><span class="n">max</span><span class="p">();</span>
</code></pre></div>
<p>For instance, <strong>buckets_</strong> will be initialized, at the construction of an empty <code>dense_hash_map</code>, with <code>node_end_index</code> since all the buckets are empty.</p>
<p>If you were attentive, you would notice that there was also an extra template parameter <code>GrowthPolicy</code> to <code>dense_hash_map</code>. Let's have a closer look at it!</p>
<h3>Growth Policy:</h3>
<p>In the last <a href="https://jguegant.github.io/blogs/tech/dense-hash-map.html">post</a>, we discovered some bit-wise operation tricks to increase the speed when mapping a <strong>key</strong> to its <strong>bucket</strong>.
We also saw that there were drawbacks in doing so. So instead of forcing this design choice on our <code>dense_hash_map</code> users, we can make this behaviour a <a href="https://en.wikipedia.org/wiki/Modern_C%2B%2B_Design#Policy-based_design">policy</a>. The default policy will be the <a href="https://github.com/Jiwan/dense_hash_map/blob/master/include/jg/details/power_of_two_growth_policy.hpp">power_of_two_growth_policy</a>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// power_of_two_growth_policy.hpp</span>
<span class="k">namespace</span> <span class="nn">jg</span><span class="o">::</span><span class="nn">details</span>
<span class="p">{</span>
<span class="k">struct</span> <span class="nc">power_of_two_growth_policy</span>
<span class="p">{</span>
<span class="c1">// Given a hash and the current capacity, use bit-wise trick to return the bucket index.</span>
<span class="k">static</span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">compute_index</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">hash</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">capacity</span><span class="p">)</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">hash</span> <span class="o">&</span> <span class="p">(</span><span class="n">capacity</span> <span class="o">-</span> <span class="mi">1</span><span class="p">);</span> <span class="c1">// We saw that trick in the previous post.</span>
<span class="p">}</span>
<span class="c1">// Given a desired new capacity for the bucket container, pick the closest power of two. </span>
<span class="k">static</span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">compute_closest_capacity</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">min_capacity</span><span class="p">)</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span>
<span class="p">{</span>
<span class="c1">// We didn't see that trick yet.</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">highest_capacity</span> <span class="o">=</span> <span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span><span class="p">{</span><span class="mi">1</span><span class="p">}</span> <span class="o"><<</span> <span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">numeric_limits</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span><span class="o">>::</span><span class="n">digits</span> <span class="o">-</span> <span class="mi">1</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="n">min_capacity</span> <span class="o">></span> <span class="n">highest_capacity</span><span class="p">)</span> <span class="p">{</span>
<span class="n">assert</span><span class="p">(</span><span class="nb">false</span> <span class="o">&&</span> <span class="s">"Maximum capacity for the dense_hash_map reached."</span><span class="p">);</span>
<span class="k">return</span> <span class="n">highest_capacity</span><span class="p">;</span>
<span class="p">}</span>
<span class="o">--</span><span class="n">min_capacity</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="k">auto</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">std</span><span class="o">::</span><span class="n">numeric_limits</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span><span class="o">>::</span><span class="n">digits</span><span class="p">;</span> <span class="n">i</span> <span class="o">*=</span> <span class="mi">2</span><span class="p">)</span> <span class="p">{</span>
<span class="n">min_capacity</span> <span class="o">|=</span> <span class="n">min_capacity</span> <span class="o">>></span> <span class="n">i</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="o">++</span><span class="n">min_capacity</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Returns the minimum capacity (~= initial capacity) the bucket container must have.</span>
<span class="k">static</span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">minimum_capacity</span><span class="p">()</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="p">{</span> <span class="k">return</span> <span class="mi">8u</span><span class="p">;</span> <span class="p">}</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div>
<p>All the growth policies will have to respect an implicit <a href="https://en.cppreference.com/w/cpp/language/constraints">C++ concept</a>: it must have the three static member functions <code>compute_index</code>, <code>compute_closest_capacity</code> and <code>minimum_capacity</code>. </p>
<p>These specific <code>compute_index</code> and <code>minimum_capacity</code> functions are self-explanatory. But <code>compute_closest_capacity</code> is interesting: it tries to find the closest upper power of two given a random capacity.
A clever way that achieve that task is to fill your capacity number in its bit representation with <code>1</code> starting from the right until you reach the highest bit already toggled on with <code>1</code>. Afterwards, you can just increment and tada... you get your upper power of two.
Here is an example with the number 18 to clarify a bit (no pun intended):</p>
<div class="highlight"><pre><span></span><code>18 (decimal) == 0001 0010 (binary)
^ Highest bit already toggle on.
After filling starting from the right:
0001 1111 (binary)
And if we add one:
0010 0000 (binary) == 32 (decimal)
</code></pre></div>
<p><code>compute_closest_capacity</code> is doing exactly that with the operator <code>|=</code> and some bit-shifting with the operator <code>>></code>. It handles the case where we already have a power two by substracting one before processing the number. We also cap the minimum capacity to the highest power of two we can represent to avoid an overflow in the result.</p>
<p>Note: you can also achieve a similar effect with some built-ins in your conpiler. For instance, GCC provides <a href="https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html">__builtin_clz</a> which can be of great help.</p>
<p>Afterwards, we rely on the growth policy in our <code>dense_hash_map</code> by "using" its functions:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">dense_hash_map</span> <span class="o">:</span> <span class="k">private</span> <span class="n">GrowthPolicy</span>
<span class="p">{</span>
<span class="k">using</span> <span class="n">GrowthPolicy</span><span class="o">::</span><span class="n">compute_closest_capacity</span><span class="p">;</span> <span class="c1">// "Import" the policy's functions in this scope.</span>
<span class="k">using</span> <span class="n">GrowthPolicy</span><span class="o">::</span><span class="n">compute_index</span><span class="p">;</span>
<span class="k">using</span> <span class="n">GrowthPolicy</span><span class="o">::</span><span class="n">minimum_capacity</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="k">constexpr</span> <span class="kt">void</span> <span class="nf">rehash</span><span class="p">(</span><span class="n">size_type</span> <span class="n">count</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="n">count</span> <span class="o">=</span> <span class="n">compute_closest_capacity</span><span class="p">(</span><span class="n">count</span><span class="p">);</span>
<span class="c1">// ...</span>
<span class="p">}</span>
<span class="p">};</span>
</code></pre></div>
<p>So what's left of the first code snippet that we didn't dive into? The node type: <code>details::node<Key, T></code>. By now, you should have a vague feeling that there is something is fishy with this one, it has its own class in the detail namespace. But is that so complicated to make a node holding a key/value pair?</p>
<h2>Dealing with a const value_type:</h2>
<p>So here is the deal, <code>std::unordered_map</code> has a peculiar <code>value_type</code> (<code>value_type</code> is the type you obtain as a reference when querying most containers in the standard library when using iterators) which is <code>std::pair<const Key, T></code>. Whenever you supply a template argument <code>Key</code>, all the key/value pairs you obtain from <code>std::unordered_map</code> will have their first member, which is the key, stored as <code>const Key</code>. And, this, ladies and gentlemen, is can of giant worms ready to devore your last bits of sanity. So... let's open it!</p>
<p><center><img width=40% height=40% src="https://jguegant.github.io/blogs/tech/images/dark-soul-open-door.jpg" alt="Opening the const value_type door"/></center></p>
<p>When you think a bit more about it, this design prevents you to put yourself in serious troubles. You cannot mutate the keys in your associative container once you have inserted them. Typically, without a <code>const Key</code>, you would be able to do such crazy moves: </p>
<div class="highlight"><pre><span></span><code><span class="n">jg</span><span class="o">::</span><span class="n">dense_hash_map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">hero</span><span class="o">></span> <span class="n">my_map</span><span class="p">{{</span><span class="s">"spongebob"</span><span class="p">,</span> <span class="n">hero</span><span class="p">{}},</span> <span class="p">{</span><span class="s">"chuck norris"</span><span class="p">,</span> <span class="n">hero</span><span class="p">{}}};</span>
<span class="k">auto</span> <span class="n">it</span> <span class="o">=</span> <span class="n">my_map</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"chuck norris"</span><span class="p">);</span>
<span class="n">it</span><span class="o">-></span><span class="n">first</span> <span class="o">=</span> <span class="s">"Chuck Norris"</span><span class="p">;</span> <span class="c1">// We are mutating the key to properly capitalise the venerable chuck.</span>
<span class="c1">// my_map after that would be in sad state.</span>
</code></pre></div>
<p>When mutating a key, your key will very likely have a new hash (unless you are the lucky winner of a collision...) which means that it probably belongs to a new bucket (unless both hashes belong to the same bucket...). Sadly, there is no way for your container to be aware that you mutated a <code>std::pair<const Key, T></code> it gave you as reference, which implies that the container cannot handle properly the hash change. Its internal data-structure would be in a broken state rather quickly. By having a <code>const Key</code>, the standard shields itself from this issue: the key becomes immutable.</p>
<p>Now, let's assume that our <code>details::node<Key, T></code> has a <code>std::pair<const Key, T></code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">node</span> <span class="p">{</span>
<span class="n">nodes_size_type</span> <span class="n">next</span> <span class="o">=</span> <span class="n">node_end_index</span><span class="p">;</span> <span class="c1">// Next index of the node in the linked-list.</span>
<span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span> <span class="n">pair</span><span class="p">;</span> <span class="c1">// Our glorious pair.</span>
<span class="p">};</span>
</code></pre></div>
<p>With our "dense" design, this becomes a <strong>performance issue</strong>. Our <code>std::pair<const Key, T></code> will have its <strong>move-constructor</strong> and <strong>move-assignment operator</strong> disabled forever and ever, and so does the node type. All our nodes are stored in a packed vector <code>nodes_</code>, which sometimes reallocate itself and send all its content into a bigger buffer. Not being to able to <strong>move</strong> our objects accross memory is a huge bummer. We would have to copy them around. That's unacceptable! Even more irritating, <strong>std::unordered_map</strong> does not suffer from this as its design allocates each nodes separately.</p>
<p>Big problems require big <strike>solutions</strike> hacks. Here are our two choices:</p>
<ul>
<li>We make a <code>std::pair</code> type on steroids that communicates back and forth with the container when the key change.
This also requires our new type to be interoperable with <code>std::pair</code>. It might be possible to partially fake a <code>std::pair</code> with some implicit conversions, but we will never make our <code>dense_hash_map</code> a drop-in replacement that way.</li>
<li>We circumvent the type system with some <a href="https://en.wikipedia.org/wiki/Type_punning">type punning</a> and create a Schrodinger <code>std::pair</code> that has both a <code>Key</code> and a <code>const Key</code> as its first member. In other words, we store internally a <code>std::pair<Key, T></code> in our node but expose it as a <code>std::pair<const Key, T></code> to the user.</li>
</ul>
<p>Obviously, the Schrodinger <code>std::pair</code> sounds the most dangerous creature of the two. So... let's pick it!</p>
<h4>The Schrodinger <strong>std::pair</strong>:</h4>
<p>C++ offers at least two official ways to perform type-punning: <a href="https://en.cppreference.com/w/cpp/language/reinterpret_cast">reinterpret_cast</a> and <a href="https://en.cppreference.com/w/cpp/language/union">unions</a>.</p>
<h5>reinterpret_cast:</h5>
<p><strong>reinterpret_cast</strong> sounds quite attractive: "it is purely a compile-time directive which instructs the compiler to treat expression as if it had the type new_type". But if you read closely <a href="http://eel.is/c++draft/expr.reinterpret.cast#:cast,reinterpret">along the standard lines</a>, it has a very restricted usage. It is often used when casting to <code>const char*</code> (<code>const std::byte*</code>) to manipulate your types as raw data. But even doing so is harder than you expect. A fun-fact is that the classic pattern in many C++ code-base of reinterpret_casting a memory buffer to a <a href="https://en.cppreference.com/w/cpp/named_req/PODType">POD</a> is actually not <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0593r2.html">allowed</a>:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">x</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">network_buffer</span> <span class="o">=</span> <span class="n">get_buffer</span><span class="p">();</span>
<span class="c1">// The following code is not allowed as explained by, you would need std:bless from p0593r2.</span>
<span class="k">const</span> <span class="n">A</span><span class="o">*</span> <span class="n">my_a</span> <span class="o">=</span> <span class="k">reinterpret_cast</span><span class="o"><</span><span class="k">const</span> <span class="n">A</span><span class="o">*></span><span class="p">(</span><span class="n">network_buffer</span><span class="p">);</span>
</code></pre></div>
<p>Clearly, <strong>reinterpret_cast</strong> is not our friend here. <a href="https://en.cppreference.com/w/cpp/language/reinterpret_cast">cppreference</a> even gently warn us:</p>
<div class="highlight"><pre><span></span><code>The behavior is undefined unless one of the following is true:
- AliasedType and DynamicType are similar.
...
std::pair<int, int> and std::pair<const int, int> are not similar.
</code></pre></div>
<h5>union:</h5>
<p>We are left with C++'s <a href="https://en.cppreference.com/w/cpp/language/union">unions</a> to perform our type-punning. Let's write one:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">union</span> <span class="nc">union_key_value_pair</span>
<span class="p">{</span>
<span class="k">using</span> <span class="n">pair_t</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">const_key_pair_t</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="n">union_key_value_pair</span><span class="p">(</span><span class="n">Args</span><span class="o">&&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span> <span class="o">:</span>
<span class="n">pair_</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o"><</span><span class="n">Args</span><span class="o">></span><span class="p">(</span><span class="n">args</span><span class="p">)...)</span> <span class="c1">// Construct the active member.</span>
<span class="p">{}</span>
<span class="c1">// Special members need to be explicitely defined for unions.</span>
<span class="n">union_key_value_pair</span><span class="p">(</span><span class="k">const</span> <span class="n">union_key_value_pair</span><span class="o">&</span> <span class="n">other</span><span class="p">)</span> <span class="p">{}</span>
<span class="n">union_key_value_pair</span><span class="p">(</span><span class="n">union_key_value_pair</span><span class="o">&&</span> <span class="n">other</span><span class="p">)</span> <span class="p">{}</span>
<span class="k">auto</span> <span class="k">operator</span><span class="o">=</span><span class="p">(</span><span class="k">const</span> <span class="n">union_key_value_pair</span><span class="o">&</span> <span class="n">other</span><span class="p">)</span> <span class="o">-></span> <span class="n">union_key_value_pair</span><span class="o">&</span>
<span class="p">{</span>
<span class="n">pair_</span> <span class="o">=</span> <span class="n">other</span><span class="p">.</span><span class="n">pair_</span><span class="p">;</span>
<span class="k">return</span> <span class="o">*</span><span class="k">this</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="k">operator</span><span class="o">=</span><span class="p">(</span><span class="n">union_key_value_pair</span><span class="o">&&</span> <span class="n">other</span><span class="p">)</span> <span class="o">-></span> <span class="n">union_key_value_pair</span><span class="o">&</span>
<span class="p">{</span>
<span class="n">pair_</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">other</span><span class="p">).</span><span class="n">pair_</span><span class="p">;</span>
<span class="k">return</span> <span class="o">*</span><span class="k">this</span><span class="p">;</span>
<span class="p">}</span>
<span class="o">~</span><span class="n">union_key_value_pair</span><span class="p">()</span> <span class="p">{</span> <span class="n">pair_</span><span class="p">.</span><span class="o">~</span><span class="n">pair_t</span><span class="p">();</span> <span class="p">}</span>
<span class="c1">// Accessors to the two members.</span>
<span class="n">pair_t</span><span class="o">&</span> <span class="n">pair</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">const</span> <span class="n">pair_t</span><span class="o">&</span> <span class="n">pair</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span> <span class="k">return</span> <span class="n">pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">const_key_pair</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">const</span> <span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">const_key_pair</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">pair_t</span> <span class="n">pair_</span><span class="p">;</span> <span class="c1">// Our active member.</span>
<span class="n">const_key_pair_t</span> <span class="n">const_key_pair_</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>Unions have this concept of an <strong>active member</strong>: the latest member you constructed. You can only have one active member at a time for a union.
In our case, the active member will always be <code>pair_</code>, a <code>std::pair<Key, T></code>, since this is how we truly want to store the pair.
It is also why we defined all the special members (copy/move constructors/assignment operators) in terms of the <code>pair_</code> member.
Now the interesting question is what can we do with the "inactive member" <code>const_key_pair_</code>? Can we expose it to the user directly?</p>
<p>The standard is suprisingly <a href="http://eel.is/c++draft/class.union#2">quite clear</a> on that topic:</p>
<blockquote>
<p>One special guarantee is made in order to simplify the use of unions:
If a standard-layout union contains several standard-layout structs that share a common initial sequence ([class.mem]), and if a non-static data member of an object of this standard-layout union type is active and is one of the standard-layout structs, it is permitted to inspect the common initial sequence of any of the standard-layout struct members; see [class.mem].</p>
</blockquote>
<p>In rough terms, we will be able to access the inactive member <code>const_key_pair_</code> if both <code>std::pair<Key, T></code> and <code>std::pair<const Key, T></code> share the <strong>same layout for their members</strong> <code>first</code> and <code>second</code> and both are <strong>standard-layout structs</strong>. If you take a look at all the three main standard libraries (<a href="https://github.com/llvm-mirror/libcxx/blob/master/include/utility#L305">libc++</a>, <a href="https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/stl_pair.h#L216">libstdc++</a>, <a href="https://github.com/microsoft/STL/blob/master/stl/inc/utility#L330">MSVC's STL</a>), we can assume that the <code>first</code> and <code>second</code> members will always be at the same position (no weird template specialisation going on). But having the guarantee that our pair types will be <strong>standard-layout structs</strong> is much harder. For that to happen, the types <code>Key</code> and <code>T</code> they store need to be <strong>standard-layout structs</strong> themselves.
Here we have to make a hard choice, to follow or not to follow the standard. Please, roll a binary dice and observe the result:</p>
<p><strong>1 - To follow the standard: </strong></p>
<p>We decided to listen to the standard and will stick to the <strong>standard-layout structs</strong> rule. In this scenario, we will conditionally enable our <code>union_key_value_pair</code> optimisation:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Our safe fallback to store key/value pairs.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">safe_key_value_pair</span>
<span class="p">{</span>
<span class="k">using</span> <span class="n">const_key_pair_t</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="n">safe_key_value_pair</span><span class="p">(</span><span class="n">Args</span><span class="o">&&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span> <span class="o">:</span> <span class="n">const_key_pair_</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o"><</span><span class="n">Args</span><span class="o">></span><span class="p">(</span><span class="n">args</span><span class="p">)...)</span> <span class="p">{}</span>
<span class="c1">// Accessors to the two members.</span>
<span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">pair</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">const</span> <span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">pair</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">const_key_pair</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">const</span> <span class="n">const_key_pair_t</span><span class="o">&</span> <span class="n">const_key_pair</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span> <span class="k">return</span> <span class="n">const_key_pair_</span><span class="p">;</span> <span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">const_key_pair_t</span> <span class="n">const_key_pair_</span><span class="p">;</span>
<span class="p">};</span>
<span class="c1">// Given a two types Key and T, checks if a pair of the two would be of standard-layout.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="kr">inline</span> <span class="k">constexpr</span> <span class="kt">bool</span> <span class="n">are_pairs_standard_layout_v</span> <span class="o">=</span>
<span class="n">std</span><span class="o">::</span><span class="n">is_standard_layout_v</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="k">const</span> <span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">>></span> <span class="o">&&</span>
<span class="n">std</span><span class="o">::</span><span class="n">is_standard_layout_v</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">>></span><span class="p">;</span>
<span class="c1">// Given a two types Key and T, returns you the best storage for your pair: the optimised one or the safe one.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">using</span> <span class="n">key_value_pair_t</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">conditional_t</span><span class="o"><</span>
<span class="n">are_pairs_standard_layout_v</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">,</span>
<span class="n">union_key_value_pair</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">,</span>
<span class="n">safe_key_value_pair</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span>
<span class="o">></span><span class="p">;</span>
</code></pre></div>
<p>We start by defining a safe container <code>safe_key_value_pair</code>. This container is <strong>not a union</strong> and has only one member of type <code>std::pair<const Key, T></code>.
The accessors to the "two members" are behind the scene redirecting to only one member. This type will always perform a copy when moved accross memory, but hey... at least it is safe. Now, when you are crafting your node type, you can use the meta-function <code>key_value_pair_t</code> that checks whether your pair is of <strong>standard-layout</strong> and if we should optimise or not the storage. If the pair's type has a <strong>standard-layout</strong>, we can use the optimised union <code>union_key_value_pair</code>, if not, we fallback to <code>safe_key_value_pair</code>. </p>
<p>Finally, if we are so zealous about the standard, we can always add an extra safety net to <code>union_key_value_pair</code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">union</span> <span class="nc">union_key_value_pair</span>
<span class="p">{</span>
<span class="k">static_assert</span><span class="p">(</span>
<span class="n">offsetof</span><span class="p">(</span><span class="n">pair_t</span><span class="p">,</span> <span class="n">first</span><span class="p">)</span> <span class="o">==</span> <span class="n">offsetof</span><span class="p">(</span><span class="n">const_key_pair_t</span><span class="p">,</span> <span class="n">first</span><span class="p">)</span> <span class="o">&&</span>
<span class="n">offsetof</span><span class="p">(</span><span class="n">pair_t</span><span class="p">,</span> <span class="n">second</span><span class="p">)</span> <span class="o">==</span> <span class="n">offsetof</span><span class="p">(</span><span class="n">const_key_pair_t</span><span class="p">,</span> <span class="n">second</span><span class="p">)</span>
<span class="p">);</span>
<span class="c1">// ...</span>
<span class="p">}</span>
</code></pre></div>
<p>This <a href="https://en.cppreference.com/w/cpp/language/static_assert">static_assert</a> ensure that the <code>first</code> and <code>second</code> members are truly located in the same memory area in both <code>std::pair<Key, T></code> and <code>std::pair<const Key, T></code>. Even if one of the three main standard library implementation were to sabotage our <code>union_key_value_pair</code> by shuffling members, we would be warned about it!</p>
<p><strong>0 - To dodge the standard: </strong></p>
<p>Note: If you are a C++ purist that never abuse the standard or write his own undefined behaviour, here is the exact point in this post where you should use your jetpack and fly away to a better place.</p>
<p>In this scenario, we are as paranoid as <a href="https://www.youtube.com/watch?v=IAdLwUXRUvg">Miro Knejp</a>: clearly, the standard committee is trying to hide the true power of unions to us. Yes, we can pun!
<center><a href="https://www.youtube.com/watch?v=IAdLwUXRUvg"><img width=35% height=35% src="https://jguegant.github.io/blogs/tech/images/non-conforming-cpp.png" alt="Miro Knejp talk"/></a></center></p>
<p>Type-punning through unions have been unofficially supported in all three major compilers for longer than I am alive.
Our case is quite safe, given that:</p>
<ul>
<li>Our types would differ from a single <code>const</code>, this shouldn't have any impact on the size of <code>first</code> nor its alignment.</li>
<li>We made the <code>const</code> version the inactive member that we expose to the user. Going from <code>T</code> to <code>const T</code> is a LOT safer than the other way around.</li>
</ul>
<p>So, we will unconditionally use the <code>union_key_value_pair</code> container, but we will keep the <code>static_assert</code> from the previous solution.
We still want to ensure that the memory layout of <code>std::pair<Key, T></code> and <code>std::pair<const Key, T></code> are strictly the same.
By doing so, we will actually disrespect the standard <a href="http://eel.is/c++draft/support.types.layout#1">once again</a>: "the offsetof macro with a type other than a standard-layout class is conditionally-supported". Thankfully, the three main compilers do seem to support it! We can silence GCC's warning on that topic with a well-placed <code>#pragma GCC diagnostic ignored "-Winvalid-offsetof"</code>.</p>
<p>Now, we have a fast container for our key/value pairs no matter the types we receive for <code>Key</code> and <code>T</code>.
Our last move will be a gesture towards our strict friends, we will wrap all our dangerously-undefined-type-punning into an <code>#ifdef</code> directive:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#ifdef JG_STRICT_TYPE_PUNNING</span>
<span class="c1">// Use the previous solution.</span>
<span class="c1">// ...</span>
<span class="cp">#else</span>
<span class="c1">// Use our hacky but fast solution.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Key</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">using</span> <span class="n">key_value_pair_t</span> <span class="o">=</span> <span class="n">union_key_value_pair</span><span class="o"><</span><span class="n">Key</span><span class="p">,</span> <span class="n">T</span><span class="o">></span><span class="p">;</span>
<span class="cp">#endif</span>
</code></pre></div>
<p>/!\ On a more serious note: you should really be aware of the undefined behaviour of this solution. If your application is used in critical operations, you should refrain yourself from defying the standard. On the other hand, if your application is non-critical (or have a rock-solid release process), you could give a try on the fast type punning. Choose wisely!</p>
<h2>Conclusion:</h2>
<p>We survived this first day of our journey! We can control the growth of our container using a policy pattern.
We also have a <strong>Schrodinger std::pair</strong> at our disposal to move our key/value pairs blazingly fast accross memory while preventing our users to shoot themselves in the feet.</p>
<p>Be ready for the next phase: <a href="https://jguegant.github.io/blogs/tech/dense-hash-map3.html">the dreaded iterators and allocators</a> are waiting for you!</p>Making a STL-compatible hash map from scratch - Part 1 - Beating std::unordered_map2020-04-06T17:40:00+02:002020-04-06T17:40:00+02:00Jean Gueganttag:jguegant.github.io,2020-04-06:/blogs/tech/dense-hash-map.html<p>This post is part of a planned series of posts:</p>
<ul>
<li><strong>Part 1 - Beating std::unordered_map (Current)</strong></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">Part 2 - Growth Policies & The Schrodinger std::pair</a></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map3.html">Part 3 - The wonderful world of iterators and allocators</a></li>
<li>Part 4 - An insertion maze (Coming Soon)</li>
</ul>
<h1>Part 1 - Beating std::unordered_map</h1>
<h2>Trivia:</h2>
<p>If C++ had an …</p><p>This post is part of a planned series of posts:</p>
<ul>
<li><strong>Part 1 - Beating std::unordered_map (Current)</strong></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">Part 2 - Growth Policies & The Schrodinger std::pair</a></li>
<li><a href="https://jguegant.github.io/blogs/tech/dense-hash-map3.html">Part 3 - The wonderful world of iterators and allocators</a></li>
<li>Part 4 - An insertion maze (Coming Soon)</li>
</ul>
<h1>Part 1 - Beating std::unordered_map</h1>
<h2>Trivia:</h2>
<p>If C++ had an equivalent in the video game world, it would be similar to Dark Souls: a dark, violent, punishing game.
Yet, the emotion you get after successfully assimilating all the moves required to overcome a challenge is absolutely impossible to replicate.
You are able to reproduce a sort of dance to defeat any boss in your way.
Speaking of which... what could be the final boss of C++?
According to <a href="https://twitter.com/stephantlavavej">Stephan T. Lavavej (STL)</a> this would be the <a href="https://cppcon2019.sched.com/event/Sft8/floating-point-charconv-making-your-code-10x-faster-with-c17s-final-boss">Floating-Point charconv of the C++17's standard</a>.
While I wouldn't be able to beat this boss by myself anytime soon, I can tackle a tiny boss: making a standard-compatible (but not necessarily standard-compliant) container from scratch.
I would like to share with you how to beat this kind of smaller bosses by yourself!</p>
<p><center><img width=35% height=35% src="https://jguegant.github.io/blogs/tech/images/dark-souls.webp" alt="Dark Souls"/></center></p>
<p>We will build a <code>unordered_map</code> together, with all the arcane beasts that come along: iterators, allocators, type traits... and we will survive it.</p>
<p>I encourage you to read this post as an introduction to the wonderful world of hash maps and the ways of the standard. You can find the code for the hash-map I am describing thorough this series of posts <a href="https://github.com/Jiwan/dense_hash_map">right here</a>.
Hopefully when you are done reading this series, you will be ready to explore a bit more what the talented people in the C++ community have produced recently.</p>
<p>Disclaimer: there are lots of extremely talented people (abseil team with their Swiss Tables, Malte Skarupke's flat hash map...)
that have been researching for years how to make the quintessence of an associative container.
While the one I am presenting here has a really decent performance (more so than the standard ones at least), it is not bleeding-edge.
On the other hand, its design is fairly simple to explain and easy to maintain.</p>
<h2>Freeing ourselves from the standard constraints:</h2>
<p>Implementing a standard associative container like <code>std::unordered_map</code> wouldn't be satisfying enough.
We have to do better! Let's make a faster one! </p>
<p>The standard library (and its ancestor the STL) is almost as old as me.
You would think that the people working on an implementation of it (libc++, libstdc++...) would have had enough to polish it until there is no room to improvement.
So how could we beat them in their own domain? Well... we are going to cheat... but in a good way.
The standard library is made to be used by the commoners.
The noble ladies and gentlemen in the standard committee stated some constraints to protect us from killing ourselves while using their <code>std::unordered_map</code>.
May this over-protective behaviour be damned! We know better!</p>
<h3>Breaking stable addressing</h3>
<p>The standard implicitly mandates <strong>stable addressing</strong> for any implementation of <code>std::unordered_map</code>.
<strong>Stable addressing</strong> means that the insertion or deletion of a <strong>key/value pair</strong> in a <code>std::unordered_map</code> should not affect the memory location of other <strong>key/value</strong> pairs in the same <code>std::unordered_map</code>. More precisely, the standard does not mention, in the <a href="https://eel.is/c++draft/unord.map.modifiers">Effects</a> sections of std::unordered_map's modifiers, anything about reference, pointer or iterator invalidation.</p>
<p>This forces any standard implementation of <code>std::unordered_map</code> to use linked-list for the buckets of pairs rather than contiguous memory.
<code>std::unordered_map</code> should look roughly like this in memory:</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/unordered_map_layout.png" alt="unordered_map layout"/></center></p>
<p>As you can see, the <strong>entries</strong> are stored in a giant <strong>linked-list</strong>. Each of the <strong>buckets</strong> are themselves <strong>sub-parts</strong> of the linked-list.
Here each colors represent different buckets of key/value pairs.
When a <strong>key/value pair</strong> is <strong>inserted</strong>, the <strong>key</strong> is somehow <strong>hashed</strong> and adjusted (usually using modulo on the amount of buckets) to find which bucket it belongs to, and the key/value pair gets inserted into that bucket.
Here, the <strong>key1</strong> and <strong>key2</strong> somehow ended-up belonging to the <strong>bucket 1</strong>.
Whereas the <strong>key3</strong> belongs to the <strong>bucket 5</strong>.<br>
When doing a lookup using a key, the key is hashed and adjusted to find the bucket it should belong to.
The bucket of the key is <strong>iterated</strong> until the key is found or the end of the bucket is reached (meaning no key is in the map).
Finally, the buckets are linked between each others to be able to do a traversal of all the key/value pairs within the <code>std::unordered_map</code>.</p>
<p>This memory layout is <strong>really bad</strong> for your CPU!
Each of the nodes of the linked-list(s) could be spread across memory and that would play against all the caches of your CPU.
Traversing buckets made of a linked-list is slow, but you could pray that your hash function save you by spreading keys as much as possible and therefore have tiny buckets.
But even the most brilliant hash function will not help you with a common use-case of an associative container: iterating through all the key/value pairs.
Each dereference of the pointer to the next node will be a huge drag on your CPU.
On the other hand, since each node are separately allocated, they can stay wherever they are in memory even if others are added or removed, which provides <strong>stable addressing</strong>.</p>
<p>So what could we obtain if we were to free ourselves from <strong>stable addressing</strong>?
Well, we could wrap our buckets into contiguous memory like so:</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/dense_hash_map_layout.png" alt="dense_hash_map layout"/></center></p>
<p>Here we are still keeping a linked-list for each buckets, but all the nodes are stored in a vector, therefore one after each others in memory.
Let's call this a <strong>dense hash map.</strong>
Instead of using pointers between nodes, we are expressing their relations with indexes within the vector: here the node with <strong>key1</strong> store a "next index" having a value of <strong>2</strong> which is the index of the node with <strong>key2</strong>. And all of that is a huge improvement! We are gaining on all fronts:</p>
<ul>
<li>Iterating over all the key/value pairs is as fast as iterating over a vector, which is lightning fast.</li>
<li>We are saving a pointer on all nodes - the "prev pointer". We don't need any sort of reverse-traversal of a given bucket, but just a global reverse-traversal of all buckets. </li>
<li>We don't need to maintain a begin and end pointer for the list of nodes.</li>
<li>Even iterating over a bucket could be faster as the node shouldn't be too scattered in memory since they all belong to one vector.</li>
</ul>
<p>All these new properties have a lot of use-cases in the domains I dabble with.
For instance, the <a href="https://en.wikipedia.org/wiki/Entity_component_system">ECS (Entity-Component-System) pattern</a> often demands a container where you can do lookup for a component associated to a given entity and at the same being able to traverse all components in one-shot. </p>
<p>With that said, the <strong>stable addressing</strong> is now gone: any insertion into the vector could produce a reallocation of its internal buffer, ending in a massive move of all the nodes across memory. So what if your user really need stable addressing? As <strong>David Wheeler</strong> would say: "just use another level of indirection".
Instead of a using a <code>dense_hash_map<Key, Value></code>, your user can always use a <code>dense_hash_map<Key, unique_ptr<Value>></code>: </p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/dense_hash_map_unique_ptr_layout.png" alt="layout"/></center></p>
<p>We are reintroducing pointers, which is obviously not great for the cache again.
But this time when iterating over all the key/value pairs you will very likely see a substantial improvement over the first layout.
The pattern of the nodes is clearly more predictable and prefetching abilities of your CPU may come to your help. </p>
<p>There a lot more layouts for hash tables that I did not mention here and could have suited my needs.
For instance, any of the <a href="https://en.wikipedia.org/wiki/Hash_table#Open_addressing">open-addressing</a> strategies could bring their own pros and cons.
Once again, if you are interested, there are a plethora of cppcon talks on that subject.</p>
<h4>Caveats</h4>
<p>By breaking stable-addressing, we cannot retain the address or the reference of a key-value pair from our map carelessly.
Some operations like adding or removing a key/value pair in the map can provoke invalidation of references or pointers to the other pairs.
This is the kind of danger I am speaking of:</p>
<div class="highlight"><pre><span></span><code><span class="n">jg</span><span class="o">::</span><span class="n">dense_hash_map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">hero</span><span class="o">></span> <span class="n">my_map</span><span class="p">{{</span><span class="s">"spongebob"</span><span class="p">,</span> <span class="n">hero</span><span class="p">{}},</span> <span class="p">{</span><span class="s">"chuck norris"</span><span class="p">,</span> <span class="n">hero</span><span class="p">{}}};</span>
<span class="k">auto</span><span class="o">&</span> <span class="n">chuck_ref</span> <span class="o">=</span> <span class="n">my_map</span><span class="p">[</span><span class="s">"chuck norris"</span><span class="p">];</span>
<span class="n">my_map</span><span class="p">.</span><span class="n">erase</span><span class="p">(</span><span class="s">"spongebob"</span><span class="p">);</span> <span class="c1">// Invalidate all the nodes after spongebob.</span>
<span class="n">chuck_ref</span><span class="p">.</span><span class="n">punch_your_face</span><span class="p">();</span> <span class="c1">// One should never try to retain "chuck norris" as a ref.</span>
</code></pre></div>
<p>Due to the call to erase, "chuck norris" is moved into the first slot of the internal vector, which ultimately makes chuck's address change.
Accessing "chuck norris" with a previously acquired reference becomes a danger.
While this can surprise at first, it is no different than trying to retain a reference to a value in a vector.
And once again, if you need stable addressing, you are welcome to store <strong>unique_ptrs</strong> within your map.</p>
<p>For the same stable-addressing reasons, implementing the <strong>node API of std::unordered_map</strong> like <a href="https://en.cppreference.com/w/cpp/container/unordered_map/extract">extract</a> becomes very challenging. To be fair, I have never seen a real usage of that API in any project.
It might be useful in scenarios where you run long instances of backend-applications and want to move things across maps without a huge performance impact.
If by any chance, you have witnessed a usage of that feature, it would be a pleasure to have some explanations in the comments section.</p>
<p>Finally, as we will see in a later post, the <strong>erase</strong> operation of such dense_hash_map becomes <strong>slower</strong>. For any erase operation, one <strong>extra swap operation</strong> is needed on top of the destruction and deallocation ones. Thankfully, a standard usage of a map will have little erase operations compared to lookup or even insertion operations. Speaking of which, how could we improve on those operations?</p>
<h3>Faster modulo operation</h3>
<p>As I mentioned previously, a lookup will require that your key is hashed and adjusted with modulo to fit in the amount of buckets available.
The amount of buckets is changing depending on how many key/value pairs are stored in your map. The more pairs, the more buckets.
In a <code>std::unordered_map</code>, the growth is triggered every time the <a href="https://en.cppreference.com/w/cpp/container/unordered_map/load_factor">load factor</a> (average number of elements per bucket) is above a certain <a href="https://en.cppreference.com/w/cpp/container/unordered_map/max_load_factor">threshold</a>.
Adjusting your hash with a "simple modulo operation" is as it turns out not a trivial operation for your hardware.
Let's <a href="https://godbolt.org/">godbolt</a> a bit! </p>
<p>If we write a simple modulo function, this is what godbolt gives to us (on GCC 9.0 with -O3):</p>
<div class="highlight"><pre><span></span><code><span class="c1">// C++</span>
<span class="kt">int</span> <span class="nf">modulo</span><span class="p">(</span><span class="kt">int</span> <span class="n">hash</span><span class="p">,</span> <span class="kt">int</span> <span class="n">bucket_count</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">hash</span> <span class="o">%</span> <span class="n">bucket_count</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// x86 assembly</span>
<span class="nf">modulo</span><span class="p">(</span><span class="no">int</span><span class="p">,</span> <span class="no">int</span><span class="p">):</span>
<span class="nf">mov</span> <span class="no">eax</span><span class="p">,</span> <span class="no">edi</span> <span class="c1">// Prepare the hash into eax</span>
<span class="nf">cdq</span> <span class="c1">// Prepare registers for an idiv operation. </span>
<span class="nf">idiv</span> <span class="no">esi</span> <span class="c1">// Divide by esi which contains bucket_count</span>
<span class="nf">mov</span> <span class="no">eax</span><span class="p">,</span> <span class="no">edx</span> <span class="c1">// Get the modulo value that ends-up in edx and return it into eax</span>
<span class="nf">ret</span>
</code></pre></div>
<p>So far things look rather good. The <code>idiv</code> operation seems to do most of the work by itself.
But, what if we made bucket_count a constant? Having more information at compile-time should help the compiler, isn't it?</p>
<div class="highlight"><pre><span></span><code><span class="c1">// C++</span>
<span class="kt">int</span> <span class="nf">modulo</span><span class="p">(</span><span class="kt">int</span> <span class="n">hash</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">index</span> <span class="o">%</span> <span class="mi">5</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// x86</span>
<span class="nf">modulo</span><span class="p">(</span><span class="no">int</span><span class="p">,</span> <span class="no">int</span><span class="p">):</span>
<span class="nf">movsx</span> <span class="no">rax</span><span class="p">,</span> <span class="no">edi</span>
<span class="nf">mov</span> <span class="no">edx</span><span class="p">,</span> <span class="no">edi</span>
<span class="nf">imul</span> <span class="no">rax</span><span class="p">,</span> <span class="no">rax</span><span class="p">,</span> <span class="mi">1717986919</span> <span class="c1">// LOTS OF SHENANIGANS GOING HERE.</span>
<span class="nf">sar</span> <span class="no">edx</span><span class="p">,</span> <span class="mi">31</span>
<span class="nf">sar</span> <span class="no">rax</span><span class="p">,</span> <span class="mi">33</span>
<span class="nf">sub</span> <span class="no">eax</span><span class="p">,</span> <span class="no">edx</span>
<span class="nf">lea</span> <span class="no">eax</span><span class="p">,</span> <span class="p">[</span><span class="no">rax</span><span class="err">+</span><span class="no">rax</span><span class="p">*</span><span class="mi">4</span><span class="p">]</span>
<span class="nf">sub</span> <span class="no">edi</span><span class="p">,</span> <span class="no">eax</span>
<span class="nf">mov</span> <span class="no">eax</span><span class="p">,</span> <span class="no">edi</span>
<span class="nf">ret</span>
</code></pre></div>
<p>Interesting! The compiler is spitting a lot more operations. Wouldn't it be simpler to keep <code>idiv</code> here? A single operation... <br>
Believe it or not, your compiler is as smart as a whip and wouldn't do this extra work without significant gains.
So we can clearly extrapolate that our innocent <code>idiv</code> must be seriously costly if it happens to be less efficient than a couple of other operations.</p>
<p>So how can we optimize this modulo operation without using <code>idiv</code> or a constant?
We can use bitwise operations and restrict ourselves to sizes made of power of two.
Assuming that <code>y</code> is a power of two in the expression <code>x % y</code>, we can replace this expression with: <code>x & (y - 1)</code>.
You can think of this bitwise operation as a "filter" on the lower bits of a number, which happen to be the same as a modulo operation when it comes to power of two.
So what do we obtain in this conditions?</p>
<div class="highlight"><pre><span></span><code><span class="c1">// C++</span>
<span class="kt">int</span> <span class="nf">fast_module</span><span class="p">(</span><span class="kt">int</span> <span class="n">hash</span><span class="p">,</span> <span class="kt">int</span> <span class="n">bucket_count</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">(</span><span class="n">hash</span> <span class="o">&</span> <span class="p">(</span><span class="n">bucket_count</span> <span class="o">-</span> <span class="mi">1</span><span class="p">));</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="c1">// x86</span>
<span class="nf">fast_module</span><span class="p">(</span><span class="no">int</span><span class="p">,</span> <span class="no">int</span><span class="p">):</span>
<span class="nf">lea</span> <span class="no">eax</span><span class="p">,</span> <span class="p">[</span><span class="no">rsi-1</span><span class="p">]</span>
<span class="nf">and</span> <span class="no">eax</span><span class="p">,</span> <span class="no">edi</span>
<span class="nf">ret</span>
</code></pre></div>
<p>Not only we have less operations, but <code>lea</code> (compute a memory address) and <code>and</code> have much less associated cost than <code>idiv</code>.
This micro-optimisation may look a bit far-fetched, but it actually matters a lot, as proved by <a href="https://www.youtube.com/watch?v=M2fKMP47slQ">Malte Skarupke</a>, when it comes to lookup operations.</p>
<p>To further convince you, I have run both implementations of modulo on <a href="http://quick-bench.com/XZGeHdIRvI8tjsgHHmPP22bN8us">quick-bench</a>.
And here are the results:</p>
<p><center><img width=70% height=70% src="https://jguegant.github.io/blogs/tech/images/modulo-benchmarks.png" alt="layout"/></center></p>
<p>There is no doubts, using <strong>bit operations</strong> instead of <strong>modulo</strong> is indeed faster!</p>
<p>Note: For the purists, the so-called modulo operator <code>%</code> in C++ is actually an integer remainder operator since it does not handle negative value as a mathematical modulo operation would. One can only imagine how costly a proper modulo would be... </p>
<h4>Caveats</h4>
<p>Unlike the previous improvement, this one does not seem to collide too much with the holly standard. So how come it is not used in every implementations?
Well, fiddling with bit operations comes with few drawbacks.</p>
<p>Firstly, the growth of the container for the buckets must always be done following a power of two. This limits you on how creative you can be with your growth strategy. In theory, this could also create subtle performance issues in some scenarios, like <a href="https://tech.labs.oliverwyman.com/blog/2013/10/08/cpu-cache-collisions-in-the-context-of-performance/">cache line collisions</a>. In practice, you should never have an access pattern that triggers such an issue on the container of buckets. It is not like you are going to iterate over that container at any time.</p>
<p>Secondly, in a subtle way you are making the first bits of your keys have more importance than they should.
Let's assume that you have some bit flags (flag sets) as a key:</p>
<div class="highlight"><pre><span></span><code> <span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="kt">uint32_t</span> <span class="n">a_precious_flag</span> <span class="o">=</span> <span class="mi">1</span> <span class="o"><<</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="kt">uint32_t</span> <span class="n">another_flag</span> <span class="o">=</span> <span class="mi">1</span> <span class="o"><<</span> <span class="mi">1</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="kt">uint32_t</span> <span class="n">yet_another_flag</span> <span class="o">=</span> <span class="mi">1</span> <span class="o"><<</span> <span class="mi">4</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">a_set_of_flags</span> <span class="o">=</span> <span class="n">yet_another_flag</span> <span class="o">|</span> <span class="n">a_precious_flag</span> <span class="o">|</span> <span class="n">another_flag</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">another_set_of_flags</span> <span class="o">=</span> <span class="n">a_precious_flag</span> <span class="o">|</span> <span class="n">another_flag</span><span class="p">;</span>
<span class="n">jg</span><span class="o">::</span><span class="n">dense_hash_map</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="kt">int</span><span class="o">></span> <span class="n">my_map</span><span class="p">;</span> <span class="c1">// A map with a buckets container of size 4 (2 ^ 2).</span>
</code></pre></div>
<p>Now, if you were to take these two flag sets and feed them directly to the modulo operation of the map, they would end-up in the same bucket simply because both of them have <code>a_precious_flag</code> and <code>another_flag</code> on. This gives way too much importance to the bits that <code>a_precious_flag</code> and <code>another_flag</code> represent.
The fact that we are using a power two (4 in this case) will always make the lower bits very significant.
It is not often that you store flag sets or bit fields as keys in a map, I will give you that, but as a good practice you should:</p>
<ul>
<li>Remind yourself not to create a corner-case with bits more important than others in your key.</li>
<li>And if it happens, you should re-write your hash function to shuffle your bits around. </li>
</ul>
<h2>First conclusion</h2>
<p>Now we have a grand master plan to beat <code>std::unordered_map</code> by removing some assumptions the standard had to make to be preserve us from some dangers.
The hacks used are not even too difficult to grasp, so where is the difficulty in writing new containers? Well, some creepy C++ features are waiting for us around the corner.</p>
<p>Be ready for the next phase: <a href="https://jguegant.github.io/blogs/tech/dense-hash-map2.html">Part 2 - Growth Policies & The Schrodinger std::pair</a>!</p>Trip report - Meeting C++ 20192019-11-29T22:30:00+01:002019-11-29T22:30:00+01:00Jean Gueganttag:jguegant.github.io,2019-11-29:/blogs/tech/trip-report-meetingcpp-2019.html<p>Time to go back to the roots!
Me and a few colleagues have been travelling across the old continent to attend the renowned <a href="https://meetingcpp.com/">Meeting C++</a> conference in Berlin.<br>
Fully backed by our employer, <a href="https://discover.king.com/about/">King</a>, we were able to enjoy a yearly dose of C++ talks at an excellent venue.
I …</p><p>Time to go back to the roots!
Me and a few colleagues have been travelling across the old continent to attend the renowned <a href="https://meetingcpp.com/">Meeting C++</a> conference in Berlin.<br>
Fully backed by our employer, <a href="https://discover.king.com/about/">King</a>, we were able to enjoy a yearly dose of C++ talks at an excellent venue.
I was thrilled at the idea of retrying the "German C++ experience" and my expectation were once again met!
<strong>C++20</strong> being right at the corner, I also had in mind to prioritise talks on this topic as much as possible.</p>
<p>As the tradition goes, I will relate a bit my experience of the event and dress-up a list of the talks I would highly suggest to search for! </p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/berlin-night.jpg"/></center></p>
<h1>A tour of Meeting (2019):</h1>
<p>I did a decent <a href="https://jguegant.github.io/blogs/tech/trip-report-meetingcpp-2017.html">introduction of the event two years ago</a> and most of it stand true.
Rather than doing another summary, I will compare and contrast my experience at <strong>Meeting C++</strong> with the one I had last year at <strong>Cppcon</strong>. </p>
<h2>The venue:</h2>
<p>When it comes to the facilities of the venue, I found the rooms in the Andels Hotel to have unequal quality.
While the two big rooms on the lower floors are excellent, the upper floor ones are really long and narrow.
If the nature made you of a short stature or you don't have the best eyesight (or both like me), you will have a hard time following some of the slides.
I cannot remember having any issue with that at <strong>CppCon</strong>. Having rooms with inclined rows helps a lot!</p>
<p>Note: apparently, the organisers are well-aware of this issue and may come up with a nice solution for the next round of Meeting C++! </p>
<p>Probably to avoid having crammed rooms and irritating noises, <strong>Meeting C++</strong> also has this concept of closing doors of the tracks as soon as the talk starts and not letting annoying enter. I am not quite convinced by this idea, it extremely frustrating to not be able to move to another talk if the current one is not to your taste. </p>
<h2>The German cuisine:</h2>
<p>Being French, a detail often comes to my mind... the food provided during the event!
Unlike <strong>CppCon</strong>, this event offers catering all day long, and tasty one mind you.
Obviously you cannot expect conference meals to deserve a <a href="https://en.wikipedia.org/wiki/Michelin">Michelin stars's</a>, but there was plenty and with a lot of variety. Is that a must-do for a conference? Probably not, but it is both really convenient and it also brings a LOT of interactions between members of the community. People will tend to have more casual conversations if they are dinning in the same place rather than going out. From what I can recall, <strong>CppCon</strong> provided snacks here and there, but nothing worth remembering.</p>
<h2>The talks:</h2>
<p>Last but not least, the speakers! On one hand, <strong>Meeting C++</strong> has more diversity: the speakers are coming from all around Europe and you may discover a lot of new names and some hidden gems that didn't/wouldn't make it to <strong>CppCon</strong> - not everyone is willing or can afford to travel to the USA. On the other hand, the amount of C++ committee members and famous speakers per square kilometre felt higher at <strong>CppCon</strong> which probably made the talks of a slightly better quality overall.</p>
<p><strong>Meeting C++</strong> offers you the possibility to go to four different tracks for a given time slot.
It is enough choices that you will always find a talk to your liking.
The talks are also explicitly tagged with a level of experience required to enjoy them: beginner to advanced.
Once again, going only to talks with topics that you like is often not the best strategy to fully enjoy this conference.
It can be very rewarding to go out of your comfort zone and try a talk not a single word on its title make sense to you.
Likewise, do not underestimate the quality of beginner talks. You may learn more than you would expect!</p>
<p>I have had a first hand, albeit short, experience on what it takes to be a speaker at such an event.
I gave a lightning talk highly inspired on the <a href="https://jguegant.github.io/blogs/tech/trip-report-meetingcpp-2017.html">performing try emplace post</a> I did at the beginning of this year.
While it went rather smoothly (if not for a small technical issue with my slides), I felt intimidated when I went on stage in front of maybe 300 people (best-guess estimation). </p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/my-lightning-talk-meeting-cpp-2019.jpg"/></center></p>
<p>You can find a pdf version of the slides right <a href="https://jguegant.github.io/blogs/tech/images/associative-containers-the-art-of-inserting-gracefully.pdf">here</a> and maybe a YouTube record of that talk will pop at one point on the <a href="https://www.youtube.com/user/MeetingCPP">Meeting C++ channel</a>.
It was interesting to see that the <strong>lightning talks</strong> were a lot more formal than at <strong>CppCon</strong> where jokes and bad-puns are a constant.
I believe that the lightning talks format was changed recently at <strong>Meeting C++</strong>: it used to be its own dedicated track on a full afternoon, and it became a more casual evening event. We, the lightning speakers, were also informed a bit late on where, when and how should the talks happen.</p>
<p>Doing a presentation on an innovative topic, interesting and understandable by all requires a lot of upfront work.
Having to do it in a foreign language, for one hour, in front of a rather massive crowd is even more demanding.
I am still amazed by what all these speakers gave us during these three days! </p>
<h1>The chief's suggestions of the year:</h1>
<p>Here comes again the menu of the talks I particularly enjoyed. The legend follow the usual rules:</p>
<ul>
<li>💀 : The difficulty of the talk (💀: Beginner friendly, 💀💀: Intermediate, 💀💀💀: High exposure to C++'s dark corners)</li>
<li>★ : My interest for the talk (★: Good talk, ★★: Tasty talk, ★★★: Legendary talk)</li>
</ul>
<p>No need to say that I will not spoil everything in the talks, but I will simply try to give an idea of what the talk was about. If the talk seems to be of your taste, I highly encourage you to watch the video record of it or the slides.
If your talk or a talk that you liked very much is not part of this menu, do not feel too disenchanted about it.
I had to curate this list to a few talks or I would have to write an entire novel.
Feel free to post your opinion about the talks in the comment section.</p>
<h3>[Keynote] Design Rationale for <chrono> - Howard Hinnant - 💀💀★★:</h3>
<ul>
<li>Slides: <a href="https://meetingcpp.com/mcpp/slides/2019/Hinnant.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>The <strong>Howard Hinnant</strong>, himself, came to present his new-born baby: the support of dates, calendars and time-zones in the <a href="https://en.cppreference.com/w/cpp/chrono">chrono header</a>.
For those out-of-the-loop, <a href="https://howardhinnant.github.io/HowardHinnant.html">Howard Hinnant</a> has been a massive contributor to what C++ is nowadays.
He is known for bringing <strong>move semantics</strong> to the language, introducing <code>std::unique_ptr</code>, being lead author of <strong>libc++</strong>... a myriad of things... and the notion of time in the standard through the header <code><chrono></code>. It should not come as surprise that Howard is also the man being the dates and calendars coming to the new C++ standard. </p>
<p>Howard's talk was very educating on this addition to <strong>C++20</strong>. Within two hours, I had a fairly good overview of what we will get and more importantly an answer to why <strong>chrono</strong> is so verbose.
Here is the gist of what we will have:</p>
<ul>
<li><a href="https://en.cppreference.com/w/cpp/chrono/time_point">std::chrono::time_point</a> (already in C++11) get a specialized alias <code>sys_days</code> equal to <code>std::chrono::time_point<std::chrono::system_clock, std::chrono::days></code>. In other word <code>sys_days</code> represent a time point using the system clock and expressed in days. This <code>sys_days</code> can be thought as an integer representing a value in days.</li>
<li>We can convert this <code>sys_days</code> time point into a <a href="https://en.cppreference.com/w/cpp/chrono/year_month_day">year_month_day</a> representation and vice-versa.
This is the same representation of that time point but in a much more human-friendly way: it uses the Gregorian calendar to express that point in time.
A time point in the Gregorian calendar is made of three parts: a year, a month and a day of the month.
The new standard offers lots of arithmetic operators, conversion operators, helpers structs... to manipulate these parts easily.</li>
<li>A new type, with a self-explanatory name, <a href="https://en.cppreference.com/w/cpp/chrono/time_zone">std::chrono::time_zone</a> has been introduced.
If you combine a <code>sys_days</code> with a <code>time_zone</code>, you obtain a <code>zoned_time</code>. <code>zoned_time</code> handles conversion from one <code>time_zone</code> to another for you.
It is highly convenient! </li>
<li>Finally, almost all the types in the <code><chrono></code> header have conversion facilities to and from strings. It even works with the new <a href="https://en.cppreference.com/w/cpp/utility/format">std::format</a> library. </li>
</ul>
<p>Equipped with this extended <code><chrono></code> header, you can easily write such things:</p>
<div class="highlight"><pre><span></span><code><span class="k">for</span> <span class="p">(</span><span class="k">auto</span> <span class="n">d</span> <span class="o">=</span> <span class="n">January</span><span class="o">/</span><span class="mi">9</span><span class="o">/</span><span class="mi">2019</span><span class="p">;</span> <span class="n">d</span><span class="p">.</span><span class="n">year</span><span class="p">()</span> <span class="o"><</span> <span class="mi">2020</span><span class="n">y</span><span class="p">;</span> <span class="n">d</span> <span class="o">=</span> <span class="n">sys_days</span><span class="p">{</span><span class="n">d</span><span class="p">}</span> <span class="o">+</span> <span class="n">weeks</span><span class="p">{</span><span class="mi">2</span><span class="p">})</span> <span class="p">{</span>
<span class="n">zoned_time</span> <span class="n">london</span><span class="p">{</span><span class="s">"Europe/London"</span><span class="p">,</span> <span class="n">local_days</span><span class="p">{</span><span class="n">d</span><span class="p">}</span> <span class="o">+</span> <span class="mi">18</span><span class="n">h</span><span class="p">};</span>
<span class="n">cout</span> <span class="o"><<</span> <span class="n">london</span> <span class="o"><<</span> <span class="sc">'\n'</span><span class="p">;</span>
<span class="n">cout</span> <span class="o"><<</span> <span class="n">zoned_time</span><span class="p">{</span><span class="s">"America/New_York"</span><span class="p">,</span> <span class="n">london</span><span class="p">}</span> <span class="o"><<</span> <span class="s">"</span><span class="se">\n\n</span><span class="s">"</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>In this example, we are displaying the time in "New York" every two weeks, starting from the 9th of January, for the year 2019, when it is 18.00 in "London".
As Howard pointed out, this is far from being trivial to do by hand: both the UK and USA have different daylight savings rules.</p>
<p>Some people may complain of the <strong>verbosity</strong>. Firstly, I find the code very readable even without prior knowledge of this new part of <code><chrono></code>
Second, <strong>safety</strong> was a key-point in Howard's design. At multiple time, he gave us example on how his design prevents wrongdoings at compile-time.
For instance: what is a time difference? Can this be considered a date? Can <code>2 hours</code> be translated to a date in a calendar? Probably not! </p>
<p>I am quite eager to get this part of C++20! </p>
<h3>[Talk] 10 techniques to understand existing code - Jonathan Boccara - 💀★★★:</h3>
<ul>
<li>Slides: <a href="https://meetingcpp.com/mcpp/slides/2019/understand_code.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p><strong>Jonathan Boccara</strong>, known for his awesome blog <a href="https://www.fluentcpp.com/">fluentcpp.com</a>, is always a safe bet when it comes to C++ talks.
You should not necessarily expect learning bleeding-edge techniques with Jonathan, but he is extremely talented at putting words on things!</p>
<p>This time, he was presenting us his favourite approaches to code you have never seen before. Especially, when it comes to intricate code-bases.
My favourite analogy was the <strong>stronghold</strong> one: start your exploration of the code from a place (a function, a routine...) that is crucial in the code-base and that you can easily understand. Like a <strong>fog of war</strong> in a video-game, the rest of the code is unknown to you at first. But slowly, you can explore the rest of the map/code-base by sending minions/yourself into the callers or the callees related to that function. One strategy to know where to "expand your territory for a maximum of profit" is by looking at <strong>call stacks</strong>: by jumping in the frames above the current function, this can give you an idea of what is the critical path of your program.</p>
<p>Here is an awesome representation, from Jonathan, of <strong>call stacks</strong> in a video-game universe:</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/call-stack-fog-of-war.png"/></center></p>
<p>Not only this applies to decipher a code-base, but it is also a very efficient technique in <strong>reverse engineering</strong>.
When trying to reverse a native application, it is often quite rewarding to put a breakpoint on the system calls: these are your strongholds.
Without source-code, you have very little places where you know what the assembly you are reading does.
System calls are well documented and exposed, so it is naturally a good place to start.</p>
<p>I believe that I am already applying most of these techniques on a daily-basis.
But if I had to explain to a junior programmer what programming on a large code-base is all about,
I would probably recommend that person to have a look at this talk.</p>
<h3>[Talk] Testing Legacy Code - Fuzzing for Better Input Data - Tina Ulbrich, Niel Waldren - 💀💀★★★:</h3>
<ul>
<li>Slides: <a href="https://meetingcpp.com/mcpp/slides/2019/Testing%20Legacy%20Code%20-%20Fuzzing%20for%20Better%20Input%20Data.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>During such conferences, you may end-up for various reasons (entering the wrong door, following someone, not getting on time to a room...) into a talk that you did not expect on your schedule. Sometimes, you may regret your unfortunate choice as much as <code>std::vector<bool></code> being in the standard, but it can also turn out into your favour: you may discover one of the best presentation of that day. This is exactly what happened to me with the talk from <strong>Tina Ulbrich</strong> and <strong>Niel Waldren</strong> about <a href="https://en.wikipedia.org/wiki/Fuzzing">fuzzing</a> and unit-testing.</p>
<p>Usually, <strong>fuzzing</strong> consists in feeding an application or part of an application with pseudo-random data to explore as much code-path as possible.
The end-goal is to find code-paths that lead to crashes or bugs which can be exploit for malicious purposes.<br>
Once an exploit found, the fuzzer will often try to reduce the "buggy input" to a minimal set to clearly isolate the corner case.</p>
<p>I have heard about fuzzing for a while, but I never had the opportunity to use it myself.
This technique is really appealing if you are working in the security business or if your application is critical part of system.
If you are developing an application a bit less demanding on the stability, like video-games, fuzzing is not necessarily the number one priority.
<strong>Tina</strong> and <strong>Niel</strong> twisted the usage of fuzzing to find exploits to improve the test-coverage of their library.
They used <a href="https://llvm.org/docs/LibFuzzer.html">libFuzzer</a> which is, from what I understand, a LLVM library that combine <a href="https://clang.llvm.org/docs/SanitizerCoverage.html">LLVM's code coverage tool - SanitizerCoverage</a> with a fuzzer engine like <a href="http://lcamtuf.coredump.cx/afl/">AFL</a>.
<strong>libFuzzer</strong> will try to maximize the code-coverage with the minimum input data.
This is really appealing for all projects relying on unit-tests. </p>
<p><strong>Tina</strong> and <strong>Niel</strong> did a step by step explanation on how to use <strong>libFuzzer</strong> and what are its benefits:</p>
<ul>
<li>How to tweak the random data you receive to valid parameters.</li>
<li>How to reuse the best input data corpus to write new unit-tests for your application.</li>
<li>Why you can use this to detect very subtle changes in your API's behaviour. Changes that you would not necessarily catch with usual unit-tests.</li>
<li>A lot more...</li>
</ul>
<p>I will have a second watch of their talk as soon as their video is available on the conference's Youtube channel.
It was a very innovative topic to me!</p>
<h3>[Talk] Modules - The Beginner's Guide - Daniela Engert - 💀💀★★:</h3>
<ul>
<li>Slides: <a href="https://meetingcpp.com/mcpp/slides/2019/modules-the-beginners-guide-meetingcpp2019.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>Out of the three major language features coming to C++20 (concepts, coroutines and modules), I was the least familiar with <strong>modules</strong>.
The committee is often very active on the standard until the last months before the release of a version.
While I had a rough idea of module were about in C++, I did not bother to learn about the syntax and the exact implementation.
You never know if the feature you read about in a white paper while look any different in the actual standard!
So, I was very grateful that <a href="https://github.com/DanielaE">Daniela Engert</a> came up with that talk right before 2020.
Being so close to the standard release, the module part was unlikely to change too much! </p>
<p>So what are modules in a few words?
Traditionally, a C++ code-base is separated into multiple <strong>compilation units</strong> which are in most case your <code>.cpp</code> files.
These compilation units are <strong>compiled</strong> separately and <strong>combined</strong> by the <strong>linker</strong>.
If you want to share functions or objects between two compilation units, you must have a <strong>common interface</strong> for them: one or more <strong>header files</strong> (.h/.hpp) that declare what is available.
The problem is that header files are:</p>
<p>1) Shared in a very primitive way. The <code>#include</code> directive is doing dumb copy-paste of the header content into your cpp file.
2) Often full of complicated content for the compiler and not just few declarations. In those, you can have macros, templates, inline functions, include of includes... </p>
<p>This results in your compiler doing a lot of unnecessary work parsing these headers for all the compilation units.
Wouldn't it be better if compilation units themselves could expose directly what they provide instead of using these hackish headers?
This is exactly what modules try to solve!</p>
<p>So writing a very basic module becomes as simple as writing a <code>.cpp</code> file with few annotations to expose what we want:</p>
<div class="highlight"><pre><span></span><code><span class="k">export</span> <span class="k">module</span> <span class="n">my</span><span class="p">.</span><span class="k">module</span><span class="p">;</span> <span class="c1">// This is the name of our module</span>
<span class="k">export</span> <span class="kt">int</span> <span class="n">e</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span> <span class="c1">// Expose a variable.</span>
<span class="k">export</span> <span class="kt">int</span> <span class="nf">bar</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">foo</span><span class="p">(</span><span class="n">e</span><span class="p">);</span> <span class="c1">// Expose a function.</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="nf">foo</span><span class="p">(</span><span class="kt">int</span> <span class="n">x</span><span class="p">){</span> <span class="k">return</span> <span class="n">x</span><span class="p">;</span> <span class="p">}</span> <span class="c1">// No export keyword == no exposition.</span>
</code></pre></div>
<p>When compiling this <code>.cpp</code> file, this will create two others files: an object file that contains binary code (OBJ) and a file for interfacing with that module called Binary Module Interface. Unlike a header file, the BMI file and its associated OBJ file can have a highly <strong>optimized representation</strong> of what is available in the module. </p>
<p>As a user of a module into another compilation unit, you will pass this BMI file as parameter to your compiler and write an import statement:</p>
<div class="highlight"><pre><span></span><code><span class="k">import</span> <span class="k">module</span> <span class="n">my</span><span class="p">.</span><span class="k">module</span><span class="p">;</span> <span class="c1">// Ask the compiler to use that module in here.</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">bar</span><span class="p">();</span> <span class="c1">// This is using bar from the other module.</span>
<span class="p">}</span>
</code></pre></div>
<p>And voilà! These are the new C++ module in all their glory.
Except that it gets quite a lot more complicated when you are mixing old header includes, namespaces, special support for standard headers...
<strong>Daniela</strong> was really good at explaining all these quirks that you may encounter in the rough transition that will happen from headers to modules.</p>
<p>As for performance, she observed an improvement from 1546 milliseconds to 62 milliseconds when using a library as a module on a huge-scale project at work.
This gives a lot of hope on what modules will offer to us when available on all major compilers!</p>
<h3>[Talk] C++20 The small things - Timur Doumler - 💀★★:</h3>
<ul>
<li>Slides: <a href="https://meetingcpp.com/mcpp/slides/2019/talk.pdf">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=Xb6u8BrfHjw">cppcon link</a></li>
</ul>
<p>You probably have heard of the major features coming to C++20: concepts, coroutines and modules. And you would think that this is enough on your C++ plate for a few years.
Well, even without these three features, C++20 still has plenty to offer. This is what <a href="https://twitter.com/timur_audio">Timur Doumler</a> demonstrated to us for one hour! </p>
<p>Amongst a plethora of small improvements, here are my favourite so far:</p>
<h4>Designated initialisers</h4>
<p>Let's imagine that you have a <code>struct</code> with quite a few members in it. Now let's pretend that we want to initialize this structure with a few variables using the <a href="https://en.cppreference.com/w/cpp/language/aggregate_initialization">aggregate initilization from C++11</a>.
What are the chances that you assign the correct members from the first try? </p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">my_struct</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">a1</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">a2</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b2</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b3</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">c</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="p">};</span>
<span class="n">my_struct</span> <span class="n">s</span><span class="p">{</span><span class="mi">42</span><span class="p">,</span> <span class="mi">43</span><span class="p">,</span> <span class="mi">1337</span><span class="p">,</span> <span class="mi">54</span><span class="p">,</span> <span class="cm">/*...*/</span><span class="p">};</span>
</code></pre></div>
<p>I tell you, the chances are low. You will have to double-check the <code>struct</code> definition more than once to order things accurately!
So what if you could specify which member you designate? This is where C++20 comes to save the day:</p>
<div class="highlight"><pre><span></span><code><span class="n">my_struct</span> <span class="n">s</span><span class="p">{.</span><span class="n">a1</span><span class="o">=</span><span class="mi">42</span><span class="p">,</span> <span class="p">.</span><span class="n">b</span><span class="o">=</span><span class="mi">43</span><span class="p">,</span> <span class="p">.</span><span class="n">a2</span><span class="o">=</span><span class="mi">1337</span><span class="p">,</span> <span class="p">.</span><span class="n">b2</span><span class="o">=</span><span class="mi">54</span><span class="p">,</span> <span class="cm">/*...*/</span><span class="p">};</span>
</code></pre></div>
<p>I can hear YOU the C programmer, this has been in C99 for a while...
And you will also point with your smug face that you must respect the order (by appearance in the definition) of the members when using C++'s designated initializers unlike C.
It happens that C++ has much more complicated rules of evaluation than C. Allowing for random order of assignment could be troublesome.
What this is feature is about is 1) <strong>safety</strong> 2) a nice way to get <strong>auto-completion</strong> from your IDE on what member you need to fill in.</p>
<h4>Improved lambdas</h4>
<p>If you want to capture a <strong>parameter pack</strong> within a lambda before C++20, you will have to be <a href="https://stackoverflow.com/questions/47496358/c-lambdas-how-to-capture-variadic-parameter-pack-from-the-upper-scope">extremely creative</a>. This is even more frustrating that lambdas are great to combine with templates.
This very tedious task becomes trivial in C++20:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span><span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="k">auto</span> <span class="n">foo</span><span class="p">(</span><span class="n">Args</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[...</span><span class="n">args</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">args</span><span class="p">)]()</span> <span class="p">{</span>
<span class="c1">// Do whatever you want with args...</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div>
<p>std::mindblowing()! Sprinkling a bit of <code>...</code> where it should... works as expected!</p>
<p>Lambdas become, in C++20, allowed in a <strong>unevaluated contexts</strong>. What are unevaluated contexts? Whenever you have an expression within a <code>sizeof(...)</code> or <code>decltype</code>.
Is that something you would frequently do? Actually yes. Whenever you want a custom deleter for your <code>std::unique_ptr</code>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Before C++20:</span>
<span class="k">const</span> <span class="k">auto</span> <span class="n">custom_deleter</span> <span class="o">=</span> <span class="p">[](</span><span class="n">handle</span><span class="o">*</span> <span class="n">h</span><span class="p">){</span> <span class="cm">/* Do something with h. */</span> <span class="n">release</span><span class="p">(</span><span class="n">h</span><span class="p">);</span> <span class="p">};</span>
<span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">handle</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">custom_deleter</span><span class="p">)</span><span class="o">></span> <span class="n">p</span><span class="p">;</span>
<span class="c1">// Becomes a one liner:</span>
<span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">handle</span><span class="p">,</span> <span class="k">decltype</span><span class="p">([](</span><span class="n">handle</span><span class="o">*</span> <span class="n">h</span><span class="p">){</span> <span class="cm">/* Do something with h. */</span> <span class="n">release</span><span class="p">(</span><span class="n">h</span><span class="p">);</span> <span class="p">})</span><span class="o">></span> <span class="n">p</span><span class="p">;</span>
</code></pre></div>
<p><strong>Timur</strong> came up with other examples like having a custom comparison template parameter for <code>std::set</code>.</p>
<h4>Extended Non-Type Template Parameter (NTTP)</h4>
<p>If you are a <strong>template meta-programming</strong> (TMP) enthusiast like me, this will be a game changer!
Right now, the standard only allow integer-like (integer, enumeration, pointers...) and types as template parameter. This is highly restrictive and frustrating at times.
When pushing template meta-programming to its limit, you will often want to manipulate strings. Right now you will need to decompose your string into <code>char</code> that you pass as template arguments.</p>
<p>For instance, these bits of code would never work in <strong>C++17</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">vec2</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">x</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">y</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="p">,</span> <span class="n">vec2</span> <span class="n">V</span><span class="o">></span> <span class="c1">// error: vec2 - illegal type for non-type template parameter </span>
<span class="k">struct</span> <span class="nc">my_struct</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="n">V</span><span class="p">.</span><span class="n">y</span><span class="o">></span><span class="p">,</span> <span class="n">V</span><span class="p">.</span><span class="n">x</span><span class="o">></span> <span class="n">bla</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">InitValue</span><span class="o">></span> <span class="c1">// error: std::string - illegal type for non-type template parameter </span>
<span class="k">struct</span> <span class="nc">my_struct2</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">s</span> <span class="o">=</span> <span class="n">InitValue</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>Good news! These restrictions have been lifted as long as your <strong>non-type template parameter</strong>'s type (I know the usage of "type" twice here is confusing):</p>
<ul>
<li>1) Has a comparison operator available (<code>operator==</code>) at compile time: it is <strong>constexpr</strong>.
The goal is that the compiler should be able to check if two template instantiations are the same by checking if all template parameters are equal. </li>
<li>2) Can be constructed at compile-time: its construction can be done in a constexpr context.</li>
</ul>
<h3>[Talk] Compile Time Regular Expressions with Deterministic Finite Automaton - Hana Dusíková - 💀💀💀★:</h3>
<p>Speaking of template meta-programming, NTTP and strings, <a href="https://twitter.com/hankadusikova">Hana Dusíková</a> improved her <a href="https://github.com/hanickadot/compile-time-regular-expressions">compile-time regular expression</a> library.
At last year's <strong>CppCon</strong>, Hana impressed the crowd with her library: it exploited template meta-programming in C++17 to its maximum to generate a regex parser at compile-time from a string literal.
This makes her library ridiculously fast compared to <code>std::regex</code>, which works at runtime. Surprisingly, the library does not affect compilation at all.
In fact, it improves a lot the compilation time compared to <code>std::regex</code>! Does this implies that there is such a thing as "Zero-cost Abstractions"? Maybe...</p>
<p>I have to admit that I was part of the people who missed the chance to see her talk live when I could have...
So this year, I took my revenge and went to her follow-up talk.
Using some of the C++20 template features, Hana succeeded to make a new regexp engine using a <a href="https://en.wikipedia.org/wiki/Deterministic_finite_automaton">Deterministic Finite Automaton</a>. Do not ask me to summarise properly what I have witnessed during this talk, it was... complicated!
But it is also fascinating, it combines a hefty does of meta-programming with language theory: that's a lot of mental stimulation.</p>
<p>If that sounds fun to you, I would probably suggest to watch her <a href="https://www.youtube.com/watch?v=QM3W36COnE4">initial talk</a> and come back to this one afterwards.</p>
<h3>[Other]:</h3>
<p>There are a lot of other talks that would be worth using your <strong>Google-Fu</strong> to find them:</p>
<ul>
<li>Using C++20's Three-way Comparison <=> - Jonathan Müller</li>
<li>Oh No! More Modern CMake - Deniz Bahadir</li>
<li>C++20 Coroutines - Milosz Warzecha </li>
<li>And many more...</li>
</ul>
<p>You will surely find the <strong>lightning talks</strong> on the <a href="https://www.youtube.com/user/MeetingCPP/videos">Meeting C++'s Youtube Channel</a> at one point.
These short talks are like snack foods, they vary a lot in their content and quality and don't take long to process. </p>
<h1>Conclusion:</h1>
<p>This was another very fruitful event for the <strong>C++</strong> community. I am glad that our beloved C++ language receive so much attention by its users and the committee.
It will be interesting to see how long it will take for the major compiler to be fully C++20 compliant: this <strong>release is massive</strong>!
I was slightly overwhelmed by the amount of new features.<br>
I am also wondering how much of the newly acquired C++20 knowledge will stay true once the standard is released.
I guess that I will have to check that by going to another C++ event next year ;) </p>How to make your maps, try_emplace and smart pointers play nicely with each others in C++17.2018-11-18T15:30:00+01:002018-11-18T15:35:00+01:00Jean Gueganttag:jguegant.github.io,2018-11-18:/blogs/tech/performing-try-emplace.html<h2>Trivia:</h2>
<p>Lately, I have been working on the reincarnation of a <strong>class</strong> at work: a hash map.
While this class had interesting internals (a sort of dense hash map) and performed really well, its interface was not up to standard both literally and metaphorically.
After much of lipstick applied to …</p><h2>Trivia:</h2>
<p>Lately, I have been working on the reincarnation of a <strong>class</strong> at work: a hash map.
While this class had interesting internals (a sort of dense hash map) and performed really well, its interface was not up to standard both literally and metaphorically.
After much of lipstick applied to it, the <strong>class</strong> now fully mimic the interface of the beloved <a href="https://en.cppreference.com/w/cpp/container/unordered_map">std::unordered_map</a> from the standard library.
A close look on <code>std::unordered_map</code> and its sister <a href="https://en.cppreference.com/w/cpp/container/map">std::map</a> reveals few interesting design choices.
Combining this interface with some smart pointer types can present some challenges to squeeze performance out of your maps.
We will explore these challenges in this blog post, and try to figure out some solutions.</p>
<p>Disclaimer: C++ being C++, I would not be suprise if 1) I wrote some unacurracies here 2) Some guru could reduce this entire article in a phantasmagoric one liner.</p>
<h3>Some peculiar modifier member functions:</h3>
<p>Note: This part of the post will serve as a reminder for some of the folks that are not well versed in the <strong>associative containers</strong> of the standard. If you are confident, you can always jump straight to the dilemma part. </p>
<p><center><img width=25% height=25% src="https://jguegant.github.io/blogs/tech/images/container-ship.png" alt="A container"/></center></p>
<h4>insert:</h4>
<p>If you observe the interface of the <strong>associative containers</strong> (like <a href="https://en.cppreference.com/w/cpp/container/map">std::map</a> or <a href="https://en.cppreference.com/w/cpp/container/unordered_map">std::unordered_map</a>) in the current standard you will notice that there are 6 member functions to map a value to a given key: <code>insert</code>, <code>insert_or_assign</code>, <code>emplace</code>, <code>emplace_hint</code>, <code>try_emplace</code> and the <strong>subscript operator</strong> (operator[]). That number does not include all the overloads for each of these member functions. It is not a wonder that a lot of C++ users will tend to do suboptimal calls to insert values in their <strong>associative containers</strong>, the choice is not always obvious when you have 6 different functions with slightly different behaviour. </p>
<p>Typically, you will often this pattern within a code-base:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">unordered_map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span> <span class="n">m</span><span class="p">;</span>
<span class="c1">// Check if the key is already in m.</span>
<span class="k">if</span> <span class="p">(</span><span class="n">m</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"johannes"</span><span class="p">)</span> <span class="o">==</span> <span class="n">m</span><span class="p">.</span><span class="n">end</span><span class="p">())</span> <span class="p">{</span> <span class="c1">// Often written as m.count("johannes") == 0</span>
<span class="n">m</span><span class="p">[</span><span class="s">"johannes"</span><span class="p">]</span> <span class="o">=</span> <span class="s">"lucio"</span><span class="p">;</span> <span class="c1">// If not insert they key</span>
<span class="p">}</span>
</code></pre></div>
<p>Little did your colleague, boss, or tired ego know that such a code will do twice a relatively costly job: checking the existence of the key in the map.
Indeed, in the case of a <code>std::unordered_map</code>, the key <code>"johannes"</code> will be hashed twice: in <code>find</code> and in the <code>operator[]</code>. In both member functions, the <code>std::unoredered_map</code> has to know in which bucket the key will fall into. Worst! If you are having collisions between your keys, checking the existence of a key may induce up to N comparisons (even your hash function can be drunk sometimes) where N is the amount of stored key-value pairs. Potentially mutiplying these comparisons by two is not something you should desire. Such a situation in <code>std::map</code> is even worst, this will always bring roughly O(log(N)) comparisons. Comparing two keys may not always be as cheap as it seems and if you add on top of that the cost of jumping through a linked list of nodes, this should be considered harmful.</p>
<p>Obviously the answer to this problem is to use <a href="https://en.cppreference.com/w/cpp/container/map/insert">insert</a>. <code>insert</code> as its name implies, will only work if it can insert the key in the <strong>associative container</strong>, meaning that the insertion will not happen if the same key is already in the map. If you really care to know whether the insertion happend, <code>insert</code> will return a <code>pair</code> of an iterator and a boolean that you can query. The iterator points to the newly inserted key-value pair or the already existing one, the boolean indicates whether the insertion happened or not.</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Use C++17 structured bindings and class template argument deduction (CTAD)</span>
<span class="c1">// See more on my previous post on that topics</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="p">(</span><span class="s">"johannes"</span><span class="p">,</span> <span class="s">"lucio"</span><span class="p">));</span> <span class="c1">// Construct a pair and insert it if needed.</span>
<span class="c1">// Do whatever you want with it and result.</span>
</code></pre></div>
<p>Here only one check for the existence will be done, that is much better, isn't it?
Well, while this snippet is shorter and performs better, there is still room for improvement.
Here we are constructing a <code>pair</code> out of the map, the same <code>pair</code> that needs to be created in a node of the <code>map</code>.
Internally a call to the <strong>move-constructor</strong> of the <code>pair</code> will be done, or way worst a call to the <strong>copy-constructor</strong> if one of the two types in the <code>pair</code> cannot be moved.
Relying on the <strong>move-constructor</strong> of a <code>pair</code> to exist AND to be performant is too much of a wishful thinking. </p>
<h4>emplace:</h4>
<p>Thanksfully, C++11 added on many containers a new member function called <a href="https://en.cppreference.com/w/cpp/container/map/emplace">emplace</a>. Given a container of type <code>T</code>, <code>emplace</code> will accept the arguments necessary for an in-situ construction of a new instance of <code>T</code>. Meaning that we can easily improve our insertion in this way: </p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">emplace</span><span class="p">(</span><span class="s">"johannes"</span><span class="p">,</span> <span class="s">"lucio"</span><span class="p">);</span> <span class="c1">// Construct the pair of "johannes", "lucio" straight into m.</span>
</code></pre></div>
<p>I will go slightly against <a href="https://abseil.io/tips/112">abseil's recommendation</a> and say that <code>emplace</code> should be prefered over <code>insert</code> in <strong>C++11</strong> for at least all the <strong>associative containers</strong>. It will perform better (as explained previously) and it also feels more natural (most users think of a key and a value, not a std::pair)! </p>
<p>Now <code>emplace</code> on <strong>associative containers</strong> has a vicious specification which <a href="https://en.cppreference.com/w/cpp/container/map/emplace">cppreference gently warns you about</a>. For some obscure reasons, even if the <strong>emplace operation</strong> does not succeed since the key already exists, your arguments passed as a <strong>r-value</strong> may have been moved anyway.
More vaguely, the standard mandates effects only on whether the insertion will happen or not and the return value:</p>
<div class="highlight"><pre><span></span><code>Effects: Inserts a value_type object t constructed with std::forward<Args>(args)... if and only if there is no element in the container with key equivalent to the key of t.
The bool component of the returned pair is true if and only if the insertion takes place, and the iterator component of the pair points to the element with key equivalent to the key of t.
</code></pre></div>
<p>This cryptic language-lawyer text does not explain in any way what happens to the arguments in the case of failure. For what we know, they could be sent over smoke signal up to Hong-Kong and inserted into some fortune cookies. Why would we care about that? Well, because that will restrain you to write such code:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">unordered_map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="kt">int</span><span class="o">>></span> <span class="n">m</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">my_precious_int</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="mi">42</span><span class="p">);</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">emplace</span><span class="p">(</span><span class="s">"ricky"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">my_precious_int</span><span class="p">));</span> <span class="c1">// We need to move unique pointers.</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">result</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Alright the insertion failed, let's do something else with my_precious_int.</span>
<span class="n">do_something_else</span><span class="p">(</span><span class="o">*</span><span class="n">my_precious_int</span><span class="p">);</span> <span class="c1">// Or can we?</span>
<span class="p">}</span>
</code></pre></div>
<p>Here <code>my_precious_int</code> is unusable right after the call to <code>emplace</code>, it may have been moved-from forever and ever, <strong>EVEN</strong> if the insertion result is <code>false</code>.
Some confused souls will tell you that this is evident, we called <code>std::move</code>, so it MUST be moved-from. Like the famous cake, <code>std::move</code> is a lie! It does not move anything, it simply casts objects to <strong>a x-value</strong> which makes them POTENTIALLY moveable-from (this world would have been better if <code>std::move</code> was named <code>std::to_xvalue</code>, <code>std::moveable</code>, <code>std::to_be_moved</code>...).</p>
<h4>try_emplace:</h4>
<p>This unleashed <code>emplace</code> is a real pain when you are trying to store move-only types in any <strong>associative container</strong>.
The standard committee was aware of the issue and fixed it in <strong>C++17</strong> with another member function called <a href="https://en.cppreference.com/w/cpp/container/map/try_emplace§">try_emplace</a>.
Here are the expected effects:</p>
<div class="highlight"><pre><span></span><code>Effects: If the map already contains an element whose key is equivalent to k, there is no effect.
Otherwise inserts an object of type value_type constructed with piecewise_construct, forward_as_tuple(std::move(k)), forward_as_tuple(std::forward<Args>(args)...).
</code></pre></div>
<p>It effectively prevents your arguments to be moved-from (as well as being packaged into fortune cookies) in the case of an insertion failure.
Why wasn't <code>emplace</code> simply patched?
If you take a look at the definition of <a href="https://en.cppreference.com/w/cpp/container/map/emplace§">emplace</a>, you will understand that it accepts as a key argument any object with a <strong>type compatible</strong> with your <strong>key type</strong>.
Unlike the value argument, the key argument ALWAYS need to be somehow converted to the <strong>key type</strong> to check for its existence in the map.
The potential conversion would defeat the "there is no effect" policy.
<code>try_emplace</code> is stricter and only takes a key of type <strong>key type</strong>, which guarantees that no conversion sequence will be triggered.
At least <code>try_emplace</code> can help us to safely rewrite the previous example:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">unordered_map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="kt">int</span><span class="o">>></span> <span class="n">m</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">my_precious_int</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="mi">42</span><span class="p">);</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"ricky"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">my_precious_int</span><span class="p">));</span> <span class="c1">// We need to move unique pointers.</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">result</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Alright the insertion failed, let's do something else with my_precious_int.</span>
<span class="n">do_something_else</span><span class="p">(</span><span class="o">*</span><span class="n">my_precious_int</span><span class="p">);</span> <span class="c1">// It is safe! </span>
<span class="p">}</span>
</code></pre></div>
<p>Hurray! After three corrections in the standard, we can effectively mix associative containers with <code>unique_ptrs</code> for a maximum of fun and profit!
Well... no. <strong>C++ being C++</strong>, it cannot be that easy. There is one last boss to slain.</p>
<h3>The last dilemma of a map of unique_ptrs:</h3>
<p><img width=20% height=20% style="float: right;" src="https://jguegant.github.io/blogs/tech/images/dilemma.jpg" alt="A container"/></p>
<p>Before I start on this topic, I would like to remind everyone a basic rule: you should try to avoid heap allocations.
You should always strive to get a <code>std::map<T1, T2></code> over a <code>std::map<T1, std::unique_ptr<T2>></code>.
Now that being said, you may have situations where you cannot do otherwise:</p>
<ul>
<li>If you need runtime polymorphism. For instance, you may need to store services into a <code>std::map<std::string, std::unique_ptr<service>></code> where <code>service</code> is an interface with multiples concrete implemetations. Although, there are always ways to hide the inheritance as explained by <a href="https://www.youtube.com/watch?v=QGcVXgEVMJg">Sean Parent</a>... </li>
<li>If your weapon of choice is a map following closely the interface of <code>std::unordered_map</code> or <code>std::map</code> minus the stable addressing part of it.
This is often the case for all the hash map with excellent performance, like <a href="https://github.com/skarupke/flat_hash_map/blob/master/flat_hash_map.hpp">skarupe's one</a>.
Not having stable addressing means that querying the address of a value in the map <code>&map["jeremy"]</code> might give you different results if you do any modifying operations (insert, erase...) on the map. In such case, having an extra indirection (heap allocation) will bring back stable addressing. </li>
</ul>
<p>Not only dealing with pointers (even smart ones) is often tedious, but it can also ruin the <code>try_emplace</code> member function of your class.
You will have to choose between a costly object creation or the dreaded double key lookup I mentioned right at the beggining of this post.
Pick your poison!</p>
<h4>Uncessary object creation or double key lookup:</h4>
<p>Let's keep the idea of a map of services: <code>std::map<std::string, std::unique_ptr<service>></code>, and you would like to register a service "file_locator" only if there was none registered earlier.
Hastily, you may write such code:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">service</span><span class="o">>></span> <span class="n">m</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="c1">// Create a file locator that explore a file system on a remote server.</span>
<span class="c1">// remote_file_locator implements the service interface.</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"file_locator"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">(</span><span class="s">"8.8.8.8"</span><span class="p">,</span> <span class="s">"/a_folder/"</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">result</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Print which file_locator is already in there. </span>
<span class="n">log</span><span class="p">(</span><span class="s">"Could not register a remote_file_locator, it has been overridden by: "</span><span class="p">,</span> <span class="n">it</span><span class="o">-></span><span class="n">second</span><span class="p">.</span><span class="n">name</span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div>
<p>If the <code>remote_file_locator</code> is successfully registered, everything is fine!
But in the other scenario where a <code>file_locator</code> is already in the map, this code has a huge pessimisation.
Here your compiler will emit code that allocate enough memory for a <code>remote_file_allocator</code>, then it will construct it, under any circumstances.
If allocating can be seen as slow in the C++ world, starting a connection to a server is pure hell when it comes to speed.
If you are not planning to use the instance of this really costly object, why would you create it in the first place?</p>
<p>So shall we revert to the double lookup?</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">it</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"file_locator"</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">it</span> <span class="o">==</span> <span class="n">m</span><span class="p">.</span><span class="n">end</span><span class="p">())</span> <span class="p">{</span>
<span class="n">m</span><span class="p">[</span><span class="s">"file_locator"</span><span class="p">]</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">(</span><span class="s">"8.8.8.8"</span><span class="p">,</span> <span class="s">"/a_folder/"</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">log</span><span class="p">(</span><span class="s">"Could not register a remote_file_locator, it has been overridden by: "</span><span class="p">,</span> <span class="n">it</span><span class="o">-></span><span class="n">second</span><span class="p">.</span><span class="n">name</span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div>
<p>Hell no! I already explained why I discourage you to use such a pattern in the first part of this post.
You could argue that here we will only pay this double lookup overhead only once, when we try to create the "remote_file_locator".
Given more time and coffee, I should be able to come up with an architecture where you could do such insertions of unique_ptrs in a loop.
In any case, C++ is all about not paying in performance for uncessary things. </p>
<p>But don't worry, C++ being C++, there surely are ways to get around this impediment.</p>
<h4>Two clumsy solutions:</h4>
<p>I, personally, could come up with two solutions. If you have a better one, you are welcome to express it in the comments.</p>
<p>The first one is actually not so hacky. You can start by trying to emplace an empty <code>unique_ptr</code>, if the insertion works you can always fix it afterwards with a real allocation:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Try to emplace an empty `unique_ptr` first.</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"file_locator"</span><span class="p">,</span> <span class="k">nullptr</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">result</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// The insertion happened, now we can safely create our remote_file_locator without wasting any performance. </span>
<span class="n">it</span><span class="o">-></span><span class="n">second</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">(</span><span class="s">"8.8.8.8"</span><span class="p">,</span> <span class="s">"/a_folder/"</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>I somehow dislike this solution. It is not consistent with the usage of <code>try_emplace</code> on more classic types, which do not require any extra step.
It really smells like some kind of two-phases initialisation pattern which are usually frowned upon.
We are temporarily putting our map into a state where "file_locator" cannot be trusted. What if the actual creation of the <code>remote_file_locator</code> throws an exception?
That would leave the map with a empty "file_locator", that's not great.</p>
<p>My second solution consists in trying to delay the construction of the <code>remote_file_locator</code>.
To do so, I wrote a very simple helper struct that I called <code>lazy_convert_construct</code>.
This struct wraps any kind of lambda that acts like factory: the <strong>lambda factory</strong> returns an instance of a given type, <code>"result_type"</code>, when called.
If at any point the struct needs to be converted to <code>result_type</code>, it will call the internal lambda to generate an instance of <code>result_type</code>.
Any code should speaks for itself, so here is the <code>lazy_convert_construct</code> beast in all its beauty:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span><span class="o"><</span><span class="k">class</span> <span class="nc">Factory</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">lazy_convert_construct</span>
<span class="p">{</span>
<span class="k">using</span> <span class="n">result_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">invoke_result_t</span><span class="o"><</span><span class="k">const</span> <span class="n">Factory</span><span class="o">&></span><span class="p">;</span> <span class="c1">// Use some traits to check what would be the return type of the lambda if called.</span>
<span class="k">constexpr</span> <span class="n">lazy_convert_construct</span><span class="p">(</span><span class="n">Factory</span><span class="o">&&</span> <span class="n">factory</span><span class="p">)</span>
<span class="o">:</span> <span class="n">factory_</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">factory</span><span class="p">))</span> <span class="c1">// Let's store the factory for a latter usage.</span>
<span class="p">{</span>
<span class="p">}</span>
<span class="c1">// ↓ Respect the same nowthrow properties as the lambda factory.</span>
<span class="k">constexpr</span> <span class="k">operator</span> <span class="n">result_type</span><span class="p">()</span> <span class="k">const</span> <span class="k">noexcept</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">is_nothrow_invocable_v</span><span class="o"><</span><span class="k">const</span> <span class="n">Factory</span><span class="o">&></span><span class="p">)</span>
<span class="c1">// ^ enable ^ the type this struct can be converted to </span>
<span class="c1">// conversion</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">factory_</span><span class="p">();</span> <span class="c1">// Delegate the conversion job to the lambda factory.</span>
<span class="p">}</span>
<span class="n">Factory</span> <span class="n">factory_</span><span class="p">;</span>
<span class="p">};</span>
<span class="c1">// Example of usage:</span>
<span class="k">auto</span> <span class="n">l</span> <span class="o">=</span> <span class="n">lazy_convert_construct</span><span class="p">([]{</span> <span class="k">return</span> <span class="mi">42</span><span class="p">;</span> <span class="p">});</span>
<span class="c1">// ^ CTAD again ^ Factory lambda that returns an int.</span>
<span class="kt">int</span> <span class="n">x</span> <span class="o">=</span> <span class="n">l</span><span class="p">;</span>
<span class="c1">// ^ Here l is forced to be converted to an int and will therefore call the lambda to do so.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span><span class="p">;</span> <span class="c1">// Prints 42. </span>
</code></pre></div>
<p>Note that the lambda will not be called if there is no conversion needed, this makes it having a <a href="https://en.wikipedia.org/wiki/Lazy_evaluation">lazy evaluation</a>.
Note also: after turning all optimisations on, the <code>lazy_convert_construct</code> entirely disappears and x will be simply initialised by 42 when needed. </p>
<p>The next step is to combine this <code>lazy_convert_construct</code> with <code>try_emplace</code>, which works like a charm:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"file_locator"</span><span class="p">,</span> <span class="n">lazy_convert_construct</span><span class="p">([]{</span> <span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unique</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">(</span><span class="s">"8.8.8.8"</span><span class="p">,</span> <span class="s">"/a_folder/"</span><span class="p">);</span> <span class="p">}));</span>
</code></pre></div>
<p><code>lazy_convert_construct</code> is now able to create a <code>unique_ptr<remote_file_locator></code> on-demand. Even with <code>lazy_convert_construct</code>, <code>try_emplace</code> will respect its contract: it will not have any side effect on the <code>lazy_convert_construct</code> object if the key <code>"file_locator"</code> is already present. Meaning that no conversion will happen if the key already exists.
This rather elegant solution fixes one of the main drawback of the previous one: it never leaves the map in a state with a <code>file_locator</code> being null.
It is also a one liner!</p>
<h4>Benchmark results:</h4>
<p>Some of you may still be a bit skeptical on the importance of optimising your queries in your associative containers.
So I wrote a very simple benchmark which explores multiple insertion scenarios on a <code>std::map<std::string, std::unique_ptr<T>></code>.
You can fetch it <a href="https://github.com/Jiwan/benchmarks_map_insertion">here</a>.</p>
<p>Using clang 6.0 on my Linux laptop, I obtain the following results:</p>
<div class="highlight"><pre><span></span><code>2018-11-18 14:16:17
Running ./fast_try_emplace
Run on (8 X 3600 MHz CPU s)
CPU Caches:
L1 Data 32K (x4)
L1 Instruction 32K (x4)
L2 Unified 256K (x4)
L3 Unified 8192K (x1)
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-----------------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------------
insertion_double_lookup 636 ns 638 ns 1085075
insertion_construct_before_try_emplace 503 ns 506 ns 1387818
insertion_lazy_convert_try_emplace 503 ns 506 ns 1380488
no_insertion_double_lookup 107 ns 107 ns 6300180
no_insertion_construct_before_try_emplace 8642 ns 8641 ns 80478
no_insertion_lazy_convert_try_emplace 32 ns 32 ns 22393508
</code></pre></div>
<p>Clearly, a sucessful insertion using the double lookup is more expensive than it should. The cost will change depending on the amount of <strong>key-value pairs</strong> already in the map.
For a failed insertion scenario, my <code>lazy_convert_construct</code> is also faster than the double lookup. I cannot explain why! Internally, <code>find</code> and <code>try_emplace</code> should have the same lookup mechanims.
And of course, creating a costly object and destroying right after is really bad choice. That explains why <code>no_insertion_construct_before_try_emplace</code>'s record is so damn huge compared to the two others cases (I purposely made the type very costly to create for the no insertion cases).</p>
<p>GCC offers similar results, without the mysterious advantage of the <code>try_emplace</code> + <code>lazy_convert_construct</code> over the double lookup in a no insertion scenario.</p>
<div class="highlight"><pre><span></span><code>2018-11-18 14:17:53
Running ./fast_try_emplace
Run on (8 X 3600 MHz CPU s)
CPU Caches:
L1 Data 32K (x4)
L1 Instruction 32K (x4)
L2 Unified 256K (x4)
L3 Unified 8192K (x1)
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-----------------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------------
insertion_double_lookup 547 ns 543 ns 1310711
insertion_construct_before_try_emplace 493 ns 496 ns 1394343
insertion_lazy_convert_try_emplace 495 ns 500 ns 1000000
no_insertion_double_lookup 44 ns 44 ns 15787599
no_insertion_construct_before_try_emplace 8659 ns 8658 ns 78869
no_insertion_lazy_convert_try_emplace 44 ns 44 ns 15895884
</code></pre></div>
<p>Now we can proudly claim that we solved all the insertion issues (at least those that I am aware of).
But somehow it still feels like something is off with the smart pointer types of the standard library.
Are we missing something here?</p>
<h4>An in_place constructor:</h4>
<p>Warning: here you will enter the somehow controversial and imaginary part of my post. I am not claiming that this is the direction <strong>C++</strong> should take on these issues, but merely raising my questions on that topic. </p>
<p>Indeed, <code>unique_ptr</code> and its other smart pointers counsins (<code>shared_ptr</code>...) are rather special types.
On one hand you could see them as simple RAII wrappers that take care of a basic resource: a pointer.
On the other hand, you could, <strong>very arguably</strong> (<== note this bold statement), see them as some funky <strong>value wrappers</strong> with a very special storage (one that implies pointer semantics). </p>
<p><strong>Value wrappers</strong> are types that enhance the property of another type(s). The most recent ones in the standard are: <a href="https://en.cppreference.com/w/cpp/utility/optional">std::optional</a>, <a href="https://en.cppreference.com/w/cpp/utility/variant">std::variant</a> and <a href="https://en.cppreference.com/w/cpp/utility/any">std::any</a>.
As expected, all of these new <strong>value wrappers</strong> have constructors that accepts an instance of the type they are wrapping:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
<span class="n">A</span><span class="p">(</span><span class="kt">int</span> <span class="n">args1</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">args2</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* do some with the args */</span><span class="p">}</span>
<span class="p">};</span>
<span class="c1">// Move construct the newly created A into o.</span>
<span class="n">std</span><span class="o">::</span><span class="n">optional</span> <span class="nf">o</span><span class="p">(</span><span class="n">A</span><span class="p">{</span><span class="mi">42</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">{</span><span class="mi">42</span><span class="p">}});</span>
</code></pre></div>
<p>While such constructors might be sufficient for most of the usages of your <strong>value wrappers</strong>, sometimes you really want to avoid any move or copy constructors.
The standard committee was proactive and provided another set of constructors to build the wrapped value <strong>in-place</strong>.
In order to disambiguate with the usual constructors, these new constructors take as a first argument a <a href="https://www.fluentcpp.com/2018/04/27/tag-dispatching/">tag type</a>: <a href="https://en.cppreference.com/w/cpp/utility/in_place">std::in_place</a> or <a href="https://en.cppreference.com/w/cpp/utility/in_place">std::in_place_type</a>. Here is how <code>std::in_place</code> works with <code>std::optional</code>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// ↓ No CTAD ↓ The arguments needed for constructing a new instance of A.</span>
<span class="n">std</span><span class="o">::</span><span class="n">optional</span><span class="o"><</span><span class="n">A</span><span class="o">></span> <span class="n">o</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">in_place</span><span class="p">,</span> <span class="mi">42</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">{</span><span class="mi">42</span><span class="p">});</span>
<span class="c1">// ^ dispatch to in place constructor.</span>
</code></pre></div>
<p>With such a constructor, we can safely assume that the wrapped instance of <code>A</code> was built directly in its storage place.
Of course, you can also use this constructor if you are dealing with <code>map::try_emplace</code>:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">map</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">optional</span><span class="o"><</span><span class="n">A</span><span class="o">>></span> <span class="n">m</span><span class="p">;</span>
<span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"bruno"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">in_place</span><span class="p">,</span> <span class="mi">42</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">{</span><span class="mi">42</span><span class="p">});</span>
<span class="c1">// ^ Will construct the wrapped A deep down in the map.</span>
</code></pre></div>
<p>At this point, if you are following my anology between <code>std::unique_ptr</code> and <strong>value wrappers</strong>, you could start to question yourself on why we could not get a similar set of constructors for our smart pointers. Maybe something similar to this:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">service</span><span class="o">></span> <span class="n">x</span><span class="p">{</span><span class="n">std</span><span class="o">::</span><span class="n">allocate_in_place</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">,</span> <span class="mi">42</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">{</span><span class="mi">42</span><span class="p">}};</span>
<span class="c1">// ^ tag ^ concret instance ^ following args.</span>
</code></pre></div>
<p>I intentionally chose a different <strong>tag type</strong> than <code>std::in_place</code> for two reasons:</p>
<ul>
<li>This constructor is doing more than constructing in place, it does allocate. The user should be informed. </li>
<li>In a similar fashion to <a href="https://en.cppreference.com/w/cpp/utility/in_place">std::in_place_type </a>, we somehow need to encode the concret type we want to instantiate.</li>
</ul>
<p>With such a constructor in the standard for <code>unique_ptr</code>, our issue with <code>try_emplace</code> would become trivial to solve. Just call it!</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="p">[</span><span class="n">it</span><span class="p">,</span> <span class="n">result</span><span class="p">]</span> <span class="o">=</span> <span class="n">m</span><span class="p">.</span><span class="n">try_emplace</span><span class="p">(</span><span class="s">"file_locator"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">allocate_in_place</span><span class="o"><</span><span class="n">remote_file_locator</span><span class="o">></span><span class="p">,</span> <span class="s">"8.8.8.8"</span><span class="p">,</span> <span class="s">"/a_folder/"</span><span class="p">);</span>
</code></pre></div>
<p>The constructor of <code>unique_ptr</code> that accept the tag <code>allocate_in_place</code> would be called only if only the key "file_locator" is not in there.
No overhead, simple syntax, you could not ask for more!</p>
<p>As a side effect, my guess is that we could also fully deprecate the usage of <code>make_unique</code> and <code>make_shared</code>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Before:</span>
<span class="k">auto</span> <span class="n">x</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_shared</span><span class="o"><</span><span class="n">something</span><span class="o">></span><span class="p">(</span><span class="s">"michel"</span><span class="p">,</span> <span class="s">"christian"</span><span class="p">);</span>
<span class="c1">// After</span>
<span class="k">auto</span> <span class="n">x</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">allocate_in_place</span><span class="o"><</span><span class="n">something</span><span class="o">></span><span class="p">,</span> <span class="s">"michel"</span><span class="p">,</span> <span class="s">"christian"</span><span class="p">);</span>
</code></pre></div>
<p>Obviously the syntax is far from being <strong>terse</strong>. The invented feature also does not take in consideration the <strong>allocator arguments</strong> you could receive in a smart pointer.
Me and a colleague promised ourselves to look a bit more into this topic. Whether we will formulate a proposal or just have some afterthoughts, be sure that I will let you informed on that in a further post! </p>
<h3>Conclusion:</h3>
<ul>
<li>If you are inserting a new pair into an <strong>associative container</strong> consider using <code>try_emplace</code> first.</li>
<li>If you cannot use <strong>C++17</strong>, prefer to use <code>emplace</code> over <code>insert</code>.</li>
<li>If you are cannot use <strong>C++11</strong>, I feel sorry for you!</li>
<li>You can borrow my <code>lazy_convert_construct</code> if you are dealing with smart pointers and <code>try_emplace</code>, to get a blazzing fast insertion.</li>
</ul>
<p>A special thanks to my colleague Yo Anes with whom I had a lot of fun discussing this specific topic.</p>Trip report - CppCon 20182018-10-07T23:09:00+02:002018-10-07T23:09:00+02:00Jean Gueganttag:jguegant.github.io,2018-10-07:/blogs/tech/trip-report-cppcon-2018.html<p>New year, new conference!
This time, my employer, <a href="https://discover.king.com/about/">King</a>, helped me to organize a first pilgrimage to <a href="https://cppcon.org/">CppCon</a> for me and another colleague.
You cannot fathom how enthusiastic I was to finally making it there!
Although I might be a bit late on the "trip-report-race", I think that it is …</p><p>New year, new conference!
This time, my employer, <a href="https://discover.king.com/about/">King</a>, helped me to organize a first pilgrimage to <a href="https://cppcon.org/">CppCon</a> for me and another colleague.
You cannot fathom how enthusiastic I was to finally making it there!
Although I might be a bit late on the "trip-report-race", I think that it is still worth to relate my overall experience of the event and then move onto a list of recommended talks you should watch-out on Youtube. </p>
<h1>About CppCon (2018):</h1>
<h2>The event:</h2>
<p><strong>CppCon</strong> is the most renowned conference for all the C++ afficionados. So far held anually in the cozy city center of Bellevue, Washington (or less precisely somewhere close-by to Seattle for those like me that are not into north-american geography), <strong>CppCon</strong> let you explore the <strong>C++</strong> world with various talks, keynotes and other activities provided by some accomplished members of the community. The event is usually happening at the end of September and lasts 5 days, or even more for those attending the training sessions. The program is really plentiful and you would need to watch presentations from 8.00am to 10.00pm, have an ubiquity capacity to be simultaneously in 6 rooms at the same time, and have infinite memory to be able to absorbe all the C++ knowledge flowing during these days.</p>
<p>Do not get me wrong, while the conference is impressive when it comes to the amount of content, that does not imply that this content is out of the reach of the commoners. On one hand you have hairy topics like "Compile-time programming" being discussed, and on the other hand you have acces to gentle introductions to some features of the language. Dedicated or novice C++ users will appreciate as much the conference but for different reasons. A novice will bring back home a lot of new keywords/concepts to plunge into. Someone who follow C++ news, may not make as many discoveries at <strong>CppCon</strong> than at <a href="http://cppnow.org/">CppNow</a>, but she/he will gain a lot of inspiration/motivation from other C++ fellows and will be exposed to other point of views on the language.</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/bellevue.jpg"/></center></p>
<p>Does this sound exciting to you, but you missed the opportunity to come this year? No worries, <strong>CppCon</strong> has one of the best <a href="https://www.youtube.com/user/CppCon/videos">Youtube channel</a> where they upload all the talks. I have always been impressed by the quality of their videos and the frequency at which they can produce such material. Now, I can say that behind the scene (or actually in front...), they have an excellent video-shooting crew in each of the conference rooms. On a side note, all keynotes were introduced by a live rock-band, which you can see at the beggining of the videos. Whether you appreciated the music or not, it is hard to deny that CppCon organizers really put a lot of efforts in the entire event! </p>
<h1>My experience over there:</h1>
<p>Right before leaving Stockholm, I had the great idea to listen to my favorite C++ podcast <a href="http://cppcast.com/">CppCast</a>. The guest for that week was <strong>Bryce Adelstein Lelbach</strong> who is one of the organiser of CppCon. Bryce had four advices for any new attendee of the conference, which matched my experience at <a href="https://jguegant.github.io/blogs/tech/trip-report-meetingcpp-2017.html">Meeting C++</a> last year and turn out to be valid for CppCon too:</p>
<ul>
<li>Have no regrets! Indeed, as an attendee you will quickly discover that two or three appealing talks will be on the same time slot. Just randomly pick one of the talk or none if you need to rest a bit. If luck is not on your side that day and you missed a terrific talk, you will naturally hear about it from others and you will have plenty of time to binge-watch it on Youtube later on. Having that in mind, I was much more relax at this conference than I was at my very intense days at <strong>Meeting C++</strong>. If you really dislike to choose you will anyway ends-up following someone, you just chit-chated with within the corridors, to the talk of his choice. Which brings us to the second advice you should follow...</li>
<li>Engage with the community! If you are planning to go to CppCon to appreciate a better version of the talks than on YouTube, you are doing it wrong! By going over there, you actually loose the possibility to watch the talk at your own pace... What you gain instead, as an attendee, is the opportunity to mingle with people with very different backgrounds, exchange tips and tricks and feel part of community bigger than the few C++ colleagues you are working with. Programmers seldom have the reputation of being extroverts, but you will always find someone that can introduce to his connections and slowly build relationships. I met quite a few people from <strong>SwedenCpp</strong> over there and it was really fun to see them in another context!</li>
<li>Be confused! You should not be afraid to be out of your confort zone when it comes to the content of the talks. You may be mortified at the idea that a guru will suddenly drop a very complicated C++ concept on stage, and you will be there lonely, not knowing what on earth he/she/it is talking about. Truth is, very few people (if not none) can claim knowing everything about such a vaste language that is C++. Usually during a "productive" C++ conference, you will dress-up a list of keywords / ideas that you will explore later-on. This year, I promised myself to look further on the edge-cases of Class Template Argument Deduction (CTAD), prepare myself for C++20's <strong>contracts</strong> and play with clang's tooling internals.</li>
<li>Be bold! The concentration of "legendary C++ devs" per square meter is even higher than in <strong>Meeting C++</strong>. While I did not shoot any selfie (not that I wanted to either) with any of these legends, I discussed briefly with few of them, and you can too! People at <strong>CppCon</strong> are here to celebrate their favorite language, not to act as elitists, which make everyone very approachable. One of the talk I attended was on a <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0052r8.pdf">proposal</a> (Woes of Scope Guards and Unique Resource - 5+ years in the making - By Peter Sommerland) I implemented at work. During the talk, Peter mentioned a relatively serious bug introduced in one of the revision. Right after that talk, I had the unique chance to have a face-to-face discussion with him. It turns out, my implementation did not suffer from that bug, but was instead hiding another one. I was so glad that I could access to that person so easily! </li>
</ul>
<p>Following these precepts, I experienced a very wonderful week made of C++ and human interactions, and I would highly recommend <strong>CppCon</strong> to anyone having a slight interest in the language.</p>
<p><center><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/cppcon.jpg"/></center></p>
<p>I still have one and only one complain about the event: the time slot for the lightning talks.
For those that are not aware, the lightning talks are short presentations of roughly 5min, easier to submit, often light-minded but also damn interesting.
Due to the short format, people often go straight to the point which is really pleasant to watch.
For instance, this year there was an epic lightning talk battle on <a href="https://arne-mertz.de/2018/05/trailing-return-types-east-const-and-code-style-consistency/">East Const vs West Const</a> or a touching "Thank You" speech from <strong>Dr. Walter E. Brown</strong>.
If that sounds interesting to you, you will have to stay awake from 8.30pm to 10.00pm which is where my grudge comes from.
After absorbing some C++ since roughly 9.00am, and with a pretty strong jetlag (=~9h for central Europeans) you really need to channel all your inner motivation to attend any of these late activities.
The lightning talks being such joyfull part of <strong>CppCon</strong>, I would argue that some of them could be moved to an earlier slot in the day...</p>
<p>Enough of my pseudo-rant on an almost perfect event and let's continue with some more concrete reporting! </p>
<h1>The chief's suggestions of the year:</h1>
<p>Once again, here is a menu of most of the talks I particulary enjoyed. The legend follow the same rules:</p>
<ul>
<li>💀 : The difficulty of the talk (💀: Begginer friendly, 💀💀: Intermediate, 💀💀💀: High exposure to C++'s dark corners)</li>
<li>★ : My interest for the talk (★: Good talk, ★★: Tasty talk, ★★★: Legendary talk)</li>
</ul>
<p>I promise you not to spoil everything in the talks, but simply try to give an overview of what you can expect within them or some conclusions. Although most of the talks are most likely worth to be seen, I forced my-very-subjective-self to pick very few of them. I have seen people with very different "favorite talk" and you should not feel sad if your own talk is not part of this not-so-prestigious list.</p>
<h3>[Keynote] Concepts: The Future of Generic Programming (the future is here) - Bjarne Stroustrup - 💀★★★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Keynotes/concepts_the_future_of_generic_programming/concepts_the_future_of_generic_programming__bjarne_stroustrup__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="https://youtu.be/HddFGPTAmtU">link</a></li>
</ul>
<p>The Bjarne himself, creator of C++ (for those really out of the loop), kick-started the conference with a keynote on a long-overdue C++ feature: <a href="https://en.wikipedia.org/wiki/Concepts_(C%2B%2B)">Concepts</a>.
Indeed, as early as 2003, Bjarne and some members of the committee have been pushing for Concepts as an extension to C++ templates to simplify the life of us poor template consumers.
Sadly, a first version, dimmed too complicated, of the feature was rejected for <strong>C++11</strong>. Nine years later, Bjarne and his crew is back with a version "Concepts Lite" that will be integrated into <strong>C++20</strong>. </p>
<p>Let's face it, C++ templates have always been a pain to use due to the absence of constraints on the template parameter types.
Take a look at this very simple piece of code:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="kt">void</span> <span class="n">foo</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span>
<span class="n">t</span><span class="p">.</span><span class="n">print</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="kt">void</span> <span class="n">bar</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span>
<span class="n">foo</span><span class="p">(</span><span class="n">t</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="nc">A</span> <span class="p">{};</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="n">bar</span><span class="p">(</span><span class="n">a</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>Your never-satisfied compiler will refuse to compile it with the following error message:</p>
<div class="highlight"><pre><span></span><code><span class="o"><</span><span class="nt">source</span><span class="o">>:</span> <span class="nt">In</span> <span class="nt">instantiation</span> <span class="nt">of</span> <span class="s1">'void foo(T&&) </span><span class="cp">[</span><span class="k">with</span> <span class="n">T</span><span class="o"> =</span> <span class="nx">A</span><span class="o">&</span><span class="cp">]</span><span class="s1">'</span><span class="o">:</span>
<span class="o"><</span><span class="nt">source</span><span class="o">></span><span class="p">:</span><span class="nd">10</span><span class="p">:</span><span class="nd">8</span><span class="o">:</span> <span class="nt">required</span> <span class="nt">from</span> <span class="s1">'void bar(T&&) </span><span class="cp">[</span><span class="k">with</span> <span class="n">T</span><span class="o"> =</span> <span class="nx">A</span><span class="o">&</span><span class="cp">]</span><span class="s1">'</span>
<span class="o"><</span><span class="nt">source</span><span class="o">></span><span class="p">:</span><span class="nd">17</span><span class="p">:</span><span class="nd">10</span><span class="o">:</span> <span class="nt">required</span> <span class="nt">from</span> <span class="nt">here</span>
<span class="o"><</span><span class="nt">source</span><span class="o">></span><span class="p">:</span><span class="nd">5</span><span class="p">:</span><span class="nd">7</span><span class="o">:</span> <span class="nt">error</span><span class="o">:</span> <span class="s1">'struct A'</span> <span class="nt">has</span> <span class="nt">no</span> <span class="nt">member</span> <span class="nt">named</span> <span class="s1">'print'</span>
<span class="nt">t</span><span class="p">.</span><span class="nc">print</span><span class="o">();</span>
<span class="o">~~^~~~~</span>
</code></pre></div>
<p>This is an horrifying warning message as we often see nowadays.
Your compiler do warn you about the absence of a <strong>print</strong> method in <strong>A</strong> far down in <strong>foo</strong>!
In this specific case, we only have two layers in the call stack which is very reasonable.
But most often when using functions provided by the Standard Library, your type will violate some constraints reaaallly deep down.
As a newcommer you will often struggle with your compiler vomiting a bizarre message of 100 lines when using templates, which is not the best first experience (to say the least).
Moreover, by looking at the signature of <strong>bar</strong>, nothing tells you that the template parameter <strong>T</strong> needs to have a <strong>print</strong> member function. That's frustrating!</p>
<p>With concepts you can define in elegant way the constraints that your parameters must respect. Have a look at we could do with Concepts in this case:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">concept</span> <span class="nc">bool</span> <span class="n">IsPrintable</span> <span class="o">=</span> <span class="k">requires</span> <span class="p">(</span><span class="n">T</span> <span class="n">a</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Define a reusable concept "IsPrintable".</span>
<span class="n">a</span><span class="p">.</span><span class="n">print</span><span class="p">();</span> <span class="c1">// Any template parameter of T respecting "IsPrintable" must have a print function. </span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="n">IsPrintable</span> <span class="n">T</span><span class="o">></span> <span class="c1">// Explicitely tells that T must respect the concept.</span>
<span class="kt">void</span> <span class="n">foo</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span>
<span class="n">t</span><span class="p">.</span><span class="n">print</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="n">IsPrintable</span> <span class="n">T</span><span class="o">></span> <span class="c1">// Same here!</span>
<span class="kt">void</span> <span class="n">bar</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span>
<span class="n">foo</span><span class="p">(</span><span class="n">t</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>Which would give us the following much more accurate error message:</p>
<div class="highlight"><pre><span></span><code><span class="o"><</span><span class="n">source</span><span class="o">></span><span class="p">:</span><span class="mi">19</span><span class="p">:</span><span class="mi">10</span><span class="p">:</span> <span class="n">error</span><span class="p">:</span> <span class="n">cannot</span> <span class="n">call</span> <span class="n">function</span> <span class="s1">'void bar(T&&) [with T = A&]'</span>
<span class="n">bar</span><span class="p">(</span><span class="n">a</span><span class="p">);</span>
<span class="o">^</span>
<span class="o"><</span><span class="n">source</span><span class="o">></span><span class="p">:</span><span class="mi">11</span><span class="p">:</span><span class="mi">6</span><span class="p">:</span> <span class="n">note</span><span class="p">:</span> <span class="n">constraints</span> <span class="ow">not</span> <span class="n">satisfied</span>
<span class="n">void</span> <span class="n">bar</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span>
<span class="o">^~~</span>
<span class="o"><</span><span class="n">source</span><span class="o">></span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="mi">33</span><span class="p">:</span> <span class="n">note</span><span class="p">:</span> <span class="n">within</span> <span class="s1">'template<class T> concept const bool IsPrintable<T> [with T = A&]'</span>
<span class="n">template</span> <span class="o"><</span><span class="k">class</span> <span class="n">T</span><span class="o">></span> <span class="n">concept</span> <span class="nb nb-Type">bool</span> <span class="n">IsPrintable</span> <span class="o">=</span> <span class="n">requires</span> <span class="p">(</span><span class="n">T</span> <span class="n">a</span><span class="p">)</span> <span class="p">{</span>
<span class="o">^~~~~~~~~~~</span>
<span class="o"><</span><span class="n">source</span><span class="o">></span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="mi">33</span><span class="p">:</span> <span class="n">note</span><span class="p">:</span> <span class="n">with</span> <span class="s1">'A& a'</span>
<span class="o"><</span><span class="n">source</span><span class="o">></span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="mi">33</span><span class="p">:</span> <span class="n">note</span><span class="p">:</span> <span class="n">the</span> <span class="n">required</span> <span class="n">expression</span> <span class="s1">'a.print()'</span> <span class="n">would</span> <span class="n">be</span> <span class="n">ill</span><span class="o">-</span><span class="n">formed</span>
</code></pre></div>
<p>Some sceptical gurus will tell you that this can be emulated with some witch-crafted <a href="https://jguegant.github.io/blogs/tech/sfinae-introduction.html">SFINAEs</a>.
Concepts were not on my christmas wish-list for <strong>C++</strong> this year, but I have to admit that Bjarne's talk hipped me a lot! </p>
<h3>[Talk] How to Write Well-Behaved Value Wrappers - Simon Brand - 💀💀💀 ★★:</h3>
<ul>
<li>Slides: <a href="">coming soon</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>Good speakers are like good wine, you are seldom disappointed.
Having attended Simon's talk on debugger internals last year and enjoyed it a lot, I took the chance to hear what he had to say on value wrappers.
You may not know what is a value wrapper, but I surely bet that you already manipulated some provided by the Standard Library: <a href="https://en.cppreference.com/w/cpp/utility/pair">std::pair</a>, <a href="https://en.cppreference.com/w/cpp/utility/tuple">std::tuple</a>, <a href="https://en.cppreference.com/w/cpp/utility/optional">std::optional</a>... </p>
<p>It might be a breeze to work with most of these wrappers (certainely not you verbose <a href="https://en.cppreference.com/w/cpp/utility/variant">std::variant</a>), but the poor souls writting implementations of these must go to great lengths to make them behave as they should. Value wrappers, as their name imply, try to mimic as closely as possible the type of a given value. All the types have some basic properties: can it be explicitely constructed? Are the <a href="https://en.wikipedia.org/wiki/Special_member_functions">special member functions</a> noexcept? Can it be called in a constant expression? Can it be compared? And much more... A wrapper must react exactly in the same way when it comes to these properties. </p>
<p>In this talk, Simon compares the old fashioned way to tackle these issues and present how it will look like as soon as the concepts and a bunch of other proposals arrive in C++20:</p>
<ul>
<li>Explicit operator akin to the noexcept operator</li>
<li><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0848r0.html">P0748R0</a> to use <strong>concepts-based require clauses</strong> on these member functions to enable or disable them. A.K.A goodbye a lot of uncessarry conditional inheritance. </li>
<li><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0847r0.html">P0847R0</a> which permits to deduce correctly the implicit <strong>this</strong> parameter type of a member function. A.K.A goodbye all the methods overloading on const, volatile and ref qualifiers. </li>
<li><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0515r2.pdf">P0515R2</a> that unify all the comparison operators into one. A.K.A goodbye all the operator overloads.</li>
<li>... Some more that I surely forgot.</li>
</ul>
<p>Regardless if you are planning to write such wrappers or not, I would suggest to watch the talk to refresh yourself on some tricky C++ mechanisms.</p>
<h3>[Talk] Fancy Pointers for Fun and Profit - Bob Steagall - 💀💀 ★★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Presentations/fancy_pointers_for_fun_and_profit/fancy_pointers_for_fun_and_profit__bob_steagall__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p><strong>Bob Steagall</strong> promoted his own talk on <strong>fancy pointers</strong> in an early September <strong>CppCast</strong> episode.
So here I was, ready to learn more about these mystical beasts!</p>
<p><strong>Allocators</strong> in C++ are rather infamous for their over-engineered interface, which is not useful 99.42% of the time. This even forced the committee to come-up, in <strong>C++17</strong>, with a lighter interface called <a href="https://youtu.be/v3dz-AKOVL8">PMR</a>. But this time, the good old full-blown interface found a very clever usage in Bob's hands. Indeed, <a href="https://en.cppreference.com/w/cpp/memory/allocator_traits">std::allocator_traits</a> has a nice type property <strong>pointer</strong>. Which means that the Standard offers a nice <a href="https://quuxplusone.github.io/blog/2018/03/19/customization-points-for-functions/">customization point</a> to switch normal pointers <strong>T*</strong> for a given allocator to a type that acts like a pointer would. These wanna-be pointers are what you call <strong>fancy pointers</strong>. Somehow, you can think of <strong>fancy pointers</strong> as more generalized concept of <a href="https://en.wikipedia.org/wiki/Smart_pointer">smart pointers</a>. </p>
<p>Now let's say that you would like to serialise to / deserialise from binary a container-like object (vector, list...) with a bunch of <a href="https://msdn.microsoft.com/en-us/library/mt767760.aspx?f=255&MSPPError=-2147217396">trivial objects</a> inside and send it through a network. Providing that you are targeting one and only one architecture, which is often the case for servers, you may be able to use <a href="https://en.cppreference.com/w/cpp/string/byte/memcpy">std::memcpy</a> to transfer the representation of this container into a <strong>char*</strong> buffer. Then, the whole buffer can be wired to another machine. At the endpoint, to deserialise the container from that buffer you can re-use <strong>std::memcpy</strong> to copy back the binary representation into a container (<a href="https://www.reddit.com/r/cpp/comments/5fk3wn/undefined_behavior_with_reinterpret_cast/">note that you cannot rely reinterpret_cast in C++ for that purpose</a>...). This will work smoothly as long as none of the stored PODs have pointers as members referencing each others! Pointers are usually not valid as soon as you cross the boundary of a process or a machine. This huge drawback can be avoided by introducing <strong>fancy pointers</strong> to your code-base. </p>
<p>As an example on how to fix that issue, Bob brings <strong>offset_ptr</strong> to the table, which permits to express some reference between two elements using their distance from each others:</p>
<div class="highlight"><pre><span></span><code><span class="c">struct obj</span>
<span class="c">{</span>
<span class="c"> value_type v;</span>
<span class="c"> offset_ptr</span><span class="nv"><</span><span class="c">value_type</span><span class="nv">></span><span class="c"> p; // optional</span>
<span class="c">}; </span>
<span class="c">Example of the layout of a container of objs:</span>
<span class="c"> </span><span class="nb">-</span><span class="c">2</span>
<span class="c"> </span><span class="nb">+-----------------------------+</span><span class="c"></span>
<span class="c"> | 1 |</span>
<span class="c"> | </span><span class="nb">+-----+</span><span class="c"> |</span>
<span class="c"> </span><span class="nb">+--------------</span><span class="c">v</span><span class="nb">-----------</span><span class="c">v</span><span class="nb">--------------------------------+</span><span class="c"></span>
<span class="c"> | v | p | v | p | v | p | V | p | v | p |</span>
<span class="c"> </span><span class="nb">+-----------------------------------------------------------+</span><span class="c"></span>
<span class="c"> | ^</span>
<span class="c"> | |</span>
<span class="c"> </span><span class="nb">+-----------------------------+</span><span class="c"></span>
<span class="c"> 3</span>
</code></pre></div>
<p>With a bit of boilerplate, this <strong>offset_ptr</strong> can be handled by a custom allocator that can be injected into a container, making different address mappings a non-issue.
I find this solution pretty elegant and is a good showcase on how extensible the Standard Library is. </p>
<h3>[Talk] RVO is Harder than it Looks: the story of -Wreturn-std-move - Arthur O'Dwyer - 💀💀 ★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Presentations/return_value_optimization_harder_than_it_looks/return_value_optimization_harder_than_it_looks__arthur_odwyer__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>It is commonly admitted that returning by value seldom has a performance impact in C++. Two mechanisms <a href="https://en.wikipedia.org/wiki/Copy_elision#Return_value_optimization">(N)RVO</a> and <a href="http://thbecker.net/articles/rvalue_references/section_02.html">move semantics</a> will most likely kick in to avoid unecessary copies:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span>
<span class="p">{</span>
<span class="c1">// ... some members</span>
<span class="p">};</span>
<span class="n">A</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="k">return</span> <span class="n">a</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">A</span> <span class="n">a</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span> <span class="c1">// The return value is directly constructed in a's stack location (RVO), can fall back onto the move-constructor otherwise. </span>
</code></pre></div>
<p>As time goes by, the C++ standard has stronger and stronger guarantees that copy ellision (RVO) will happen in these situations.
At the same time, forcefully moving the return value can a pretty huge pessimisation and is taught as an anti-pattern:</p>
<div class="highlight"><pre><span></span><code><span class="n">A</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">a</span><span class="p">);</span> <span class="c1">// Will hinder the compiler to choose RVO instead of move constructor.</span>
<span class="p">}</span>
</code></pre></div>
<p>In the worst case scenario, if the object has no move-constructor, the compiler might resort to use the copy constructor, which could have been avoided with RVO.</p>
<p>Now in the <strong>C++ land</strong> nothing really holds true if you look closer at some corner cases. And the "no-move-on-return-values" rule mentioned right above can be debated for that reason. Arthur was valiant enough to inquire into this topic and found few cases where a call to <strong>std::move</strong> will BE an optimization. Notably, if you return a value with a type convertible to the function's return type thanks to an <a href="https://en.cppreference.com/w/cpp/language/cast_operator">explicit conversion operator</a>, you should apply <strong>std::move</strong>. Arthur introduced a new warning in clang <a href="https://reviews.llvm.org/D43322">-Wreturn-std-move</a> to avoid this pitfall. I will gadly turn that warning on as soon as I can.</p>
<p>I liked the talk for delving into such precise topics ; although, Arthur rushed on quite a few slides and even skipped a whole bunch of them, meaning that there was more to say on this theme. </p>
<h3>[Talk] State Machines Battlefield - Naive vs STL vs Boost - Kris Jusiak - 💀💀 ★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Presentations/state_machines_battlefield_naive_vs_stl_vs_boost/state_machines_battlefield_naive_vs_stl_vs_boost__kris_jusiak__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=yZVby-PuXM0">link</a></li>
</ul>
<p><strong>Kris Jusiak</strong> is the proud author of two library aspiring to be into <a href="https://www.boost.org/">Boost</a>: <a href="http://boost-experimental.github.io/di/">Boost.DI</a> and <a href="https://github.com/boost-experimental/sml">Boost.SML</a>. This talk was partly based on the later one. More precisely <strong>Kris</strong> compared how different implementations of a state machine would fare in term of performance and ease to maintain.</p>
<p><strong>Kris</strong> started with a good ol' implementation designed around a giant switch case roughly similar to this code:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">connection</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="kt">void</span> <span class="nf">update</span><span class="p">(</span><span class="n">event</span> <span class="n">e</span><span class="p">)</span> <span class="p">{</span>
<span class="k">switch</span><span class="p">(</span><span class="n">state_</span><span class="p">)</span> <span class="p">{</span>
<span class="k">case</span> <span class="nl">connecting</span><span class="p">:</span>
<span class="k">if</span> <span class="p">(</span><span class="n">e</span> <span class="o">&&</span> <span class="n">e</span><span class="o">-></span><span class="n">type</span> <span class="o">==</span> <span class="n">event_type</span><span class="o">::</span><span class="n">established</span><span class="p">)</span> <span class="p">{</span>
<span class="n">state_</span> <span class="o">=</span> <span class="n">state</span><span class="o">::</span><span class="n">connected</span><span class="p">;</span>
<span class="n">log</span><span class="p">(</span><span class="s">"connected"</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="nl">connected</span><span class="p">:</span>
<span class="c1">// ...</span>
<span class="n">do_connected_things</span><span class="p">();</span>
<span class="c1">// ...</span>
<span class="k">break</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="k">default</span><span class="o">:</span>
<span class="k">throw</span> <span class="n">runtime_exception</span><span class="p">(</span><span class="s">"bad state"</span><span class="p">);</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="kt">int</span> <span class="n">variable_for_connected_state</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">another_variable_for_connected_state</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">variable_for_disconnected_state</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="n">state</span> <span class="n">state_</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>Surely, this implementation will perform rather decently, but at the cost of being extremely hard to maintain if the amount of states increase. Sadly, a lot of code-bases for games or networking have plenty of these ugly state machines sprinkled around. <strong>C++</strong> is all about <strong>zero-cost abstractions</strong>, which means that if you want to avoid some serious posttraumatic stress disorders after working on such projects, you may want to look at other choices than switch.</p>
<p>Therefore, <strong>Kris</strong> jumped onto other implementations. One of the them is using <a href="https://en.cppreference.com/w/cpp/utility/variant">std::variant</a> which reminded me a lot a blog post from <a href="https://khuttun.github.io/2017/02/04/implementing-state-machines-with-std-variant.html">Kalle Huttunen</a>. <strong>std::variant</strong> will permit you to isolate the variables necessary for your different states and will enforce a stricter handling of your state with <a href="https://en.cppreference.com/w/cpp/utility/variant/visit">std::visit</a>. In my opinion this solution is huge improvement compared to using a switch and does not require the introduction of an external library into your project. As I will explain later, <strong>std::variant</strong> may or may not have a slight performance impact.</p>
<p>After dwelling with two oldish and rather slow Boost libraries that can help to design state machines, Kris presented us his work. I have to admit that the <a href="https://en.wikipedia.org/wiki/Domain-specific_language">DSL</a> provided by his library looks very pleasant to use: </p>
<div class="highlight"><pre><span></span><code><span class="c1">// Coming straight from Kris's slides:</span>
<span class="n">sml</span><span class="o">::</span><span class="n">sm</span> <span class="n">connection</span> <span class="o">=</span> <span class="p">[]{</span>
<span class="k">using</span> <span class="k">namespace</span> <span class="nn">sml</span><span class="p">;</span>
<span class="k">return</span> <span class="n">transition_table</span><span class="p">{</span>
<span class="o">*</span> <span class="s">"Disconnected"</span><span class="n">_s</span> <span class="o">+</span> <span class="n">event</span><span class="o"><</span><span class="n">connect</span><span class="o">></span> <span class="o">/</span> <span class="n">establish</span> <span class="o">=</span> <span class="s">"Connecting"</span><span class="n">_s</span><span class="p">,</span>
<span class="s">"Connecting"</span><span class="n">_s</span> <span class="o">+</span> <span class="n">event</span><span class="o"><</span><span class="n">established</span><span class="o">></span> <span class="o">=</span> <span class="s">"Connected"</span><span class="n">_s</span><span class="p">,</span>
<span class="s">"Connected"</span><span class="n">_s</span> <span class="o">+</span> <span class="n">event</span><span class="o"><</span><span class="n">ping</span><span class="o">></span> <span class="p">[</span> <span class="n">is_valid</span> <span class="p">]</span> <span class="o">/</span> <span class="n">reset_timeout</span><span class="p">,</span>
<span class="s">"Connected"</span><span class="n">_s</span> <span class="o">+</span> <span class="n">event</span><span class="o"><</span><span class="n">timeout</span><span class="o">></span> <span class="o">/</span> <span class="n">establish</span> <span class="o">=</span> <span class="s">"Connecting"</span><span class="n">_s</span><span class="p">,</span>
<span class="s">"Connected"</span><span class="n">_s</span> <span class="o">+</span> <span class="n">event</span><span class="o"><</span><span class="n">disconnect</span><span class="o">></span> <span class="o">/</span> <span class="n">close</span> <span class="o">=</span> <span class="s">"Disconnected"</span><span class="n">_s</span>
<span class="p">};</span>
<span class="p">};</span>
</code></pre></div>
<p><strong>Boost.DI</strong> is performing very well according to Kris and is on par with the switch solution according to his benchmark.
<strong>Boost.DI</strong> offers different dispatch strategies to get the current state: <strong>recursive branching</strong>, <a href="https://mpark.github.io/programming/2015/07/07/variant-visitation/">jump table</a>, <a href="https://en.cppreference.com/w/cpp/language/fold">fold expressions</a>... It turns out that the <strong>recursive branching</strong> is amongst the fastest, yelding results as close as the giant switch strategy. I am not so surprised by these results, since we observed a similar pattern at work with our custom implementation of <strong>std::visit</strong>. As far as I know, <strong>clang</strong> and <strong>gcc</strong> visit their <strong>std::variant</strong> using a <strong>jump table</strong>, which may explain the slight performance drop compared to a giant switch. These are good news though, it means that there is room to improve the Quality of Implementation (QoI) of <strong>std::visit</strong> in our favorite libraries. </p>
<h3>[Talk] Compile-time programming and reflection in C++20 and beyond - Louis Dionne - 💀💀💀 ★★★:</h3>
<ul>
<li>Slides: <a href="">coming soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=CRDNPwXDVp0">link</a></li>
</ul>
<p>Three skulls, three stars, nothing unusual when it comes to my judgement on <strong>Louis Dionne's</strong> talks. I am very fond of (template) meta-programming, and I have always been in awe of Louis's work on <a href="https://www.boost.org/doc/libs/1_61_0/libs/hana/doc/html/index.html">Boost.Hana</a> and more recently <a href="https://github.com/ldionne/dyno">dyno</a>. This year, he was on stage to give us an overview on what we could expect in the upcomming standards concerning <a href="https://en.cppreference.com/w/cpp/language/constexpr">constexpr</a>, and how this would unlock a better interface for reflection.</p>
<p>We are slowly but surely reaching the point where we will be able to "allocate" at compile-time and convert most of our code-bases to <strong>constexpr</strong> within a blink. Louis explained what are the necessary changes we need to apply to <a href="https://en.cppreference.com/w/cpp/language/constexpr">constexpr</a> to be able to use it in expressions where we do allocate:</p>
<ul>
<li>Allowing constexpr non-trivial destructors, allowing heap allocation and placement new that you will find in <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0784r0.md">P0784R0</a>.</li>
<li>Having the new trait <strong>std::is_constant_evaluated</strong> from <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0595r1.html">P0595R0</a> that queries whether the compiler is currently evaluating the function in a constexpr context or not. Surprisingly, you will NOT use that trait within a <strong>if constexpr</strong> statement ; this would always be evaluated as constexpr and return <strong>true</strong>, a simple <strong>if</strong> does the job. This trait is an absolute necessity if we want to share a single interface for both a <strong>constexpr</strong> and runtime implementation of a feature (a std::vector...). Behind the scene, <strong>constexpr</strong> code usually has very different demands to perform corretly than standard runtime code. </li>
<li>Support <strong>try-catch</strong> statements within a <strong>constexpr</strong> expression which we would get from <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1002r0.pdf">P1002R0</a>. Note that this does imply that the compiler</li>
<li>Some other minor changes that must appear in some other hairy white papers ...</li>
</ul>
<p>Taking all these changes in consideration, we should be able to slap <strong>constexpr</strong> on many containers and algorithms from the STL (vector, string...). That would make the usage of <strong>constexpr</strong> very trivial to any decent <strong>C++</strong> developer.</p>
<p>It will also be a great paradigm shift for the planned reflection within the language. The standard committee used to formulate a reflection proposal based on <strong>template meta-programming</strong>, which dreadfully reminds you some kind of <a href="https://www.boost.org/doc/libs/1_68_0/libs/mpl/doc/index.html">Boost.MPL</a>.
While templates are powerfull, the syntax to manipulate types appears alienesque to most of the human coders.
<strong>Constexpr-based metaprogramming</strong> looks a lot more natural and having proper containers was the last missing part of the puzzle to use that syntax for reflection.
If you are in doubt, have a look at this very short example from Louis: </p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">my_struct</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">x</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">y</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="p">};</span>
<span class="c1">// Get the type of the first member of my_struct using the old template-based syntax:</span>
<span class="k">using</span> <span class="n">my_struct_meta</span> <span class="o">=</span> <span class="n">reflexpr</span><span class="p">(</span><span class="n">my_struct</span><span class="p">);</span>
<span class="k">using</span> <span class="n">members</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">reflect</span><span class="o">::</span><span class="n">get_data_members_t</span><span class="o"><</span><span class="n">my_struct</span><span class="o">></span><span class="p">;</span> <span class="c1">// Some weird template list-like type.</span>
<span class="k">using</span> <span class="n">x_meta</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">reflect</span><span class="o">::</span><span class="n">get_element_t</span><span class="o"><</span><span class="mi">0</span><span class="p">,</span> <span class="n">members</span><span class="o">></span><span class="p">;</span> <span class="c1">// Some ideaous index accessor.</span>
<span class="k">using</span> <span class="n">x_type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">reflect</span><span class="o">::</span><span class="n">get_reflected_type_t</span><span class="o"><</span><span class="n">x_meta</span><span class="o">></span><span class="p">;</span>
<span class="c1">// Get the type of the first member of my_struct with the new fancy constexpr-based syntax:</span>
<span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">reflect</span><span class="o">::</span><span class="n">Record</span> <span class="n">my_struct_meta</span> <span class="o">=</span> <span class="n">reflexp</span><span class="p">(</span><span class="n">my_struct</span><span class="p">);</span>
<span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span> <span class="n">members</span> <span class="o">=</span> <span class="n">my_struct_meta</span><span class="p">.</span><span class="n">get_data_members</span><span class="p">();</span> <span class="c1">// Uses the good ol' vector and class template argument deduction guides from C++17.</span>
<span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">reflect</span><span class="o">::</span><span class="n">RecordMember</span> <span class="n">x_meta</span> <span class="o">=</span> <span class="n">members</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span> <span class="c1">// Just use the operator[] as usual... </span>
<span class="k">using</span> <span class="n">type</span> <span class="o">=</span> <span class="n">unreflexpr</span><span class="p">(</span><span class="n">x_meta</span><span class="p">.</span><span class="n">get_reflected_type</span><span class="p">());</span> <span class="c1">// Get that actual type of x.</span>
</code></pre></div>
<p>If you want to have a better understanding on the proposed syntax, have a look at <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0953r0.html">P0962R0</a>.</p>
<h3>[Keynote] Thoughts on a More Powerful and Simpler C++ (5 of N) - Herb Sutter - 💀💀 ★★★:</h3>
<ul>
<li>Slides: <a href="">coming soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=80BZxujhY38">link</a></li>
</ul>
<p>The last two years, at CppCon, Herb brought us his vision on a future <strong>C++</strong> full of promises.
Both of these talks were accompanied with some concrete actions (white-papers, guidelines, proof-of-concepts..) that Herb was working on with the rest of the fellowship of the C++s. This year, Herb shared with us some more results on his goals.
It might not sound like a thrilling talk... but that would be under-appreciating the two main ideas Herb was initially pushing for: lifetimes and meta-classes.</p>
<p>Lifetimes are some implicit or explicit rules that directly concern the ownership of an object.
If such lifetime rules are adjusted correctly, your code should be bulletproof when it comes to a huge range of bugs related to memory: user after free, dandling pointers...
Some languages like <strong>Rust</strong> even make it a core concept of the language. Arguably, Herb's lifetimes will be slightly more relaxed (no annotations on everything) and more natural to use, at the price of not covering some extreme cases.
Let's have a look at what these so-called lifetimes may protect you from:</p>
<div class="highlight"><pre><span></span><code><span class="kt">int</span><span class="o">&</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">a</span><span class="p">;</span>
<span class="k">return</span> <span class="n">a</span><span class="p">;</span> <span class="c1">// Oups I am returning a reference to a local variable that will die right after that function execution.</span>
<span class="c1">// Some compilers may warn you about it, some may not! </span>
<span class="p">}</span>
<span class="n">std</span><span class="o">::</span><span class="n">reference_wrapper</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">a</span><span class="p">;</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">reference_wrapper</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="n">a</span><span class="p">);</span> <span class="c1">// Same issue. No compiler warns you about it! </span>
<span class="p">}</span>
</code></pre></div>
<p>After applying the rules elaborated by Herb and his crew, the lifetime of <strong>a</strong> would be dimmed as ending at the end of foo and the compiler would yield a strong warning or a plain error.
Here <a href="https://en.cppreference.com/w/cpp/utility/functional/reference_wrapper">std::reference</a> is considered as a pointer/reference type and will be highly scrutinised by the compiler.
If you combine the lifetimes and the concepts, your compiler or linter may be able to discover the pointer types automagically!</p>
<p>Another trivial bug yet often spawning nastily in your code is the dreaded "use-after-move" situation. Here again, lifetimes would avoid an easy shoot in the feet situation:</p>
<div class="highlight"><pre><span></span><code><span class="n">my_class</span> <span class="n">my_obj</span><span class="p">;</span>
<span class="n">my_class</span> <span class="n">another_obj</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">my_obj</span><span class="p">);</span>
<span class="n">my_obj</span><span class="p">.</span><span class="n">x</span><span class="o">-></span><span class="n">bla</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span> <span class="c1">// lifetime warning: using a moved-from obj is seldom a good idea.</span>
</code></pre></div>
<p>All these smart lifetime rules are often based on recommendations that you may find in <a href="https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines">C++ Core Guidelines</a>.
Having them enforced within your projects is amazing. I am eager to try the clang implementation of it. Later in the day <strong>Matthias Gehre</strong> and <strong>Gabor Horvath</strong> did show us the internals of clang that will support this new feature.</p>
<p>After mesmering the crowd with the lifetimes, Herb gave us some updates on the <a href="https://www.youtube.com/watch?v=4AfRAVcThyA&t=4016s">meta-classes</a>, which were mainly some changes in the syntax.
While I really appreciate the efforts put into <strong>meta-classes</strong>, I still have doubts that I will enjoy such a feature before I am retiring (roughly in 50 years from now). The lifetimes were much more concrete and fathomable when it comes to my daily C++ life. </p>
<h3>[Talk] Better C++ using Machine Learning on Large Projects - Nicolas Fleury and Mathieu Nayrolles - 💀 ★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Presentations/better_cpp_using_machine_learning_on_large_projects/better_cpp_using_machine_learning_on_large_projects__nicolas_fleury_mathieu_nayrolles__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p>You can certainely rely on C++ to improve your AI projects, but can you use an AI or machine learning to improve your C++ project?
The two "cousin-frenchies" <strong>Nicolas</strong> and <strong>Mathieu</strong> had the smart idea to detect bugs in pull-requests using some kind of machine learning that analyse previously reported issues in their code-base.</p>
<p>The presentation did not contain much of actual C++ code, but was more focused on the process they invented to automatically fetch, analyse and post feedback on any submitted code.
I am not an expert on these topics and would not dare to emit any comment on what they presented us.
It seems that after training, the classifying algorithm they put in place was able to predict with a success rate of 70% whether a piece of code would have a negative impact or not.
Their next step would be to add some automatic code correction facilities by applying machine learning on the fixed cases.
Triple A games tend to reuse a lot of variations of the same code across multiple titles, WITHOUT actually sharing it (new games are just cloned from old ones). With this process in place, the projects are spreading the awareness of some issues very easily. It seems like a huge time saver.</p>
<p>In any case, it was a breeze to attend a slightly less C++ oriented talk.
There was a lot of questions regarding the human aspect of that technology.
Is 70% of success rate high enough not to piss-off the users experimenting with the bot?
My experience is that a lot of false positive in a linter, will invariably make people turn it off at the earliest opportunity...
Would you be able to spot the bad programmers in your team with such a tool? Thanksfully, the labor rights in Canada (Montréal) should protect the employees on that topic...
And many other interesting facts that you can discover in the video.</p>
<h3>[Talk] Class Template Argument Deduction for Everyone - Stephan T. Lavavej - 💀💀 ★★:</h3>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2018/blob/master/Presentations/class_template_argument_deduction_for_everyone/class_template_argument_deduction_for_everyone__stephan_t_lavavej__cppcon_2018.pdf">link</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p><a href="https://en.cppreference.com/w/cpp/language/class_template_argument_deduction">Class Template Argument Deduction</a>, also known as CTAD, is new tiny feature added into the C++17.
While not being an amazing game changer, CTAD can been seen as some very tasty syntaxic sugar that avoid you to specific the template argument of a class template when instantiating it.
In other simpler words: it can avoid you to call <strong>make_xxx</strong> function when there is enough information for the compiler to deduce the template paramters of a class template.
Here is what the CTAD lipstick looks like on <a href="https://en.cppreference.com/w/cpp/utility/pair">std::pair</a>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Before C++17:</span>
<span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*></span> <span class="n">p1</span> <span class="o">=</span> <span class="p">{</span><span class="mi">42</span><span class="p">,</span> <span class="s">"test"</span><span class="p">};</span>
<span class="k">auto</span> <span class="n">p2</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_pair</span><span class="p">(</span><span class="mi">42</span><span class="p">,</span> <span class="s">"test"</span><span class="p">);</span>
<span class="c1">// After C++17, with CTAD:</span>
<span class="n">std</span><span class="o">::</span><span class="n">pair</span> <span class="n">p3</span> <span class="o">=</span> <span class="p">{</span><span class="mi">42</span><span class="p">,</span> <span class="s">"test"</span><span class="p">};</span> <span class="c1">// No needs to specify the template argument "int" and "const char*".</span>
<span class="k">auto</span> <span class="n">p4</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="p">(</span><span class="mi">42</span><span class="p">,</span> <span class="s">"test"</span><span class="p">);</span> <span class="c1">// Another way to construct following the always auto rule.</span>
</code></pre></div>
<p>In many instances, you do not have to update your class template to benefit from CTAD.
But when you do, one must understand how to help the compiler using some deduction guides.
<strong>STL</strong>'s (known as Stephan T. Lavavej) dedicated and still dedicate his life to maintain the <strong>STL</strong> (also known as Standard Template Library) for MSVC.
<strong>Stephan</strong> apparently had a first hand experience on the deduction guides when adding CTAD to the standard containers in the STL and wanted to explain the gist of it.</p>
<p>Deduction guides are "pseudo-constructors" declared out of the targeted class, that are evaluated right before going to the steps of template parameter deduction, substitution and all the subsequent mess.
When instantiating a given class, all the deduction guides follow the overload resolution and template argument deduction rules that you would expect if applied on normal functions.
The return type of the chosen deduction guide, will be the one used by the following steps.
Now this sounds very wordy, but it is actually fairly trivial to write:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">foo</span> <span class="c1">// A trivial class template. </span>
<span class="p">{</span>
<span class="n">foo</span><span class="p">(</span><span class="n">T</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{}</span> <span class="c1">// Do something with t...</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">foo</span><span class="p">(</span><span class="n">T</span> <span class="n">t</span><span class="p">)</span> <span class="o">-></span> <span class="n">foo</span><span class="o"><</span><span class="n">T</span><span class="o">&></span><span class="p">;</span> <span class="c1">// Declare a deduction guide: given a type T, I will help to deduce a foo with a first parameter being a reference to this type T. </span>
<span class="kt">int</span> <span class="n">a</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">bar</span> <span class="o">=</span> <span class="n">foo</span><span class="p">(</span><span class="n">a</span><span class="p">);</span> <span class="c1">// Bar will be foo<int&> thanks to the CTAD and this deduction guide.</span>
</code></pre></div>
<p>I chose this example as it has two interesting tidbits. First you will notice that I apply a transformation on <strong>T</strong> in the return type: the template parameter becomes a <strong>T*</strong>.
It turns out that you can do a lot more in this place: you can invoke <s>satan</s> some traits or inject a SFINAE expression (Oh my...! I really have to push that idea further).
The second unexpected part is that my guide does not have the same signature as my constructor. Indeed, one takes T as an r-value reference, the other one by value.
That's really fortunate, unlike the <strong>make_xxx</strong> functions which would take universal references and <a href="https://en.cppreference.com/w/cpp/types/decay">decay</a> the arguments, the deductions guides can rely on the automatic decaying of template parameters taken by value. <strong>Stephan</strong> has a lot more of nitty-gritty details on how <strong>deduction guides</strong> behave and it would take a full a post to explain some of them, just watch his talk instead!</p>
<h3>[Talk] The Bits Between the Bits: How We Get to main() - Matt Godbolt - 💀💀 ★★★:</h3>
<ul>
<li>Slides: <a href="">coming soon</a></li>
<li>Video: <a href="">coming soon</a></li>
</ul>
<p><strong>Matt Godbolt</strong>, author of the amazing website <a href="https://godbolt.org/">godbolt.org</a>, has a very pedagogic mindset and is a naturally gifted speaker (if you ever wanted to understand the <a href="https://meltdownattack.com/">Meltdown Attack</a>, you should make a detour on his <a href="https://www.youtube.com/watch?v=IPhvL3A-e6E&t=692s">YouTube video</a> and come back to this blog post afterwards).
This time <strong>Matt</strong> wanted us to have a closer look at the hidden steps from the linking stage of your build until the beggining of the execution of your main entry point.
More precisely, how your linker select the symbols that will appear into your application and which are the mechanisms that allow the creation of global variables right before entering <strong>main</strong>. </p>
<p>Being didactic as usual, <strong>Matt</strong> did some live debugging sessions, using GDB, objdump, readelf and such, to make us conceive how things work under the hood.
It covered the sections you can find within an application (.data, .text, ...), the <a href="https://en.wikipedia.org/wiki/One_Definition_Rule">One Definition Rule</a> (ODR), the linker's gathering of informations, the mysterious <strong>__static_initialization_and_destruction_0</strong> and its associates... His approach of solving one problem at a time using simple tools make it very easy to comprehend fully what is going on in there. </p>
<p>I made two discoveries during that one hour debugging session:</p>
<ul>
<li>LD, the GNU linker (and very likely clang's one too) uses a <a href="https://sourceware.org/binutils/docs/ld/Scripts.html">scripting language</a> to define the rules for each section of your binary. I wish to never have to dabble in this language for work purpose. </li>
<li><a href="http://man7.org/linux/man-pages/man8/ld.so.8.html">ld.so</a>, the GNU dynamic linker reacts on an environment variable called <strong>LD_DEBUG</strong>. By setting this variable to =all (or something more precise), the dynamic linker will output all operations and some extra info when loading a dynamic library. It is very convenient if you want to know which libraries get loaded by your process, which symbols it will use, etc... Here is what the output would look like if your process is somehow fetching <a href="http://man7.org/linux/man-pages/man3/getenv.3.html">getenv</a>:</li>
</ul>
<div class="highlight"><pre><span></span><code> 15257: binding file wc [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `getenv' [GLIBC_2.2.5]
15257: symbol=abort; lookup in file=wc [0]
15257: symbol=abort; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
</code></pre></div>
<p>Here we know the <strong>getenv</strong> resides within the libc.so library which is the C runtime library.</p>
<h3>[Other]:</h3>
<p>Sadly, I would need to (std::)allocate myself a lot of more time to be able to cover all the goodies you can get from this <strong>CppCon</strong>.
I have watched many different talks and had great echoes from some others. Here is what I would consider worth to google for:</p>
<ul>
<li>Spectre: Secrets, Side-Channels, Sandboxes, and Security - by Chandler Carruth</li>
<li>The Nightmare of Initialization in C++ - by Nicolai Josuttis</li>
<li>The Salami Method for Cross Platform Development - by Adi Shavit</li>
<li>Compile Time Regular Expressions - by Hana Dusíková</li>
<li>And many more...</li>
</ul>
<h1>Conclusion:</h1>
<p>The week I spent fully immersed in C++ was really special! A bunch of passionated and dedicated persons can build really great events.
Hopefully, I will be able to join in denver next year or the year after, maybe as a <strong>speaker</strong> this time (I should not stay a simple lurker forever ;))!
If not, be sure to find me at some other C++ conferences in Europe (Meeting C++, C++ On Sea...) or some local meetup groups, this community is trully amazing.</p>Meta Crush Saga: a C++17 compile-time game2018-05-19T23:30:00+02:002018-05-19T23:30:00+02:00Jean Gueganttag:jguegant.github.io,2018-05-19:/blogs/tech/meta-crush-saga.html<h2>Trivia:</h2>
<p>As a quest to obtain the highly coveted title of <strong>Lead Senior C++ Over-Engineer</strong>, I decided last year to rewrite the game I work on during daytime (Candy Crush Saga) using the quintessence of modern C++ (C++17). And... thus was born <a href="https://github.com/Jiwan/meta_crush_saga">Meta Crush Saga</a>: a <strong>compile-time game</strong>. I …</p><h2>Trivia:</h2>
<p>As a quest to obtain the highly coveted title of <strong>Lead Senior C++ Over-Engineer</strong>, I decided last year to rewrite the game I work on during daytime (Candy Crush Saga) using the quintessence of modern C++ (C++17). And... thus was born <a href="https://github.com/Jiwan/meta_crush_saga">Meta Crush Saga</a>: a <strong>compile-time game</strong>. I was highly inspired by <a href="https://blog.mattbierner.com/stupid-template-tricks-snake/">Matt Bernier's Nibbler game</a> that used pure template meta-programming to recreate our the famous snake game we could play on our Nokia 3310 back in the days.</p>
<p>"What the <s>hell</s> heck is a <strong>compile-time game</strong>?", "How does it looks like?", "What <strong>C++17</strong> features did you use in this project?", "What was your learnings?" might come to your mind. To answer these questions you can either read the rest of this post or accept your inner laziness and watch the video version of this post, which is a talk I made during a <a href="https://www.meetup.com/swedencpp/events/246069743/">Meetup event</a> in Stockholm:</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/XV1lXtB3sqg" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>Disclaimer: for the sake of your sanity and the fact that <em>errare humanum est</em>, this article might contain some alternative facts.</p>
<h2>A compile-time game you said?</h2>
<p><img width=20% height=20% src="https://jguegant.github.io/blogs/tech/images/what-does-this-mean.png"/></p>
<p>I believe that it's easier to understand what I mean by the "concept" of a <strong>compile-time game</strong> if you compare the life cycle of such a game with the life cycle of a normal game. So here it is!</p>
<h3>Life cycle of a normal game:</h3>
<p>As a normal game developer with a normal life working at a normal job with a normal sanity level you would usually start by writing your <strong>game logic</strong> using your favorite language (C++ of course!), then fire your <strong>compiler</strong> to transform this, far too often spaghetti-like, logic into an <strong>executable</strong>. As soon as you double-click on your <strong>executable</strong> (or use the console), a <strong>process</strong> will be spawned by your operating system. This <strong>process</strong> will execute your <strong>game logic</strong> which 99.42% of time consists of a <strong>game loop</strong>. A <strong>game loop</strong> will <strong>update</strong> the state of your game according to some rules and the <strong>user inputs</strong>, <strong>render</strong> the newly computed state of your game in some flashy pixels, again, again and again.</p>
<p><img alt="Life cycle of a normal game" src="https://jguegant.github.io/blogs/tech/images/life-cycle-normal-game.png"></p>
<h3>Life cycle of a compile-time game:</h3>
<p>As an over-engineer cooking the next big compile-time game, you will still have use of your favorite language (still C++ of course!) to write your <strong>game logic</strong>. You will still have a <strong>compilation phase</strong> but... then... here comes the plot twist: you will <strong>execute</strong> your <strong>game logic</strong> within this compilation step. One could call it a compilutation. This is where C++ truly comes in handy ; it has a some features like <a href="https://en.wikipedia.org/wiki/Template_metaprogramming">Template Meta Programming (TMP)</a> or <a href="http://en.cppreference.com/w/cpp/language/constexpr">constexpr</a> to actually have <strong>computations</strong> happening during the <strong>compilation phase</strong>. We will dive later on the features you can use to do so. As we are executing the <strong>logic</strong> of the game during this phase, we must also inject the <strong>user inputs</strong> at that point in time. Obviously, our compiler will still output an <strong>executable</strong>. What could it be used for? Well, the executable will not contain any <strong>game loop</strong> anymore, but it will have a very simple mission: output the newly <strong>computed state</strong>. Let's name this <strong>executable</strong> a <strong>renderer</strong> and its <strong>output</strong> the <strong>rendering</strong>. Our <strong>rendering</strong> won't contain any fancy particule effect nor ambiant occlusion shadows, it will be in ASCII. An ASCII <strong>rendering</strong> of your newly computed <strong>state</strong> has the nice property that you can easily show it to your player, but you also copy it into a text file. Why a text file? Obviously because you can combine it with your <strong>code</strong> in some way, redo all the previous steps and therefore have a <strong>loop</strong>.</p>
<p>As you may understand now, a <strong>compile-time</strong> game is made of a <strong>game-loop</strong> where each <strong>frame</strong> of your game is a <strong>compilation step</strong>. Each <strong>compilation step</strong> is computing a new <strong>state</strong> of your game, that you can present to your player and also inject to the following <strong>frame</strong> / <strong>compilation step</strong>. </p>
<p>I let you contemplate this magnificient diagram for as much time as it takes you to understand what I just wrote above:
<img alt="Life cycle of a compile-time game" src="https://jguegant.github.io/blogs/tech/images/life-cycle-compile-time-game.png"></p>
<p>Before we move on the implementation details of such a loop, I am sure that you have one question you would like to shoot at me...</p>
<h3>"Why would you even do that?"</h3>
<p><img width=25% height=25% src="https://jguegant.github.io/blogs/tech/images/why-would-you-even-do-that.png"/></p>
<p>Do you really think I would let you break my C++ meta-programming idyll with such a fundamental question? Never!</p>
<ul>
<li>First and foremost, a <strong>compile-time game</strong> will have amazing runtime performances since most of the computations are done during the <strong>compilation phase</strong>. Runtime performance is a key to the success of your AAA game in ASCII art! </li>
<li>You lessen the probability that a wild crustacean appears in your github repository and ask you to rewrite your game in <strong>Rust</strong>. His well-prepared speech on security will fall appart as soon as you explain that a dangling pointer cannot exist at compile-time. Smug <strong>Haskell</strong> programmers might even approve the <strong>type-safety</strong> of your code.</li>
<li>You will gain respect from the <strong>Javascript</strong> hipster kingdom, where any over-complicated framework with a strong NIH syndrom can reign as long as it has a catchy name.</li>
<li>One of my friend used to say that any line of code from a Perl program provides you de-facto a very strong password. I surely bet that he never tried generating credentials from <strong>compile-time C++</strong>.</li>
</ul>
<p>So what? Aren't you satisfied with my answers? Then, maybe your question should have been: "Why could you even do that?".</p>
<p>As a matter of fact, I really wanted to play with the features introduced by <strong>C++17</strong>. Quite a few of them focus on improving the expressiveness of the language as well as the meta-programming facilities (mainly constexpr). Instead of writing small code samples, I thought that it would be more fun to turn all of this into a game. Pet projects are a nice way to learn concepts that you may not use before quite some time at work. Being able to run the core logic of your game at compile-time proves once a again that templates and constepxr are <a href="https://en.wikipedia.org/wiki/Turing_completeness">turing complete</a> subsets of the C++ language.</p>
<h2>Meta Crush Saga: an overview</h2>
<h3>A Match-3 game:</h3>
<p><strong>Meta Crush Saga</strong> is a <a href="https://en.wikipedia.org/wiki/Tile-matching_video_game">tiled-matching game</a> similar to <strong>Bejeweled</strong> or <strong>Candy Crush Saga</strong>. The core of the rules consists in matching three or more tiles of the same pattern to increase your scores. Here is sneak peek of a <strong>game state</strong> I "dumped" (dumping ASCII is pretty damn easy): </p>
<div class="highlight"><pre><span></span><code><span class="sa">R</span><span class="s">"</span><span class="dl">(</span><span class="s"></span>
<span class="s"> Meta crush saga </span>
<span class="s">------------------------ </span>
<span class="s">| | </span>
<span class="s">| R B G B B Y G R | </span>
<span class="s">| | </span>
<span class="s">| | </span>
<span class="s">| Y Y G R B G B R | </span>
<span class="s">| | </span>
<span class="s">| | </span>
<span class="s">| R B Y R G R Y G | </span>
<span class="s">| | </span>
<span class="s">| | </span>
<span class="s">| R Y B Y (R) Y G Y | </span>
<span class="s">| | </span>
<span class="s">| | </span>
<span class="s">| B G Y R Y G G R | </span>
<span class="s">| | </span>
<span class="s">| | </span>
<span class="s">| R Y B G Y B B G | </span>
<span class="s">| | </span>
<span class="s">------------------------ </span>
<span class="s">> score: 9009</span>
<span class="s">> moves: 27</span>
<span class="dl">)</span><span class="s">"</span>
</code></pre></div>
<p>The game-play of this specific Match-3 is not so interesting in itself, but what about the architecture running all of this? To understand it, I will try to explain each part of the life cycle of this <strong>compile-time</strong> game in term of code.</p>
<h3>Injecting the game state:</h3>
<p><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/injecting-game-state.png"/></p>
<p>As a C++ afficionados or a nitpicker, you may have noticed that my previous dumped game state started with the following pattern: <strong>R"(</strong>. This is indeed a <a href="http://en.cppreference.com/w/cpp/language/string_literal">C++11 raw string literal</a>, meaning that I do not have to escape special characters like <strong>line feed</strong>. This raw string literal is stored in a file called <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/current_state.txt">current_state.txt</a>.</p>
<p>How do we inject this current game state into a compile state? Let's just include it into the loop inputs!</p>
<div class="highlight"><pre><span></span><code><span class="c1">// loop_inputs.hpp</span>
<span class="k">constexpr</span> <span class="n">KeyboardInput</span> <span class="n">keyboard_input</span> <span class="o">=</span> <span class="n">KeyboardInput</span><span class="o">::</span><span class="n">KEYBOARD_INPUT</span><span class="p">;</span> <span class="c1">// Get the current keyboard input as a macro</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">get_game_state_string</span> <span class="o">=</span> <span class="p">[]()</span> <span class="k">constexpr</span>
<span class="p">{</span>
<span class="k">auto</span> <span class="n">game_state_string</span> <span class="o">=</span> <span class="n">constexpr_string</span><span class="p">(</span>
<span class="c1">// Include the raw string literal into a variable</span>
<span class="cp">#include</span> <span class="cpf">"current_state.txt"</span><span class="cp"></span>
<span class="p">);</span>
<span class="k">return</span> <span class="n">game_state_string</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>Whether it is a <em>.txt</em> file or a <em>.h</em> file, the <strong>include</strong> directive from C preprocessor will work exactly the same: it will copy the content of the file at its location. Here I am copying the ascii-game-state-raw-string-literal into a variable named <strong>game_state_string</strong>. </p>
<p>Note that this header file <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/loop_inputs.hpp">loop_inputs.hpp</a> also exposes the keyboard inputs for the current frame / compilation. Unlike the game state, the keyboard state is fairly small and can be easily received as a preprocessor definition.</p>
<h3>Compile time computation of the new state:</h3>
<p><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/compile-time-computation-new-state.png"/></p>
<p>Now that we have gathered enough data, we can compute a new state. And finally, we reach the point where we have to write a <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/main.cpp">main.cpp file</a>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// main.cpp</span>
<span class="cp">#include</span> <span class="cpf">"loop_inputs.hpp"</span><span class="c1"> // Get all the data necessary for our computation.</span><span class="cp"></span>
<span class="c1">// Start: compile-time computation.</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">current_state</span> <span class="o">=</span> <span class="n">parse_game_state</span><span class="p">(</span><span class="n">get_game_state_string</span><span class="p">);</span> <span class="c1">// Parse the game state into a convenient object.</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">new_state</span> <span class="o">=</span> <span class="n">game_engine</span><span class="p">(</span><span class="n">current_state</span><span class="p">)</span> <span class="c1">// Feed the engine with the parsed state,</span>
<span class="p">.</span><span class="n">update</span><span class="p">(</span><span class="n">keyboard_input</span><span class="p">);</span> <span class="c1">// Update the engine to obtain a new state.</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">array</span> <span class="o">=</span> <span class="n">print_game_state</span><span class="p">(</span><span class="n">new_state</span><span class="p">);</span> <span class="c1">// Convert the new state into a std::array<char> representation.</span>
<span class="c1">// End: compile-time computation.</span>
<span class="c1">// Runtime: just render the state.</span>
<span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="o">&</span> <span class="nl">c</span> <span class="p">:</span> <span class="n">array</span><span class="p">)</span> <span class="p">{</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">c</span><span class="p">;</span> <span class="p">}</span>
</code></pre></div>
<p>Strangely, this C++ code does not look too convoluted for what it does. Most of this code is run during the compilation phase and yet follows traditional OOP and procedural paradigms. Only the rendering, the last line, is an impediment to a pure compile-time computation. By sprinkling a bit of <strong>constexpr</strong> where it should, you can have pretty elegant meta-programming in C++17 as we will see later-on. I find it fascinating the freedom C++ gives us when it comes to mix runtime and compile-time execution.</p>
<p>You will also notice that this code only execute one frame, there is no <strong>game-loop</strong> in here. Let's solve that issue!</p>
<h3>Gluing things together:</h3>
<p><img width=50% height=50% src="https://jguegant.github.io/blogs/tech/images/gluing-things-together.png"/></p>
<p>If you are revulsed by my <strong>C++</strong> tricks, I wish you do not mind to contemplate my <strong>Bash</strong> skills. Indeed, my <strong>game loop</strong> is nothing more than a <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/meta_crush_saga.sh">bash script</a> executing repeatidly some compilations.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># It is a loop! No wait, it is a game loop!!!</span>
<span class="k">while</span><span class="p">;</span> <span class="k">do</span> :
<span class="c1"># Start a compilation step using G++</span>
g++ -o renderer main.cpp -DKEYBOARD_INPUT<span class="o">=</span><span class="s2">"</span><span class="nv">$keypressed</span><span class="s2">"</span>
<span class="nv">keypressed</span><span class="o">=</span>get_key_pressed<span class="o">()</span>
<span class="c1"># Clean the screen.</span>
clear
<span class="c1"># Obtain the rendering</span>
<span class="nv">current_state</span><span class="o">=</span><span class="k">$(</span>./renderer<span class="k">)</span>
<span class="nb">echo</span> <span class="nv">$current_state</span> <span class="c1"># Show the rendering to the player</span>
<span class="c1"># Place the rendering into a current_state.txt file and wrap it into a raw string literal.</span>
<span class="nb">echo</span> <span class="s2">"R\"("</span> > current_state.txt
<span class="nb">echo</span> <span class="nv">$current_state</span> >> current_state.txt
<span class="nb">echo</span> <span class="s2">")\""</span> >> current_state.txt
<span class="k">done</span>
</code></pre></div>
<p>I actually struggled a bit to get the keyboard inputs from the console. I initially wanted to receive the inputs in parallel of the compilation. After lots trial and error, I got something working-ish using the <code>read</code> <strong>Bash</strong> command. I would never dare to duel a <strong>Bash</strong> wizard, that language is way too maleficent!</p>
<p>Now let's agree, I had to resort to use another language to handle my game loop. Although, technically, nothing would prevent me to write that part of the code in C++. It also does not cancel the fact that 90% of the logic of my game is done within this <strong>g++</strong> compilation command, which is pretty awesome!</p>
<h3>A bit of gameplay to soften your eyes:</h3>
<p>Now that you have suffered your way into my explanations on the game's architecture, here comes a bit of eye candy:</p>
<p><img alt="Meta Crush Saga in action" src="https://jguegant.github.io/blogs/tech/images/meta-crush-saga.gif"></p>
<p>This pixelated gif is a record of me playing <strong>Meta Crush Saga</strong>. As you can see, the game runs smoothly enough to make it playable in real-time. It is clearly not attractive enough to be able to stream it on Twitch and for me to become the new Pewdiepie, but hey... it works!
One of the funny aspect of having a <strong>game state</strong> stored as a <em>.txt</em> is the ability to cheat or test edge-cases really easily.</p>
<p>Now that I sketched the architecture, we will dive a bit more in the C++17 features used within that project. I will not focus on the game logic, as it is very specific to a Match-3, but will instead discuss subjects of C++ that be could applied in other projects too.</p>
<h2>My C++17 learnings:</h2>
<p><img width=25% height=25% style="float: left;" src="https://jguegant.github.io/blogs/tech/images/spoiled-kids.png"/></p>
<p>Unlike C++14 which mainly contained minor fixes, the new C++17 standard has a lot to offer. There were hopes that some long-overdue features would land this time (modules, coroutines, concepts...) and... well... they did not ; which disappointed quite a few of us. But after the mourning, we discovered a myriad of small unexpected gems that made their way through.</p>
<p>I would dare to say that all the meta-programming kids were spoiled this year! Few minor tweaks and additions in language now permit you to write code very similar weither it is running during compilation or afterwards during runtime.</p>
<div style="clear: both;"></div>
<h3>Constepxr all the things:</h3>
<p>As Ben Deane and Jason Turner foretold in their <a href="https://www.youtube.com/watch?v=PJwd4JLYJJY">C++14 talk</a>,
C++ is quickly improving value-computations at compile-time using the almighty <a href="http://en.cppreference.com/w/cpp/language/constexpr">constexpr</a> keyword. By placing this keyword in the appropriate places you can hint to your compiler that an expression is constant and <strong>could</strong> be directly evaluated at compile-time. In <strong>C++11</strong> you could already write such code:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="kt">int</span> <span class="nf">factorial</span><span class="p">(</span><span class="kt">int</span> <span class="n">n</span><span class="p">)</span> <span class="c1">// Combining a function with constexpr make it potentially evaluable at compile-time.</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">n</span> <span class="o"><=</span> <span class="mi">1</span><span class="o">?</span> <span class="mi">1</span> <span class="o">:</span> <span class="p">(</span><span class="n">n</span> <span class="o">*</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">));</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="n">factorial</span><span class="p">(</span><span class="mi">5</span><span class="p">);</span> <span class="c1">// Call to a constexpr function.</span>
<span class="c1">// Can be replace by a good compiler by:</span>
<span class="c1">// int i = 120;</span>
</code></pre></div>
<p>While powerful, <strong>constexpr</strong> had quite a lot of restrictions on its usage and made it cumbersome to write expressive code in this way.
<strong>C++14</strong> relaxed a lot <strong>constexpr</strong> and felt much more natural to use. Our previous factorial function could be rewritten this way:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="kt">int</span> <span class="nf">factorial</span><span class="p">(</span><span class="kt">int</span> <span class="n">n</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">n</span> <span class="o"><=</span> <span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">n</span> <span class="o">*</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>Indeed, <strong>C++14</strong> lifted the rule stipulating that a <strong>constexpr function</strong> must only consist of one return statement, which forced us to use the <a href="https://en.wikipedia.org/wiki/%3F:">ternary operator</a> as a basic building block. Now <strong>C++17</strong> brought even more placements for the <strong>constexpr</strong> keyword that we can explore!</p>
<h4>Compile-time branching:</h4>
<p>Did you ever end-up in a situation where you wish that you could have different behavior according to the template parameter you are manipulating? Let's say that you wanted a generic <code>serialize</code> function that would call <code>.serialize()</code> if your object provides one, otherwise fall back on calling <code>to_string</code> on it. As explained in more details in this <a href="https://jguegant.github.io/blogs/tech/sfinae-introduction.html">post about SFINAE</a> you would very likely write such a lovely alien code:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o"><</span><span class="n">has_serialize_v</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span>
<span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o"><!</span><span class="n">has_serialize_v</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span>
<span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>In your dreams you may be able to rewrite that awkward <strong>SFINAE trick</strong> into such a magestic piece of code in <strong>C++14</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// has_serialize is a constexpr function that test the of serialize on a object.</span>
<span class="c1">// See my post on SFINAE to understand how to write such a function. </span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">constexpr</span> <span class="kt">bool</span> <span class="n">has_serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="cm">/*t*/</span><span class="p">);</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// We know that constexpr can be placed before functions.</span>
<span class="k">if</span> <span class="p">(</span><span class="n">has_serialize</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>Sadly, as soon as you wake-up and start writing <strong>C++14</strong> for real, your compiler will vomit you a displeasant message regarding the call <code>serialize(42);</code>. It will explain that the object <code>obj</code> of type <code>int</code> does not have a <code>serialize()</code> member function. As much as you hate it, your compiler is right! Given the current code, it will always try to compile both of the branches <code>return obj.serialize();</code> and
<code>return std::to_string(obj);</code>. For an <code>int</code>, the branch <code>return obj.serialize();</code> might well be some dead-code since <code>has_serialize(obj)</code> will always return <code>false</code>, but your compiler will still need to compile-it.</p>
<p>As you may expect, <strong>C++17</strong> save us from such an embarassing situation by introducing the possibility to add <strong>constexpr</strong> after an if statement to "force" a compile-time branching and discard the unused statements:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// has_serialize...</span>
<span class="c1">// ...</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="k">if</span> <span class="k">constexpr</span> <span class="p">(</span><span class="n">has_serialize</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span> <span class="p">{</span> <span class="c1">// Now we can place constexpr on the 'if' directly.</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="c1">// This branch will be discarded and therefore not compiled if obj is an int.</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span><span class="n">branch</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p><img width=25% height=25% src="https://jguegant.github.io/blogs/tech/images/constexpr-all-the-things.png"/></p>
<p>This is clearly a huge improvement compared to the <strong>SFINAE trick</strong> we had to employ until now. After that, you will start to get the same addiction as Ben and Jason which consists in <strong>constexpr</strong> everything, everywhere at anytime. Alas, there is still one place where the <strong>constexpr</strong> keyword would well fit in but cannot be done yet: <strong>constexpr parameters</strong>.</p>
<h4>Constexpr parameters:</h4>
<p>If you are assiduous, you may have noticed a strange pattern in one my previous code sample. I am talking about the loop inputs:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// loop_inputs.hpp</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">get_game_state_string</span> <span class="o">=</span> <span class="p">[]()</span> <span class="k">constexpr</span> <span class="c1">// Why?</span>
<span class="p">{</span>
<span class="k">auto</span> <span class="n">game_state_string</span> <span class="o">=</span> <span class="n">constexpr_string</span><span class="p">(</span>
<span class="c1">// Include the raw string literal into a variable</span>
<span class="cp">#include</span> <span class="cpf">"current_state.txt"</span><span class="cp"></span>
<span class="p">);</span>
<span class="k">return</span> <span class="n">game_state_string</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>Why is the variable <strong>game_state_string</strong> encapsulated into a constexpr lambda? Why not making it a <strong>constexpr global variable</strong>?</p>
<p>Well, I wanted to pass this variable and its content deep down to some functions. For instance, my <strong>parse_board</strong> needed to be fed with it and used it in some constant expressions:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="kt">int</span> <span class="nf">parse_board_size</span><span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">game_state_string</span><span class="p">);</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">parse_board</span><span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">game_state_string</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="n">GemType</span><span class="p">,</span> <span class="n">parse_board_size</span><span class="p">(</span><span class="n">game_state_string</span><span class="p">)</span><span class="o">></span> <span class="n">board</span><span class="p">{};</span>
<span class="c1">// ^ ‘game_state_string’ is not a constant expression</span>
<span class="c1">// ... </span>
<span class="p">}</span>
<span class="n">parse_board</span><span class="p">(</span><span class="err">“</span><span class="p">...</span><span class="n">something</span><span class="p">...</span><span class="err">”</span><span class="p">);</span>
</code></pre></div>
<p>If you are doing it this way, your grumpy compiler will complain that the parameter <strong>game_state_string</strong> is not a constant expression. When I am building my array of Gems, I need to compute its fixed capacity directly (you cannot use vectors at compile-time as it requires to allocate) and pass it as a value-template-argument to <strong>std::array</strong>. The expression <strong>parse_board_size(game_state_string)</strong> therefore needs to be a constant expression. While <strong>parse_board_size</strong> is clearly marked as <strong>constexpr</strong>, <strong>game_state_string</strong> is not AND cannot be! Two rules are annoying us in this case:</p>
<ul>
<li>Arguments of a constexpr function are not constexpr!</li>
<li>And you cannot add constexpr in front of them!</li>
</ul>
<p>It boils down to the fact that <strong>constexpr functions</strong> MUST be usable for both runtime or compile-time computations. Allowing <strong>constexpr parameters</strong> would discard the possibility to use them at runtime.</p>
<p><center><img width=25% height=25% src="https://jguegant.github.io/blogs/tech/images/facepalm.jpg"/></center></p>
<p>Thanksfully, there is a way to mitigate that issue. Instead of accepting the value as a normal function parameter, you can encapsulate that value into a type and pass that type as a template parameter:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">GameStringType</span><span class="o">></span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">parse_board</span><span class="p">(</span><span class="n">GameStringType</span><span class="o">&&</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="n">CellType</span><span class="p">,</span> <span class="n">parse_board_size</span><span class="p">(</span><span class="n">GameStringType</span><span class="o">::</span><span class="n">value</span><span class="p">())</span><span class="o">></span> <span class="n">board</span><span class="p">{};</span>
<span class="c1">// ...</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="nc">GameString</span> <span class="p">{</span>
<span class="k">static</span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">value</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="s">"...something..."</span><span class="p">;</span> <span class="p">}</span>
<span class="p">};</span>
<span class="n">parse_board</span><span class="p">(</span><span class="n">GameString</span><span class="p">{});</span>
</code></pre></div>
<p>In this code sample, I am creating a struct type <strong>GameString</strong> which has a static constexpr member function <strong>value()</strong> that returns the string literal I want to pass to <strong>parse_board</strong>. In <strong>parse_board</strong> I receive this type through the template parameter <strong>GameStringType</strong> thanks to template argument deduction rules. Having <strong>GameStringType</strong>, I can simply call the static member function <strong>value()</strong> whenever I want to get the string literal, even in locations where constant expressions are necessary since <strong>value()</strong> is constexpr. </p>
<p>We succeeded to encapsulate our literal to somehow pass it to <strong>parse_board</strong> in a constexpr way. Now, it gets very annoying to have to define a new type everytime you want to send a new literal to <strong>parse_board</strong>: "...something1...", "...something2...". To solve that issue in <strong>C++11</strong>, you can rely on some ugly macros and few indirection using an anonymous union and a lambda. Mikael Park has a nice explanation on this topic in <a href="https://mpark.github.io/programming/2017/05/26/constexpr-function-parameters/">one of his post</a>. </p>
<p>We can do even better in <strong>C++17</strong>. If you list our current requirements to pass our string literal, we need:</p>
<ul>
<li>A generated function</li>
<li>Which is constexpr</li>
<li>With a unique or anonymous name</li>
</ul>
<p>This requirements should ring a bell to you. What we need is a <strong>constexpr lambda</strong>! And <strong>C++17</strong> rightfully added the possibility to use the <strong>constexpr</strong> keyword on a lambda. We could rewrite the code sample in such a way:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">LambdaType</span><span class="o">></span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">parse_board</span><span class="p">(</span><span class="n">LambdaType</span><span class="o">&&</span> <span class="n">get_game_state_string</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="n">CellType</span><span class="p">,</span> <span class="n">parse_board_size</span><span class="p">(</span><span class="n">get_game_state_string</span><span class="p">())</span><span class="o">></span> <span class="n">board</span><span class="p">{};</span>
<span class="c1">// ^ Allowed to call a constexpr lambda in this context.</span>
<span class="p">}</span>
<span class="n">parse_board</span><span class="p">([]()</span> <span class="k">constexpr</span> <span class="o">-></span> <span class="p">{</span> <span class="k">return</span> <span class="err">“</span><span class="p">...</span><span class="n">something</span><span class="p">...</span><span class="err">”</span><span class="p">;</span> <span class="p">});</span>
<span class="c1">// ^ Make our lambda constexpr.</span>
</code></pre></div>
<p>Believe me, this feels already much neater than the previous <strong>C++11</strong> hackery using macros. I discovered this awesome trick thanks to <strong>Björn Fahller</strong>, a member of the C++ meetup group I participate in. You can read more about this trick on his <a href="http://playfulprogramming.blogspot.se/2016/08/strings-as-types-with-c17-constexpr.html">blog</a>. Note also that the <strong>constexpr</strong> keyword is actually not necessary in this case: all the <strong>lambdas</strong> with the capacity to be <strong>constexpr</strong> will be by default. Having an explicit <strong>constexpr</strong> in the signature just makes it easier to catch mistakes.</p>
<p>Now you should understand why I was forced to use a <strong>constexpr</strong> lambda to pass down the string representing my game state. Have a look at this lambda and one question should arise again. What is this <strong>constexpr_string</strong> type I also use to wrap the string literal?</p>
<h5><strong>constexpr_string</strong> and <strong>constexpr_string_view</strong>:</h5>
<p>When you are dealing with strings, you do not want to deal with them the C way. All these pesky algorithms iterating in a raw manner and checking null ending should be forbidden! The alternative offered by <strong>C++</strong> is the almighty <strong>std::string</strong> and <strong>STL algorithms</strong>. Sadly, <strong>std::string</strong> may have to allocate on the heap (even with Small String Optimization) to store their content. One or two standards from now we may benefit from <strong>constexpr new/delete</strong> or being able to pass <strong>constexpr allocators</strong> to <strong>std::string</strong>, but for now we have to find another solution.</p>
<p>My approach was to write a <strong>constexpr_string</strong> class which has a fixed capacity. This capacity is passed as a value template parameter. Here is a short overview of my class:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">N</span><span class="o">></span> <span class="c1">// N is the capacity of my string.</span>
<span class="k">class</span> <span class="nc">constexpr_string</span> <span class="p">{</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">std</span><span class="o">::</span><span class="n">array</span><span class="o"><</span><span class="kt">char</span><span class="p">,</span> <span class="n">N</span><span class="o">></span> <span class="n">data_</span><span class="p">;</span> <span class="c1">// Reserve N chars to store anything. </span>
<span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">size_</span><span class="p">;</span> <span class="c1">// The actual size of the string.</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">constexpr</span> <span class="n">constexpr_string</span><span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="p">(</span><span class="o">&</span><span class="n">a</span><span class="p">)[</span><span class="n">N</span><span class="p">])</span><span class="o">:</span> <span class="n">data_</span><span class="p">{},</span> <span class="n">size_</span><span class="p">(</span><span class="n">N</span> <span class="mi">-1</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// copy a into data_ }</span>
<span class="c1">// ...</span>
<span class="k">constexpr</span> <span class="n">iterator</span> <span class="nf">begin</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">data_</span><span class="p">;</span> <span class="p">}</span> <span class="c1">// Points at the beggining of the storage.</span>
<span class="k">constexpr</span> <span class="n">iterator</span> <span class="nf">end</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="n">data_</span> <span class="o">+</span> <span class="n">size_</span><span class="p">;</span> <span class="p">}</span> <span class="c1">// Points at the end of the stored string.</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p>My <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/constexpr_string.hpp">constexpr_string</a> class tries to mimic as closely as possible the interface of <strong>std::string</strong> (for the operations that I needed): you can query the <strong>begin and end iterators</strong>, retrive the <strong>size</strong>, get access to <strong>data</strong>, <strong>erase</strong> part of it, get a substring using <strong>substr</strong>, etc. It makes it very straightforward to transform a piece of code from <strong>std::string</strong> to <strong>constexpr_string</strong>. You may wonder what happens when you want to do use operations that would normally requires an allocation in <strong>std::string</strong>. In such cases, I was forced to transform them into <strong>immutable operations</strong> that would create new instance of <strong>constexpr_string</strong>.</p>
<p>Let's have a look at the <strong>append</strong> operation:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">N</span><span class="o">></span> <span class="c1">// N is the capacity of my string.</span>
<span class="k">class</span> <span class="nc">constexpr_string</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">template</span> <span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">M</span><span class="o">></span> <span class="c1">// M the capacity of the other string.</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">append</span><span class="p">(</span><span class="k">const</span> <span class="n">constexpr_string</span><span class="o"><</span><span class="n">M</span><span class="o">>&</span> <span class="n">other</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">constexpr_string</span><span class="o"><</span><span class="n">N</span> <span class="o">+</span> <span class="n">M</span><span class="o">></span> <span class="n">output</span><span class="p">(</span><span class="o">*</span><span class="k">this</span><span class="p">,</span> <span class="n">size</span><span class="p">()</span> <span class="o">+</span> <span class="n">other</span><span class="p">.</span><span class="n">size</span><span class="p">());</span>
<span class="c1">// ^ Enough capacity for both. ^ Copy the first string into the output.</span>
<span class="k">for</span> <span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="kt">size_t</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">other</span><span class="p">.</span><span class="n">size</span><span class="p">();</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
<span class="n">output</span><span class="p">[</span><span class="n">size</span><span class="p">()</span> <span class="o">+</span> <span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">other</span><span class="p">[</span><span class="n">i</span><span class="p">];</span>
<span class="o">^</span> <span class="n">Copy</span> <span class="n">the</span> <span class="n">second</span> <span class="n">string</span> <span class="n">into</span> <span class="n">the</span> <span class="n">output</span><span class="p">.</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">output</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div>
<p><img width=25% height=25% style="float:right;" src="https://jguegant.github.io/blogs/tech/images/einstein.jpg"/></p>
<p>No needs to have a Fields Medal to assume that if we have a string of size <strong>N</strong> and a string or size <strong>M</strong>, a string of size <strong>N + M</strong> should be enough to store the concatenation. You may waste a bit of "compile-time storage" since both of your strings may not use their full capacity, but that is a fairly small price to pay for a lot of convenience. I, obviously, also wrote the counterpart of <strong>std::string_view</strong> which I named <a href="https://github.com/Jiwan/meta_crush_saga/blob/master/constexpr_string_view.hpp">constexpr_string_view</a>.</p>
<div style="clear: both;"></div>
<p>Having these two classes, I was ready to write elegant code to parse my <strong>game state</strong>. Think about something like that:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="k">auto</span> <span class="n">game_state</span> <span class="o">=</span> <span class="n">constexpr_string</span><span class="p">(</span><span class="err">“</span><span class="p">...</span><span class="n">something</span><span class="p">...</span><span class="err">”</span><span class="p">);</span>
<span class="c1">// Let's find the first blue gem occurence within my string:</span>
<span class="k">constexpr</span> <span class="k">auto</span> <span class="n">blue_gem</span> <span class="o">=</span> <span class="n">find_if</span><span class="p">(</span><span class="n">game_state</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">game_state</span><span class="p">.</span><span class="n">end</span><span class="p">(),</span>
<span class="p">[](</span><span class="kt">char</span> <span class="n">c</span><span class="p">)</span> <span class="k">constexpr</span> <span class="o">-></span> <span class="p">{</span> <span class="k">return</span> <span class="n">c</span> <span class="o">==</span> <span class="err">‘</span><span class="n">B</span><span class="err">’</span><span class="p">;</span> <span class="p">}</span>
<span class="p">);</span>
</code></pre></div>
<p>It was fairly simple to iterate over the gems on my board - speaking of which, did you notice another <strong>C++17</strong> gem in that code sample?</p>
<p>Yes! I did not have to explicitely specify the capacity of my <strong>constexpr_string</strong> when constructing it. In the past, you had to explicitely specify the arguments of a <strong>class template</strong> when using it. To avoid this pain, we would provide <em>make_xxx</em> functions since parameters of <strong>function templates</strong> could be deduced. Have a look on how <a href="http://en.cppreference.com/w/cpp/language/class_template_argument_deduction">class template argument deduction</a> is changing our life for the better:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="kt">int</span> <span class="n">N</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">constexpr_string</span> <span class="p">{</span>
<span class="n">constexpr_string</span><span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="p">(</span><span class="o">&</span><span class="n">a</span><span class="p">)[</span><span class="n">N</span><span class="p">])</span> <span class="p">{}</span>
<span class="c1">// ..</span>
<span class="p">};</span>
<span class="c1">// **** Pre C++17 ****</span>
<span class="k">template</span> <span class="o"><</span><span class="kt">int</span> <span class="n">N</span><span class="o">></span>
<span class="n">constexpr_string</span><span class="o"><</span><span class="n">N</span><span class="o">></span> <span class="n">make_constexpr_string</span><span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="p">(</span><span class="o">&</span><span class="n">a</span><span class="p">)[</span><span class="n">N</span><span class="p">])</span> <span class="p">{</span>
<span class="c1">// Provide a function template to deduce N ^ right here</span>
<span class="k">return</span> <span class="n">constexpr_string</span><span class="o"><</span><span class="n">N</span><span class="o">></span><span class="p">(</span><span class="n">a</span><span class="p">);</span>
<span class="c1">// ^ Forward the parameter to the class template.</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">test2</span> <span class="o">=</span> <span class="n">make_constexpr_string</span><span class="p">(</span><span class="s">"blablabla"</span><span class="p">);</span>
<span class="c1">// ^ use our function template to for the deduction.</span>
<span class="n">constexpr_string</span><span class="o"><</span><span class="mi">7</span><span class="o">></span> <span class="n">test</span><span class="p">(</span><span class="s">"blabla"</span><span class="p">);</span>
<span class="c1">// ^ or feed the argument directly and pray that it was the good one.</span>
<span class="c1">// **** With C++17 ****</span>
<span class="n">constexpr_string</span> <span class="nf">test</span><span class="p">(</span><span class="s">"blabla"</span><span class="p">);</span>
<span class="c1">// ^ Really smmoth to use, the argument is deduced.</span>
</code></pre></div>
<p>In some tricky situations, you may still need to help your compiler to deduce correctly your arguments. If you encouter such an issue, have a look at <a href="http://en.cppreference.com/w/cpp/language/class_template_argument_deduction#User-defined_deduction_guides">user-defined deduction guides</a>.</p>
<h4>Free food from the STL:</h4>
<p>Alright, you can always rewrite things by yourself. But did the committee members kindly cooked something for us in the standard library?</p>
<h5>New utility types:</h5>
<p><strong>C++17</strong> introduced <a href="http://en.cppreference.com/w/cpp/utility/variant">std::variant</a> and <a href="http://en.cppreference.com/w/cpp/utility/optional">std::optional</a> to the common vocabulary types, with <strong>constexpr</strong> in mind. While the former one is really interesting since it permits you to express type-safe unions, the implementation provided in the <strong>libstdc++</strong> library with <strong>GCC 7.2</strong> had issues when used in constant expressions. Therefore, I gave up the idea to introduce <strong>std::variant</strong> in my code and solely utilised <strong>std::optional</strong>.</p>
<p>Given a type <strong>T</strong>, <strong>std::optional</strong> allows you to create a new type <strong>std::optional<T></strong> which may either hold a value of type <strong>T</strong> or nothing. It is pretty similar to <a href="https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/nullable-types/">nullable value types</a> in <strong>C#</strong>.
Let's consider a <strong>find_in_board</strong> function that return the position of the first item in the board that validate a predicate. You may not have such an item in the board at all. To handle such a situation, the position type must be optional:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Predicate</span><span class="o">></span>
<span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">optional</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="kt">int</span><span class="o">>></span> <span class="n">find_in_board</span><span class="p">(</span><span class="n">GameBoard</span><span class="o">&&</span> <span class="n">g</span><span class="p">,</span> <span class="n">Predicate</span><span class="o">&&</span> <span class="n">p</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">(</span><span class="k">auto</span> <span class="nl">item</span> <span class="p">:</span> <span class="n">g</span><span class="p">.</span><span class="n">items</span><span class="p">())</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">p</span><span class="p">(</span><span class="n">item</span><span class="p">))</span> <span class="p">{</span> <span class="k">return</span> <span class="p">{</span><span class="n">item</span><span class="p">.</span><span class="n">x</span><span class="p">,</span> <span class="n">item</span><span class="p">.</span><span class="n">y</span><span class="p">};</span> <span class="p">}</span> <span class="c1">// Return by value if we find such an item.</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">nullopt</span><span class="p">;</span> <span class="c1">// Return the empty state otherwise.</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">item</span> <span class="o">=</span> <span class="n">find_in_board</span><span class="p">(</span><span class="n">g</span><span class="p">,</span> <span class="p">[](</span><span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="n">item</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nb">true</span><span class="p">;</span> <span class="p">});</span>
<span class="k">if</span> <span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Test if the optional is empty or not.</span>
<span class="n">do_something</span><span class="p">(</span><span class="o">*</span><span class="n">item</span><span class="p">);</span> <span class="c1">// Can safely use the optional by 'deferencing' with '*'.</span>
<span class="cm">/* ... */</span>
<span class="p">}</span>
</code></pre></div>
<p>Previously, you would have recourse to either <strong>pointer semantics</strong> or add an "empty state" directly into your position type or return a boolean and take an <strong>out parameter</strong>. Let's face it, it was pretty clumsy!</p>
<p>Some already existing types also received a <strong>constexpr</strong> lifting: <a href="http://en.cppreference.com/w/cpp/utility/tuple">tuple</a> and <a href="http://en.cppreference.com/w/cpp/utility/pair">pair</a>. I will not explain their usage as a lot have been already written on these two, but I will share you one of my disapointment. The committee added to the standard a <strong>syntactic sugar</strong> to extract the values hold by a <strong>tuple</strong> or <strong>pair</strong>. Called <a href="http://en.cppreference.com/w/cpp/language/structured_binding">structured binding</a>, this new kind of declaration uses brackets to define in which variables you would like to store the exploded <strong>tuple</strong> or <strong>pair</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="kt">int</span><span class="o">></span> <span class="n">foo</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span><span class="mi">42</span><span class="p">,</span> <span class="mi">1337</span><span class="p">};</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
<span class="c1">// x = 42 and y = 1337.</span>
</code></pre></div>
<p>Really clever! It is just a pity that the committee members [could not, would not, did not have the time, forgot, enjoyed not] to make it <strong>constexpr</strong> friendly. I would have expected something along the way:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="k">auto</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span> <span class="c1">// OR</span>
<span class="k">auto</span> <span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">]</span> <span class="k">constexpr</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
</code></pre></div>
<p>We now have fancy containers and utilities, how can we manipulate them easily?</p>
<h5>Algorithms:</h5>
<p>Upgrading a container to a handle <strong>constexpr</strong> is a rather tedious work. Comparatively, bringing <strong>constexpr</strong> to <strong>non-modifying algorithms</strong> seems rather straightforward. But strangely enough, <strong>C++17</strong> did not see any progress in that domain, it is actually coming in <strong>C++20</strong>. For instance, the supreme <a href="http://en.cppreference.com/w/cpp/algorithm/find">std::find</a> algorithms did not receive its <strong>constexpr</strong> signature.</p>
<p>Worry not (or "Qu'à cela ne tienne" in French)! As explained by Ben and Jason, you can easily turn an algorithm in <strong>constexpr</strong> by simply copying a current implementation (checkout for copyrights though) ; cppreference being a good fit. Ladies and gentlemen, I present you a <strong>constexpr std::find</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span><span class="o"><</span><span class="k">class</span> <span class="nc">InputIt</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="k">constexpr</span> <span class="n">InputIt</span> <span class="n">find</span><span class="p">(</span><span class="n">InputIt</span> <span class="n">first</span><span class="p">,</span> <span class="n">InputIt</span> <span class="n">last</span><span class="p">,</span> <span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">value</span><span class="p">)</span>
<span class="c1">// ^ TADAMMMM!!! I added constexpr here.</span>
<span class="p">{</span>
<span class="k">for</span> <span class="p">(;</span> <span class="n">first</span> <span class="o">!=</span> <span class="n">last</span><span class="p">;</span> <span class="o">++</span><span class="n">first</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">*</span><span class="n">first</span> <span class="o">==</span> <span class="n">value</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">first</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">last</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Thanks to: http://en.cppreference.com/w/cpp/algorithm/find</span>
</code></pre></div>
<p>I already hear optimisation affionados screaming on their chair! Yes, just adding <strong>constexpr</strong> in front of a code sample gently provided by <strong>cppreference</strong> may not give you the perfect <strong>runtime performances</strong>. But if we really had to polish this algorithm it would be for <strong>compile-time performances</strong>. Keeping things simple usually give the best results when it comes to speed of <strong>compilation</strong> from what I observed.</p>
<h3>Performance & bugs:</h3>
<p>Every triple A games must put efforts in these topics, right?</p>
<h4>Performance:</h4>
<p>When I achieved a first half-workingish version of <strong>Meta Crush Saga</strong> things ran rather smoothly. It actually reached a bit more than <strong>3 FPS</strong> (Frame Per Second) on my old laptop with an i5 clocked at 1.80GHz (frequency do matter in this case). As in any project, I quickly found my previously written code unsavoury and started to rewrite the parsing of my game state using <strong>constexpr_string</strong> and standard algorithms. Although it made my code much more maintenable it also had a severe impact on the performance ; <strong>0.5 FPS</strong> was the new ceiling. </p>
<p><center><img width=35% height=35% src="https://jguegant.github.io/blogs/tech/images/performance-rating.png"/></center></p>
<p>Unlike the old C++ adage, "zero-head abstractions" did not apply to <strong>compile-time computations</strong>. Which really makes sense if you see your compiler as an interpreter of some "compile-time code". There is still room to improve for the various compilers, but also for us writers of such code. Here is a non-exhaustive list of few observations and tips, maybe specific to GCC, that I figured out:</p>
<ul>
<li><strong>C arrays</strong> performed significantly better than <strong>std::array</strong>. <strong>std::array</strong> is a bit of modern C++ cosmetic over a <strong>C-style array</strong> and one must pay a certain price to use it in such circumstances. </li>
<li>It felt like <strong>recursive functions</strong> had the advantage (speed-wise) over writing <strong>functions with loops</strong>. It could well be that writing recursive algorithms forces you to tackle your problems in another way, which behaves better. To put in my two penny worth, I believe that the cost of compile-time calls might be smaller than executing a complicated function body especially that compilers (and their implementors) have been exposed to decades of abusive recursions we used for our own template-metaprogramming ends. </li>
<li>Copying data around is also quite expensive, notably if you are dealing with value types. If I wanted to futher optimise my game, I would mainly focus on that problem.</li>
<li>I only <s>ab</s>used one of my CPU core to do the job. Having only one compilation unit restricted me to spawn only one instance of GCC at a time. I am not quite sure if you could parallelise my compilutation.</li>
</ul>
<h4>Bugs:</h4>
<p><img width=20% height=20% style="float: left;" src="https://jguegant.github.io/blogs/tech/images/no-bugs.png"/>
More than once, my compiler regurgitated terrible compilation errors, my code logic being flawed. But how could I find where the bug was located? Without <strong>debugger</strong> and <strong>printf</strong>, things get complicated. If your metaphoric programming beard is not up to your knees (both my metaphoric and physical one did not reach the expectations), you may not have the motivation to use <a href="https://github.com/mikael-s-persson/templight">templight</a> nor to debug your compiler.</p>
<p>Your first friend will be <a href="http://en.cppreference.com/w/cpp/language/static_assert">static_assert</a>, which gives you the possibility to test the value of a compile-time boolean. Your second friend will be a macro to turn on and off <strong>constexpr</strong> wherever possible:</p>
<div style="clear: both;"></div>
<div class="highlight"><pre><span></span><code><span class="cp">#define CONSTEXPR constexpr </span><span class="c1">// compile-time No debug possible</span>
<span class="c1">// OR</span>
<span class="cp">#define CONSTEXPR </span><span class="c1">// Debug at runtime</span>
</code></pre></div>
<p>By using this macro, you can force the logic to execute at runtime and therefore attach a debugger to it.</p>
<h2>Meta Crush Saga II - Looking for a pure compile-time experience:</h2>
<p>Clearly, <strong>Meta Crush Saga</strong> will not win <a href="https://en.wikipedia.org/wiki/The_Game_Awards">The Game Awards</a> this year. It has some great potential, but the experience is not fully <strong>compile-time</strong> YET, which may be a showstopper for hardcore gamers... I cannot get rid-of the bash script, unless someone add <strong>keyboard inputs</strong> and impure logic during the compilation-phase (pure madness!). But I believe, that one day, I could entirely bypass the <strong>renderer</strong> executable and print my <strong>game state</strong> at <strong>compile-time</strong>:</p>
<p><img alt="Life cycle of a fully compile-time game" src="https://jguegant.github.io/blogs/tech/images/life-cycle-fully-compile-time-game.png"></p>
<p>A crazy fellow pseudo-named <strong>saarraz</strong>, <a href="https://github.com/saarraz/static-print">extended GCC</a> to add a <strong>static_print</strong> statement to the language. This statement would take a few constant expressions or string literals and output them during the compilation. I would be glad if such tool would be added to the standard, or at least extend <strong>static_assert</strong> to accept constant expressions.</p>
<p>Meanwhile, there might be a <strong>C++17</strong> way to obtain that result. Compilers already ouput two things, <strong>errors</strong> and <strong>warnings</strong>! If we could somehow control or bend a <strong>warning</strong> to our needs, we may already have a decent output. I tried few solutions, notably the <a href="http://en.cppreference.com/w/cpp/language/attributes">deprecated attribute</a>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="kt">char</span><span class="p">...</span> <span class="n">words</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">useless</span> <span class="p">{</span>
<span class="p">[[</span><span class="n">deprecated</span><span class="p">]]</span> <span class="kt">void</span> <span class="n">call</span><span class="p">()</span> <span class="p">{}</span> <span class="c1">// Will trigger a warning.</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="kt">char</span><span class="p">...</span> <span class="n">words</span><span class="o">></span> <span class="kt">void</span> <span class="n">output_as_warning</span><span class="p">()</span> <span class="p">{</span> <span class="n">useless</span><span class="o"><</span><span class="n">words</span><span class="p">...</span><span class="o">></span><span class="p">().</span><span class="n">call</span><span class="p">();</span> <span class="p">}</span>
<span class="n">output_as_warning</span><span class="o"><</span><span class="err">’</span><span class="n">a</span><span class="err">’</span><span class="p">,</span> <span class="err">‘</span><span class="n">b</span><span class="err">’</span><span class="p">,</span> <span class="err">‘</span><span class="n">c</span><span class="err">’</span><span class="o">></span><span class="p">();</span>
<span class="c1">// warning: 'void useless<words>::call() [with char ...words = {'a', 'b', 'c'}]' is deprecated </span>
<span class="c1">// [-Wdeprecated-declarations]</span>
</code></pre></div>
<p>While the output is clearly there and parsable, it is unfortunately not playable! If by sheer luck, you are part of the secret-community-of-c++-programmers-that-can-output-things-during-compilation, I would be glad to recruit you in my team to create the perfect <strong>Meta Crush Saga II</strong>!</p>
<h2>Conclusion:</h2>
<p>I am done selling you my <s>scam</s> game. Hopefully you found this post ludic and learned things along the reading. If you find any mistake or think of any potential improvement, please to do not hesitate to reach me.</p>
<p>I would like to thanks the <strong>SwedenCpp team</strong> for letting me having my talk on this project during one of their event. And I particularly would like to express my gratitude to <a href="https://www.linkedin.com/in/alexandre-gordeev/">Alexandre Gourdeev</a> who helped me improve <strong>Meta Crush Saga</strong> in quite a few significant aspects.</p>Distributed C++ Meetup 0x022018-04-07T21:04:00+02:002018-04-07T21:04:00+02:00Jean Gueganttag:jguegant.github.io,2018-04-07:/blogs/tech/distributed-c++-meetup-0x02.html<p>Here is a quick follow-up of the event I announced in my previous <a href="https://jguegant.github.io/blogs/tech/distributed-c++-meetup.html">post</a>: the <strong>Distributed C++ Meetup 0x02</strong>. A quick explanation for those too lazy to click a link or scroll down a bit to read my previous post (not judging you here, I would do the same); the …</p><p>Here is a quick follow-up of the event I announced in my previous <a href="https://jguegant.github.io/blogs/tech/distributed-c++-meetup.html">post</a>: the <strong>Distributed C++ Meetup 0x02</strong>. A quick explanation for those too lazy to click a link or scroll down a bit to read my previous post (not judging you here, I would do the same); the concept of a Distibuted C++ Meetup consists in gathering multiple C++ user groups from around the world in one event using video-conference facilities. This time we had the pleasure to bring together the <a href="https://www.meetup.com/berlincplusplus/">Berlin</a>, <a href="https://www.meetup.com/CppLondon/">London</a> and <a href="https://www.meetup.com/swedencpp/">Stockholm</a> Meetup groups using <a href="https://king.com/">King's</a> offices.</p>
<h3>Distributed C++ Meetup 0x02:</h3>
<p>Once again <strong>Phil Nash</strong> took on himself to edit a full footage of the event, which you can enjoy right here:</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/31mQmUryw50" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>We also have altenative footage of the two first talks made by <strong>Harald Achitz</strong>:</p>
<iframe width="300" height="250" src="https://www.youtube.com/embed/Os3i-N_Pz2U" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<iframe width="300" height="250" src="https://www.youtube.com/embed/NGrnKr9rSz4" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>Being the host of the event makes you easily distracted, and so I was! But thanks to Phil's and Harald's work, I can enjoy a rewatch of the talks in a serene environment.</p>
<p><strong>Harald Achitz</strong> gently explained us the <a href="http://open-std.org/JTC1/SC22/WG21/docs/papers/2017/p0515r0.pdf">space-ship operator</a> that will arrive in C++20 and what to expect from it. Then came the introduction on <strong>Type Safe Flags</strong> made by <strong>Arvid Norberg</strong> that I particulary enjoyed : simple and very practical! <strong>Richard Spindler</strong> related us his experience with <a href="https://www.boost.org/doc/libs/1_66_0/libs/compute/doc/html/index.html">Boost.Compute</a>. Finally, <strong>Dominic Jones</strong> made a talk on <strong>Expression Tree Transform</strong> which reminded me a lot <a href="https://www.youtube.com/watch?v=QYNRoeWJuAw">Reactive Equations by André Bergner</a> that I had the chance to see during <strong>Meeting C++ 2017</strong>. Although, I always enjoy template meta-programming, it was very alien to anyone at the event who wasn't familiar to <a href="https://en.wikipedia.org/wiki/Expression_templates">expression templates</a>. I always admire anyone willing to be on stage to share her/his knowledge and I appreciate the honor our speakers did to us by doing so that evening.</p>
<h3>Polirazed opinions:</h3>
<p>As the tradition goes, we sent a survey after our Meetup event. The feedback we received was substantially more divided than last time. Although the numbers indicate that a majority of the people are willing to redo such an event ; 47.50% really want a re-run, 35.00% liked it as much as a normal event, 17.50% would have prefer to stick to a normal one; it was clearly less popular than the last event with respectively 82.50%, 15% and 2%. Surprisingly people changed their minds and now would prefer going back to lightning talks (instead of 30min ones), as suggested by few feedback comments. As a comparison you can also read more about the previous event's results on Phil Nash's <a href="http://www.levelofindirection.com/journal/2017/11/6/the-worlds-first-distributed-c-meet-up.html">blog</a>. People also expressed a strong feeling of being less "connected with the remote crowd" this time, which is sadly one of our main goal for a Distributed Meetup. It was especially surprising that this time we had questions crossing the virtual borders.</p>
<p>I have been reflecting on what changed in our event's recipe and what could have be done better. Here are my thoughs as well as observations from other co-organizers:</p>
<ul>
<li>The more cities we gather, the less time each location has talks happening physically in front of them. No matter how good is the video-conference setup, anyone would prefer to have it "at home". While as an attendee you will appreciate spending <strong>half</strong> of the event learning from speakers at another location, <strong>two-thirds</strong> of the event might be too long. Having more cities also increase the risk of cumulating technical issues. I would suggest to come back to a two cities format.</li>
<li>We need to have room for a break in our schedule. Even Supertemplateman would not stand two hours of C++ talks on a cramped bench. Having 15min to make our transitions between countries would be pretty sweet too.</li>
<li>Lightning talks may be somehow frustrating as they barely touch the surface of a topic. But a 35min talk can also be pretty damn long if you do not like the subject, even more so when projected on a screen. We would like to try 15min for a next run of the event. Shorter talks also have a desirable side effect: speakers will tend to write slides that are very expressive. Which bring us to the next issue.</li>
<li>I noticed that slides were much harder to read especially on the weird 4-TVs setup we have in the Stockholm's office. This was corroborated by the feedback specific to Stockholm. Tiny fonts force you to concentrate to extrapolate information that you cannot obtain with your eyes, it's exhausting and frustrating. We should really advise the speakers to use a minimum font size prior to the event. There is also the possibility to have multiple screens in the Stockholm office, to improve the experience for the people that always sit in our very attractive carousels in the back (every King employee did that mistake once).</li>
<li>People don't like the mandatory NDA to access the London office. I am not sure how this could be solved, as I am not familiar to this office. Be sure that we will explore that topic later on.</li>
<li>Finally, as a reminder, never forget to turn the light on! As much as we are used to darkness in Sweden, it was still a challenge for us to see what happened in the tenebrous London office. </li>
</ul>
<h3>Conclusion:</h3>
<p>I would like to run a third <strong>Distributed C++ Meetup</strong> with the aforementioned "bug-fixes". I am eager to see if the concept is actually viable or if the first run was simply benefiting from the novelty effect. As we are pioneer in these virtual meetups, it is pretty obvious that we will have to <s>use our attendees as guinea pigs</s> slowly refine our ideas .</p>
<p>Thanks to all people that helped during this event: Mårten Möller, Vasil Georgiev, Phil Nash, Harald Achitz, Tindy Hellman, Bruno Mikus, our tech and video-crew...</p>Distributed C++ Meetup2018-03-20T21:04:00+01:002018-03-20T21:04:00+01:00Jean Gueganttag:jguegant.github.io,2018-03-20:/blogs/tech/distributed-c++-meetup.html<p>As mentioned in an early <a href="https://jguegant.github.io/blogs/tech/c++-stockholm-0x02.html">post</a>, I am active member of a C++ Meetup group in Stockholm: <a href="https://www.meetup.com/swedencpp/">SwedenCpp</a>. Last year me and the main organizer of <strong>SwedenCpp</strong> came up with the idea to gather different Meetup groups from Europe for a special event. Knowing that we are not as famous …</p><p>As mentioned in an early <a href="https://jguegant.github.io/blogs/tech/c++-stockholm-0x02.html">post</a>, I am active member of a C++ Meetup group in Stockholm: <a href="https://www.meetup.com/swedencpp/">SwedenCpp</a>. Last year me and the main organizer of <strong>SwedenCpp</strong> came up with the idea to gather different Meetup groups from Europe for a special event. Knowing that we are not as famous as the <a href="https://cppcon.org/">cppcon</a>, and therefore could not attract people easily to come to visit us in our dark and cold country, we had to come up with a special plan. It happens that my current company <a href="https://discover.king.com/about/">King</a> has offices of decent size in few of the major tech hubs of Europe: Berlin, Stockholm, London and Barcelona. It also happens that we have pretty good video-conference facilities that we frequently use to do company-wide calls. AND It also happens that a huge share of the mobile games (Candy Crush Saga, Bubble Witch Saga...) made at <strong>King</strong> rely heavily on a <strong>C++</strong> tech stack. By using the video-conference facilities, having some drinks and food, all gently provided by <strong>King</strong>, we were confident that we could run an event with multiple Meetup groups participating remotely and have a lot of fun.</p>
<p>And thus was born the concept of a <strong>Distributed C++ Meetup</strong>!</p>
<p>We contacted the <a href="https://www.meetup.com/CppLondon/events/248091281/">London Meetup group</a> and asked them if they wanted to join us in this aventure. Sure enough, Phil Nash, the organizer, said yes. Greatly helped by few colleagues (Mårten Möller, Tindy Hellman, Bruno Mikus, Brittany Pinder...) we ran a first event in November 2017 with a connection between London and Stockholm. </p>
<h3>Distributed C++ Meetup 0x01:</h3>
<p>Here is a full video of the first event (Distributed C++ Meetup 0x01) where we had lightning talks from speakers on both sites:</p>
<iframe width="854" height="480" src="https://www.youtube.com/embed/vIVZsVneVDg" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>Or if you want to see some specific talks, you can also watch this <a href="https://www.youtube.com/playlist?list=PLBKFkLKL89uLmO22p7vWVrWO5eGzDM7PE">youtube playlist</a> with videos nicely crafted by our amazing <strong>SwedenCpp</strong> video crew.</p>
<p>I was pretty nervous to be the host of the event, but I think that everything ran pretty smoothly and the vast majority of the attendees enjoyed the concept.
We had few learnings though:</p>
<ul>
<li>Some people prefer the long format for the talks instead of lightning talks we used this time.</li>
<li>I had a failed attempt in creating an interaction between the crowd and the speakers. No one really dares to speak to remote speakers! This could be solved by using apps like slido.</li>
<li>It always nice to use have backups for video recording and sound recording. The audio recorded in London was not as good as expected.</li>
<li>Do not drink the beers and ciders originally bought for the halloween party!</li>
</ul>
<h3>Distributed C++ Meetup 0x02:</h3>
<p>The reason I decided to write about this event few months after is that 1) I am lazy 2) we will soon have another run of this concept.
Next week, on the 28th of March 2018, the Distributed C++ Meetup 0x02 will happen. This time not only London and Stockholm will be involved but we also bring with us the <a href="https://www.meetup.com/berlincplusplus/">Berlin Meetup Group</a> managed by Sebastian Theophil and Vasil Georgiev (from King)!</p>
<p>If you are reading this post and want to join the event, it is still not too late to sign-up:</p>
<ul>
<li><a href="https://www.meetup.com/berlincplusplus/events/248093069/">Berlin</a></li>
<li><a href="https://www.meetup.com/de-DE/CppLondon/events/248091281/">London</a></li>
<li><a href="https://www.meetup.com/swedencpp/events/248092613/">Stockholm</a></li>
</ul>Trip report - Meeting C++ 20172017-11-16T23:09:00+01:002017-11-16T23:09:00+01:00Jean Gueganttag:jguegant.github.io,2017-11-16:/blogs/tech/trip-report-meetingcpp-2017.html<p>Finally, after years of watching youtube videos on that topic, I made it to my first <strong>C++</strong> international conference!
Thanks to my current employer <a href="https://discover.king.com/about/">King</a>, I went last week to <a href="http://meetingcpp.com/">Meeting C++</a> in <strong>Berlin</strong>, which is, as far as I know, the biggest <strong>C++</strong> event in Europe. I really enjoyed …</p><p>Finally, after years of watching youtube videos on that topic, I made it to my first <strong>C++</strong> international conference!
Thanks to my current employer <a href="https://discover.king.com/about/">King</a>, I went last week to <a href="http://meetingcpp.com/">Meeting C++</a> in <strong>Berlin</strong>, which is, as far as I know, the biggest <strong>C++</strong> event in Europe. I really enjoyed my time there with few hundred other fellow <strong>C++</strong> enthusiasts. In this post, I will try to relate how I experienced the event and dress a list of the must-watch talks.</p>
<h1>About meeting C++:</h1>
<h2>The concept:</h2>
<p><img style="float: right;" width=200 height=250 src="https://jguegant.github.io/blogs/tech/images/badge.jpg"/></p>
<p>Held in the magnificient <a href="http://meetingcpp.com/2017/Location.html">Andels Hotel</a> in <strong>Berlin</strong>, <strong>Meeting C++</strong> offers the possibility to attend keynotes, talks and lightning talks (respectively lasting 2 hours, 50min and 5min) about our favourite language <strong>C++</strong> for 3 days (one extra day added for the 6th edition of the event). <strong>C++</strong> being multi-paradigm and a general-purpose programming language, the variety of the topics being discussed is pretty wide. It ranges from subject on "(Template) Meta-programming" to a deep dive on "How a C++ debugger work", from begginner friendly talks to hairy discussions on the yet-to-be-standardised white paper on <strong>std::expected<T, E></strong>.</p>
<p>As some talks happen simulteanously in different rooms, you cannot physically attend all the talks. Instead, you would usually prepare your own schedule by trying to guess the content of the talks from their summary. It is always a dilemna to choose between a topic that you like with the risk to have nothing new to learn, and a brand-new topic for you, where sleepness may kick-in midway to the presentation. If you are curious and daring, the lightning talks on the last day permit you to randomly discover antyhing about C++ in succint presentations. In any case, you can always catch-up the missed talks by checking the <a href="https://www.youtube.com/user/MeetingCPP">Youtube channel</a>.</p>
<p>Generally, I was not disapointed by the quality of the slides and the speakers. I picked up quite a few new concepts, prepared myself for the future of <strong>C++</strong> (C++20) and refreshed myself on some fields I did not touch for a while.</p>
<h2>More than just talks:</h2>
<p>Where <strong>Meeting C++</strong> really shines is in its capacity to gather roughly 600 passionated developers from various backgrounds (university, gaming industry, finance, Sillicon Valley's giants...) in one building and <strong>share</strong>. Just share anything about C++, about reverse-engineering, about your job, about your country, about the german food in your plate! The C++ community represented at this event is very active, open-minded and willing to help. The catering between the sessions and the dinner parties permit you to meet anyone easily. The even team also oganises some really fun events</p>
<p>In a room full of "world-class developers", it is easy to be intimidated, but you should not hesitate to reach them. They will not nitpick your words nor snob you. For some people, it is a dream to meet an Hollywood Star in the street, for me it was really delightful to have casual conversations with these "legendary" coders from the web.</p>
<h1>The chief's suggestions of the day:</h1>
<p>Here is a menu of most of the talks I attended. The legend is pretty simple:</p>
<ul>
<li>💀 : The difficulty of the talk (💀: Begginer friendly, 💀💀: Intermediate, 💀💀💀: High exposure to C++'s dark corners)</li>
<li>★ : My interest for the talk (★: Good talk, ★★: Tasty talk, ★★★: Legendary talk)</li>
</ul>
<p>I will not spoil all the talks, but simply try to give an overview of what you can expect within them. Note that all the talks are generally of high quality and my appreciation very subjective. I have seen people with very different "favorite talk".</p>
<h3>[Keynote] Better Code: Human interface - By Sean Parent - 💀 ★★</h3>
<ul>
<li>Slides: <a href="https://github.com/sean-parent/sean-parent.github.io/blob/master/presentations/2017-11-09-human-interface/2017-11-09-human-interface.pdf">link</a></li>
<li>Video: <a href="">coming-soon</a></li>
</ul>
<p>Sean Parent is a Principal Scientist at Adobe Systems and has been working on the famous software <strong>Photoshop</strong> for more than 15 years. Sean Parent is a regular and prominent speaker at C++ conferences, one of his recently most famous talk being <a href="https://www.youtube.com/watch?v=QGcVXgEVMJg">Better Code: Runtime Polyphormism</a> from the same series of talks (Better Code) as the one he gave during <strong>Meeting C++</strong>.</p>
<p>Thorough his keynote, Sean was conveing the message that in order to have ergonomic human interfaces, you must design your code to reflects its usage through UI. By following such a principle, one can easily come-up with good namings, semantics and grouping of your UI components.</p>
<p>For instance, Sean was explaining that most of the menu actions in <strong>Photoshop</strong> somehow mapped some object, their methods, properties and most importantly their <strong>relations</strong>:</p>
<ul>
<li>The menu action <strong>Create New Layer</strong> will somehow call the constructor of a class called something like <strong>Layer</strong>.</li>
<li>Likewise the action <strong>Delete A Layer</strong> would call its destructor.</li>
<li>A selection in <strong>Photoshop</strong> will most likely translate in a container of objects.</li>
</ul>
<p>As a counter-example he explained that the old version of <strong>Gmail</strong> used to have a confusing flow when it comes to the most trivial usage of a mail service: creating a mail. A <strong>link</strong>, which implies <strong>navigation</strong>, was used instead of <strong>button</strong> for the "compose message" action.</p>
<p><img alt="Gmail failed compose button" src="https://jguegant.github.io/blogs/tech/images/gmail-failure.png"></p>
<p>Sean put a strong emphasis that relationships are the most difficult part of an architecture to represent. He came up with few examples on how <a href="http://en.cppreference.com/w/cpp/algorithm/stable_partition">std::stable_partition</a> can be used to solve in an elegant way the gathering and display of items </p>
<p>Overall a very nice talk, but on a very abstract topic, since not much has been explored on that subject yet! This is worth paying attention in game-programming where a good UI is a key-part of the success of a game.</p>
<h2>[Talk] Threads and Locks must Go - Rainer Grimm - 💀💀 ★</h2>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=fkqVRzy4JhA">link</a></li>
</ul>
<p>In this talk <strong>Rainer Grimm</strong>, a German author of multiple <a href="https://www.goodreads.com/author/show/7496329.Rainer_Grimm">C++ books</a>, brought under the spotlight the concurrency features introduced by the new C++ standard <strong>C++17</strong> and the coming one <strong>C++20</strong>. Here is a short summary of my favourite features:</p>
<h3>For C++17 (and concurrency TS):</h3>
<ul>
<li>The new parallel algorithms in <a href="http://en.cppreference.com/w/cpp/algorithm"><algorithm></a> and the associated execution policies <a href="http://en.cppreference.com/w/cpp/algorithm/execution_policy_tag_t">seq, par and par_unseq</a>.</li>
</ul>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">v</span> <span class="o">=</span> <span class="p">{...};</span> <span class="c1">// A bit vector... </span>
<span class="n">std</span><span class="o">::</span><span class="n">sort</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">execution</span><span class="o">::</span><span class="n">par</span><span class="p">{},</span> <span class="n">v</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">v</span><span class="p">.</span><span class="n">end</span><span class="p">());</span>
<span class="c1">// Due to "par", this **might** execute the sort in parallel using a thread pool of some sort.</span>
</code></pre></div>
<ul>
<li>The new <a href="http://en.cppreference.com/w/cpp/experimental/future">std::future</a> interface which permits to add continuations to the shared state using the <a href="http://en.cppreference.com/w/cpp/experimental/future/then">then member function</a>.</li>
</ul>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">future</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">foo</span><span class="p">();</span>
<span class="k">auto</span> <span class="n">f</span> <span class="o">=</span> <span class="n">foo</span><span class="p">();</span>
<span class="n">f</span><span class="p">.</span><span class="n">then</span><span class="p">([](</span><span class="n">std</span><span class="o">::</span><span class="n">future</span><span class="o"><</span><span class="kt">int</span><span class="o">>&</span> <span class="n">f</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Will be called when f is done.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">f</span><span class="p">.</span><span class="n">get</span><span class="p">();</span> <span class="c1">// Will therefore not block.</span>
<span class="p">});</span>
</code></pre></div>
<h3>Hopefully for C++20:</h3>
<ul>
<li>The stackless <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4680.pdf">coroutines</a> as implemented by MSVC and clang. This introduce two keywords <strong>co_await</strong> and <strong>co_yield</strong>. Considering the previous example using std::future, it could be rewritten in the following way:</li>
</ul>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">future</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">foo</span><span class="p">();</span>
<span class="kt">int</span> <span class="n">x</span> <span class="o">=</span> <span class="k">co_await</span> <span class="n">foo</span><span class="p">();</span> <span class="c1">// The continuation is "generated" by the compiler using the keyword co_await.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span><span class="p">;</span> <span class="c1">// Everything after co_await is implicitely part of the continuation.</span>
</code></pre></div>
<ul>
<li>A new <a href="http://en.cppreference.com/w/cpp/language/transactional_memory">synchronized keyword</a> applying the transactional memory concept on a group of statements.</li>
</ul>
<div class="highlight"><pre><span></span><code><span class="kt">int</span> <span class="n">x</span><span class="p">;</span> <span class="n">y</span><span class="p">;</span> <span class="c1">// Global vars</span>
<span class="kt">void</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span> <span class="c1">// Foo can be called by multiple thread simuteanously.</span>
<span class="c1">// Reads-writes on x and y are now thread-safe and synchronized.</span>
<span class="n">synchronized</span> <span class="p">{</span>
<span class="o">++</span><span class="n">x</span><span class="p">;</span>
<span class="o">++</span><span class="n">y</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>As explained by <strong>Rainer Grimm</strong>, we will have the possibility to easily bring concurrency to our C++ codebases without ressorting to the low-level, and tricky to get right, features like thread and locks. While I appreciated the talk, it lacked a bit of novelty as I was already aware of most of the features.</p>
<h2>[Talk] Strong types for strong interfaces - Johnathan Boccora - 💀 ★★★</h2>
<ul>
<li>Related blog post: <a href="https://www.fluentcpp.com/2016/12/08/strong-types-for-strong-interfaces/">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=WVleZqzTw2k&t=1536s">link</a></li>
</ul>
<p>A <strong>must watch</strong>! Even when facing some technical issues, <strong>Johnathan</strong> is very good speaker and I was quickly captivated by the topic of <strong>strong types</strong>. Jonathan is also a talented writer with his famous blog <a href="https://www.fluentcpp.com/">fluentcpp</a> (I would really suggest to have a look at it once in a while).</p>
<p>As C++ developers, we heavily rely on the language's type system to express our intentions to other developers, to optimise our code and avoid shooting ourselves into our feet. Yet, we always reuse some <strong>types</strong> to express very different properties of our system. For instance to describe a person, you would use an <strong>int</strong> for her/his age and his/her weight. Did it ever come to you that the unit <strong>year</strong> (for age) should be a very different type than <strong>kg</strong> (for weight)? The concept of <strong>strong type</strong> would solve this problem by introducing new int-compatible types:</p>
<div class="highlight"><pre><span></span><code> <span class="c1">// We need to have these empty tag types to create entirely new types and not just weakly-typed aliases.</span>
<span class="k">using</span> <span class="n">kg</span> <span class="o">=</span> <span class="n">strong_type</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="k">struct</span> <span class="nc">KgTag</span><span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">years</span> <span class="o">=</span> <span class="n">strong_type</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="k">struct</span> <span class="nc">YearsTag</span><span class="o">></span><span class="p">;</span>
<span class="kt">void</span> <span class="nf">unsafe_create_person</span><span class="p">(</span><span class="kt">int</span> <span class="n">age</span><span class="p">,</span> <span class="kt">int</span> <span class="n">weight</span><span class="p">);</span>
<span class="kt">void</span> <span class="nf">create_person</span><span class="p">(</span><span class="n">years</span> <span class="n">age</span><span class="p">,</span> <span class="n">kg</span> <span class="n">weight</span><span class="p">);</span> <span class="c1">// Explicit interface.</span>
<span class="kt">int</span> <span class="n">age</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">weight</span> <span class="o">=</span> <span class="mi">1337</span><span class="p">;</span>
<span class="n">unsafe_create_person</span><span class="p">(</span><span class="n">weight</span><span class="p">,</span> <span class="n">age</span><span class="p">);</span> <span class="c1">// Oops I inversed the arguments but no compiler error.</span>
<span class="n">create_person</span><span class="p">(</span><span class="n">years</span><span class="p">(</span><span class="n">age</span><span class="p">),</span> <span class="n">kg</span><span class="p">(</span><span class="n">weight</span><span class="p">)));</span> <span class="c1">// Much less error-prone.</span>
</code></pre></div>
<p>As a bonus, <strong>strong types</strong> can actually affect positively the performances of your codebase as the compiler can agressively optimise without violating <a href="http://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html">strict-aliasing rules</a>, since the types are now strictly unrelated.</p>
<p>This concept is not new and is already used in <a href="http://en.cppreference.com/w/cpp/chrono">std::chrono</a> or <a href="http://www.boost.org/doc/libs/1_65_1/doc/html/boost_units.html">Boost.Unit</a>, but it was really refreshing to have an explanation with simple words and good examples! I am now very keen to use this in my personal projects and at work too.</p>
<h2>[Talk] How C++ Debuggers Work - Simon Brand (4/5) - 💀💀 ★★</h2>
<ul>
<li>Related blog post: <a href="https://blog.tartanllama.xyz/writing-a-linux-debugger-setup/">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=Q3Rm95Mk03c">link</a></li>
</ul>
<p><strong>Simon Brand</strong>, also known as <a href="https://blog.tartanllama.xyz/">TartanLlama</a> (a really fancy fictious name for a Scott), presented us how a mixture of calls to <a href="http://man7.org/linux/man-pages/man2/ptrace.2.html">ptrace</a>, injection of the <a href="https://en.wikipedia.org/wiki/INT_(x86_instruction)">int3 opcode</a>, parsing of the <a href="https://en.wikipedia.org/wiki/DWARF">DWARF format</a> and perseverance is the base to create a debugger on a <strong>x86</strong>(_64) architecture with a <strong>Unix</strong> platform (or <strong>Linux</strong> platform only if you OS specific calls like <a href="http://man7.org/linux/man-pages/man2/process_vm_readv.2.html">process_vm_readv, process_vm_writev</a>).</p>
<p>Unlike some of the other talks, it would be hard to give succinct code examples, but I trully appreciated his presentation! When it comes to low-level APIs for debbuging and reverse-engineering, I have a better understanding of the Windows platform. I think that <strong>Simon</strong> did an excellent job to help me transfer my knowledge to the Unix world.</p>
<p>If one day I have to tackle the creation of a debugger on Unix, I would certainely come back to this talk or follow his series of <a href="https://blog.tartanllama.xyz/writing-a-linux-debugger-setup/">blog posts</a> on the same subject. I also think that as programmer, it is always beneficial to have some knowledge on the underlying mechanims of the tools you use (gdb, or lldb in that case). I would, therefore suggest to watch that talk to any C++ enthusiast willing to progress in their art of programming.</p>
<h3>[Talk] The Three Little Dots and the Big Bad Lambdas - Joel Falcou - 💀💀💀 ★★★</h3>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=WU5v4FT7CMM">link</a></li>
</ul>
<p>I am always excited by watching a talk from <a href="https://twitter.com/joel_f?lang=en">Joel Falcou</a>: he is a venerable (template) metra-programmer wizzard with a very didactic approach to explain things (and also we share the same nationality \O/). Once again, I was not disapointed by his session.</p>
<p>With a lot of humour, Joel introduced a new facet to <strong>meta-programming</strong> in C++. We used to have <a href="https://en.wikipedia.org/wiki/Template_metaprogramming">template meta-programming</a> to manipulate types at compile-time (and with difficulties values), then came <a href="http://en.cppreference.com/w/cpp/language/constexpr">constexpr</a> to ease value computation, and recently a <strong>Louis Dionne</strong> came-up with a powerful combo of these two cadrants with <a href="http://www.boost.org/doc/libs/1_61_0/libs/hana/doc/html/index.html#tutorial-introduction-quadrants">Boost.Hana</a>. <strong>Joel</strong> statement was that <a href="http://en.cppreference.com/w/cpp/language/lambda">lambdas expressions</a> combined with <a href="http://en.cppreference.com/w/cpp/language/auto">auto</a> and <a href="http://en.cppreference.com/w/cpp/language/parameter_pack">parameter packs</a> are powerful enough to replace some cases where we would have resort to use <strong>template meta-programming</strong> or the uglier infamous <strong>macros</strong>! <strong>Joel</strong> came to that conclusion after being inspired by the language <a href="http://okmij.org/ftp/ML/MetaOCaml.html">MetaOCaml</a>.</p>
<p>Let's say that you want to fill a vector with push-back instruction generated at compile-time:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><tuple></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><array></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><utility></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><vector></span><span class="cp"></span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">F</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="kt">size_t</span><span class="p">...</span> <span class="n">I</span><span class="o">></span>
<span class="kt">void</span> <span class="n">apply_imp</span><span class="p">(</span><span class="n">F</span><span class="o">&&</span> <span class="n">f</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">index_sequence</span><span class="o"><</span><span class="n">I</span><span class="p">...</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="n">I</span><span class="p">),</span> <span class="p">...);</span> <span class="c1">// C++17 fold expression.</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="kt">int</span> <span class="n">N</span><span class="p">,</span> <span class="k">class</span> <span class="nc">F</span><span class="o">></span>
<span class="kt">void</span> <span class="n">apply</span><span class="p">(</span><span class="n">F</span><span class="o">&&</span> <span class="n">f</span><span class="p">)</span> <span class="p">{</span>
<span class="n">apply_imp</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">make_index_sequence</span><span class="o"><</span><span class="n">N</span><span class="o">></span><span class="p">{});</span>
<span class="p">}</span>
<span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">bob</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">bind</span> <span class="o">=</span> <span class="p">[](</span><span class="k">auto</span><span class="o">&</span> <span class="n">v</span><span class="p">,</span> <span class="k">auto</span> <span class="n">f</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="p">[</span><span class="o">&</span><span class="n">v</span><span class="p">,</span> <span class="n">f</span><span class="p">](</span><span class="k">auto</span> <span class="n">x</span><span class="p">){</span> <span class="n">f</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">x</span><span class="p">);</span> <span class="p">};</span> <span class="p">};</span>
<span class="k">auto</span> <span class="n">push_back</span> <span class="o">=</span> <span class="n">bind</span><span class="p">(</span><span class="n">bob</span><span class="p">,</span> <span class="p">[](</span><span class="k">auto</span><span class="o">&</span> <span class="n">v</span><span class="p">,</span> <span class="kt">int</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="n">v</span><span class="p">.</span><span class="n">push_back</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="mi">3</span><span class="p">);</span> <span class="p">});</span>
<span class="n">apply</span><span class="o"><</span><span class="mi">3</span><span class="o">></span><span class="p">(</span><span class="n">push_back</span><span class="p">);</span>
<span class="c1">// Will generate at compile time:</span>
<span class="c1">// bob.push_back(0);</span>
<span class="c1">// bob.push_back(3);</span>
<span class="c1">// bob.push_back(6);</span>
</code></pre></div>
<p>This example is fairly trivial and there would be a high chance that you would reach the same assembly output using a simple <strong>for loop</strong>. But it is very interesting to notice that <strong>lambdas</strong> are reusable type-safe <strong>units of code</strong> that you transport, combine and "instantiate" at any time. Performance-wise, <strong>lambdas</strong> are pretty incredible according to Joel's measurement on his linear-algebra project. <strong>C++17</strong> <strong>constexpr lambdas</strong> could also help on that topic. One drawback might be the debugging complexity when navigating in nested lambdas. I still need to wrap my head around this new concept and I am eager to rewatch Joel's talk to explore it more!</p>
<h2>[Keynote] Its complicated! - Kate Gregory - 💀 ★★★</h2>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=tTexD26jIN4">link</a></li>
</ul>
<p>While excellent, <a href="http://www.gregcons.com/kateblog/">Kate</a>'s keynote would be very hard to summarise correctly within few paragraphs. It makes you reflect on the difficulties to introduce <strong>C++</strong> to newcomers. You would hope that there is a subset of the language that could be easily assimilate by anyone, <strong>Kate</strong> argues that the reality is sadly more <strong>complicated</strong> than that. Just have a look at how long are the <a href="http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#fcall-parameter-passing">C++ core guidelines</a> on passing parameters to function calls. One day or another, a begginer must learn on how to pass parameters with good semantics and in an optimised fashio. Well, good luck to her/him! On the other hand, it does not mean that the language could have been designed in a simpler way. What we should strive for instead might be better naming of these concepts: the acronym <a href="http://en.cppreference.com/w/cpp/language/raii">RAII (Resource Acquisition Is Initialization)</a> is obviously not as straightforward as <a href="https://en.wikipedia.org/wiki/Copy-on-write">COW (Copy-on-write)</a>. Whether you are a "newbie" or the best "lead over-engineer" of your company, this talk is really worth a look!</p>
<h2>[Talk] There Is A New Future - Felix Petriconi - 💀💀 ★★</h2>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=vDmQlIeY4z0">link</a></li>
</ul>
<p><a href="https://petriconi.net/?author=1">Felix Petriconi</a> and <strong>Sean Parent</strong> have been working on a the <a href="http://stlab.cc/libraries/concurrency/">stlab</a> library for quite some time. <strong>stlab</strong> takes the best of the various <strong>future</strong> implementations <a href="http://en.cppreference.com/w/cpp/thread/future">std::future (C++11)</a>, <a href="http://en.cppreference.com/w/cpp/thread/future">std::future (C++14)</a> or <a href="http://www.boost.org/doc/libs/1_65_0/doc/html/thread/synchronization.html#thread.synchronization.futures">boost::future</a>, and adds a bit of it owns features on top of it. For instance, <strong>stlabs</strong> supports passing explicit executors to control where <a href="http://stlab.cc/libraries/concurrency/future/async.html">async</a> will execute the tasks, and where the continuation associated with <a href="http://stlab.cc/libraries/concurrency/future/future/then.html">then</a> will be executed too. <strong>Executors</strong> are akin to <strong>event-loops</strong> (or message-pumps in the .Net world) that will process the tasks.</p>
<div class="highlight"><pre><span></span><code><span class="c1">// This task will be executed on ``an_executor``</span>
<span class="k">auto</span> <span class="n">f</span> <span class="o">=</span> <span class="n">stlab</span><span class="o">::</span><span class="n">async</span><span class="p">(</span><span class="n">an_executor</span><span class="p">,</span> <span class="p">[]</span> <span class="p">{</span> <span class="k">return</span> <span class="mi">42</span><span class="p">;</span> <span class="p">});</span>
<span class="c1">// The continuation on another executor.</span>
<span class="n">f</span><span class="p">.</span><span class="n">then</span><span class="p">(</span><span class="n">another_executor</span><span class="p">,</span> <span class="p">[](</span><span class="kt">int</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span><span class="p">;</span> <span class="p">});</span>
</code></pre></div>
<p>While <strong>executors</strong> are present in <a href="http://www.boost.org/doc/libs/1_65_1/doc/html/thread.html">Boost.Thread</a>, <strong>stlabs's</strong> <a href="http://stlab.cc/libraries/concurrency/channel/channel.html">channels</a> are unique to this <strong>future</strong> library. <strong>Channels</strong> are one of the Go language's favorite <a href="https://gobyexample.com/channels">toy</a>. It is a neat way to create communication between a <strong>sender</strong> and a <strong>receiver</strong> on different <strong>executors</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="p">[</span><span class="n">sender</span><span class="p">,</span> <span class="n">receiver</span><span class="p">]</span> <span class="o">=</span> <span class="n">channel</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="n">receive_executor</span><span class="p">);</span> <span class="c1">// Create the link.</span>
<span class="n">receiver</span> <span class="o">|</span> <span class="p">[](</span><span class="kt">int</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">x</span><span class="p">;</span> <span class="p">};</span> <span class="c1">// Define what should happen at the reception.</span>
<span class="c1">// Establish the connections.</span>
<span class="n">receiver</span><span class="p">.</span><span class="n">set_ready</span><span class="p">();</span>
<span class="n">sender</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span> <span class="c1">// Sent 42 through the channel.</span>
<span class="c1">// Receiver will print 42 when executing the task.</span>
</code></pre></div>
<p>I really like some of the features in <strong>stlabs</strong>, hopefully this could be incorporated into the <strong>C++</strong> standard (the executors are down in the pipe of the standardisation process).</p>
<h3>[Talk] Introduction to proposed std::expected<T, E> - Niall Douglas - 💀💀💀 ★</h3>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=JfMBLx7qE0I">link</a></li>
</ul>
<p>Are you the kind of person that would rather have errors on the return values rather than using <strong>exceptions</strong>. <a href="https://twitter.com/ned14?lang=en">Niall</a> has a solution for you: <a href="https://github.com/viboes/std-make/blob/master/doc/proposal/expected/p0323r3.pdf">std::expected<T, E></a>. You can see <strong>std::expected<T, E></strong> either as a <a href="http://en.cppreference.com/w/cpp/utility/optional">std::optional<T></a> with a empty state containing an error for being empty, or as a <a href="http://en.cppreference.com/w/cpp/utility/variant">std::variant<T, E></a> where you agree that the first alternative is the return value and the second alternative is the potential error. Example:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">expected</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span> <span class="n">foo</span><span class="p">(</span><span class="kt">int</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">x</span> <span class="o"><</span> <span class="mi">0</span><span class="p">)</span> <span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">make_unexpected</span><span class="p">(</span><span class="s">"x < 0"</span><span class="p">);</span>
<span class="k">return</span> <span class="mi">42</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">result</span> <span class="o">=</span> <span class="n">foo</span><span class="p">(</span><span class="mi">-1</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">result</span><span class="p">)</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">result</span><span class="p">.</span><span class="n">error</span><span class="p">().</span><span class="n">value</span><span class="p">();</span> <span class="c1">// Prints " x < 0".</span>
</code></pre></div>
<p><strong>std::expected</strong> starts to be cumbersome to use when combining or propogating returned results. To palliate this problem, <strong>std::expected</strong> exposes a <a href="https://en.wikipedia.org/wiki/Monad_(functional_programming)">Monad interface</a>, with the bind member function coming directly to your mind. If you are a Haskell user, <strong>std::expected</strong> should remind you of the <a href="https://en.wikibooks.org/wiki/Haskell/Understanding_monads/Maybe">Maybe Monad</a>. Using <strong>bind</strong> is still verbose and hopefully we obtain a dedicated keyword <strong>try</strong> to ease our pain.</p>
<h2>[Talk] The most valuable values - Juan Pedro Bolívar Puente - 💀💀 ★★</h2>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="">coming-soon</a></li>
</ul>
<p>During his presentation, <a href="https://sinusoid.es/do">Juan</a> actively promoted <a href="https://isocpp.org/wiki/faq/value-vs-ref-semantics#val-vs-ref-semantics">value semantic over reference semantic</a> and did so with some analogies from our physical world (from our dear philosopher Platos) and code examples. The talk quickly moved onto immutability and functionnal programming applied to user interfaces. There is a trend in the web sphere to follow a software architectural pattern called <a href="https://facebook.github.io/flux/">flux</a> with a main implementation the <a href="https://redux.js.org/">redux</a> framework. Arguably, <strong>flux</strong> is a glorified good old <a href="https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller">MVC (Model View Controller) architecture</a> with a strong emphasis on the immutability of the <strong>model</strong> and strict <strong>flow</strong> on the interactions between the components of <strong>MVC</strong>. <strong>Model</strong>, <strong>View</strong> and <strong>Controller</strong> also get respectively renamed to <strong>Store</strong>, <strong>View</strong> and <strong>Dispatcher</strong>. An action submitted to the <strong>Dispatcher</strong> will update the <strong>Store</strong> with a new <strong>Store</strong> in a determinstic way, which will imply a redraw of the <strong>View</strong>. </p>
<p><strong>Juan</strong> succeeded to mimic <strong>redux</strong> in the <strong>C++</strong> world using his library <a href="https://sinusoid.es/immer/">immer</a>. To demonstrate the capabilities of his library, <strong>Juan</strong> recreated an emacs-like <a href="https://github.com/arximboldi/ewig">editor</a>. The beauty of having an immutable <strong>Store</strong> is truly expressed in the <strong>time-traveling machine</strong> that you can create from it: by saving all the states of the <strong>Store</strong>, you can easily come back to a previous state of your application (similar to undo / redo). You should absolutely watch the video to understand how easy it seems to implement this. On top of that you will have the chance to watch what might be the most audacious ending of a <strong>C++</strong> talk I have ever seen.</p>
<h2>[Talk] Reactive Equations - André Bergner - 💀💀💀 ★★★</h2>
<ul>
<li>Github project: <a href="https://github.com/andre-bergner/zignal/tree/master/reactive_equations">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=QYNRoeWJuAw">link</a></li>
</ul>
<p>As a meta-programmer aficionado, this was the most "devilish" talk, and therefore highly thrilling, I attended during the event. It was not the first time I heard from <strong>André Bergner</strong>, he attended the cppcon in 2015 and I remembered that he <a href="https://www.youtube.com/watch?v=zVLLdGlbCSw">presented</a> a nice way to have <a href="https://en.wikipedia.org/wiki/Currying">currying</a> on your function. This time, <strong>André</strong> focused on reactive equations. If this sounds foreign to you, you might be more familiar with <a href="https://en.wikipedia.org/wiki/Data_binding">data binding</a> in <a href="http://doc.qt.io/qt-5/qtqml-javascript-expressions.html#javascript-in-property-bindings">Qt QML</a> using Javascript expression. <strong>André</strong>'s reactive equations are similar but with simpler expressions:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// ***** Using a pseudo-language, let's define the equations *****</span>
<span class="nl">x</span> <span class="p">:</span> <span class="kt">float</span>
<span class="nl">y</span> <span class="p">:</span> <span class="kt">float</span>
<span class="nl">z</span> <span class="p">:</span> <span class="kt">float</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">x</span> <span class="o">*</span> <span class="mi">42</span> <span class="c1">// When x is updated, y will be automatically updated.</span>
<span class="n">z</span> <span class="o">=</span> <span class="n">y</span> <span class="o">+</span> <span class="n">x</span> <span class="c1">// When y or x is updated, z will be automatically updated.</span>
</code></pre></div>
<p>You may notice that I didn't write any <strong>C++</strong> code. By default <strong>C++</strong> would not permit to have expressions that update themselves by "magic" if one variable changes. You could write the update logic manually, but with a lot of long equations, this becomes very error prone. Instead <strong>André</strong> created a <a href="https://en.wikipedia.org/wiki/Domain-specific_language">DSL (Domain Specific Language)</a>, which is equivalent to create a language within <strong>C++</strong> itself. To define his DSL, <strong>André</strong> used <a href="https://en.wikipedia.org/wiki/Expression_templates">expression templates</a>. <strong>Expression templates</strong> are tricky creatures, which roughly consist in encapsulating <strong>C++</strong> expressions into a type at compile-time. This type will retain all the operators / functions (let's call them operations) that you applied in your expression. These operations can be queried at compile-time to generate other expression that you will execute at runtime. In <strong>André</strong>'s case, the encapsulated operations from his reactive equations would be used to automagically generate the update logic. To facilitate his task, <strong>André</strong> heavily relied on <a href="http://www.boost.org/doc/libs/1_65_1/doc/html/proto.html">Boost.Proto</a>. If you are versed in the art of meta-programming, this will certainely be entertaining to you!</p>
<h2>[Talk] Free your functions - Klaus Iglberger - 💀 ★★★</h2>
<ul>
<li>Slides: <a href="https://github.com/CppCon/CppCon2017/raw/master/Presentations/Free%20Your%20Functions/Free%20Your%20Functions%20-%20Klaus%20Iglberger%20-%20CppCon%202017.pdf">link</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=nWJHhtmWYcY">link</a></li>
</ul>
<p>This was a glorious hymn to our beloved free functions by <a href="https://www.linkedin.com/in/klaus-iglberger-2133694/">Klaus Iglberger</a>. Programmers often resort to use <strong>member functions</strong> and inheritance to provide polyphormism in <strong>C++</strong>, often overlooking that <strong>free functions</strong> and <strong>overloading</strong> would be a smarter choice.</p>
<p>Let's take a situation where you would need to implement a <strong>serialise</strong> function for a bunch of unrelated types. Would rather use the implementation <strong>1</strong>?</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">serialisable</span> <span class="p">{</span>
<span class="k">virtual</span> <span class="o">~</span><span class="n">serialisable</span><span class="p">();</span>
<span class="k">virtual</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialise</span><span class="p">();</span>
<span class="p">};</span>
<span class="k">struct</span> <span class="nc">A</span> <span class="o">:</span> <span class="n">serialisable</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialise</span><span class="p">()</span> <span class="k">override</span> <span class="p">{</span> <span class="cm">/* return something... */</span> <span class="p">};</span>
<span class="p">};</span>
<span class="k">struct</span> <span class="nc">B</span> <span class="o">:</span> <span class="n">serialisable</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialise</span><span class="p">()</span> <span class="k">override</span> <span class="p">{</span> <span class="cm">/* return something... */</span> <span class="p">};</span>
<span class="p">};</span>
</code></pre></div>
<p>Or the solution <strong>2</strong>?</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{};</span>
<span class="k">struct</span> <span class="nc">B</span> <span class="p">{};</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialise</span><span class="p">(</span><span class="k">const</span> <span class="n">A</span><span class="o">&</span> <span class="n">a</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* return something... */</span> <span class="p">}</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialise</span><span class="p">(</span><span class="k">const</span> <span class="n">B</span><span class="o">&</span> <span class="n">b</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* return something... */</span> <span class="p">}</span>
</code></pre></div>
<p>As <strong>Kate</strong> explained, it is complicated! If you are looking for <strong>runtime polyphormism</strong>, then you will certainely use the solution <strong>1</strong>. If not, the solution <strong>2</strong> is actually preferable. It has a lot of advantages that <strong>Klaus</strong> explained for one full hour. My favorite one being that you can extend your polyphormism to types that you do not own. Let's say that you want to serialise <strong>std::vector</strong>, you can simply write an overload for it:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialise</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">T</span><span class="o">>&</span> <span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* return something... */</span> <span class="p">}</span>
</code></pre></div>
<p>In practice, nothing prevent you from mixing a both solutions to your needs. One counter-argument being that <strong>free functions</strong> have an ugly syntax: <code>v.serialise();</code> feels more natural than <code>serialize(v);</code>. That issue could have been solve with the <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0251r0.pdf">unified call syntax</a> proposal by <strong>Bjarne Stroustrup</strong> and <strong>Herb Sutter</strong>. Sadly, it was rejected by the <strong>C++</strong> committee.</p>
<h2>[Talk] Reader-Write Lock versus Mutex - Understanding a Lost Bet - Jeffrey Mendelsohn - 💀💀💀 ★★</h2>
<ul>
<li>Slides: <a href="">coming-soon</a></li>
<li>Video: <a href="https://www.youtube.com/watch?v=GzLsHTgZ-XI">link</a></li>
</ul>
<p><a href="https://www.linkedin.com/in/jeffrey-mendelsohn-ph-d-8417454">Jeffrey Mendelson</a> from <strong>Bloomberg</strong> had a bet with a colleague on whether a <a href="https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock">readers-writer lock</a> would be faster than a <a href="https://en.wikipedia.org/wiki/Lock_(computer_science)">mutex</a> to protect the access on a <strong>resource</strong> that could be written by a single writer, but could have multiple readers simultaneously. The <strong>readers-writer lock</strong> would follow exactly that behaviour (multiple readers, single writer). The <strong>mutex</strong> would keep the exclusivity to one writer or one reader only! <strong>Jeffrey</strong> lost the bet, but that did not hindered him from exploring the reasons behind his lost. It was challenging for me to grasp all the implications on this topic, but here is what I understood:</p>
<ul>
<li><strong>Jeffrey</strong>'s <strong>reader-writer lock</strong> was made of <a href="http://en.cppreference.com/w/cpp/atomic/atomic">atomic variables</a> to keep track of the amount of readers and writers. If the <strong>resource</strong> was currently written onto, the <strong>readers</strong> and other <strong>writers</strong> would wait onto a semaphor to be waken-up later on.</li>
<li>If the amount of time spent by the <strong>readers</strong> or the <strong>writers</strong> on the resource is fairly long, the <strong>readers-writer lock</strong> will actually perform better than the <strong>mutex</strong> as multiple <strong>reader</strong> can process simultanesouly.</li>
<li>On the other hand, if the <strong>writing</strong> and <strong>reading operations</strong> are very fast, the <strong>atomic operations</strong> on the counters will start to be costly comparatively. <strong>Atomics</strong> tend to have nonnegligible effects on the cache lines of your CPU(s). In this case, loosing the ability to have multiple <strong>readers</strong> is actually not as dramatic as you would think in comparison of stressing your cache.</li>
<li><strong>Jeffrey</strong> came up with an hybrid solution that combines both a <strong>readers-writer lock</strong> and a fallback to a <strong>mutex</strong> that outperformed the previous solutions.</li>
</ul>
<p>Once the video of this talk is uploaded, I must have a complete rewatch of it. It always amusing that our intuitions can be entirely wrong when it comes to concurrency and programming. </p>
<h2>[Other] The not-so-secret lightning talks</h2>
<p>Videos: <a href="https://www.youtube.com/watch?v=ARosL9xrozk">link</a>, <a href="https://www.youtube.com/watch?v=UNSkp6KpMNk">link</a>, <a href="https://www.youtube.com/watch?v=2KGkcGtGVM4">link</a>, <a href="https://www.youtube.com/watch?v=rj_BSmixRYg">link</a>, <a href="https://www.youtube.com/watch?v=Kq8lOymvRcE">link</a>, <a href="https://www.youtube.com/watch?v=7GIZN03-_6w">link</a></p>
<p>Before the last Keynote, we had the pleasure to listen to some not-so-unexpected lightning talks. I will not spoil too much of it!</p>
<p>I just want to express my gratitude to <strong>Guy Davidson</strong> and <strong>Sean Parent</strong> for bringing <strong>diversity in the C++ community</strong> under the spotlight. It was more than welcome and I am glad of these initiatives. </p>
<h1>Conclusion:</h1>
<p>Once again, I was amazed by the <strong>C++</strong> community and how a group of dedicated persons can build such nice event. I am already eager to fly to one of the big conference next year: <strong>Meeting C++</strong> or <strong>cppcon</strong>. I would also encourage anyone with a bit of passion for this language or programming in general to give a try to conferences or local groups, you will discover more than you would expect!</p>C++ Stockholm 0x022017-02-20T21:04:00+01:002017-02-20T21:04:00+01:00Jean Gueganttag:jguegant.github.io,2017-02-20:/blogs/tech/c++-stockholm-0x02.html<p>I had the chance to participate last week in a <strong>C++ meetup</strong>.
This <a href="https://www.meetup.com/swedencpp/">Stockholm meetup group</a> is fairly recent but has a lot of potential!
I am always enjoying my time during these sessions, surrounded by the smart and passionated people.
In the unlikely chance that you are living around …</p><p>I had the chance to participate last week in a <strong>C++ meetup</strong>.
This <a href="https://www.meetup.com/swedencpp/">Stockholm meetup group</a> is fairly recent but has a lot of potential!
I am always enjoying my time during these sessions, surrounded by the smart and passionated people.
In the unlikely chance that you are living around <strong>Stockholm</strong>, or decided to visit the capital of Scandinavia (and start missing your keyboard), I would highly recommend you to join this group.</p>
<p>For the second edition of our conference <strong>C++ Stockholm 0x02</strong>, I volunteered for talk about a subject I already adressed in the blog: <a href="https://jguegant.github.io/blogs/tech/sfinae-introduction.html">SFINAE and compile-time introspection</a>.
Overall, it was a good exercise for a first talk but I definitely need more training :).
I discovered that it is much harder than I expected to explain code in live.
Thanks to the meetup team, I have a video of this talk with a very nice editing job done on it.
It is always a weird experience to watch yourself on a video and was not so keen to put it here in the first place.
But since this video has already <em>leaked</em> among my friends (or foes in that case) and colleagues, I guess I might place it for the posterity!
So, here it is:</p>
<iframe width="854" height="480" src="https://www.youtube.com/embed/YgbTBqS-bHg" frameborder="0" allowfullscreen></iframe>
<p>You can find a pdf version of the slides right <a href="https://jguegant.github.io/blogs/tech/images/SFINAE-compile-time-introspection.pdf">here</a>.</p>An introduction to C++'s variadic templates: a thread-safe multi-type map2016-02-01T14:00:00+01:002016-02-01T14:00:00+01:00Jean Gueganttag:jguegant.github.io,2016-02-01:/blogs/tech/thread-safe-multi-type-map.html<h3>Trivia:</h3>
<p>One of our favorite motto in our C++ team at work is: you shall use <strong>dependency injections</strong> instead of <strong>singletons</strong>! It actually comes with our unit-testing strategy. If the various components of your architecture are too tightly coupled, it becomes a tremendous effort to deeply test small critical chunks …</p><h3>Trivia:</h3>
<p>One of our favorite motto in our C++ team at work is: you shall use <strong>dependency injections</strong> instead of <strong>singletons</strong>! It actually comes with our unit-testing strategy. If the various components of your architecture are too tightly coupled, it becomes a tremendous effort to deeply test small critical chunks of your code. <strong>Singletons</strong> are that kind of beast that revives itself without your permission and comes from hell to haunt your lovely unit-tests. Our main project being multi-threaded (hence highly bug-prone) and vital for the company, "<strong>singleton</strong>" became a forbidden word. Yet, our team recently started going down the dark path. Thanks to C++11 and its variadic templates, I carefully crafted a <strong>thread-safe multi-type map container</strong> that simplified our configuration reloading system and saved us from the dark side of the coder force. If you always wondered what are <strong>variadic templates</strong>, how <strong>C++11's tuples</strong> can be implemented, I am going to present these concepts in this post using my container as a cobaye.</p>
<p>Note: for the sake of your sanity and the fact that <em>errare humanum est</em>, this article might not be 100% accurate!</p>
<h3>Why would I use a thread-safe multi-type map?</h3>
<p>Let me explain our odyssey: we are working on a highly modular and multi-threaded application. One of its core feature is the ability to reload various configuration files or assets used by some components spread accross many threads and a giant hierarchy of objects. The reloading process is automic using Linux's <a href="http://man7.org/linux/man-pages/man7/inotify.7.html">inotify</a> monitoring filesystem events. One thread is dedicated to the reception of filesystem events and must react accordingly by parsing any changes and pushing them to other threads. At first, we used, to pass-by any newly parsed asset, some thread-safe <strong>queues</strong> or something analog to <a href="https://tour.golang.org/concurrency/2">go channels</a>. Since we did not want to use <strong>singletons</strong>, we had to pass references to our queues all along our object hierarchy. Sadly, our <strong>queue</strong> implementation is <strong>one to one</strong> and supports only <strong>one type</strong>, none of our config/asset types share the same <strong>base-type</strong>. For each asset type and each component using this asset, we had to create a new <strong>queue</strong> and pass-it all along our hierarchy. That is certainely not convenient! What we really wanted was a hybrid class between a <a href="http://en.cppreference.com/w/cpp/container/map">std::map</a> and a <a href="http://en.cppreference.com/w/cpp/utility/tuple">std::tuple</a>.</p>
<p>We could have used a <strong>std::map</strong> with <a href="http://www.boost.org/doc/libs/1_60_0/doc/html/variant.html">Boost.Variant</a> to store our items, using a type like the following <strong>"std::map< std::string, std::shared_ptr< Boost.Variant < ConfigType1, ConfigType2>>>"</strong>. <strong>Boost.Variant</strong> permits to encapsulate a <strong>heterogeneous set of types</strong> without <strong>common base-type or base-class</strong>, which solves one of our point. Another solution would be to encapsulate manually all our configuration classes in the same familly of classes, that is pretty cumbersome. But anyway, <strong>std::map</strong> does not guaranty any safety if you are writing and reading at the same time on a map slot. Secondly, <strong>std::shared_ptr</strong> does guaranty a thread-safe destruction of the pointee object (i.e: the reference counter is thread-safe) but nothing for the <strong>std::shared_ptr</strong> object itself. It means that copying a <strong>std::shared_ptr</strong> that could potentially be modified from another thread, might lead to an undefined behaviour. Even if we were to encapsulate all these unsafe accesses with mutexes, we are still lacking a nice mechanism to get update notifications for our objects. We do not want to constantly poll the latest version and propagate it through our code. And finally, if that solution were elegant enough, why would I currently write this blog post?</p>
<p><strong>C++11</strong> brings another collection type called <strong>std::tuple</strong>. It permits to store a set of elements of <strong>heterogeneous types</strong>. Take a look at this short example:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">myTuple</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">make_tuple</span><span class="p">(</span><span class="s">"Foo"</span><span class="p">,</span> <span class="mi">1337</span><span class="p">,</span> <span class="mi">42</span><span class="p">);</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">0</span><span class="o">></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Access element by index: "Foo"</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">1</span><span class="o">></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Access element by index: 1337</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">2</span><span class="o">></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Access element by index: 42</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="k">const</span> <span class="kt">char</span><span class="o">*></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Access element by type: "Foo"</span>
<span class="c1">// compilation error: static_assert failed "tuple_element index out of range"</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="mi">3</span><span class="o">></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="c1">// compilation error: static_assert failed "type can only occur once in type list"</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="n">myTuple</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
</code></pre></div>
<p><strong>Tuples</strong> are that kind of <strong>C++11</strong> jewelry that should decide your old-fashioned boss to upgrade your team's compiler (and his ugly tie). Not only I could store a <strong>const char* </strong> and two <strong>ints</strong> without any compiling error, but I could also access them using compile-time mechanisms. In some way, you can see tuples as a compile-time map using indexes or types as keys to reach its elements. You cannot use an index out of bands, it will be catched at compile-time anyway! Sadly, using a type as a key to retrieve an element is only possible if the type is unique in the <strong>tuple</strong>. At my work, we do have few config objects sharing the same class. Anyway, tuples weren't fitting our needs regarding thread safety and update events. Let's see what we could create using tasty <strong>tuples</strong> as an inspiration.</p>
<p>Note that some <strong>tuples</strong> implementations were already available before <strong>C++11</strong>, notably in <a href="http://www.boost.org/doc/libs/1_60_0/libs/tuple/doc/tuple_users_guide.html">boost</a>. <strong>C++11</strong> variadic templates are just very handy, as you will see, to construct such a class.</p>
<h3>A teaser for my repository class:</h3>
<p>To keep your attention for the rest of this post, here is my <strong>thread-safe multi-type map</strong> in action:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><memory></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><string></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf">"repository.hpp"</span><span class="cp"></span>
<span class="c1">// Incomplete types used as compile-time keys.</span>
<span class="k">struct</span> <span class="nc">Key1</span><span class="p">;</span>
<span class="k">struct</span> <span class="nc">Key2</span><span class="p">;</span>
<span class="c1">// Create a type for our repository.</span>
<span class="k">using</span> <span class="n">MyRepository</span> <span class="o">=</span> <span class="n">Repository</span>
<span class="o"><</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">,</span> <span class="c1">// One slot for std::string.</span>
<span class="n">Slot</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key1</span><span class="o">></span><span class="p">,</span> <span class="c1">// Two slots for int.</span>
<span class="n">Slot</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key2</span><span class="o">></span> <span class="c1">// Must be differentiate using "type keys" (Key1, Key2).</span>
<span class="o">></span><span class="p">;</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">MyRepository</span> <span class="n">myRepository</span><span class="p">;</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">emplace</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">(</span><span class="s">"test"</span><span class="p">);</span> <span class="c1">// Construct the shared_ptr within the repository.</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">emplace</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key1</span><span class="o">></span><span class="p">(</span><span class="mi">1337</span><span class="p">);</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">set</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key2</span><span class="o">></span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">make_shared</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">(</span><span class="mi">42</span><span class="p">));</span> <span class="c1">// Set the shared_ptr manually.</span>
<span class="c1">// Note: I use '*' as get returns a shared_ptr.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="o">*</span><span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print "test".</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="o">*</span><span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key1</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print 1337.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="o">*</span><span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">Key2</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print 42.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="o">*</span><span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="c1">// ^^^ Compilation error: which int shall be selected? Key1 or Key2?</span>
<span class="k">auto</span> <span class="n">watcher</span> <span class="o">=</span> <span class="n">myRepository</span><span class="p">.</span><span class="n">getWatcher</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">();</span> <span class="c1">// Create a watcher object to observe changes on std::string.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">watcher</span><span class="o">-></span><span class="n">hasBeenChanged</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// 0: no changes since the watcher creation.</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">emplace</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">(</span><span class="s">"yo"</span><span class="p">);</span> <span class="c1">// Emplace a new value into the std::string slot.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">watcher</span><span class="o">-></span><span class="n">hasBeenChanged</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// 1: the std::string slot has been changed.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="o">*</span><span class="n">watcher</span><span class="o">-></span><span class="n">get</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Poll the value and print "yo".</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">watcher</span><span class="o">-></span><span class="n">hasBeenChanged</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// 0: no changes since the last polling.</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>First and foremost, its name <strong>repository</strong> might not be well-suited for its responsibility. If your native language is the same as shakespeare and come-up with a better term, please feel free to submit it. In our internal usage, <strong>config repository</strong> sounded great!</p>
<p>I start by describing the slots necessary for my application by creating a new type <strong>MyRepository</strong> using a <a href="http://en.cppreference.com/w/cpp/language/type_alias">type alias</a>. As you can see, I use the type of the slots as a key for accessing elements. But in case of contention, I must use a second key: an "empty type" ; like <strong>Key1</strong> and <strong>Key2</strong> in this example.
If using types as keys seems odd for you, fear not! Here is the most rational explanation I can share with you: we are trying to benefit from our "know-it-all compiler". Your compiler is mainly manipulating types, one can change its flow using these types during the compilation process. Note that these structs are not even complete (no definition), it has <strong>no impact</strong> for the <strong>runtime memory</strong> or <strong>runtime execution</strong> and that's the amazing part of <strong>meta-programming</strong>. The dispatch of an expression such as <strong>"myRepository.get< int, Key1>()"</strong> is done during your build-time.</p>
<p>You may also notice that every slot is actually a <a href="http://en.cppreference.com/w/cpp/memory/shared_ptr">std::shared_ptr</a>. It enforces a clean ressource management: in a multithreaded application, one must be really careful of the lifetime of heap objects. <strong>std::shared_ptr</strong> in this case permits me to ensure that even if someone replaces a value in a slot, other components on other threads manipulating the old value won't end up with a <strong>dangling pointer/reference</strong> bomb in their hands. Another solution would be to use plain value objects, but not only it would require copying big objects in every other components but it would also remove polymorphism.</p>
<p>As for the updates signalisation, you first create a watcher object that establishes a contract between a desired slot to watch and your context. You can thereafter query in thread-safe way weither an update has been made and, if so, poll the latest changes. The watcher object is actually a <a href="http://en.cppreference.com/w/cpp/memory/unique_ptr">std::unique_ptr</a> for a special class, it cannot be moved nor copied without your permission and will automagically disable the signalisation contract between the slot and your context, once destroyed. We will dive deeper in this topic in the comming sections.</p>
<p>Within our application, the repository object is encapsulated into a RuntimeContext object. This RuntimeContext object is created explicitely within our main entry point and passed as a reference to a great part of our components. We therefore keep the possibility to test our code easily by setting this RuntimeContext with different implementations. Here is a simplified version of our usage:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// runtimecontext.hpp</span>
<span class="cp">#include</span> <span class="cpf">"repository.hpp"</span><span class="cp"></span>
<span class="c1">// Incomplete types used as compile-time keys.</span>
<span class="k">struct</span> <span class="nc">Key1</span><span class="p">;</span>
<span class="k">struct</span> <span class="nc">Key2</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">ConfigType1</span><span class="p">;</span> <span class="c1">// Defined in another file.</span>
<span class="k">class</span> <span class="nc">ConfigType2</span><span class="p">;</span> <span class="c1">// Defined in another file.</span>
<span class="c1">// Create a type for our repository.</span>
<span class="k">using</span> <span class="n">ConfigRepository</span> <span class="o">=</span> <span class="n">Repository</span>
<span class="o"><</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">ConfigType1</span><span class="o">></span><span class="p">,</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">ConfigType2</span><span class="p">,</span> <span class="n">Key1</span><span class="o">></span><span class="p">,</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">ConfigType2</span><span class="p">,</span> <span class="n">Key2</span><span class="o">></span>
<span class="o">></span><span class="p">;</span>
<span class="k">struct</span> <span class="nc">RuntimeContext</span>
<span class="p">{</span>
<span class="n">ILogger</span><span class="o">*</span> <span class="n">logger</span><span class="p">;</span>
<span class="c1">// ...</span>
<span class="n">ConfigRepository</span> <span class="n">configRepository</span><span class="p">;</span>
<span class="p">};</span>
<span class="c1">// Main.cpp</span>
<span class="cp">#include</span> <span class="cpf">"runtimecontext.hpp"</span><span class="cp"></span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">RuntimeContext</span> <span class="n">runtimeContext</span><span class="p">;</span>
<span class="c1">// Setup:</span>
<span class="n">runtimeContext</span><span class="p">.</span><span class="n">logger</span> <span class="o">=</span> <span class="k">new</span> <span class="n">StdOutLogger</span><span class="p">();</span>
<span class="c1">// ...</span>
<span class="c1">// Let's take a reference to the context and change the configuration repository when necessary. </span>
<span class="n">startConfigurationMonitorThread</span><span class="p">(</span><span class="n">runtimeContext</span><span class="p">);</span>
<span class="c1">// Let's take a reference and pass it down to all our components in various threads.</span>
<span class="n">startOurApplicationLogic</span><span class="p">(</span><span class="n">runtimeContext</span><span class="p">);</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<h3>Time for a C++11 implementation:</h3>
<p>We can decompose the solution in 3 steps: at first we need to implement a map that accepts <strong>multiple types</strong>, we then need to work on the <strong>thread safety</strong> and finish by the <strong>watcher mechanism</strong>. Let's first fulfill the mission of this post: introducing you to <strong>variadic templates</strong> to solve the multiple-type problem.</p>
<h4>Variadic templates:</h4>
<p>You may not have heard of <strong>variadic templates</strong> in <strong>C++11</strong> but I bet that you already used <strong>variadic functions</strong> like <strong>printf</strong> in <strong>C</strong> (maybe in a previous unsafe life). As <a href="https://en.wikipedia.org/wiki/Variadic_function">wikipedia</a> kindly explains "a variadic function is a function of indefinite which accepts a variable number of arguments". In other words, a <strong>variadic function</strong> has potentially an infinite number of <strong>parameters</strong>. Likewise, a <strong>variadic template</strong> has potentially an infinite number of <strong>parameters</strong>. Let's see how to use them!</p>
<h5>Usage for variadic function templates:</h5>
<p>Let's say that you wish to create a template that accept an infinite number of class as arguments. You will use the following notation:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span>
</code></pre></div>
<p>You specify a group of template parameters using the ellipsis notation named <strong>T</strong>. Note that this <strong>ellipsis</strong> notation is consistent with <strong>C</strong>'s variadic function notation. This group of parameters, called a <strong>parameter-pack</strong>, can then be used in your function template or your class template by <strong>expanding</strong> them. One must use the <strong>ellipsis</strong> notation again (this time after T) to <strong>expand</strong> the parameter pack <strong>T</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span><span class="p">...)</span>
<span class="c1">// ^ pack T ^expansion</span>
<span class="p">{</span>
<span class="c1">// Your function content.</span>
<span class="p">}</span>
</code></pre></div>
<p>Now that we have expanded <strong>T</strong>, what can we do Sir? Well, first you give to your expanded parameter <strong>types</strong>, a fancy <strong>name</strong> like <strong>t</strong>.</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span><span class="p">...</span> <span class="n">t</span><span class="p">)</span>
<span class="c1">// ^ your fancy t.</span>
<span class="p">{</span>
<span class="c1">// Your function content.</span>
<span class="p">}</span>
</code></pre></div>
<p>If <strong>T = T1, T2</strong>, then <strong>T... t = T1 t1, T2 t2</strong> and <strong>t = t1, t2</strong>. Brilliant, but is that all? Sure no! You can then <strong>expand</strong> again <strong>t</strong> using an "suffix-ellipsis" again:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span><span class="p">...</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">anotherFunction</span><span class="p">(</span><span class="n">t</span><span class="p">...);</span>
<span class="c1">// ^ t is expanded here! </span>
<span class="p">}</span>
</code></pre></div>
<p>Finally, you can call this function <strong>f</strong> as you would with a normal function template:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span><span class="p">...</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">anotherFunction</span><span class="p">(</span><span class="n">t</span><span class="p">...);</span>
<span class="p">}</span>
<span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s">"foo"</span><span class="p">,</span> <span class="s">"bar"</span><span class="p">);</span> <span class="c1">// Note: the argument deduction avoids us to use f<int, const char*, const char*></span>
<span class="c1">// f(1, "foo", "bar") calls a generated f(int t1, const char* t2, const char* t3)</span>
<span class="c1">// with T1 = int, T2 = const char* and T3 = const char*,</span>
<span class="c1">// that itself calls anotherFunction(t1, t2, t3) equivalent to call anotherFunction(1, "foo", "bar");</span>
</code></pre></div>
<p>Actually, the <strong>expansion mechanism</strong> is creating <strong>comma-separated</strong> replication of the <strong>pattern</strong> you apply the <strong>ellipsis</strong> onto. If you think I am tripping out with template-related wording, here is a much more concret example:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">g</span><span class="p">(</span><span class="n">T</span><span class="p">...</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">anotherFunction</span><span class="p">(</span><span class="n">t</span><span class="p">...);</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span><span class="o">*</span><span class="p">...</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">g</span><span class="p">(</span><span class="k">static_cast</span><span class="o"><</span><span class="kt">double</span><span class="o">></span><span class="p">(</span><span class="o">*</span><span class="n">t</span><span class="p">)...);</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">a</span> <span class="o">=</span> <span class="mi">2</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">b</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="n">f</span><span class="p">(</span><span class="o">&</span><span class="n">a</span><span class="p">,</span> <span class="o">&</span><span class="n">b</span><span class="p">);</span> <span class="c1">// Call f(int* t1, int* t2).</span>
<span class="c1">// Do a subcall to g(static_cast<double>(*t1), static_cast<double>(*t2)).</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>I could use the pattern <strong>'*'</strong> for <strong>f</strong> parameters and therefore take them as a pointer! In the same manner, I applied the pattern <strong>'static_cast< double>(*)</strong> to get the value of each arguments and cast them as doubles before forwarding them to <strong>g</strong>.</p>
<p>One last example before moving to <strong>variadic class templates</strong>. One can combine "normal" template parameters with parameter packs and initiate a compile recursion on function templates. Let's take a look at this printing function:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">HEAD</span><span class="o">></span> <span class="kt">void</span> <span class="n">print</span><span class="p">(</span><span class="n">HEAD</span> <span class="n">head</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Stop: "</span> <span class="o"><<</span> <span class="n">head</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">HEAD</span><span class="p">,</span> <span class="k">class</span><span class="p">...</span> <span class="n">TAIL</span><span class="o">></span> <span class="kt">void</span> <span class="n">print</span><span class="p">(</span><span class="n">HEAD</span> <span class="n">head</span><span class="p">,</span> <span class="n">TAIL</span><span class="p">...</span> <span class="n">tail</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Recurse: "</span> <span class="o"><<</span> <span class="n">head</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">print</span><span class="p">(</span><span class="n">tail</span><span class="p">...);</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">print</span><span class="p">(</span><span class="mi">42</span><span class="p">,</span> <span class="mi">1337</span><span class="p">,</span> <span class="s">"foo"</span><span class="p">);</span>
<span class="c1">// Print:</span>
<span class="c1">// Recurse: 42</span>
<span class="c1">// Recurse: 1337</span>
<span class="c1">// Stop: foo</span>
<span class="c1">// Call print<int, int, const char*> (second version of print).</span>
<span class="c1">// The first int (head) is printed and we call print<int, const char*> (second version of print).</span>
<span class="c1">// The second int (head again) is printed and we call print<const char*> (first version of print).</span>
<span class="c1">// We reach recursion stopping condition, only one element left.</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p><strong>Variadic templates</strong> are very interesting and I wouldn't be able to cover all their features within this post. It roughly feels like functional programming using your compiler, and even some <strong>Haskellers</strong> might listen to you if you bring that topic during a dinner. For those interested, I would challenge them to write a type-safe version of <strong>printf</strong> using variadic templates with the help of this <a href="http://en.cppreference.com/w/cpp/language/parameter_pack">reference</a>. After that, you will run and scream of fear at the precense of <strong>C</strong>'s <strong>vargs</strong>.</p>
<h5>"Variadic" inheritance:</h5>
<p>Sometimes during my programming sessions, I have a very awkward sensation that my crazy code will never compile and, yet, I finally see "build finished" in my terminal. I am talking about that kind of Frankenstein constructions:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span> <span class="p">};</span>
<span class="k">struct</span> <span class="nc">B</span> <span class="p">{</span> <span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">C</span><span class="o">:</span> <span class="k">public</span> <span class="n">T</span><span class="p">...</span> <span class="c1">// Variadic inheritance</span>
<span class="p">{</span>
<span class="p">};</span>
<span class="n">C</span><span class="o"><</span><span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="o">></span> <span class="n">c</span><span class="p">;</span>
</code></pre></div>
<p>Yes, we can now create a class inheriting of an infinite number of bases. If you remember my explanation about pattern replications separated by commas, you can imaginge that <strong>struct C: public T...</strong> will be "transformed" in <strong>struct C: public A, public B</strong>, <strong>public T</strong> being the pattern. We start to be able to combine multiple types, each exposing a small amount of methods, to create a flexible concret type. That's one step closer to our multi-type map, and if you are interested in this concept, take a look at <a href="https://en.wikipedia.org/wiki/Mixin">mixins</a>.</p>
<p>Instead of inheriting directly from multiple types, couldn't we inherit from some types that encapsulate our types? Absolutely! A traditional map has some <strong>slots</strong> accessible using keys and these slots contain a value. If you give me base-class you are looking for, I can give you access to the value it contains:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="k">struct</span> <span class="nc">SlotA</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">value</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">struct</span> <span class="nc">SlotB</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">value</span><span class="p">;</span>
<span class="p">};</span>
<span class="c1">// Note: private inheritance, no one can access directly to the slots other than C itself.</span>
<span class="k">struct</span> <span class="nc">Repository</span><span class="o">:</span> <span class="k">private</span> <span class="n">SlotA</span><span class="p">,</span> <span class="k">private</span> <span class="n">SlotB</span>
<span class="p">{</span>
<span class="kt">void</span> <span class="nf">setSlotA</span><span class="p">(</span><span class="k">const</span> <span class="kt">int</span><span class="o">&</span> <span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// I access the base-class's value</span>
<span class="c1">// Since we have multiple base with a value field, we need to "force" the access to SlotA.</span>
<span class="n">SlotA</span><span class="o">::</span><span class="n">value</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="nf">getSlotA</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">SlotA</span><span class="o">::</span><span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="nf">setSlotB</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&</span> <span class="n">b</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">SlotB</span><span class="o">::</span><span class="n">value</span> <span class="o">=</span> <span class="n">b</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">getSlotB</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">SlotB</span><span class="o">::</span><span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Repository</span> <span class="n">r</span><span class="p">;</span>
<span class="n">r</span><span class="p">.</span><span class="n">setSlotA</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">r</span><span class="p">.</span><span class="n">getSlotA</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print: 42.</span>
<span class="n">r</span><span class="p">.</span><span class="n">setSlotB</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">(</span><span class="s">"toto"</span><span class="p">));</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">r</span><span class="p">.</span><span class="n">getSlotB</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print: "toto".</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>This code is not generic at all! We know how to create a generic <strong>Slot</strong> using a simple template, and we acquired the magic "create varidiac inheritance" skill. If my <strong>Repository</strong> class inherit from <strong>Slot< TypeA></strong> and you call a method template with <strong>TypeA</strong> as a template argument, I can call the <strong>doGet</strong> method of the <strong>Slot< TypeA></strong> base-class and give you back the <strong>value</strong> of <strong>TypeA</strong> in that repository. Let's fix the previous ugly copy-paste code:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><string></span><span class="cp"></span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Slot</span>
<span class="p">{</span>
<span class="k">protected</span><span class="o">:</span>
<span class="n">Type</span><span class="o">&</span> <span class="n">doGet</span><span class="p">()</span> <span class="c1">// A nice encapsulation, that will be usefull later on.</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">value_</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">doSet</span><span class="p">(</span><span class="k">const</span> <span class="n">Type</span><span class="o">&</span> <span class="n">value</span><span class="p">)</span> <span class="c1">// Same encapsulation.</span>
<span class="p">{</span>
<span class="n">value_</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">Type</span> <span class="n">value_</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Slots</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Repository</span> <span class="o">:</span> <span class="k">private</span> <span class="n">Slots</span><span class="p">...</span> <span class="c1">// inherit from our slots...</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="o">></span> <span class="c1">// Give me a type and,</span>
<span class="n">Type</span><span class="o">&</span> <span class="n">get</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="o">>::</span><span class="n">doGet</span><span class="p">();</span> <span class="c1">// I can select the Base class.</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="o">></span>
<span class="kt">void</span> <span class="n">set</span><span class="p">(</span><span class="k">const</span> <span class="n">Type</span><span class="o">&</span> <span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="o">>::</span><span class="n">doSet</span><span class="p">(</span><span class="n">value</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="c1">// Incomplete types used as compile-time keys.</span>
<span class="k">struct</span> <span class="nc">Key1</span><span class="p">;</span>
<span class="k">struct</span> <span class="nc">Key2</span><span class="p">;</span>
<span class="c1">// Create a type for our repository.</span>
<span class="k">using</span> <span class="n">MyRepository</span> <span class="o">=</span> <span class="n">Repository</span>
<span class="o"><</span>
<span class="n">Slot</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">,</span> <span class="c1">// Let's pick the type of our slots.</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span>
<span class="o">></span><span class="p">;</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">MyRepository</span> <span class="n">myRepository</span><span class="p">;</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">set</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">(</span><span class="s">"toto"</span><span class="p">);</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">set</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span> <span class="c1">// Notice the type deduction: we pass an int, so it writes in the int slot.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print: "toto".</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Print: 42.</span>
<span class="k">return</span> <span class="n">EXIT_SUCCESS</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>This repository starts to take shape, but we are not yet done! If you try to have two int slots, you will raise a compilation error: "base class 'Slot<int>' specified more than once as a direct base class". We need to add another key-type to our slot class with a default value and we need to modify our repository methods to handle it:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">DefaultSlotKey</span><span class="p">;</span> <span class="c1">// No needs for a definition</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span> <span class="o">=</span> <span class="n">DefaultSlotKey</span><span class="o">></span> <span class="c1">// The Key type will never be trully used. </span>
<span class="k">class</span> <span class="nc">Slot</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Slots</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Repository</span> <span class="o">:</span> <span class="k">private</span> <span class="n">Slots</span><span class="p">...</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span> <span class="o">=</span> <span class="n">DefaultSlotKey</span><span class="o">></span> <span class="c1">// The default key must be here too.</span>
<span class="n">Type</span><span class="o">&</span> <span class="n">get</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doGet</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span> <span class="o">=</span> <span class="n">DefaultSlotKey</span><span class="o">></span>
<span class="kt">void</span> <span class="n">set</span><span class="p">(</span><span class="k">const</span> <span class="n">Type</span><span class="o">&</span> <span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doSet</span><span class="p">(</span><span class="n">value</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">struct</span> <span class="nc">Key1</span><span class="p">;</span> <span class="c1">// No need for definition.</span>
<span class="k">struct</span> <span class="nc">Key2</span><span class="p">;</span>
<span class="c1">// Now you can do:</span>
<span class="k">using</span> <span class="n">MyRepository</span> <span class="o">=</span> <span class="n">Repository</span>
<span class="o"><</span>
<span class="n">Slot</span><span class="o"><</span><span class="kt">int</span><span class="o">></span><span class="p">,</span> <span class="c1">// Let's pick the type of our slots.</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">Key1</span><span class="o">></span><span class="p">,</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">Key2</span><span class="o">></span>
<span class="o">></span><span class="p">;</span>
</code></pre></div>
<p>Here is a UML representation of this Repository using distinct <strong>Keys</strong> for the type <strong>std::string</strong>:
<img alt="A nice UML diagram of my classes" src="https://jguegant.github.io/blogs/tech/images/repository_uml.png"></p>
<p>Our repository class is missing an <strong>emplace</strong> method, right? <strong>emplace</strong> is taking a variable number of arguments with different types and <strong>forward</strong> them to create an object within one of our slots. A variable number of arguments and types must remind you something... <strong>variadic templates</strong>! Let's create this variadic <strong>emplace</strong> method as well as its equivalent in the Slot class:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// In class Slot:</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="kt">void</span> <span class="n">doEmplace</span><span class="p">(</span><span class="k">const</span> <span class="n">Args</span><span class="o">&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span> <span class="c1">// Here the pattern is const &.</span>
<span class="p">{</span>
<span class="n">value_</span> <span class="o">=</span> <span class="n">Type</span><span class="p">(</span><span class="n">args</span><span class="p">...);</span> <span class="c1">// copy-operator (might use move semantics).</span>
<span class="p">}</span>
<span class="c1">// In class Repository:</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span> <span class="o">=</span> <span class="n">DefaultSlotKey</span><span class="p">,</span> <span class="k">class</span><span class="p">...</span> <span class="n">Args</span><span class="o">></span>
<span class="kt">void</span> <span class="n">emplace</span><span class="p">(</span><span class="k">const</span> <span class="n">Args</span><span class="o">&</span><span class="p">...</span> <span class="n">args</span><span class="p">)</span> <span class="c1">// Here the pattern is const &.</span>
<span class="p">{</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doEmplace</span><span class="p">(</span><span class="n">args</span><span class="p">...);</span>
<span class="p">}</span>
<span class="c1">// Usage:</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">emplace</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="sc">'a'</span><span class="p">);</span> <span class="c1">// Create a std::string "aaaa".</span>
</code></pre></div>
<p>One last improvement for the future users of your repositories! If one morning, badly awake, a coworker of yours is trying to get a type or key that doesn't exist (like myRepository.get< double>();), he might be welcomed by such a message:</p>
<div class="highlight"><pre><span></span><code><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">jguegant</span><span class="o">/</span><span class="n">Coding</span><span class="o">/</span><span class="n">ConfigsRepo</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="nl">cpp</span><span class="p">:</span><span class="mi">36</span><span class="o">:</span><span class="mi">33</span><span class="o">:</span> <span class="nl">error</span><span class="p">:</span> <span class="n">call</span> <span class="n">to</span> <span class="n">non</span><span class="o">-</span><span class="k">static</span> <span class="n">member</span> <span class="n">function</span> <span class="n">without</span> <span class="n">an</span> <span class="n">object</span> <span class="n">argument</span>
<span class="k">return</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doGet</span><span class="p">();</span>
<span class="o">~~~~~~~~~~~~~~~~~^~~~~</span>
<span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">jguegant</span><span class="o">/</span><span class="n">Coding</span><span class="o">/</span><span class="n">ConfigsRepo</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="nl">cpp</span><span class="p">:</span><span class="mi">67</span><span class="o">:</span><span class="mi">18</span><span class="o">:</span> <span class="nl">note</span><span class="p">:</span> <span class="n">in</span> <span class="n">instantiation</span> <span class="n">of</span> <span class="n">function</span> <span class="k">template</span> <span class="n">specialization</span> <span class="err">'</span><span class="n">Repository</span><span class="o"><</span><span class="n">Slot</span><span class="o"><</span><span class="kt">int</span><span class="p">,</span> <span class="n">DefaultSlotKey</span><span class="o">></span><span class="p">,</span> <span class="n">Slot</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">__1</span><span class="o">::</span><span class="n">basic_string</span><span class="o"><</span><span class="kt">char</span><span class="o">></span><span class="p">,</span> <span class="n">DefaultSlotKey</span><span class="o">></span> <span class="o">>::</span><span class="n">get</span><span class="o"><</span><span class="kt">double</span><span class="p">,</span> <span class="n">DefaultSlotKey</span><span class="o">></span><span class="err">'</span> <span class="n">requested</span> <span class="n">here</span>
<span class="n">myRepository</span><span class="p">.</span><span class="n">get</span><span class="o"><</span><span class="kt">double</span><span class="o">></span><span class="p">();</span>
<span class="o">^</span>
<span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">jguegant</span><span class="o">/</span><span class="n">Coding</span><span class="o">/</span><span class="n">ConfigsRepo</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="nl">cpp</span><span class="p">:</span><span class="mi">36</span><span class="o">:</span><span class="mi">33</span><span class="o">:</span> <span class="nl">error</span><span class="p">:</span> <span class="err">'</span><span class="n">doGet</span><span class="err">'</span> <span class="n">is</span> <span class="n">a</span> <span class="k">protected</span> <span class="n">member</span> <span class="n">of</span> <span class="err">'</span><span class="n">Slot</span><span class="o"><</span><span class="kt">double</span><span class="p">,</span> <span class="n">DefaultSlotKey</span><span class="o">></span><span class="err">'</span>
<span class="k">return</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doGet</span><span class="p">();</span>
<span class="o">^</span>
<span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">jguegant</span><span class="o">/</span><span class="n">Coding</span><span class="o">/</span><span class="n">ConfigsRepo</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="nl">cpp</span><span class="p">:</span><span class="mi">10</span><span class="o">:</span><span class="mi">11</span><span class="o">:</span> <span class="nl">note</span><span class="p">:</span> <span class="n">declared</span> <span class="k">protected</span> <span class="n">here</span>
<span class="n">Type</span><span class="o">&</span> <span class="n">doGet</span><span class="p">()</span>
<span class="o">^</span>
<span class="mi">2</span> <span class="n">errors</span> <span class="n">generated</span><span class="p">.</span>
</code></pre></div>
<p>This message is very confusing, our class does not inherit from <strong>Slot< double, DefaultSlotKey></strong>! And we are talking about a <strong>clang</strong> output, I wonder what <strong>gcc</strong> or <strong>MSVC</strong> could produce... If you do not want to be assinated from your moody colleague with a spoon, here is a nice solution using <strong>C++11</strong>'s <a href="http://en.cppreference.com/w/cpp/language/static_assert">static_asserts</a>. <strong>Static asserts</strong> give you the possibility to generate your own compiler error messages in the same fashion as normal asserts but at compile-time. Using a the trait like <strong>std::is_base_of</strong>, you can suggest the user of your repository to check twice his type. Let's put this <strong>static_assert</strong> at the beggining of all the methods of <strong>Repository</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">static_assert</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">is_base_of</span><span class="o"><</span><span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">></span><span class="p">,</span> <span class="n">Repository</span><span class="o"><</span><span class="n">Slots</span><span class="p">...</span><span class="o">>>::</span><span class="n">value</span><span class="p">,</span>
<span class="s">"Please ensure that this type or this key exists in this repository"</span><span class="p">);</span>
</code></pre></div>
<p>We are done for this part (finally...), time to think about multi-threading! If you want to know more about the magic behind <strong>std::is_base_of</strong>, I would suggest you to read my previous post on <a href="https://jguegant.github.io/blogs/tech/sfinae-introduction.html">SFINAE</a>, it might give you few hints. Here is a <a href="https://gist.github.com/Jiwan/c1daf7a80ebeb166dc61">gist</a> of what we achieved so far. Did you notice the change on <strong>emplace</strong>? If you do not understand it, have a look at <a href="http://thbecker.net/articles/rvalue_references/section_07.html">this explanation on perfect forwarding</a>. Sadly, it would be a way too long topic for this post (trust me on that point!) and has a minor impact on our repository right now.</p>
<h4>Let's play safe:</h4>
<p>The repository we just succeeded to craft can now be used in a single-thread environment without further investigation. But the initial decision was to make this class manipulable from multiple-threads without any worries considering the safety of our operations. As explained in the beginning of this post, we will not use direct values as we currently do, but instead allocate our objects on the heap and use some <strong>shared pointers</strong> to strictly control their <strong>lifetime</strong>. No matter which version (recent or deprecated) of the object a thread is manipulating, it's lifetime will be extended until the last thread using it definitely release it. It also implies that the objects themselves are thread-safe. In the case of read-only objects like configs or assets, it shouldn't be too much a burden. In this <a href="https://gist.github.com/Jiwan/cb66d01c38128a351f42">gist</a>, you will find a repository version using <strong>std::shared_ptrs</strong>.</p>
<p><strong>std::shared_ptr</strong> is an amazing feature of <strong>C++11</strong> when dealing with multi-threading, but has its weakness. Within my code (in the previous gist link) a race condition can occur:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// What if I try to copy value_ at the return point...</span>
<span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="n">doGet</span><span class="p">()</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">value_</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// ... meanwhile another thread is changing value_ to value?</span>
<span class="kt">void</span> <span class="n">doSet</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="o">&</span><span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">value_</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>As specified: "If multiple threads of execution access the <strong>same</strong> std::shared_ptr object without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur". Note that we are talking about the <strong>same</strong> shared pointer. Multiple shared pointer <strong>copies</strong> pointing to the same object are fine, as long as these copies originated from the same shared pointer in first place. Copies are sharing the same <strong>control block</strong>, where the <strong>reference counters</strong> (one for shared_ptr and one for weak_ptr) are located, and the specification says "the control block of a shared_ptr is thread-safe: different std::shared_ptr objects can be accessed using mutable operations, such as operator= or reset, simultaneously by multiple threads, even when these instances are copies, and share the same control block internally.".</p>
<p>Depending on the age of your compiler and its standard library, I suggest two solutions:</p>
<h5>1) A global mutex:</h5>
<p>A straightforward solution relies on a <a href="http://en.cppreference.com/w/cpp/thread/mutex">std::mutex</a> that we lock during <strong>doGet</strong> and <strong>doSet</strong> execution:</p>
<div class="highlight"><pre><span></span><code><span class="p">...</span>
<span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="n">doGet</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// The lock is enabled until value_ has been copied!</span>
<span class="n">std</span><span class="o">::</span><span class="n">lock_guard</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">mutex</span><span class="o">></span> <span class="n">lock</span><span class="p">(</span><span class="n">mutex_</span><span class="p">);</span>
<span class="k">return</span> <span class="n">value_</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">doSet</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="o">&</span><span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// The lock is enabled until value has been copied into value!</span>
<span class="n">std</span><span class="o">::</span><span class="n">lock_guard</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">mutex</span><span class="o">></span> <span class="n">lock</span><span class="p">(</span><span class="n">mutex_</span><span class="p">);</span>
<span class="n">value_</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">std</span><span class="o">::</span><span class="n">mutex</span> <span class="n">mutex_</span><span class="p">;</span>
<span class="p">...</span>
</code></pre></div>
<p>This solution is ideal if you have a <strong>Linux</strong> distribution that only ships <strong>gcc 4.8.x</strong> like mine. While not particularly elegant, it doesn't have a great impact on performances compared to the next solution. </p>
<h5>2) Atomic access functions:</h5>
<p>Starting from <strong>gcc 4.9</strong>, one can use <a href="http://en.cppreference.com/w/cpp/memory/shared_ptr/atomic">atomic access functions</a> to manipulate shared pointers. I dream of a day where a specialisation for <strong>std::atomic< std::shared_ptr<T>></strong> exists, but from now, we will resort to use <strong>std::atomic_load</strong> and <strong>std::atomic_exchange</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="p">...</span>
<span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="n">doGet</span><span class="p">()</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">atomic_load</span><span class="p">(</span><span class="o">&</span><span class="n">value_</span><span class="p">);</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">doSet</span><span class="p">(</span><span class="k">const</span> <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="o">&</span><span class="n">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">atomic_exchange</span><span class="p">(</span><span class="o">&</span><span class="n">value_</span><span class="p">,</span> <span class="n">value</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">Type</span><span class="o">></span> <span class="n">value_</span><span class="p">;</span>
<span class="p">...</span>
</code></pre></div>
<p><strong>Atomics</strong> are elegants and can often bring a great increase of performances if using lock-free instructions internally. Sadly, in the case of <strong>shared_ptrs</strong>, <strong>atomic_is_lock_free</strong> will return you <strong>false</strong>. By digging in <strong>libstdc++</strong> and <strong>libc++</strong>, you will find some mutexes. <strong>gcc</strong> seems to use a fixed size "pool" of mutexes attributed to a shared_ptr according to a hash of its pointee address, when dealing with atomic operations. In other words, no rocket-science for atomic shared pointers until now.</p>
<h4>Our own watchers:</h4>
<p>"...I shall live and die at my post. I am the sword in the darkness. I am the watcher on the walls. I am the shield that guards the realms of men..." <strong>-- The Night's Watch oath</strong></p>
<p>We want to be able to seal a bond between one of the slot and a context. By context, I mean the lifetime of an object in a thread, a function or a method. If an update has been made on that slot, we must be signaled in that context and to retrieve the new update. The bond must be destroyed if the context does not exist anymore. It should reminds you the Night's Watch oath ... as well as the <a href="https://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization">RAII idiom</a>: "holding a resource is tied to object lifetime: resource acquisition is done during object creation, by the constructor, while resource deallocation is done during object destruction, by the destructor. If objects are destroyed properly, resource leaks do not occur.". A strong ownership policy can be obtained with the help of a <a href="http://en.cppreference.com/w/cpp/memory/unique_ptr">std::unique_ptr</a> and the signalisation can be done using a <strong>boolean flag</strong>.</p>
<p>We will, therefore, encapsulate a <a href="http://en.cppreference.com/w/cpp/atomic/atomic">std::atomic_bool</a> into a class <strong>Watcher</strong> automagically registered to a slot once created, and unregistered once destructed. This <strong>Watcher</strong> class also takes as a reference the slot in order to query its value as you can see:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Watcher</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="n">Watcher</span><span class="p">(</span><span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>&</span> <span class="n">slot</span><span class="p">)</span><span class="o">:</span>
<span class="n">slot_</span><span class="p">(</span><span class="n">slot</span><span class="p">),</span>
<span class="n">hasBeenChanged_</span><span class="p">(</span><span class="nb">false</span><span class="p">)</span>
<span class="p">{</span>
<span class="p">}</span>
<span class="n">Watcher</span><span class="p">(</span><span class="k">const</span> <span class="n">Watcher</span><span class="o">&</span><span class="p">)</span> <span class="o">=</span> <span class="k">delete</span><span class="p">;</span> <span class="c1">// Impossible to copy that class.</span>
<span class="n">Watcher</span> <span class="o">&</span> <span class="k">operator</span><span class="o">=</span><span class="p">(</span><span class="k">const</span> <span class="n">Watcher</span><span class="o">&</span><span class="p">)</span> <span class="o">=</span> <span class="k">delete</span><span class="p">;</span> <span class="c1">// Impossible to copy that class.</span>
<span class="kt">bool</span> <span class="nf">hasBeenChanged</span><span class="p">()</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">hasBeenChanged_</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="nf">triggerChanges</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">hasBeenChanged_</span> <span class="o">=</span> <span class="nb">true</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">get</span><span class="p">()</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>></span><span class="p">().</span><span class="n">doGet</span><span class="p">())</span>
<span class="p">{</span>
<span class="n">hasBeenChanged_</span> <span class="o">=</span> <span class="nb">false</span><span class="p">;</span> <span class="c1">// Note: even if there is an update of the value between this line and the getValue one,</span>
<span class="c1">// we will still have the latest version.</span>
<span class="c1">// Note 2: atomic_bool automatically use a barrier and the two operations can't be inversed.</span>
<span class="k">return</span> <span class="n">slot_</span><span class="p">.</span><span class="n">doGet</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>&</span> <span class="n">slot_</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="kt">atomic_bool</span> <span class="n">hasBeenChanged_</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>As for the automatic registration, we will add two private methods <strong>registerWatcher</strong> and <strong>unregisterWatcher</strong> to our <strong>Slot</strong> class that add or remove a watcher from an internal list. The list is always protected, when accessed, with a <strong>std::mutex</strong> and tracks all the current watchers that must be signaled when <strong>set</strong> is called on that slot. </p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Slot</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">using</span> <span class="n">ThisType</span> <span class="o">=</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">WatcherType</span> <span class="o">=</span> <span class="n">Watcher</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">></span><span class="p">;</span>
<span class="p">...</span>
<span class="k">private</span><span class="o">:</span>
<span class="kt">void</span> <span class="n">registerWatcher</span><span class="p">(</span><span class="n">WatcherType</span><span class="o">*</span> <span class="n">newWatcher</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">lock_guard</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">mutex</span><span class="o">></span> <span class="n">l</span><span class="p">(</span><span class="n">watchers_mutex_</span><span class="p">);</span>
<span class="n">watchers_</span><span class="p">.</span><span class="n">push_back</span><span class="p">(</span><span class="n">newWatcher</span><span class="p">);</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">unregisterWatcher</span><span class="p">(</span><span class="n">WatcherType</span> <span class="o">*</span><span class="n">toBeDelete</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">lock_guard</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">mutex</span><span class="o">></span> <span class="n">l</span><span class="p">(</span><span class="n">watchers_mutex_</span><span class="p">);</span>
<span class="n">watchers_</span><span class="p">.</span><span class="n">erase</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">remove</span><span class="p">(</span><span class="n">watchers_</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">watchers_</span><span class="p">.</span><span class="n">end</span><span class="p">(),</span> <span class="n">toBeDelete</span><span class="p">),</span> <span class="n">watchers_</span><span class="p">.</span><span class="n">end</span><span class="p">());</span>
<span class="k">delete</span> <span class="n">toBeDelete</span><span class="p">;</span> <span class="c1">// Now that we removed the watcher from the list, we can proceed to delete it.</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">signal</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">lock_guard</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">mutex</span><span class="o">></span> <span class="n">l</span><span class="p">(</span><span class="n">watchers_mutex_</span><span class="p">);</span>
<span class="k">for</span> <span class="p">(</span><span class="k">auto</span> <span class="nl">watcher</span> <span class="p">:</span> <span class="n">watchers_</span><span class="p">)</span> <span class="p">{</span>
<span class="n">watcher</span><span class="o">-></span><span class="n">triggerChanges</span><span class="p">();</span> <span class="c1">// Let's raise the hasBeenChanged_ atomic boolean flag. </span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">private</span><span class="o">:</span>
<span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">WatcherType</span><span class="o">*></span> <span class="n">watchers_</span><span class="p">;</span> <span class="c1">// All the registered watchers are in that list.</span>
<span class="p">...</span>
<span class="p">};</span>
</code></pre></div>
<p>You may have notice that we are passing a bare <strong>WatcherType</strong> pointers. The ownership is actually given to whoever is using that watcher encapsulated within a <strong>std::unique_ptr</strong>. <strong>C++11</strong>'s unique pointers are designed such as you can pass a <strong>custom deleter</strong>, or a <strong>delete callback</strong> so to speak. Hence, we can create a method that get a <strong>Watcher</strong> for a <strong>Slot</strong>, and register as the deleter of that <strong>Watcher</strong> a <strong>lambda function</strong> designed to call <strong>unregisterWatcher</strong>. Note that the slot MUST always lives longer than the unique pointer and its associated watcher (it should not be a problem in most cases). Let's finish that <strong>Slot</strong> class forever and ever:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span><span class="o">></span>
<span class="k">class</span> <span class="nc">Slot</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">using</span> <span class="n">ThisType</span> <span class="o">=</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">></span><span class="p">;</span>
<span class="k">using</span> <span class="n">WatcherType</span> <span class="o">=</span> <span class="n">Watcher</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">></span><span class="p">;</span>
<span class="c1">// We use unique_ptr for a strong ownership policy.</span>
<span class="c1">// We use std::function to declare the type of our deleter.</span>
<span class="k">using</span> <span class="n">WatcherTypePtr</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">unique_ptr</span><span class="o"><</span><span class="n">WatcherType</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">function</span><span class="o"><</span><span class="kt">void</span><span class="p">(</span><span class="n">WatcherType</span><span class="o">*</span><span class="p">)</span><span class="o">>></span> <span class="p">;</span>
<span class="p">...</span>
<span class="k">public</span><span class="o">:</span>
<span class="n">WatcherTypePtr</span> <span class="n">doGetWatcher</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// Create a unique_ptr and pass a lambda as a deleter.</span>
<span class="c1">// The lambda capture "this" and will call unregisterWatcher.</span>
<span class="n">WatcherTypePtr</span> <span class="nf">watcher</span><span class="p">(</span><span class="k">new</span> <span class="n">WatcherType</span><span class="p">(</span><span class="o">*</span><span class="k">this</span><span class="p">),</span> <span class="p">[</span><span class="k">this</span><span class="p">](</span><span class="n">WatcherType</span><span class="o">*</span> <span class="n">toBeDelete</span><span class="p">)</span> <span class="p">{</span>
<span class="k">this</span><span class="o">-></span><span class="n">unregisterWatcher</span><span class="p">(</span><span class="n">toBeDelete</span><span class="p">);});</span>
<span class="n">registerWatcher</span><span class="p">(</span><span class="n">watcher</span><span class="p">.</span><span class="n">get</span><span class="p">());</span>
<span class="k">return</span> <span class="n">watcher</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">...</span>
<span class="p">};</span>
</code></pre></div>
<p>Are we done? Hell no, but we will be really soon. All we need is to expose the possibility to acquire a watcher from the repository itself. In the same manner as <strong>set</strong> and <strong>get</strong>, we simply dispatch using the type and the key on one of our slot:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">Type</span><span class="p">,</span> <span class="k">class</span> <span class="nc">Key</span> <span class="o">=</span> <span class="n">DefaultSlotKey</span><span class="o">></span>
<span class="k">typename</span> <span class="nc">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">WatcherTypePtr</span> <span class="n">getWatcher</span><span class="p">()</span> <span class="c1">// typename is used for disambiguate</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">Slot</span><span class="o"><</span><span class="n">Type</span><span class="p">,</span> <span class="n">Key</span><span class="o">>::</span><span class="n">doGetWatcher</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div>
<p><strong>WAIT</strong>, don't close that page too fast. If you want to be able to snub everyone, you can replace this ugly <strong>typename Slot<Type, Key>::WatcherTypePtr</strong> with <strong>auto</strong> and claim that your repository class is <strong>C++14</strong> only! Grab the full code of what we build together on <a href="https://gist.github.com/Jiwan/31f8f837e4f4b90fed13">gist</a> and enjoy!</p>
<h3>Conclusion:</h3>
<p>Once again, I hope you enjoyed this post about one of my favourite subject: C++. I might not be the best teacher nor the best author but I wish that you learnt something today! Please, if you any suggestions or questions, feel free to post anything in the commentaries. My broken English being what it is, I kindly accept any help for my written mistakes.</p>
<p>Many thanks to my colleagues that greatly helped me by reviewing my code and for the time together.</p>An introduction to C++'s SFINAE concept: compile-time introspection of a class member2015-10-31T14:00:00+01:002015-10-18T14:00:00+02:00Jean Gueganttag:jguegant.github.io,2015-10-31:/blogs/tech/sfinae-introduction.html<!-- http://stackoverflow.com/questions/18570285/using-sfinae-to-detect-a-member-function -->
<h3>Trivia:</h3>
<p>As a C++ enthusiast, I usually follow the annual C++ conference <a href="http://cppcon.org/">cppconf</a> or at least try to keep myself up-to-date with the major events that happen there. One way to catch up, if you can't afford a plane ticket or the ticket, is to follow the <a href="https://www.youtube.com/channel/UCMlGfpWw-RUdWX_JbLCukXg">youtube channel</a> dedicated …</p><!-- http://stackoverflow.com/questions/18570285/using-sfinae-to-detect-a-member-function -->
<h3>Trivia:</h3>
<p>As a C++ enthusiast, I usually follow the annual C++ conference <a href="http://cppcon.org/">cppconf</a> or at least try to keep myself up-to-date with the major events that happen there. One way to catch up, if you can't afford a plane ticket or the ticket, is to follow the <a href="https://www.youtube.com/channel/UCMlGfpWw-RUdWX_JbLCukXg">youtube channel</a> dedicated to this conference. This year, I was impressed by <strong>Louis Dionne</strong> talk entitled "C++ Metaprogramming: A Paradigm Shift". One feature called <strong>is_valid</strong> that can be found in Louis's <a href="http://github.com/boostorg/hana">Boost.Hana</a> library particulary caught my attention. This genious <strong>is_valid</strong> function heavily rely on an even more "magic" C++ programming technique coined with the term <strong>SFINAE</strong> discovered at the end of the previous century. If this acronym doesn't speak to you, don't be scared, we are going to dive straight in the subject.</p>
<p>Note: for the sake of your sanity and the fact that <em>errare humanum est</em>, this article might not be 100% accurate!</p>
<h3>Introspection in C++?</h3>
<p>Before explaining what is <strong>SFINAE</strong>, let's explore one of its main usage: <strong>introspection</strong>. As you might be aware, C++ doesn't excel when it comes to examine the type or properties of an object at runtime. The best ability provided by default would be <a href="https://en.wikipedia.org/wiki/Run-time_type_information">RTTI</a>. Not only <strong>RTTI</strong> isn't always available, but it also gives you barely more than the current type of the manipulated object. Dynamic languages or those having <strong>reflection</strong> on the other hand are really convenient in some situations like <strong>serialization</strong>.</p>
<p>For instance, in Python, using reflection, one can do the following:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="c1"># Simply overrides the 'object.__str__' method.</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"I am a A"</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="c1"># A custom method for my custom objects that I want to serialize.</span>
<span class="k">def</span> <span class="nf">serialize</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"I am a B"</span>
<span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Oups! 'serialize' is not a method. </span>
<span class="bp">self</span><span class="o">.</span><span class="n">serialize</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"I am a C"</span>
<span class="k">def</span> <span class="nf">serialize</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="c1"># Let's check if obj has an attribute called 'serialize'.</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">"serialize"</span><span class="p">):</span>
<span class="c1"># Let's check if this 'serialize' attribute is a method.</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">serialize</span><span class="p">,</span> <span class="s2">"__call__"</span><span class="p">):</span>
<span class="k">return</span> <span class="n">obj</span><span class="o">.</span><span class="n">serialize</span><span class="p">()</span>
<span class="c1"># Else we call the __str__ method.</span>
<span class="k">return</span> <span class="nb">str</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">A</span><span class="p">()</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">B</span><span class="p">()</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">C</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">serialize</span><span class="p">(</span><span class="n">a</span><span class="p">))</span> <span class="c1"># output: I am a A.</span>
<span class="nb">print</span><span class="p">(</span><span class="n">serialize</span><span class="p">(</span><span class="n">b</span><span class="p">))</span> <span class="c1"># output: I am a B.</span>
<span class="nb">print</span><span class="p">(</span><span class="n">serialize</span><span class="p">(</span><span class="n">c</span><span class="p">))</span> <span class="c1"># output: I am a C.</span>
</code></pre></div>
<p>As you can see, during serialization, it comes pretty handy to be able to check if an object has an attribute and to query the type of this attribute. In our case, it permits us to use the <strong>serialize</strong> method if available and fall back to the more generic method <strong>str</strong> otherwise. Powerful, isn't it? Well, we can do it <strong>in plain C++</strong>!</p>
<p>Here is the <strong>C++14</strong> solution mentionned in <strong>Boost.Hana</strong> documentation, using <strong>is_valid</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#include</span> <span class="cpf"><boost/hana.hpp></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><iostream></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><string></span><span class="cp"></span>
<span class="k">using</span> <span class="k">namespace</span> <span class="nn">std</span><span class="p">;</span>
<span class="k">namespace</span> <span class="nn">hana</span> <span class="o">=</span> <span class="nn">boost</span><span class="o">::</span><span class="nn">hana</span><span class="p">;</span>
<span class="c1">// Check if a type has a serialize method.</span>
<span class="k">auto</span> <span class="n">hasSerialize</span> <span class="o">=</span> <span class="n">hana</span><span class="o">::</span><span class="n">is_valid</span><span class="p">([](</span><span class="k">auto</span><span class="o">&&</span> <span class="n">x</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="p">});</span>
<span class="c1">// Serialize any kind of objects.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialize</span><span class="p">(</span><span class="n">T</span> <span class="k">const</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">hana</span><span class="o">::</span><span class="n">if_</span><span class="p">(</span><span class="n">hasSerialize</span><span class="p">(</span><span class="n">obj</span><span class="p">),</span> <span class="c1">// Serialize is selected if available!</span>
<span class="p">[](</span><span class="k">auto</span><span class="o">&</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">x</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="p">},</span>
<span class="p">[](</span><span class="k">auto</span><span class="o">&</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">to_string</span><span class="p">(</span><span class="n">x</span><span class="p">);</span> <span class="p">}</span>
<span class="p">)(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Type A with only a to_string overload.</span>
<span class="k">struct</span> <span class="nc">A</span> <span class="p">{};</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">to_string</span><span class="p">(</span><span class="k">const</span> <span class="n">A</span><span class="o">&</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"I am a A!"</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Type B with a serialize method.</span>
<span class="k">struct</span> <span class="nc">B</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialize</span><span class="p">()</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"I am a B!"</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="c1">// Type C with a "wrong" serialize member (not a method) and a to_string overload.</span>
<span class="k">struct</span> <span class="nc">C</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialize</span><span class="p">;</span>
<span class="p">};</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">to_string</span><span class="p">(</span><span class="k">const</span> <span class="n">C</span><span class="o">&</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"I am a C!"</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="n">B</span> <span class="n">b</span><span class="p">;</span>
<span class="n">C</span> <span class="n">c</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">c</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>As you can see, it only requires a bit more of boilerplate than Python, but not as much as you would expect from a language as complexe as C++. How does it work? Well if you are too lazy to read the rest, here is the simplest answer I can give you: unlike dynamically typed languages, your compiler has access a lot of static type information once fired. It makes sense that we can constraint your compiler to do a bit of work on these types! The next question that comes to your mind is "How to?". Well, right below we are going to explore the various options we have to enslave our favorite compiler for fun and profit! And we will eventually recreate our own <strong>is_valid</strong>.</p>
<h3>The old-fashioned C++98-way:</h3>
<p>Whether your compiler is a dinosaur, your boss refuses to pay for the latest Visual Studio license or you simply love archeology, this chapter will interest you. It's also interesting for the people stuck between C++11 and C++14. The solution in C++98 relies on 3 key concepts: <strong>overload resolution</strong>, <strong>SFINAE</strong> and the static behavior of <strong>sizeof</strong>. </p>
<h4>Overload resolution:</h4>
<p>A simple function call like "<strong>f(obj);</strong>"" in <strong>C++</strong> activates a mechanism to figure out which <strong>f</strong> function shoud be called according to the argument <strong>obj</strong>. If a <strong>set</strong> of <strong>f</strong> functions could accept <strong>obj</strong> as an argument, the compiler must choose the most appropriate function, or in other words <strong>resolve</strong> the best <strong>overload</strong>! Here is a good cppreference page explaining the full process: <a href="http://en.cppreference.com/w/cpp/language/overload_resolution">Overload resolution</a>. The rule of thumb in this case is <em>the compiler picks the candidate function whose parameters match the arguments most closely is the one that is called</em>. Nothing is better than a good example:</p>
<div class="highlight"><pre><span></span><code><span class="kt">void</span> <span class="nf">f</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">s</span><span class="p">);</span> <span class="c1">// int can't be convert into a string.</span>
<span class="kt">void</span> <span class="nf">f</span><span class="p">(</span><span class="kt">double</span> <span class="n">d</span><span class="p">);</span> <span class="c1">// int can be implicitly convert into a double, so this version could be selected, but...</span>
<span class="kt">void</span> <span class="nf">f</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span><span class="p">);</span> <span class="c1">// ... this version using the type int directly is even more close!</span>
<span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="c1">// Call f(int i);</span>
</code></pre></div>
<p>In C++ you also have some sink-hole functions that accept everything. First, function templates accept any kind of parameter (let's say T). But the true black-hole of your compiler, the devil variable vacuum, the oblivion of the forgotten types are the <a href="http://en.cppreference.com/w/cpp/utility/variadic">variadic functions</a>. Yes, exactly like the horrible C <strong>printf</strong>.</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">f</span><span class="p">(...);</span> <span class="c1">// Variadic functions are so "untyped" that...</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">f</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">t</span><span class="p">);</span> <span class="c1">// ...this templated function got the precedence!</span>
<span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="c1">// Call the templated function version of f.</span>
</code></pre></div>
<p>The fact that function templates are less generic than variadic functions is the first point you must remember!</p>
<p>Note: A <strong>templated function</strong> can actually be more precise than a <strong>normal function</strong>. However, in case of a draw, the <strong>normal function</strong> will have the precedence. </p>
<h4>SFINAE:</h4>
<p>I am already teasing you with the power for already few paragraphs and here finally comes the explanation of this not so complex acronym. <strong>SFINAE</strong> stands for <strong>S</strong>ubstitution <strong>F</strong>ailure <strong>I</strong>s <strong>N</strong>ot <strong>A</strong>n <strong>E</strong>rror. In rough terms, a <strong>substitution</strong> is the mechanism that tries to replace the template parameters with the provided types or values. In some cases, if the <strong>substitution</strong> leads to an invalid code, the compiler shouldn't throw a massive amount of errors but simply continue to try the other available <strong>overloads</strong>. The <strong>SFINAE</strong> concept simply guaranties such a "sane" behavior for a "sane" compiler. For instance:</p>
<div class="highlight"><pre><span></span><code><span class="cm">/*</span>
<span class="cm"> The compiler will try this overload since it's less generic than the variadic.</span>
<span class="cm"> T will be replace by int which gives us void f(const int& t, int::iterator* b = nullptr);</span>
<span class="cm"> int doesn't have an iterator sub-type, but the compiler doesn't throw a bunch of errors.</span>
<span class="cm"> It simply tries the next overload. </span>
<span class="cm">*/</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">t</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">T</span><span class="o">::</span><span class="n">iterator</span><span class="o">*</span> <span class="n">it</span> <span class="o">=</span> <span class="k">nullptr</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span>
<span class="c1">// The sink-hole.</span>
<span class="kt">void</span> <span class="n">f</span><span class="p">(...)</span> <span class="p">{</span> <span class="p">}</span>
<span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="c1">// Calls void f(...) { }</span>
</code></pre></div>
<p>All the expressions won't lead to a <strong>SFINAE</strong>. A broad rule would be to say that all the <strong>substitutions</strong> out of the function/methods <strong>body</strong> are "safes". For a better list, please take a look at this <a href="http://en.cppreference.com/w/cpp/language/sfinae">wiki page</a>. For instance, a wrong substitution within a function <strong>body</strong> will lead to a horrible C++ template error:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// The compiler will be really unhappy when it will later discover the call to hahahaICrash. </span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="kt">void</span> <span class="n">f</span><span class="p">(</span><span class="n">T</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span> <span class="n">t</span><span class="p">.</span><span class="n">hahahaICrash</span><span class="p">();</span> <span class="p">}</span>
<span class="kt">void</span> <span class="n">f</span><span class="p">(...)</span> <span class="p">{</span> <span class="p">}</span> <span class="c1">// The sink-hole wasn't even considered.</span>
<span class="n">f</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</code></pre></div>
<h4>The operator sizeof:</h4>
<p>The <strong>sizeof operator</strong> is really a nice tool! It permits us to returns the size in bytes of a type or an expression at compilation time. <strong>sizeof</strong> is really interesting as it accurately evaluates an expression as precisely as if it were compiled.
One can for instance do:</p>
<div class="highlight"><pre><span></span><code><span class="k">typedef</span> <span class="kt">char</span> <span class="n">type_test</span><span class="p">[</span><span class="mi">42</span><span class="p">];</span>
<span class="n">type_test</span><span class="o">&</span> <span class="nf">f</span><span class="p">();</span>
<span class="c1">// In the following lines f won't even be truly called but we can still access to the size of its return type.</span>
<span class="c1">// Thanks to the "fake evaluation" of the sizeof operator.</span>
<span class="kt">char</span> <span class="n">arrayTest</span><span class="p">[</span><span class="k">sizeof</span><span class="p">(</span><span class="n">f</span><span class="p">())];</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">f</span><span class="p">())</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 42.</span>
</code></pre></div>
<p>But wait! If we can manipulate some compile-time integers, couldn't we do some compile-time comparison? The answer is: absolutely yes, my dear reader! Here we are:</p>
<div class="highlight"><pre><span></span><code><span class="k">typedef</span> <span class="kt">char</span> <span class="n">yes</span><span class="p">;</span> <span class="c1">// Size: 1 byte.</span>
<span class="k">typedef</span> <span class="n">yes</span> <span class="n">no</span><span class="p">[</span><span class="mi">2</span><span class="p">];</span> <span class="c1">// Size: 2 bytes.</span>
<span class="c1">// Two functions using our type with different size.</span>
<span class="n">yes</span><span class="o">&</span> <span class="nf">f1</span><span class="p">();</span>
<span class="n">no</span><span class="o">&</span> <span class="nf">f2</span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">f1</span><span class="p">())</span> <span class="o">==</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">f2</span><span class="p">()))</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 0.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">f1</span><span class="p">())</span> <span class="o">==</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">f1</span><span class="p">()))</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 1.</span>
</code></pre></div>
<h4>Combining everything:</h4>
<p>Now we have all the tools to create a solution to check the existence of a method within a type at compile time. You might even have already figured it out most of it by yourself. So let's create it:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">struct</span> <span class="nc">hasSerialize</span>
<span class="p">{</span>
<span class="c1">// For the compile time comparison.</span>
<span class="k">typedef</span> <span class="kt">char</span> <span class="n">yes</span><span class="p">[</span><span class="mi">1</span><span class="p">];</span>
<span class="k">typedef</span> <span class="n">yes</span> <span class="n">no</span><span class="p">[</span><span class="mi">2</span><span class="p">];</span>
<span class="c1">// This helper struct permits us to check that serialize is truly a method.</span>
<span class="c1">// The second argument must be of the type of the first.</span>
<span class="c1">// For instance reallyHas<int, 10> would be substituted by reallyHas<int, int 10> and works!</span>
<span class="c1">// reallyHas<int, &C::serialize> would be substituted by reallyHas<int, int &C::serialize> and fail!</span>
<span class="c1">// Note: It only works with integral constants and pointers (so function pointers work).</span>
<span class="c1">// In our case we check that &C::serialize has the same signature as the first argument!</span>
<span class="c1">// reallyHas<std::string (C::*)(), &C::serialize> should be substituted by </span>
<span class="c1">// reallyHas<std::string (C::*)(), std::string (C::*)() &C::serialize> and work!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">U</span><span class="p">,</span> <span class="n">U</span> <span class="n">u</span><span class="o">></span> <span class="k">struct</span> <span class="nc">reallyHas</span><span class="p">;</span>
<span class="c1">// Two overloads for yes: one for the signature of a normal method, one is for the signature of a const method.</span>
<span class="c1">// We accept a pointer to our helper struct, in order to avoid to instantiate a real instance of this type.</span>
<span class="c1">// std::string (C::*)() is function pointer declaration.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">C</span><span class="o">></span> <span class="k">static</span> <span class="n">yes</span><span class="o">&</span> <span class="n">test</span><span class="p">(</span><span class="n">reallyHas</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="p">(</span><span class="n">C</span><span class="o">::*</span><span class="p">)(),</span> <span class="o">&</span><span class="n">C</span><span class="o">::</span><span class="n">serialize</span><span class="o">>*</span> <span class="cm">/*unused*/</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">C</span><span class="o">></span> <span class="k">static</span> <span class="n">yes</span><span class="o">&</span> <span class="n">test</span><span class="p">(</span><span class="n">reallyHas</span><span class="o"><</span><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="p">(</span><span class="n">C</span><span class="o">::*</span><span class="p">)()</span> <span class="k">const</span><span class="p">,</span> <span class="o">&</span><span class="n">C</span><span class="o">::</span><span class="n">serialize</span><span class="o">>*</span> <span class="cm">/*unused*/</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span>
<span class="c1">// The famous C++ sink-hole.</span>
<span class="c1">// Note that sink-hole must be templated too as we are testing test<T>(0).</span>
<span class="c1">// If the method serialize isn't available, we will end up in this method.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span><span class="o">></span> <span class="k">static</span> <span class="n">no</span><span class="o">&</span> <span class="n">test</span><span class="p">(...)</span> <span class="p">{</span> <span class="cm">/* dark matter */</span> <span class="p">}</span>
<span class="c1">// The constant used as a return value for the test.</span>
<span class="c1">// The test is actually done here, thanks to the sizeof compile-time evaluation.</span>
<span class="k">static</span> <span class="k">const</span> <span class="kt">bool</span> <span class="n">value</span> <span class="o">=</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">test</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="mi">0</span><span class="p">))</span> <span class="o">==</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">yes</span><span class="p">);</span>
<span class="p">};</span>
<span class="c1">// Using the struct A, B, C defined in the previous hasSerialize example.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">hasSerialize</span><span class="o"><</span><span class="n">A</span><span class="o">>::</span><span class="n">value</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">hasSerialize</span><span class="o"><</span><span class="n">B</span><span class="o">>::</span><span class="n">value</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">hasSerialize</span><span class="o"><</span><span class="n">C</span><span class="o">>::</span><span class="n">value</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
</code></pre></div>
<p>The <strong>reallyHas</strong> struct is kinda tricky but necessary to ensure that serialize is a method and not a simple member of the type. You can do a lot of test on a type using variants of this solution (test a member, a sub-type...) and I suggest you to google a bit more about <strong>SFINAE</strong> tricks. Note: if you truly want a pure compile-time constant and avoid some errors on old compilers, you can replace the last <strong>value</strong> evaluation by: "<strong>enum { value = sizeof(test<T>(0)) == sizeof(yes) };</strong>". </p>
<p>You might also wonder why it doesn't work with <strong>inheritence</strong>. <strong>Inheritence</strong> in C++ and <strong>dynamic polymorphism</strong> is a concept available at runtime, or in other words, a data that the compiler won't have and can't guess! However, compile time type inspection is much more efficient (0 impact at runtime) and almost as powerful as if it were at runtime.
For instance:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Using the previous A struct and hasSerialize helper.</span>
<span class="k">struct</span> <span class="nc">D</span> <span class="o">:</span> <span class="n">A</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialize</span><span class="p">()</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"I am a D!"</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="kt">bool</span> <span class="n">testHasSerialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="cm">/*t*/</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">hasSerialize</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">value</span><span class="p">;</span> <span class="p">}</span>
<span class="n">D</span> <span class="n">d</span><span class="p">;</span>
<span class="n">A</span><span class="o">&</span> <span class="n">a</span> <span class="o">=</span> <span class="n">d</span><span class="p">;</span> <span class="c1">// Here we lost the type of d at compile time.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">testHasSerialize</span><span class="p">(</span><span class="n">d</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 1.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">testHasSerialize</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 0.</span>
</code></pre></div>
<p>Last but no least, our test cover the main cases but not the tricky ones like a Functor: </p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">E</span>
<span class="p">{</span>
<span class="k">struct</span> <span class="nc">Functor</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">operator</span><span class="p">()()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="s">"I am a E!"</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="n">Functor</span> <span class="n">serialize</span><span class="p">;</span>
<span class="p">};</span>
<span class="n">E</span> <span class="n">e</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">e</span><span class="p">.</span><span class="n">serialize</span><span class="p">()</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Succefully call the functor.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">testHasSerialize</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 0.</span>
</code></pre></div>
<p>The trade-off for a full coverage would be the readability. As you will see, C++11 shines in that domain!</p>
<h4>Time to use our genius idea:</h4>
<p>Now you would think that it will be super easy to use our <strong>hasSerialize</strong> to create a <strong>serialize</strong> function! Okay let's try it:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">hasSerialize</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">value</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="c1">// error: no member named 'serialize' in 'A'.</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="n">serialize</span><span class="p">(</span><span class="n">a</span><span class="p">);</span>
</code></pre></div>
<p>It might be hard to accept, but the error raised by your compiler is absolutely normal! If you consider the code that you will obtain after substitution and compile-time evaluation:</p>
<div class="highlight"><pre><span></span><code><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="nf">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">A</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Dead branching, but the compiler will still consider it!</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="c1">// error: no member named 'serialize' in 'A'.</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>Your compiler is really a good guy and won't drop any dead-branch, and <strong>obj</strong> must therefore have both a <strong>serialize method</strong> and a <strong>to_string overload</strong> in this case. The solution consists in spliting the serialize function into two different functions: one where we solely use <strong>obj.serialize()</strong> and one where we use <strong>to_string</strong> according to <strong>obj's type</strong>. We come back to an earlier problem that we already solved, how to split according to a type? <strong>SFINAE</strong>, for sure! At that point we could re-work our <strong>hasSerialize</strong> function into a <strong>serialize</strong> function and make it return a <strong>std::string</strong> instead of compile time <strong>boolean</strong>. But we won't do it that way! It's cleaner to separate the <strong>hasSerialize</strong> test from its usage <strong>serialize</strong>.</p>
<p>We need to find a clever <strong>SFINAE</strong> solution on the signature of "<strong>template <class T> std::string serialize(const T& obj)</strong>". I bring you the last piece of the puzzle called <strong>enable_if</strong>.</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span><span class="o"><</span><span class="kt">bool</span> <span class="n">B</span><span class="p">,</span> <span class="k">class</span> <span class="nc">T</span> <span class="o">=</span> <span class="kt">void</span><span class="o">></span> <span class="c1">// Default template version.</span>
<span class="k">struct</span> <span class="nc">enable_if</span> <span class="p">{};</span> <span class="c1">// This struct doesn't define "type" and the substitution will fail if you try to access it.</span>
<span class="k">template</span><span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="c1">// A specialisation used if the expression is true. </span>
<span class="k">struct</span> <span class="nc">enable_if</span><span class="o"><</span><span class="nb">true</span><span class="p">,</span> <span class="n">T</span><span class="o">></span> <span class="p">{</span> <span class="k">typedef</span> <span class="n">T</span> <span class="n">type</span><span class="p">;</span> <span class="p">};</span> <span class="c1">// This struct do have a "type" and won't fail on access.</span>
<span class="c1">// Usage:</span>
<span class="n">enable_if</span><span class="o"><</span><span class="nb">true</span><span class="p">,</span> <span class="kt">int</span><span class="o">>::</span><span class="n">type</span> <span class="n">t1</span><span class="p">;</span> <span class="c1">// Compiler happy. t's type is int.</span>
<span class="n">enable_if</span><span class="o"><</span><span class="n">hasSerialize</span><span class="o"><</span><span class="n">B</span><span class="o">>::</span><span class="n">value</span><span class="p">,</span> <span class="kt">int</span><span class="o">>::</span><span class="n">type</span> <span class="n">t2</span><span class="p">;</span> <span class="c1">// Compiler happy. t's type is int.</span>
<span class="n">enable_if</span><span class="o"><</span><span class="nb">false</span><span class="p">,</span> <span class="kt">int</span><span class="o">>::</span><span class="n">type</span> <span class="n">t3</span><span class="p">;</span> <span class="c1">// Compiler unhappy. no type named 'type' in 'enable_if<false, int>';</span>
<span class="n">enable_if</span><span class="o"><</span><span class="n">hasSerialize</span><span class="o"><</span><span class="n">A</span><span class="o">>::</span><span class="n">value</span><span class="p">,</span> <span class="kt">int</span><span class="o">>::</span><span class="n">type</span> <span class="n">t4</span><span class="p">;</span> <span class="c1">// no type named 'type' in 'enable_if<false, int>';</span>
</code></pre></div>
<p>As you can see, we can trigger a substitution failure according to a compile time expression with <strong>enable_if</strong>. Now we can use this failure on the "<strong>template <class T> std::string serialize(const T& obj)</strong>" signature to dispatch to the right version. Finally, we have the true solution of our problem:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">typename</span> <span class="nc">enable_if</span><span class="o"><</span><span class="n">hasSerialize</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">value</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">>::</span><span class="n">type</span> <span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">typename</span> <span class="nc">enable_if</span><span class="o"><!</span><span class="n">hasSerialize</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">value</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">>::</span><span class="n">type</span> <span class="n">serialize</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="n">B</span> <span class="n">b</span><span class="p">;</span>
<span class="n">C</span> <span class="n">c</span><span class="p">;</span>
<span class="c1">// The following lines work like a charm!</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">serialize</span><span class="p">(</span><span class="n">c</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
</code></pre></div>
<p>Two details worth being noted! Firstly we use <strong>enable_if</strong> on the return type, in order to keep the paramater deduction, otherwise we would have to specify the type explicitely "<strong>serialize<A>(a)</strong>". Second, even the version using <strong>to_string</strong> must use the <strong>enable_if</strong>, otherwise <strong>serialize(b)</strong> would have two potential overloads available and raise an ambiguity. If you want to check the full code of this C++98 version, here is a <a href="https://gist.github.com/Jiwan/2573fc47e4fa5025306b">gist</a>.
Life is much easier in C++11, so let's see the beauty of this new standard!</p>
<p>Note: it's also important to know that this code creates a <strong>SFINAE</strong> on an expression ("<strong>&C::serialize</strong>"). Whilst this feature wasn't required by the <strong>C++98</strong> standard, it was already in use depending on your compiler. It trully became a safe choice in <strong>C++11</strong>.</p>
<h3>When C++11 came to our help:</h3>
<p>After the great century leap year in 2000, people were fairly optimistic about the coming years. Some even decided to design a new standard for the next generation of <strong>C++</strong> coders like me! Not only this standard would ease <strong>TMP</strong> headaches (<strong>T</strong>emplate <strong>M</strong>eta <strong>P</strong>rogramming side-effects), but it would be available in the first decade, hence its code-name <strong>C++0x</strong>. Well, the standard sadly came the next decade (2011 ==> <strong>C++11</strong>), but it brought a lot of features interesting for the purpose of this article. Let's review them!</p>
<h4>decltype, declval, auto & co:</h4>
<p>Do you remember that the <strong>sizeof operator</strong> does a "fake evaluation" of the expression that you pass to it, and return gives you the size of the type of the expression? Well <strong>C++11</strong> adds a new operator called <strong>decltype</strong>. <a href="http://en.cppreference.com/w/cpp/language/decltype">decltype</a> gives you the type of the of the expression it will evaluate. As I am kind, I won't let you google an example and give it to you directly:</p>
<div class="highlight"><pre><span></span><code><span class="n">B</span> <span class="n">b</span><span class="p">;</span>
<span class="k">decltype</span><span class="p">(</span><span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="n">test</span> <span class="o">=</span> <span class="s">"test"</span><span class="p">;</span> <span class="c1">// Evaluate b.serialize(), which is typed as std::string.</span>
<span class="c1">// Equivalent to std::string test = "test";</span>
</code></pre></div>
<p><a href="http://en.cppreference.com/w/cpp/utility/declval">declval</a> is an utility that gives you a "fake reference" to an object of a type that couldn't be easily construct. <strong>declval</strong> is really handy for our <strong>SFINAE</strong> constructions. <strong>cppreference</strong> example is really straightforward, so here is a copy:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Default</span> <span class="p">{</span>
<span class="kt">int</span> <span class="nf">foo</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span><span class="k">return</span> <span class="mi">1</span><span class="p">;}</span>
<span class="p">};</span>
<span class="k">struct</span> <span class="nc">NonDefault</span> <span class="p">{</span>
<span class="n">NonDefault</span><span class="p">(</span><span class="k">const</span> <span class="n">NonDefault</span><span class="o">&</span><span class="p">)</span> <span class="p">{}</span>
<span class="kt">int</span> <span class="n">foo</span><span class="p">()</span> <span class="k">const</span> <span class="p">{</span><span class="k">return</span> <span class="mi">1</span><span class="p">;}</span>
<span class="p">};</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">decltype</span><span class="p">(</span><span class="n">Default</span><span class="p">().</span><span class="n">foo</span><span class="p">())</span> <span class="n">n1</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span> <span class="c1">// int n1</span>
<span class="c1">// decltype(NonDefault().foo()) n2 = n1; // error: no default constructor</span>
<span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">NonDefault</span><span class="o">></span><span class="p">().</span><span class="n">foo</span><span class="p">())</span> <span class="n">n2</span> <span class="o">=</span> <span class="n">n1</span><span class="p">;</span> <span class="c1">// int n2</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"n2 = "</span> <span class="o"><<</span> <span class="n">n2</span> <span class="o"><<</span> <span class="sc">'\n'</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>The <strong>auto specifier</strong> <em>specifies that the type of the variable that is being declared will be automatically deduced</em>. <a href="http://en.cppreference.com/w/cpp/language/auto">auto</a> is equivalent of <strong>var</strong> in C#. <strong>auto</strong> in <strong>C++11</strong> has also a less famous but nonetheless usage for function declaration. Here is a good example:</p>
<div class="highlight"><pre><span></span><code><span class="kt">bool</span> <span class="nf">f</span><span class="p">();</span>
<span class="k">auto</span> <span class="n">test</span> <span class="o">=</span> <span class="n">f</span><span class="p">();</span> <span class="c1">// Famous usage, auto deduced that test is a boolean, hurray!</span>
<span class="c1">// vvv t wasn't declare at that point, it will be after as a parameter!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="k">decltype</span><span class="p">(</span><span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="n">g</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span> <span class="p">{</span> <span class="p">}</span> <span class="c1">// Compilation error</span>
<span class="c1">// Less famous usage:</span>
<span class="c1">// vvv auto delayed the return type specification!</span>
<span class="c1">// vvv vvv the return type is specified here and use t!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="k">auto</span> <span class="n">g</span><span class="p">(</span><span class="k">const</span> <span class="n">T</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="p">}</span> <span class="c1">// No compilation error.</span>
</code></pre></div>
<p>As you can see, <strong>auto</strong> permits to use the trailing return type syntax and use <strong>decltype</strong> coupled with an expression involving one of the function argument. Does it means that we can use it to test the existence of <strong>serialize</strong> with a SFINAE? Yes Dr. Watson! <strong>decltype</strong> will shine really soon, you will have to wait for the <strong>C++14</strong> for this tricky <strong>auto</strong> usage (but since it's a C++11 feature, it ends up here).</p>
<h4>constexpr:</h4>
<p>C++11 also came with a new way to do compile-time computations! The new keyword <strong>constexpr</strong> is a hint for your compiler, meaning that this expression is constant and could be evaluate directly at compile time. In C++11, <a href="http://en.cppreference.com/w/cpp/language/constexpr">constexpr</a> has a lot of rules and only a small subset of VIEs (Very Important Expression) expressions can be used (no loops...)! We still have enough for creating a compile-time factorial function:</p>
<div class="highlight"><pre><span></span><code><span class="k">constexpr</span> <span class="kt">int</span> <span class="nf">factorial</span><span class="p">(</span><span class="kt">int</span> <span class="n">n</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">n</span> <span class="o"><=</span> <span class="mi">1</span><span class="o">?</span> <span class="mi">1</span> <span class="o">:</span> <span class="p">(</span><span class="n">n</span> <span class="o">*</span> <span class="n">factorial</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">));</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="n">factorial</span><span class="p">(</span><span class="mi">5</span><span class="p">);</span> <span class="c1">// Call to a constexpr function.</span>
<span class="c1">// Will be replace by a good compiler by:</span>
<span class="c1">// int i = 120;</span>
</code></pre></div>
<p><strong>constexpr</strong> increased the usage of <strong>std::true_type</strong> & <strong>std::false_type</strong> from the STL. As their name suggest, these types encapsulate a constexpr boolean "true" and a constrexpr boolean "false". Their most important property is that a class or a struct can inherit from them. For instance:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">testStruct</span> <span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span> <span class="p">{</span> <span class="p">};</span> <span class="c1">// Inherit from the true type.</span>
<span class="k">constexpr</span> <span class="kt">bool</span> <span class="n">testVar</span> <span class="o">=</span> <span class="n">testStruct</span><span class="p">();</span> <span class="c1">// Generate a compile-time testStruct.</span>
<span class="kt">bool</span> <span class="n">test</span> <span class="o">=</span> <span class="n">testStruct</span><span class="o">::</span><span class="n">value</span><span class="p">;</span> <span class="c1">// Equivalent to: test = true;</span>
<span class="n">test</span> <span class="o">=</span> <span class="n">testVar</span><span class="p">;</span> <span class="c1">// true_type has a constexpr converter operator, equivalent to: test = true;</span>
</code></pre></div>
<h4>Blending time:</h4>
<h5>First solution:</h5>
<p>In cooking, a good recipe requires to mix all the best ingredients in the right proportions. If you don't want to have a spaghetti code dating from 1998 for dinner, let's revisit our C++98 <strong>hasSerialize</strong> and <strong>serialize</strong> functions with "fresh" ingredients from 2011. Let's start by removing the rotting <strong>reallyHas</strong> trick with a tasty <strong>decltype</strong> and bake a bit of <strong>constexpr</strong> instead of <strong>sizeof</strong>. After 15min in the oven (or fighting with a new headache), you will obtain:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">struct</span> <span class="nc">hasSerialize</span>
<span class="p">{</span>
<span class="c1">// We test if the type has serialize using decltype and declval.</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">C</span><span class="o">></span> <span class="k">static</span> <span class="k">constexpr</span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">C</span><span class="o">></span><span class="p">().</span><span class="n">serialize</span><span class="p">(),</span> <span class="kt">bool</span><span class="p">())</span> <span class="n">test</span><span class="p">(</span><span class="kt">int</span> <span class="cm">/* unused */</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// We can return values, thanks to constexpr instead of playing with sizeof.</span>
<span class="k">return</span> <span class="nb">true</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">C</span><span class="o">></span> <span class="k">static</span> <span class="k">constexpr</span> <span class="kt">bool</span> <span class="n">test</span><span class="p">(...)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nb">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// int is used to give the precedence!</span>
<span class="k">static</span> <span class="k">constexpr</span> <span class="kt">bool</span> <span class="n">value</span> <span class="o">=</span> <span class="n">test</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="kt">int</span><span class="p">());</span>
<span class="p">};</span>
</code></pre></div>
<p>You might be a bit puzzled by my usage of <strong>decltype</strong>. The C++ comma operator "<strong>,</strong>" can create a chain of multiple expressions. In <strong>decltype</strong>, all the expressions will be evaluated, but only the last expression will be considered for the type. The <strong>serialize</strong> doesn't need any changes, minus the fact that the <strong>enable_if</strong> function is now provided in the <strong>STL</strong>. For your tests, here is a <a href="https://gist.github.com/Jiwan/21f65ddbd91e7ce93384">gist</a>.</p>
<h5>Second solution:</h5>
<p>Another C++11 solution described in <strong>Boost.Hanna</strong> documentation and using <strong>std::true_type</strong> and <strong>std::false_type</strong>, would be this one:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Primary template, inherit from std::false_type.</span>
<span class="c1">// ::value will return false. </span>
<span class="c1">// Note: the second unused template parameter is set to default as std::string!!!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="p">,</span> <span class="k">typename</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">hasSerialize</span>
<span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span>
<span class="p">{</span>
<span class="p">};</span>
<span class="c1">// Partial template specialisation, inherit from std::true_type.</span>
<span class="c1">// ::value will return true. </span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span>
<span class="k">struct</span> <span class="nc">hasSerialize</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">().</span><span class="n">serialize</span><span class="p">())</span><span class="o">></span>
<span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span>
<span class="p">{</span>
<span class="p">};</span>
</code></pre></div>
<p>This solution is, in my own opinion, more sneaky! It relies on a not-so-famous-property of default template parameters. But if your soul is already (stack-)corrupted, you may be aware that the <strong>default parameters</strong> are propagated in the <strong>specialisations</strong>. So when we use <strong>hasSerialize<OurType>::value</strong>, the default parameter comes into play and we are actually looking for <strong>hasSerialize<OurType, std::string>::value</strong> both on the <strong>primary template</strong> and the <strong>specialisation</strong>. In the meantime, the <strong>substitution</strong> and the evaluation of <strong>decltype</strong> are processed and our <strong>specialisation</strong> has the signature <strong>hasSerialize<OurType, std::string></strong> if <strong>OurType</strong> has a <strong>serialize</strong> method that returns a <strong>std::string</strong>, otherwise the substitution fails. The <strong>specialisation</strong> has therefore the precedence in the good cases. One will be able to use the <a href="http://en.cppreference.com/w/cpp/types/void_t">std::void_t</a> C++17 helper in these cases. Anyway, here is a <a href="https://gist.github.com/Jiwan/160a64a5d1d25e4bdf6b">gist</a> you can play with!</p>
<p>I told you that this second solution hides a lot of complexity, and we still have a lot of C++11 features unexploited like <strong>nullptr</strong>, <strong>lambda</strong>, <strong>r-values</strong>. No worries, we are going to use some of them in <strong>C++14</strong>!</p>
<h3>The supremacy of C++14:</h3>
<p>According to the Gregorian calendar in the upper-right corner of my XFCE environment, we are in 2015! I can turn on the <strong>C++14</strong> compilation flag on my favorite compiler safely, isn't it? Well, I can with <strong>clang</strong> (is <strong>MSVC</strong> using a maya calendar?).
Once again, let's explore the new features, and use them to build something wonderful! We will even recreate an <strong>is_valid</strong>, like I promised at the beggining of this article.</p>
<h4>auto & lambdas:</h4>
<h6>Return type inference:</h6>
<p>Some cool features in <strong>C++14</strong> come from the relaxed usage of the <strong>auto</strong> keyword (the one used for type inference).</p>
<p>Now, <strong>auto</strong> can be used on the return type of a function or a method. For instance:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">myFunction</span><span class="p">()</span> <span class="c1">// Automagically figures out that myFunction returns ints.</span>
<span class="p">{</span>
<span class="k">return</span> <span class="kt">int</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div>
<p>It works as long as the type is easily "guessable" by the compiler. We are coding in C++ after all, not OCaml! </p>
<h6>A feature for functional lovers:</h6>
<p><strong>C++11</strong> introduced <a href="http://en.cppreference.com/w/cpp/language/lambda">lambdas</a>. A lambda has the following syntax: </p>
<div class="highlight"><pre><span></span><code><span class="p">[</span><span class="n">capture</span><span class="o">-</span><span class="n">list</span><span class="p">](</span><span class="n">params</span><span class="p">)</span> <span class="o">-></span> <span class="n">non</span><span class="o">-</span><span class="n">mandatory</span><span class="o">-</span><span class="kr">return</span><span class="o">-</span><span class="kr">type</span> <span class="p">{</span> <span class="p">...</span><span class="n">body</span><span class="p">...</span> <span class="p">}</span>
</code></pre></div>
<p>A useful example in our case would be:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">l1</span> <span class="o">=</span> <span class="p">[](</span><span class="n">B</span><span class="o">&</span> <span class="n">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="p">};</span> <span class="c1">// Return type figured-out by the return statement.</span>
<span class="k">auto</span> <span class="n">l3</span> <span class="o">=</span> <span class="p">[](</span><span class="n">B</span><span class="o">&</span> <span class="n">b</span><span class="p">)</span> <span class="o">-></span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="p">{</span> <span class="k">return</span> <span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="p">};</span> <span class="c1">// Fixed return type.</span>
<span class="k">auto</span> <span class="n">l2</span> <span class="o">=</span> <span class="p">[](</span><span class="n">B</span><span class="o">&</span> <span class="n">b</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="k">return</span> <span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="p">};</span> <span class="c1">// Return type dependant to the B type.</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l1</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output: I am a B!</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l2</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output: I am a B!</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l3</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output: I am a B!</span>
</code></pre></div>
<p><strong>C++14</strong> brings a small change to the <strong>lambdas</strong> but with a big impact! <strong>Lambdas</strong> accept <strong>auto parameters</strong>: the parameter type is deduced according the argument. <strong>Lambdas</strong> are implemented as an object having an newly created <strong>unnamed type</strong>, also called <strong>closure type</strong>. If a <strong>lambda</strong> has some <strong>auto parameters</strong>, its "Functor operator" <strong>operator()</strong> will be simply templated. Let's take a look:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// ***** Simple lambda unamed type *****</span>
<span class="k">auto</span> <span class="n">l4</span> <span class="o">=</span> <span class="p">[](</span><span class="kt">int</span> <span class="n">a</span><span class="p">,</span> <span class="kt">int</span> <span class="n">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span><span class="p">;</span> <span class="p">};</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l4</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 9.</span>
<span class="c1">// Equivalent to:</span>
<span class="k">struct</span> <span class="nc">l4UnamedType</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="nf">operator</span><span class="p">()(</span><span class="kt">int</span> <span class="n">a</span><span class="p">,</span> <span class="kt">int</span> <span class="n">b</span><span class="p">)</span> <span class="k">const</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="n">l4UnamedType</span> <span class="n">l4Equivalent</span> <span class="o">=</span> <span class="n">l4UnamedType</span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l4Equivalent</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output 9 too.</span>
<span class="c1">// ***** auto parameters lambda unnamed type *****</span>
<span class="c1">// b's type is automagically deduced!</span>
<span class="k">auto</span> <span class="n">l5</span> <span class="o">=</span> <span class="p">[](</span><span class="k">auto</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="k">return</span> <span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span> <span class="p">};</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l5</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output: I am a B!</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l5</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Error: no member named 'serialize' in 'A'.</span>
<span class="c1">// Equivalent to:</span>
<span class="k">struct</span> <span class="nc">l5UnamedType</span>
<span class="p">{</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">T</span><span class="o">></span> <span class="k">auto</span> <span class="k">operator</span><span class="p">()(</span><span class="n">T</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span> <span class="k">const</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="c1">// /!\ This signature is nice for a SFINAE!</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="n">l5UnamedType</span> <span class="n">l5Equivalent</span> <span class="o">=</span> <span class="n">l5UnamedType</span><span class="p">();</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l5Equivalent</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Output: I am a B!</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">l5Equivalent</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span> <span class="c1">// Error: no member named 'serialize' in 'A'.</span>
</code></pre></div>
<p>More than the <strong>lambda</strong> itself, we are interested by the generated <strong>unnamed type</strong>: its lambda <strong>operator()</strong> can be used as a SFINAE! And as you can see, writing a <strong>lambda</strong> is less cumbersome than writing the equivalent type. It should remind you the beggining of my initial solution:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Check if a type has a serialize method.</span>
<span class="k">auto</span> <span class="n">hasSerialize</span> <span class="o">=</span> <span class="n">hana</span><span class="o">::</span><span class="n">is_valid</span><span class="p">([](</span><span class="k">auto</span><span class="o">&&</span> <span class="n">x</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="p">});</span>
</code></pre></div>
<p>And the good new is that we have everything to recreate <strong>is_valid</strong>, right now!</p>
<h4>The making-of a valid <strong>is_valid</strong>:</h4>
<p>Now that we have a really stylish manner to generate a <strong>unnamed types</strong> with potential <strong>SFINAE</strong> properties using <strong>lambdas</strong>, we need to figure out how to use them! As you can see, <strong>hana::is_valid</strong> is a function that takes our lambda as a parameter and return a type. We will call the type returned by <strong>is_valid</strong> the <strong>container</strong>. The <strong>container</strong> will be in charge to keep the lambda's <strong>unnamed type</strong> for a later usage. Let's start by writing the <strong>is_valid</strong> function and its the <strong>containter</strong>:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">struct</span> <span class="nc">container</span>
<span class="p">{</span>
<span class="c1">// Remembers UnnamedType.</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">is_valid</span><span class="p">(</span><span class="k">const</span> <span class="n">UnnamedType</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// We used auto for the return type: it will be deduced here.</span>
<span class="k">return</span> <span class="n">container</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">();</span>
<span class="p">}</span>
<span class="k">auto</span> <span class="n">test</span> <span class="o">=</span> <span class="n">is_valid</span><span class="p">([](</span><span class="k">const</span> <span class="k">auto</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">t</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{})</span>
<span class="c1">// Now 'test' remembers the type of the lambda and the signature of its operator()!</span>
</code></pre></div>
<p>The next step consists at extending <strong>container</strong> with the operator <strong>operator()</strong> such as we can call it with an argument. This argument type will be tested against the <strong>UnnamedType</strong>! In order to do a test on the argument type, we can use once again a SFINAE on a reacreated 'UnnamedType' object! It gives us this solution:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">struct</span> <span class="nc">container</span>
<span class="p">{</span>
<span class="c1">// Let's put the test in private.</span>
<span class="k">private</span><span class="o">:</span>
<span class="c1">// We use std::declval to 'recreate' an object of 'UnnamedType'.</span>
<span class="c1">// We use std::declval to also 'recreate' an object of type 'Param'.</span>
<span class="c1">// We can use both of these recreated objects to test the validity!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">Param</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">testValidity</span><span class="p">(</span><span class="kt">int</span> <span class="cm">/* unused */</span><span class="p">)</span>
<span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">()(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">Param</span><span class="o">></span><span class="p">()),</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span><span class="p">())</span>
<span class="p">{</span>
<span class="c1">// If substitution didn't fail, we can return a true_type.</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">Param</span><span class="o">></span> <span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span> <span class="n">testValidity</span><span class="p">(...)</span>
<span class="p">{</span>
<span class="c1">// Our sink-hole returns a false_type.</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">public</span><span class="o">:</span>
<span class="c1">// A public operator() that accept the argument we wish to test onto the UnnamedType.</span>
<span class="c1">// Notice that the return type is automatic!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">Param</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">Param</span><span class="o">&</span> <span class="n">p</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// The argument is forwarded to one of the two overloads.</span>
<span class="c1">// The SFINAE on the 'true_type' will come into play to dispatch.</span>
<span class="c1">// Once again, we use the int for the precedence.</span>
<span class="k">return</span> <span class="n">testValidity</span><span class="o"><</span><span class="n">Param</span><span class="o">></span><span class="p">(</span><span class="kt">int</span><span class="p">());</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">is_valid</span><span class="p">(</span><span class="k">const</span> <span class="n">UnnamedType</span><span class="o">&</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// We used auto for the return type: it will be deduced here.</span>
<span class="k">return</span> <span class="n">container</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">();</span>
<span class="p">}</span>
<span class="c1">// Check if a type has a serialize method.</span>
<span class="k">auto</span> <span class="n">hasSerialize</span> <span class="o">=</span> <span class="n">is_valid</span><span class="p">([](</span><span class="k">auto</span><span class="o">&&</span> <span class="n">x</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="p">});</span>
</code></pre></div>
<p>If you are a bit lost at that point, I suggest you take your time and re-read all the previous example. You have all the weapons you need, now fight <strong>C++</strong>!</p>
<p>Our <strong>hasSerialize</strong> now takes an argument, we therefore need some changes for our serialize function. We can simply post-pone the return type using <strong>auto</strong> and use the argument in a <strong>decltype</strong> as we learn. Which gives us:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Notice how I simply swapped the return type on the right?</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">auto</span> <span class="n">serialize</span><span class="p">(</span><span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="o">-></span> <span class="k">typename</span> <span class="nc">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o"><</span><span class="k">decltype</span><span class="p">(</span><span class="n">hasSerialize</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span><span class="o">::</span><span class="n">value</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">>::</span><span class="n">type</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">obj</span><span class="p">.</span><span class="n">serialize</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="o">></span> <span class="k">auto</span> <span class="n">serialize</span><span class="p">(</span><span class="n">T</span><span class="o">&</span> <span class="n">obj</span><span class="p">)</span>
<span class="o">-></span> <span class="k">typename</span> <span class="nc">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o"><!</span><span class="k">decltype</span><span class="p">(</span><span class="n">hasSerialize</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span><span class="o">::</span><span class="n">value</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">>::</span><span class="n">type</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">to_string</span><span class="p">(</span><span class="n">obj</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p><strong>FINALLY!!!</strong> We do have a working <strong>is_valid</strong> and we could use it for serialization! If I were as vicious as my SFINAE tricks, I would let you copy each code pieces to recreate a fully working solution. But today, Halloween's spirit is with me and here is <a href="https://gist.github.com/Jiwan/7a586c739a30dd90d259">gist</a>. Hey, hey! Don't close this article so fast! If you are true a warrior, you can read the last part!</p>
<h4>For the fun:</h4>
<p>There are few things I didn't tell you, on purpose. This article would otherwise be twice longer, I fear. I highly suggest you to google a bit more about what I am going to speak about.</p>
<ul>
<li>
<p>Firstly, if you wish to have a solution that works with the <strong>Boost.Hana</strong> static <strong>if_</strong>, you need to change the return type of our <strong>testValidity</strong> methods by Hana's equivalents, like the following:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">Param</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">test_validity</span><span class="p">(</span><span class="kt">int</span> <span class="cm">/* unused */</span><span class="p">)</span>
<span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">()(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">Param</span><span class="o">></span><span class="p">()),</span> <span class="n">boost</span><span class="o">::</span><span class="n">hana</span><span class="o">::</span><span class="n">true_c</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// If substitution didn't fail, we can return a true_type.</span>
<span class="k">return</span> <span class="n">boost</span><span class="o">::</span><span class="n">hana</span><span class="o">::</span><span class="n">true_c</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">Param</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">decltype</span><span class="p">(</span><span class="n">boost</span><span class="o">::</span><span class="n">hana</span><span class="o">::</span><span class="n">false_c</span><span class="p">)</span> <span class="n">test_validity</span><span class="p">(...)</span>
<span class="p">{</span>
<span class="c1">// Our sink-hole returns a false_type.</span>
<span class="k">return</span> <span class="n">boost</span><span class="o">::</span><span class="n">hana</span><span class="o">::</span><span class="n">false_c</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>The static <strong>if_</strong> implementation is really interesting, but at least as hard as our <strong>is_valid</strong> problem solved in this article. I might dedicate another article about it, one day!</p>
</li>
<li>
<p>Did you noticed that we only check one argument at a time? Couldn't we do something like:</p>
<div class="highlight"><pre><span></span><code><span class="k">auto</span> <span class="n">test</span> <span class="o">=</span> <span class="n">is_valid</span><span class="p">([](</span><span class="k">auto</span><span class="o">&&</span> <span class="n">a</span><span class="p">,</span> <span class="k">auto</span><span class="o">&&</span> <span class="n">b</span><span class="p">)</span> <span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">a</span><span class="p">.</span><span class="n">serialize</span><span class="p">(),</span> <span class="n">b</span><span class="p">.</span><span class="n">serialize</span><span class="p">())</span> <span class="p">{</span> <span class="p">});</span>
<span class="n">A</span> <span class="n">a</span><span class="p">;</span>
<span class="n">B</span> <span class="n">b</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">test</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
</code></pre></div>
<p>Actually we can, using some <a href="http://en.cppreference.com/w/cpp/language/parameter_pack">parameter packs</a>. Here is the solution:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">struct</span> <span class="nc">container</span>
<span class="p">{</span>
<span class="c1">// Let's put the test in private.</span>
<span class="k">private</span><span class="o">:</span>
<span class="c1">// We use std::declval to 'recreate' an object of 'UnnamedType'.</span>
<span class="c1">// We use std::declval to also 'recreate' an object of type 'Param'.</span>
<span class="c1">// We can use both of these recreated objects to test the validity!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span><span class="p">...</span> <span class="n">Params</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">test_validity</span><span class="p">(</span><span class="kt">int</span> <span class="cm">/* unused */</span><span class="p">)</span>
<span class="o">-></span> <span class="k">decltype</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">()(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o"><</span><span class="n">Params</span><span class="o">></span><span class="p">()...),</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span><span class="p">())</span>
<span class="p">{</span>
<span class="c1">// If substitution didn't fail, we can return a true_type.</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span><span class="p">...</span> <span class="n">Params</span><span class="o">></span> <span class="k">constexpr</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span> <span class="n">test_validity</span><span class="p">(...)</span>
<span class="p">{</span>
<span class="c1">// Our sink-hole returns a false_type.</span>
<span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">public</span><span class="o">:</span>
<span class="c1">// A public operator() that accept the argument we wish to test onto the UnnamedType.</span>
<span class="c1">// Notice that the return type is automatic!</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span><span class="p">...</span> <span class="n">Params</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="k">operator</span><span class="p">()(</span><span class="n">Params</span><span class="o">&&</span> <span class="p">...)</span>
<span class="p">{</span>
<span class="c1">// The argument is forwarded to one of the two overloads.</span>
<span class="c1">// The SFINAE on the 'true_type' will come into play to dispatch.</span>
<span class="k">return</span> <span class="n">test_validity</span><span class="o"><</span><span class="n">Params</span><span class="p">...</span><span class="o">></span><span class="p">(</span><span class="kt">int</span><span class="p">());</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">template</span> <span class="o"><</span><span class="k">typename</span> <span class="nc">UnnamedType</span><span class="o">></span> <span class="k">constexpr</span> <span class="k">auto</span> <span class="n">is_valid</span><span class="p">(</span><span class="n">UnnamedType</span><span class="o">&&</span> <span class="n">t</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// We used auto for the return type: it will be deduced here.</span>
<span class="k">return</span> <span class="n">container</span><span class="o"><</span><span class="n">UnnamedType</span><span class="o">></span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div>
</li>
<li>
<p>This code is working even if my types are incomplete, for instance a forward declaration, or a normal declaration but with a missing definition. What can I do? Well, you can insert a check on the size of your type either in the <strong>SFINAE</strong> construction or before calling it: "<strong>static_assert( sizeof( T ), "type is incomplete." );</strong>".</p>
</li>
<li>
<p>Finally, why are using the notation "<strong>&&</strong>" for the <strong>lambdas</strong> parameters? Well, these are called <strong>forwarding references</strong>. It's a really complex topic, and if you are interested, here is good <a href="http://thbecker.net/articles/rvalue_references/section_01.html">article</a> about it. You need to use "<strong>auto&&</strong>" due to the way <strong>declval</strong> is working in our <strong>is_valid</strong> implementation!</p>
</li>
</ul>
<h4>Notes:</h4>
<p>This is my first serious article about <strong>C++</strong> on the web and I hope you enjoyed it! I would be glad if you have any suggestions or questions and that you wish to share with me in the commentaries.</p>
<p>Anyway, thanks to <a href="https://github.com/Naav">Naav</a> and <a href="https://github.com/superboum">Superboum</a> for rereading this article and theirs suggestions. Few suggestions were also provided by the reddit community or in the commentaries of this post, thanks a lot guys!</p>Why starting this blog?2015-10-17T21:04:00+02:002015-10-17T21:04:00+02:00Jean Gueganttag:jguegant.github.io,2015-10-17:/blogs/tech/first-post.html<p>Whilst I never had an urge to daily browse the various social networks, I have always liked to read technical articles and to follow few blogs. I wanted, since few years, to write my thoughts somewhere in order to clarify them and be able to easily fetch them later on …</p><p>Whilst I never had an urge to daily browse the various social networks, I have always liked to read technical articles and to follow few blogs. I wanted, since few years, to write my thoughts somewhere in order to clarify them and be able to easily fetch them later on. I procrastinated the idea of running a tech blog for few years, my aversion for the over-engineered web technologies and a lack of motivation restrained me. However, I have more spare time these days and I stumbled upon a simple and elegant Python static blog generator called <a href="http://blog.getpelican.com/">Pelican</a>. With <strong>Pelican</strong> I don't have to pull a thousand of npm packages for a sanatizing function neither to use the latest language for hipsters with an eccentric syntax. Static pages also greatly simplify the maintenance and decrease the security burdens of dynamic websites.</p>
<p>If this blog is mostly for my own purpose, I also hope that I could get an external opinion on my posts, my methods or even my skills in the beautiful English language.</p>