Towards C++ kwargs

28 Dec 2024

§ Motivation

One of my favorite features from Python is the **kwargs syntax, for example, if we're given

def start_server(hostname: str, port: int = 8000, ssl: bool = False):
    # do things here

If, for instance, we wanted to just turn on SSL while using the default port, we could selectively define the ssl argument

start_server("localhost", ssl=True)

P2996 "Reflection for C++" is a proposal aiming to include static reflection in the upcoming C++26 standard.

More importantly, it aims for more than read-only reflection; with P2996 we can perform reflective metaprogramming.

After some reading, I realized that it'd be possible to get reasonably close to kwargs with function reflection (P3096).

A working example can be found on Godbolt.

§ C++ Metaprogramming

Everything here is subject to change as the proposals are finalized.

Revisions as of 28 Dec. 2024:

Bloomberg's clang-p2996 experimental compiler (commit c437bb0)
P2996 rev. 8
P3096 rev. 5

The syntax for reflection is the unary ^^ operator [P3381R0].

Because reflection is such a fundamental (and primitive) operation for metaprogramming, it should have a simple spelling. The ^^ is intended to imply "lifting" or "raising" a term into the metaprogramming environment [P2320R0]

The operator yields the meta::info scalar type, and we say that ^^thing is the reflection of thing.

We for now restrict the space of reflectable values to those of structural type in order to meet two requirements:

The compiler must know how to mangle any reflectable value (i.e., when a reflection thereof is used as a template argument).

The compiler must know how to compare any two reflectable values, ideally without interpreting user-defined comparison operators (i.e., to implement comparison between reflections).

[P2996R8]

Let us write a start_server in C++ for which we may wish to use kwargs with.

void start_server(std::string_view hostname, short port=8000, bool ssl=false) {
    std::print("Listening on http{}://{}:{}"sv,
                ssl ? "s" : "", hostname, port);

    // actually start the server
}

§ Function Parameters

With [P3096R5] we can obtain the metaprogramming info for a given function, and inspect its parameters with parameters_of.

constexpr auto parameters = std::meta::parameters_of(^^start_server);

This gives us a std::vector<meta::info> of each of the parameters accepted by the function.

§ Default Parameters

On the topic of default parameters, the proposal states

"In our research, we have encountered the need to know whether a parameter has a default value, but not necessarily what the value is. [...] Therefore, we only propose to add the has_default_argument as it was already in [P1240R2]." [P3096R5].

For start_server we see that port and ssl have default values, but we don't know what they are.

Until function reflection is extended to include this feature, we have to workaround it with a default_argument<T, default_value> template. Unfortunately, this workaround means that we must rewrite our function to accomodate the kwargs metafunction.

template <typename T, T default_value>
struct default_argument {
    T value;
    constexpr default_argument(T t = default_value) : val(t) {};
    constexpr T operator*() { return value; }
};

void start_server(
    std::string_view hostname,
    default_argument<short, 8000> port={},
    default_argument<bool, false> ssl={}
) {
    std::print("Listening on http{}://{}:{}"sv,
                *ssl ? "s" : "", hostname, *port);
}

This creates a container where the constructor will take the default value from the template argument.

However, this only works if T is a structural type; so we can't use things like std::string_view. In C++20, we can create a string-literal template; so it should be possible to create MyStingLiteral such that we can use default_argument<MyStringLiteral, "hostname">. Other types would require specialized containers.

§ Parameter Vector

Now, we can iterate through each of the function parameters with template for

template for(constexpr auto e : meta::parameters_of(^^start_server)) { /* loop */}

Unfortunately, there are a few pieces unimplemented as of c437bb0 so we can't use template for on a std::vector<meta::info> quite yet.

So we have to use the expand workaround [P2996R8] and convert the constexpr vector to a std::span.

constexpr auto parameters = meta::define_static_array(
                                meta::parameters_of(^^start_server));

[:expand(parameters):] >> [&]<auto e> { /* loop */ };

Now we can print things like the type and name of the parameters

[:expand(parameters):] >> [&]<auto e> {
    std::print("{} {}\n"sv,
                meta::identifier_of(meta::type_of(e)),
                meta::identifier_of(e));
};

As suggested by the names,

type_of gives us ^^type (i.e. the reflection of the type)
identifier_of gives us the representation, i.e. variable name or type name

§ Parameter Structs

[P2996R8] gives an example of named-tuples.

For example, creating a named tuple for a 2D vector

struct Vec2D;
make_named_tuple(^^Vec2D, pair<double, "x">{}, pair<double, "y">{});

This gave me the idea of creating function parameter tuples, for our revised start_server example we could have a struct containing the function parameters (referred to as FuncKwargs)

struct start_server_parameters {
    std::string_view hostname;
    default_argument<short, 8000> port;
    default_argument<bool, false> ssl;
}

Ideally, we could just define make_function_struct(func_Ty, struct_Ty) and use reflection to generate FuncKwargs.

But, if we pass in the reflection of the function as a parameter, then we can't define a constexpr variable using func_Ty because it's not considered to be non-constexpr.

As a result we must define make_function_struct<func_Ty>(struct_Ty).

template<meta::info func_Ty>
consteval void make_function_struct(meta::info struct_Ty) {
    std::vector<meta::data_member_spec> nsdms;

    template for(constexpr auto e : meta::parameters_of(func_Ty)) {
        nsdms.push_back({
            meta::type_of(e),
            {.name = meta::identifier_of(e)}
        });
    }

    return define_aggregate(struct_Ty, nsdms);
}

#define make_function_struct(funcname)                                            \
struct funcname##_parameters;                                                     \
static_assert(is_type(__make_function_struct<^^funcname>(funcname##_parameters)));

Even if we are able to get the default argument value that [P3096R5] omits; meta::data_member_spec [P2996R8] does not have the ability to set a default value for a member.

The above code does not currently work [c437bb0]; simply replace template for with expand().

The define_aggregate injects declarations defined in the nsdms vector into the [:struct_Ty:] struct.

As far as I am aware, this is the most that we are able to do in terms of injection as static reflection is defined in [P2996R8].

Until we get fancier code injection capabilities, we need a macro to create the funcname##_parameters struct and the new overload for funcname that uses the struct.

Ideally, with code injection we can create a consteval metafunction that takes a reflection of a namespace, and (1) gets all functions of the namespace, (2) injects FuncKwargs structs for each function, (3) injects the FuncKwargs overload into the namespace.

§ kwargs

Finally we can define an overload that accepts FuncKwargs as its only parameter, e.g.

void start_server(start_server_parameters&& params) {
    /* start_server(**params) */
}

Essentially, we want to create a parameter pack from our start_server_parameters struct we created using reflection.

The first thing I thought of was to create a std::tuple of each parameter in the correct order, and use std::get<> in a fold-expression to pass the parameters from the struct.

Instead of get-ing on a std::tuple, I decided it'd be better to implement my own get specialized for FuncKwargs.

In this case, I would need to:

create a get<I> to get the value of the I^th parameter (as ordered in the function decl) from FuncKwargs, and
call start_server using a fold expr.

§ get

First we can define a member_at to get the i^th data member of any struct, not just function parameter structs.

consteval auto member_at(meta::info Ty, size_t i)
{
    meta::nonstatic_data_members_of(Ty)[i];
}

Next, we can define get<i> using a member access slice

template<size_t i, typename FuncKwargs>
auto get(FuncKwargs const& kwargs)
{
    return kwargs.[:member_at(^^FuncKwargs, i):];
}

§ Folding

We can obtain the number of parameters at compile time; it is simply the size of the std::vector of parameter reflections for the function.

Then, we can create a std::index_sequence based on the number of parameters.

Using the fold expression, we can fold the get<i> we defined previously on the index sequence to extract a list of arguments from FuncKwargs.

void start_server(start_server_parameters&& params)
{
    constexpr auto num_params = meta::nonstatic_data_members_of(
                                    ^^funcname##_parameters).size();

    [&]<size_t... I>(std::index_sequence<I...>) {
        start_server(get<I>(params)...);
    }(std::make_index_sequence<num_params> {});
}

§ Examples

Non-exhaustive list of examples

// Example 1: original
start_server("nguyen.vc"sv, 443, true);
// Output: Listening on https://nguyen.vc:443

// Example 2: original
start_server("localhost"sv);
// Output: Listening on http://localhost:8000

// Example 3: original, but only want to change ssl=true
start_server("localhost"sv, 8000, true);
// Output: Listening on https://localhost:8000

// Example 4: kwargs changing ssl only
start_server({ .hostname = "localhost"sv, .ssl = true });
// Output: Listening on https://localhost:8000

§ Positional Arguments

It is possible to extend the above to create an overload that accepts the non-default positional arguments with the FuncKwargs object.

Currently, FuncKwargs must preceed the positional arguments if we use a variadic template to define the overload which leads to an awkward syntax.

template<typename... Args>
start_server(start_server_parameters&&, Args... args)

Should non-terminal variadic template parameters [P0478R0] make it into the standard, we can then define an overload with func(Args... args, FuncKwargs&&), which brings us pretty close to Python's kwargs.

This is a pretty trivial modification to the existing overload

template <
    size_t paramc = meta::nonstatic_data_members_of(^^start_server_parameters).size(),
    typename... Args>
decltype(auto) start_server(start_server_parameters&& params, Args... args)
{
    constexpr auto argc = sizeof...(Args);

    return [&]<size_t... I>(std::index_sequence<I...>) {
        return start_server(std::forward<Args>(args)..., get<argc + I>(params)...);
    }(std::make_index_sequence<paramc - argc> {});
}

Here we pass paramc as a template parameter of the number of parameters.Then, we pass to the original function, a total of argc positional arguments and N - argc kwarg parameters.

I'm not entirely sure why I couldn't just define a constexpr auto paramc variable equal to the size of the nonstatic members range inside of the overload, instead of including it as a template parameter.

To me, this seems like an issue with the current implementation.

§ Duplicated Arguments

As it stands, providing a positional argument that is user-defined in the FuncKwargs object is not an error. The current behavior would to take the corresponding positional argument over the kwarg.

One potential way to detect this is to modify default_argument to have a user_defined flag.

template <typename T, T default_value>
struct default_argument {
    bool user_defined = false;
    struct empty_t {};

    union {
        empty_t __empty;
        T value;
    };

    constexpr default_argument() = default;
    constexpr default_argument(T t)
        : user_defined(true)
        , value(t) { };

    constexpr T operator*(void) { return user_defined ? value : default_value; }
};

Next, in the FuncKwargs overload we can check over each positional argument, ensuring that there is not a value in FuncKwargs that is flagged as user-defined.

[&]<size_t... I>(std::index_sequence<I...>) {
    (assert(!get<I>(params).user_defined), ...);
}(std::make_index_sequence<argc> {});

However, this only works if every argument is encapsulated with default_argument; as before, only arguments with structural types can be.

§ Examples

start_server({ .ssl = true }, "localhost"sv);
// Output: Listening on https://localhost:8000

start_server({ .port = 7777 }, "127.0.0.1"sv);
// Output: Listening on http://127.0.0.1:7777

start_server({ .hostname = "localhost"sv, .port = 7777, .ssl = true }, "127.0.0.1"sv);
// Output: Listening on https://127.0.0.1:7777

§ Conclusion

Static reflection and metaprogramming are powerful tools, and now that constexpr and consteval have matured this is now a viable and promising path for future C++.

This was an exercise for me to get familiar with the features, as they currently stand; I probably won't update this page unless both [P2996] and [P3096] make it into the C++26 standard.

And, I really want [P0478] to be merged into the IS, and to amend §10 of [P3096] (or make a proposal) to include a get_default_argument alongside has_default_argument.

§ Code

Again, a runnable example is on Godbolt; hopefully it remains runnable in the future.

Below is the definition for the kwargs_for macro using the capabilities of [P2996R8] and [P3096R5].

namespace meta = std::meta;
namespace views = std::views;

template<class...>
constexpr std::false_type always_false {};

consteval auto member_at(meta::info Ty, size_t i)
{
    return meta::nonstatic_data_members_of(Ty)[i];
}

template <size_t i, typename FuncKwargs>
auto get(FuncKwargs const& kwargs)
{
    return kwargs.[:member_at(^^FuncKwargs, i):];
}

template <typename T, T default_value>
struct default_argument {
    bool user_defined = false;
    struct empty_t {};

    union {
        empty_t __empty;
        T value;
    };

    constexpr default_argument() { };
    constexpr default_argument(T t)
        : user_defined(true)
        , value(t) { };

    constexpr T operator*()
    {
        return user_defined ? value : default_value;
    }
};

template <meta::info func_Ty>
consteval auto __make_function_struct(meta::info struct_Ty)
{
    std::vector<meta::info> nsdms;

    template for (constexpr auto e : meta::parameters_of(func_Ty)) {
        nsdms.push_back(meta::data_member_spec(
            meta::type_of(e), { .name = meta::identifier_of(e); }
        ));
    }

    return define_aggregate(struct_Ty, nsdms);
}

#define make_function_struct(funcname) \
    struct funcname##_parameters;      \
    static_assert(is_type(             \
        __make_function_struct<^^funcname>(^^funcname##_parameters)));

#define create_function_kwarg_overload(funcname) \
    template <
        size_t paramc = meta::nonstatic_data_members_of(^^funcname##_parameters).size(),
        typename...Args>
    decltype(auto) funcname(funcname##_parameters&& params, Args... args)
    {
        constexpr auto argc = sizeof...(Args);

        return [&]<size_t... I>(std::index_sequence<I...>) {
            return funcname(std::forward<Args>(args)..., get<argc + I>(params)....);
        }(std::make_index_sequence<paramc - argc> {});
    }

#define kwargs_for(funcname)        \
    make_function_struct(funcname); \
    create_function_kwarg_overload(funcname);

§ Notes

Inside the FuncKwargs overload, I used nonstatic_members_of on ^^FuncKwargs instead of using parameters_of on ^^func.

Because FuncKwargs was generated based off of the original definition of func, there will always be the same number of parameters, and in the correct order – enabling the usage of the fold expression.

Defining an overload this way is a bit of a hack; inside the start_server overload context, the token start_server now refers to an overload set and the compiler is unable to resolve which overload to use for ^^start_server [P3096R5].

The better thing to do is to define a function with a different name such as func$(FuncKwargs&&) to avoid overloading; then we can have a semantic association with functions suffixed by $ with using kwargs.

Another limitation, as stated previously, is that default_argument<T, default_value> workaround only works with structural types; we can get around this with type wrappers and specializations.