Use semi-constexpr signatures on MSVC

MSCV does not allow `&typeid(T)` in constexpr contexts, but the string part of the type signature can still be constexpr. In order to avoid `typeid` as long as possible, `descr` is modified to collect type information as template parameters instead of constexpr `typeid`. The actual `std::type_info` pointers are only collected in the end, as a `constexpr` (gcc/clang) or regular (MSVC) function call. Not only does it significantly reduce binary size on MSVC, gcc/clang benefit a little bit as well, since they can skip some intermediate `std::type_info*` arrays.
tschijnmo · Sep 16, 2017 · 5661394 · 5661394
1 parent c10ac6c
commit 5661394
Show file tree

Hide file tree

Showing 7 changed files with 67 additions and 109 deletions.
diff --git a/README.md b/README.md
@@ -87,9 +87,8 @@ In addition to the core functionality, pybind11 provides some extra goodies:
   [reported](http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf) a binary
   size reduction of **5.4x** and compile time reduction by **5.8x**.
 
-- When supported by the compiler, two new C++14 features (relaxed constexpr and
-  return value deduction) are used to precompute function signatures at compile
-  time, leading to smaller binaries.
+- Function signatures are precomputed at compile time (using ``constexpr``),
+  leading to smaller binaries.
 
 - With little extra effort, C++ types can be pickled and unpickled similar to
   regular Python objects.

diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -9,7 +9,11 @@ Starting with version 1.8.0, pybind11 releases use a `semantic versioning
 v2.3.0 (Not yet released)
 -----------------------------------------------------
 
-* TBD
+* Significantly reduced module binary size (10-20%) when compiled in C++11 mode
+  with GCC/Clang, or in any mode with MSVC. Function signatures are now always
+  precomputed at compile time (this was previously only available in C++14 mode
+  for non-MSVC compilers).
+  `#934 <https://github.com/pybind/pybind11/pull/934>`_.
 
 v2.2.1 (September 14, 2017)
 -----------------------------------------------------

diff --git a/docs/faq.rst b/docs/faq.rst
@@ -228,36 +228,6 @@ In addition to decreasing binary size, ``-fvisibility=hidden`` also avoids
 potential serious issues when loading multiple modules and is required for
 proper pybind operation.  See the previous FAQ entry for more details.
 
-Another aspect that can require a fair bit of code are function signature
-descriptions. pybind11 automatically generates human-readable function
-signatures for docstrings, e.g.:
-
-.. code-block:: none
-
-     |  __init__(...)
-     |      __init__(*args, **kwargs)
-     |      Overloaded function.
-     |
-     |      1. __init__(example.Example1) -> NoneType
-     |
-     |      Docstring for overload #1 goes here
-     |
-     |      2. __init__(example.Example1, int) -> NoneType
-     |
-     |      Docstring for overload #2 goes here
-     |
-     |      3. __init__(example.Example1, example.Example1) -> NoneType
-     |
-     |      Docstring for overload #3 goes here
-
-
-In C++11 mode, these are generated at run time using string concatenation,
-which can amount to 10-20% of the size of the resulting binary. If you can,
-enable C++14 language features (using ``-std=c++14`` for GCC/Clang), in which
-case signatures are efficiently pre-generated at compile time. Unfortunately,
-Visual Studio's C++14 support (``constexpr``) is not good enough as of April
-2016, so it always uses the more expensive run-time approach.
-
 Working with ancient Visual Studio 2009 builds on Windows
 =========================================================
 

diff --git a/docs/intro.rst b/docs/intro.rst
@@ -77,9 +77,8 @@ In addition to the core functionality, pybind11 provides some extra goodies:
   of `PyRosetta`_, an enormous Boost.Python binding project, reported a binary
   size reduction of **5.4x** and compile time reduction by **5.8x**.
 
-- When supported by the compiler, two new C++14 features (relaxed constexpr and
-  return value deduction) are used to precompute function signatures at compile
-  time, leading to smaller binaries.
+- Function signatures are precomputed at compile time (using ``constexpr``),
+  leading to smaller binaries.
 
 - With little extra effort, C++ types can be pickled and unpickled similar to
   regular Python objects.

diff --git a/include/pybind11/detail/descr.h b/include/pybind11/detail/descr.h
@@ -14,99 +14,85 @@
 NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
 NAMESPACE_BEGIN(detail)
 
+#if !defined(_MSC_VER)
+#  define PYBIND11_DESCR_CONSTEXPR static constexpr
+#else
+#  define PYBIND11_DESCR_CONSTEXPR const
+#endif
+
 /* Concatenate type signatures at compile time */
-template <size_t Size1, size_t Size2> class descr {
-    template <size_t Size1_, size_t Size2_> friend class descr;
-public:
-    constexpr descr() = default;
-
-    constexpr descr(char const (&text) [Size1+1], const std::type_info * const (&types)[Size2+1])
-        : descr(text, types,
-                make_index_sequence<Size1>(),
-                make_index_sequence<Size2>()) { }
-
-    constexpr const char *text() const { return m_text; }
-    constexpr const std::type_info * const * types() const { return m_types; }
-
-    template <size_t OtherSize1, size_t OtherSize2>
-    constexpr descr<Size1 + OtherSize1, Size2 + OtherSize2> operator+(const descr<OtherSize1, OtherSize2> &other) const {
-        return concat(other,
-                      make_index_sequence<Size1>(),
-                      make_index_sequence<Size2>(),
-                      make_index_sequence<OtherSize1>(),
-                      make_index_sequence<OtherSize2>());
-    }
+template <size_t N, typename... Ts>
+struct descr {
+    char text[N + 1];
 
-protected:
-    template <size_t... Indices1, size_t... Indices2>
-    constexpr descr(
-        char const (&text) [Size1+1],
-        const std::type_info * const (&types) [Size2+1],
-        index_sequence<Indices1...>, index_sequence<Indices2...>)
-        : m_text{text[Indices1]..., '\0'},
-          m_types{types[Indices2]...,  nullptr } {}
-
-    template <size_t OtherSize1, size_t OtherSize2, size_t... Indices1,
-              size_t... Indices2, size_t... OtherIndices1, size_t... OtherIndices2>
-    constexpr descr<Size1 + OtherSize1, Size2 + OtherSize2>
-    concat(const descr<OtherSize1, OtherSize2> &other,
-           index_sequence<Indices1...>, index_sequence<Indices2...>,
-           index_sequence<OtherIndices1...>, index_sequence<OtherIndices2...>) const {
-        return descr<Size1 + OtherSize1, Size2 + OtherSize2>(
-            { m_text[Indices1]..., other.m_text[OtherIndices1]..., '\0' },
-            { m_types[Indices2]..., other.m_types[OtherIndices2]..., nullptr }
-        );
-    }
+    constexpr descr() : text{'\0'} { }
+    constexpr descr(char const (&s)[N+1]) : descr(s, make_index_sequence<N>()) { }
 
-protected:
-    char m_text[Size1 + 1];
-    const std::type_info * m_types[Size2 + 1];
+    template <size_t... Is>
+    constexpr descr(char const (&s)[N+1], index_sequence<Is...>) : text{s[Is]..., '\0'} { }
+
+    template <typename... Chars>
+    constexpr descr(char c, Chars... cs) : text{c, static_cast<char>(cs)..., '\0'} { }
+
+    static constexpr std::array<const std::type_info *, sizeof...(Ts) + 1> types() {
+        return {{&typeid(Ts)..., nullptr}};
+    }
 };
 
-template <size_t Size> constexpr descr<Size - 1, 0> _(char const(&text)[Size]) {
-    return descr<Size - 1, 0>(text, { nullptr });
+template <size_t N1, size_t N2, typename... Ts1, typename... Ts2, size_t... Is1, size_t... Is2>
+constexpr descr<N1 + N2, Ts1..., Ts2...> plus_impl(const descr<N1, Ts1...> &a, const descr<N2, Ts2...> &b,
+                                                   index_sequence<Is1...>, index_sequence<Is2...>) {
+    return {a.text[Is1]..., b.text[Is2]...};
 }
 
+template <size_t N1, size_t N2, typename... Ts1, typename... Ts2>
+constexpr descr<N1 + N2, Ts1..., Ts2...> operator+(const descr<N1, Ts1...> &a, const descr<N2, Ts2...> &b) {
+    return plus_impl(a, b, make_index_sequence<N1>(), make_index_sequence<N2>());
+}
+
+template <size_t N>
+constexpr descr<N - 1> _(char const(&text)[N]) { return descr<N - 1>(text); }
+constexpr descr<0> _(char const(&)[1]) { return {}; }
+
 template <size_t Rem, size_t... Digits> struct int_to_str : int_to_str<Rem/10, Rem%10, Digits...> { };
 template <size_t...Digits> struct int_to_str<0, Digits...> {
-    static constexpr auto digits = descr<sizeof...(Digits), 0>({ ('0' + Digits)..., '\0' }, { nullptr });
+    static constexpr auto digits = descr<sizeof...(Digits)>(('0' + Digits)...);
 };
 
 // Ternary description (like std::conditional)
-template <bool B, size_t Size1, size_t Size2>
-constexpr enable_if_t<B, descr<Size1 - 1, 0>> _(char const(&text1)[Size1], char const(&)[Size2]) {
+template <bool B, size_t N1, size_t N2>
+constexpr enable_if_t<B, descr<N1 - 1>> _(char const(&text1)[N1], char const(&)[N2]) {
     return _(text1);
 }
-template <bool B, size_t Size1, size_t Size2>
-constexpr enable_if_t<!B, descr<Size2 - 1, 0>> _(char const(&)[Size1], char const(&text2)[Size2]) {
+template <bool B, size_t N1, size_t N2>
+constexpr enable_if_t<!B, descr<N2 - 1>> _(char const(&)[N1], char const(&text2)[N2]) {
     return _(text2);
 }
-template <bool B, size_t SizeA1, size_t SizeA2, size_t SizeB1, size_t SizeB2>
-constexpr enable_if_t<B, descr<SizeA1, SizeA2>> _(descr<SizeA1, SizeA2> d, descr<SizeB1, SizeB2>) { return d; }
-template <bool B, size_t SizeA1, size_t SizeA2, size_t SizeB1, size_t SizeB2>
-constexpr enable_if_t<!B, descr<SizeB1, SizeB2>> _(descr<SizeA1, SizeA2>, descr<SizeB1, SizeB2> d) { return d; }
+
+template <bool B, typename T1, typename T2>
+constexpr enable_if_t<B, T1> _(const T1 &d, const T2 &) { return d; }
+template <bool B, typename T1, typename T2>
+constexpr enable_if_t<!B, T2> _(const T1 &, const T2 &d) { return d; }
 
 template <size_t Size> auto constexpr _() -> decltype(int_to_str<Size / 10, Size % 10>::digits) {
     return int_to_str<Size / 10, Size % 10>::digits;
 }
 
-template <typename Type> constexpr descr<1, 1> _() {
-    return descr<1, 1>({ '%', '\0' }, { &typeid(Type), nullptr });
-}
+template <typename Type> constexpr descr<1, Type> _() { return {'%'}; }
 
-constexpr descr<0, 0> concat() { return _(""); }
+constexpr descr<0> concat() { return {}; }
 
-template <size_t Size1, size_t Size2>
-constexpr descr<Size1, Size2> concat(descr<Size1, Size2> descr) { return descr; }
+template <size_t N, typename... Ts>
+constexpr descr<N, Ts...> concat(const descr<N, Ts...> &descr) { return descr; }
 
-template <size_t Size1, size_t Size2, typename... Args>
-constexpr auto concat(descr<Size1, Size2> d, Args... args)
-    -> decltype(descr<Size1 + 2, Size2>{} + concat(args...)) {
+template <size_t N, typename... Ts, typename... Args>
+constexpr auto concat(const descr<N, Ts...> &d, const Args &...args)
+    -> decltype(std::declval<descr<N + 2, Ts...>>() + concat(args...)) {
     return d + _(", ") + concat(args...);
 }
 
-template <size_t Size1, size_t Size2>
-constexpr descr<Size1 + 2, Size2> type_descr(descr<Size1, Size2> descr) {
+template <size_t N, typename... Ts>
+constexpr descr<N + 2, Ts...> type_descr(const descr<N, Ts...> &descr) {
     return _("{") + descr + _("}");
 }
 

diff --git a/include/pybind11/numpy.h b/include/pybind11/numpy.h
@@ -941,8 +941,8 @@ template <typename T>
 struct format_descriptor<T, detail::enable_if_t<detail::array_info<T>::is_array>> {
     static std::string format() {
         using detail::_;
-        constexpr auto extents = _("(") + detail::array_info<T>::extents + _(")");
-        return extents.text() + format_descriptor<detail::remove_all_extents_t<T>>::format();
+        static constexpr auto extents = _("(") + detail::array_info<T>::extents + _(")");
+        return extents.text + format_descriptor<detail::remove_all_extents_t<T>>::format();
     }
 };
 

diff --git a/include/pybind11/pybind11.h b/include/pybind11/pybind11.h
@@ -92,7 +92,7 @@ class cpp_function : public function {
     /// Special internal constructor for functors, lambda functions, etc.
     template <typename Func, typename Return, typename... Args, typename... Extra>
     void initialize(Func &&f, Return (*)(Args...), const Extra&... extra) {
-
+        using namespace detail;
         struct capture { detail::remove_reference_t<Func> f; };
 
         /* Store the function including any extra state it might have (e.g. a lambda capture object) */
@@ -163,11 +163,11 @@ class cpp_function : public function {
         detail::process_attributes<Extra...>::init(extra..., rec);
 
         /* Generate a readable signature describing the function's arguments and return value types */
-        using detail::descr; using detail::_;
-        constexpr auto signature = _("(") + cast_in::arg_names + _(") -> ") + cast_out::name;
+        static constexpr auto signature = _("(") + cast_in::arg_names + _(") -> ") + cast_out::name;
+        PYBIND11_DESCR_CONSTEXPR auto types = decltype(signature)::types();
 
         /* Register the function with Python from generic (non-templated) code */
-        initialize_generic(rec, signature.text(), signature.types(), sizeof...(Args));
+        initialize_generic(rec, signature.text, types.data(), sizeof...(Args));
 
         if (cast_in::has_args) rec->has_args = true;
         if (cast_in::has_kwargs) rec->has_kwargs = true;