You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1613 lines
60 KiB

  1. .. _advanced:
  2. Advanced topics
  3. ###############
  4. For brevity, the rest of this chapter assumes that the following two lines are
  5. present:
  6. .. code-block:: cpp
  7. #include <pybind11/pybind11.h>
  8. namespace py = pybind11;
  9. Exporting constants and mutable objects
  10. =======================================
  11. To expose a C++ constant, use the ``attr`` function to register it in a module
  12. as shown below. The ``int_`` class is one of many small wrapper objects defined
  13. in ``pybind11/pytypes.h``. General objects (including integers) can also be
  14. converted using the function ``cast``.
  15. .. code-block:: cpp
  16. PYBIND11_PLUGIN(example) {
  17. py::module m("example", "pybind11 example plugin");
  18. m.attr("MY_CONSTANT") = py::int_(123);
  19. m.attr("MY_CONSTANT_2") = py::cast(new MyObject());
  20. }
  21. Operator overloading
  22. ====================
  23. Suppose that we're given the following ``Vector2`` class with a vector addition
  24. and scalar multiplication operation, all implemented using overloaded operators
  25. in C++.
  26. .. code-block:: cpp
  27. class Vector2 {
  28. public:
  29. Vector2(float x, float y) : x(x), y(y) { }
  30. Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); }
  31. Vector2 operator*(float value) const { return Vector2(x * value, y * value); }
  32. Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; }
  33. Vector2& operator*=(float v) { x *= v; y *= v; return *this; }
  34. friend Vector2 operator*(float f, const Vector2 &v) {
  35. return Vector2(f * v.x, f * v.y);
  36. }
  37. std::string toString() const {
  38. return "[" + std::to_string(x) + ", " + std::to_string(y) + "]";
  39. }
  40. private:
  41. float x, y;
  42. };
  43. The following snippet shows how the above operators can be conveniently exposed
  44. to Python.
  45. .. code-block:: cpp
  46. #include <pybind11/operators.h>
  47. PYBIND11_PLUGIN(example) {
  48. py::module m("example", "pybind11 example plugin");
  49. py::class_<Vector2>(m, "Vector2")
  50. .def(py::init<float, float>())
  51. .def(py::self + py::self)
  52. .def(py::self += py::self)
  53. .def(py::self *= float())
  54. .def(float() * py::self)
  55. .def("__repr__", &Vector2::toString);
  56. return m.ptr();
  57. }
  58. Note that a line like
  59. .. code-block:: cpp
  60. .def(py::self * float())
  61. is really just short hand notation for
  62. .. code-block:: cpp
  63. .def("__mul__", [](const Vector2 &a, float b) {
  64. return a * b;
  65. })
  66. This can be useful for exposing additional operators that don't exist on the
  67. C++ side, or to perform other types of customization.
  68. .. note::
  69. To use the more convenient ``py::self`` notation, the additional
  70. header file :file:`pybind11/operators.h` must be included.
  71. .. seealso::
  72. The file :file:`example/example3.cpp` contains a complete example that
  73. demonstrates how to work with overloaded operators in more detail.
  74. Callbacks and passing anonymous functions
  75. =========================================
  76. The C++11 standard brought lambda functions and the generic polymorphic
  77. function wrapper ``std::function<>`` to the C++ programming language, which
  78. enable powerful new ways of working with functions. Lambda functions come in
  79. two flavors: stateless lambda function resemble classic function pointers that
  80. link to an anonymous piece of code, while stateful lambda functions
  81. additionally depend on captured variables that are stored in an anonymous
  82. *lambda closure object*.
  83. Here is a simple example of a C++ function that takes an arbitrary function
  84. (stateful or stateless) with signature ``int -> int`` as an argument and runs
  85. it with the value 10.
  86. .. code-block:: cpp
  87. int func_arg(const std::function<int(int)> &f) {
  88. return f(10);
  89. }
  90. The example below is more involved: it takes a function of signature ``int -> int``
  91. and returns another function of the same kind. The return value is a stateful
  92. lambda function, which stores the value ``f`` in the capture object and adds 1 to
  93. its return value upon execution.
  94. .. code-block:: cpp
  95. std::function<int(int)> func_ret(const std::function<int(int)> &f) {
  96. return [f](int i) {
  97. return f(i) + 1;
  98. };
  99. }
  100. This example demonstrates using python named parameters in C++ callbacks which
  101. requires using ``py::cpp_function`` as a wrapper. Usage is similar to defining
  102. methods of classes:
  103. .. code-block:: cpp
  104. py::cpp_function func_cpp() {
  105. return py::cpp_function([](int i) { return i+1; },
  106. py::arg("number"));
  107. }
  108. After including the extra header file :file:`pybind11/functional.h`, it is almost
  109. trivial to generate binding code for all of these functions.
  110. .. code-block:: cpp
  111. #include <pybind11/functional.h>
  112. PYBIND11_PLUGIN(example) {
  113. py::module m("example", "pybind11 example plugin");
  114. m.def("func_arg", &func_arg);
  115. m.def("func_ret", &func_ret);
  116. m.def("func_cpp", &func_cpp);
  117. return m.ptr();
  118. }
  119. The following interactive session shows how to call them from Python.
  120. .. code-block:: pycon
  121. $ python
  122. >>> import example
  123. >>> def square(i):
  124. ... return i * i
  125. ...
  126. >>> example.func_arg(square)
  127. 100L
  128. >>> square_plus_1 = example.func_ret(square)
  129. >>> square_plus_1(4)
  130. 17L
  131. >>> plus_1 = func_cpp()
  132. >>> plus_1(number=43)
  133. 44L
  134. .. note::
  135. This functionality is very useful when generating bindings for callbacks in
  136. C++ libraries (e.g. a graphical user interface library).
  137. The file :file:`example/example5.cpp` contains a complete example that
  138. demonstrates how to work with callbacks and anonymous functions in more detail.
  139. .. warning::
  140. Keep in mind that passing a function from C++ to Python (or vice versa)
  141. will instantiate a piece of wrapper code that translates function
  142. invocations between the two languages. Copying the same function back and
  143. forth between Python and C++ many times in a row will cause these wrappers
  144. to accumulate, which can decrease performance.
  145. Overriding virtual functions in Python
  146. ======================================
  147. Suppose that a C++ class or interface has a virtual function that we'd like to
  148. to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is
  149. given as a specific example of how one would do this with traditional C++
  150. code).
  151. .. code-block:: cpp
  152. class Animal {
  153. public:
  154. virtual ~Animal() { }
  155. virtual std::string go(int n_times) = 0;
  156. };
  157. class Dog : public Animal {
  158. public:
  159. std::string go(int n_times) {
  160. std::string result;
  161. for (int i=0; i<n_times; ++i)
  162. result += "woof! ";
  163. return result;
  164. }
  165. };
  166. Let's also suppose that we are given a plain function which calls the
  167. function ``go()`` on an arbitrary ``Animal`` instance.
  168. .. code-block:: cpp
  169. std::string call_go(Animal *animal) {
  170. return animal->go(3);
  171. }
  172. Normally, the binding code for these classes would look as follows:
  173. .. code-block:: cpp
  174. PYBIND11_PLUGIN(example) {
  175. py::module m("example", "pybind11 example plugin");
  176. py::class_<Animal> animal(m, "Animal");
  177. animal
  178. .def("go", &Animal::go);
  179. py::class_<Dog>(m, "Dog", animal)
  180. .def(py::init<>());
  181. m.def("call_go", &call_go);
  182. return m.ptr();
  183. }
  184. However, these bindings are impossible to extend: ``Animal`` is not
  185. constructible, and we clearly require some kind of "trampoline" that
  186. redirects virtual calls back to Python.
  187. Defining a new type of ``Animal`` from within Python is possible but requires a
  188. helper class that is defined as follows:
  189. .. code-block:: cpp
  190. class PyAnimal : public Animal {
  191. public:
  192. /* Inherit the constructors */
  193. using Animal::Animal;
  194. /* Trampoline (need one for each virtual function) */
  195. std::string go(int n_times) {
  196. PYBIND11_OVERLOAD_PURE(
  197. std::string, /* Return type */
  198. Animal, /* Parent class */
  199. go, /* Name of function */
  200. n_times /* Argument(s) */
  201. );
  202. }
  203. };
  204. The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual
  205. functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have
  206. a default implementation.
  207. There are also two alternate macros :func:`PYBIND11_OVERLOAD_PURE_NAME` and
  208. :func:`PYBIND11_OVERLOAD_NAME` which take a string-valued name argument
  209. after the *Name of the function* slot. This is useful when the C++ and Python
  210. versions of the function have different names, e.g. ``operator()`` vs ``__call__``.
  211. The binding code also needs a few minor adaptations (highlighted):
  212. .. code-block:: cpp
  213. :emphasize-lines: 4,6,7
  214. PYBIND11_PLUGIN(example) {
  215. py::module m("example", "pybind11 example plugin");
  216. py::class_<Animal, std::unique_ptr<Animal>, PyAnimal /* <--- trampoline*/> animal(m, "Animal");
  217. animal
  218. .def(py::init<>())
  219. .def("go", &Animal::go);
  220. py::class_<Dog>(m, "Dog", animal)
  221. .def(py::init<>());
  222. m.def("call_go", &call_go);
  223. return m.ptr();
  224. }
  225. Importantly, pybind11 is made aware of the trampoline trampoline helper class
  226. by specifying it as the *third* template argument to :class:`class_`. The
  227. second argument with the unique pointer is simply the default holder type used
  228. by pybind11. Following this, we are able to define a constructor as usual.
  229. The Python session below shows how to override ``Animal::go`` and invoke it via
  230. a virtual method call.
  231. .. code-block:: pycon
  232. >>> from example import *
  233. >>> d = Dog()
  234. >>> call_go(d)
  235. u'woof! woof! woof! '
  236. >>> class Cat(Animal):
  237. ... def go(self, n_times):
  238. ... return "meow! " * n_times
  239. ...
  240. >>> c = Cat()
  241. >>> call_go(c)
  242. u'meow! meow! meow! '
  243. Please take a look at the :ref:`macro_notes` before using this feature.
  244. .. seealso::
  245. The file :file:`example/example12.cpp` contains a complete example that
  246. demonstrates how to override virtual functions using pybind11 in more
  247. detail.
  248. .. _macro_notes:
  249. General notes regarding convenience macros
  250. ==========================================
  251. pybind11 provides a few convenience macros such as
  252. :func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and
  253. ``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated
  254. in the preprocessor (which has no concept of types), they *will* get confused
  255. by commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1,
  256. T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates
  257. the beginnning of the next parameter. Use a ``typedef`` to bind the template to
  258. another name and use it in the macro to avoid this problem.
  259. Global Interpreter Lock (GIL)
  260. =============================
  261. The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be
  262. used to acquire and release the global interpreter lock in the body of a C++
  263. function call. In this way, long-running C++ code can be parallelized using
  264. multiple Python threads. Taking the previous section as an example, this could
  265. be realized as follows (important changes highlighted):
  266. .. code-block:: cpp
  267. :emphasize-lines: 8,9,33,34
  268. class PyAnimal : public Animal {
  269. public:
  270. /* Inherit the constructors */
  271. using Animal::Animal;
  272. /* Trampoline (need one for each virtual function) */
  273. std::string go(int n_times) {
  274. /* Acquire GIL before calling Python code */
  275. py::gil_scoped_acquire acquire;
  276. PYBIND11_OVERLOAD_PURE(
  277. std::string, /* Return type */
  278. Animal, /* Parent class */
  279. go, /* Name of function */
  280. n_times /* Argument(s) */
  281. );
  282. }
  283. };
  284. PYBIND11_PLUGIN(example) {
  285. py::module m("example", "pybind11 example plugin");
  286. py::class_<Animal, std::unique_ptr<Animal>, PyAnimal> animal(m, "Animal");
  287. animal
  288. .def(py::init<>())
  289. .def("go", &Animal::go);
  290. py::class_<Dog>(m, "Dog", animal)
  291. .def(py::init<>());
  292. m.def("call_go", [](Animal *animal) -> std::string {
  293. /* Release GIL before calling into (potentially long-running) C++ code */
  294. py::gil_scoped_release release;
  295. return call_go(animal);
  296. });
  297. return m.ptr();
  298. }
  299. Passing STL data structures
  300. ===========================
  301. When including the additional header file :file:`pybind11/stl.h`, conversions
  302. between ``std::vector<>``, ``std::list<>``, ``std::set<>``, and ``std::map<>``
  303. and the Python ``list``, ``set`` and ``dict`` data structures are automatically
  304. enabled. The types ``std::pair<>`` and ``std::tuple<>`` are already supported
  305. out of the box with just the core :file:`pybind11/pybind11.h` header.
  306. .. note::
  307. Arbitrary nesting of any of these types is supported.
  308. .. seealso::
  309. The file :file:`example/example2.cpp` contains a complete example that
  310. demonstrates how to pass STL data types in more detail.
  311. Binding sequence data types, iterators, the slicing protocol, etc.
  312. ==================================================================
  313. Please refer to the supplemental example for details.
  314. .. seealso::
  315. The file :file:`example/example6.cpp` contains a complete example that
  316. shows how to bind a sequence data type, including length queries
  317. (``__len__``), iterators (``__iter__``), the slicing protocol and other
  318. kinds of useful operations.
  319. Return value policies
  320. =====================
  321. Python and C++ use wildly different ways of managing the memory and lifetime of
  322. objects managed by them. This can lead to issues when creating bindings for
  323. functions that return a non-trivial type. Just by looking at the type
  324. information, it is not clear whether Python should take charge of the returned
  325. value and eventually free its resources, or if this is handled on the C++ side.
  326. For this reason, pybind11 provides a several `return value policy` annotations
  327. that can be passed to the :func:`module::def` and :func:`class_::def`
  328. functions. The default policy is :enum:`return_value_policy::automatic`.
  329. .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
  330. +--------------------------------------------------+----------------------------------------------------------------------------+
  331. | Return value policy | Description |
  332. +==================================================+============================================================================+
  333. | :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
  334. | | :enum:`return_value_policy::take_ownership` when the return value is a |
  335. | | pointer. Otherwise, it uses :enum:`return_value::move` or |
  336. | | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
  337. | | See below for a description of what all of these different policies do. |
  338. +--------------------------------------------------+----------------------------------------------------------------------------+
  339. | :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
  340. | | return value is a pointer. This is the default conversion policy for |
  341. | | function arguments when calling Python functions manually from C++ code |
  342. | | (i.e. via handle::operator()). You probably won't need to use this. |
  343. +--------------------------------------------------+----------------------------------------------------------------------------+
  344. | :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take |
  345. | | ownership. Python will call the destructor and delete operator when the |
  346. | | object's reference count reaches zero. Undefined behavior ensues when the |
  347. | | C++ side does the same. |
  348. +--------------------------------------------------+----------------------------------------------------------------------------+
  349. | :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. |
  350. | | This policy is comparably safe because the lifetimes of the two instances |
  351. | | are decoupled. |
  352. +--------------------------------------------------+----------------------------------------------------------------------------+
  353. | :enum:`return_value_policy::move` | Use ``std::move`` to move the return value contents into a new instance |
  354. | | that will be owned by Python. This policy is comparably safe because the |
  355. | | lifetimes of the two instances (move source and destination) are decoupled.|
  356. +--------------------------------------------------+----------------------------------------------------------------------------+
  357. | :enum:`return_value_policy::reference` | Reference an existing object, but do not take ownership. The C++ side is |
  358. | | responsible for managing the object's lifetime and deallocating it when |
  359. | | it is no longer used. Warning: undefined behavior will ensue when the C++ |
  360. | | side deletes an object that is still referenced and used by Python. |
  361. +--------------------------------------------------+----------------------------------------------------------------------------+
  362. | :enum:`return_value_policy::reference_internal` | This policy only applies to methods and properties. It references the |
  363. | | object without taking ownership similar to the above |
  364. | | :enum:`return_value_policy::reference` policy. In contrast to that policy, |
  365. | | the function or property's implicit ``this`` argument (called the *parent*)|
  366. | | is considered to be the the owner of the return value (the *child*). |
  367. | | pybind11 then couples the lifetime of the parent to the child via a |
  368. | | reference relationship that ensures that the parent cannot be garbage |
  369. | | collected while Python is still using the child. More advanced variations |
  370. | | of this scheme are also possible using combinations of |
  371. | | :enum:`return_value_policy::reference` and the :class:`keep_alive` call |
  372. | | policy described next. |
  373. +--------------------------------------------------+----------------------------------------------------------------------------+
  374. The following example snippet shows a use case of the
  375. :enum:`return_value_policy::reference_internal` policy.
  376. .. code-block:: cpp
  377. class Example {
  378. public:
  379. Internal &get_internal() { return internal; }
  380. private:
  381. Internal internal;
  382. };
  383. PYBIND11_PLUGIN(example) {
  384. py::module m("example", "pybind11 example plugin");
  385. py::class_<Example>(m, "Example")
  386. .def(py::init<>())
  387. .def("get_internal", &Example::get_internal, "Return the internal data",
  388. py::return_value_policy::reference_internal);
  389. return m.ptr();
  390. }
  391. .. warning::
  392. Code with invalid call policies might access unitialized memory or free
  393. data structures multiple times, which can lead to hard-to-debug
  394. non-determinism and segmentation faults, hence it is worth spending the
  395. time to understand all the different options in the table above.
  396. It is worth highlighting one common issue where a method (e.g. a getter)
  397. returns a reference (or pointer) to the first attribute of a class. In this
  398. case, the class and attribute will be located at the same address in
  399. memory, which pybind11 will recongnize and return the parent instance
  400. instead of creating a new Python object that represents the attribute.
  401. Here, the :enum:`return_value_policy::reference_internal` policy should be
  402. used rather than relying on the automatic one.
  403. .. note::
  404. The next section on :ref:`call_policies` discusses *call policies* that can be
  405. specified *in addition* to a return value policy from the list above. Call
  406. policies indicate reference relationships that can involve both return values
  407. and parameters of functions.
  408. .. note::
  409. As an alternative to elaborate call policies and lifetime management logic,
  410. consider using smart pointers (see the section on :ref:`smart_pointers` for
  411. details). Smart pointers can tell whether an object is still referenced from
  412. C++ or Python, which generally eliminates the kinds of inconsistencies that
  413. can lead to crashes or undefined behavior. For functions returning smart
  414. pointers, it is not necessary to specify a return value policy.
  415. .. _call_policies:
  416. Additional call policies
  417. ========================
  418. In addition to the above return value policies, further `call policies` can be
  419. specified to indicate dependencies between parameters. There is currently just
  420. one policy named ``keep_alive<Nurse, Patient>``, which indicates that the
  421. argument with index ``Patient`` should be kept alive at least until the
  422. argument with index ``Nurse`` is freed by the garbage collector; argument
  423. indices start at one, while zero refers to the return value. For methods, index
  424. one refers to the implicit ``this`` pointer, while regular arguments begin at
  425. index two. Arbitrarily many call policies can be specified.
  426. Consider the following example: the binding code for a list append operation
  427. that ties the lifetime of the newly added element to the underlying container
  428. might be declared as follows:
  429. .. code-block:: cpp
  430. py::class_<List>(m, "List")
  431. .def("append", &List::append, py::keep_alive<1, 2>());
  432. .. note::
  433. ``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse,
  434. Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient ==
  435. 0) policies from Boost.Python.
  436. .. seealso::
  437. The file :file:`example/example13.cpp` contains a complete example that
  438. demonstrates using :class:`keep_alive` in more detail.
  439. Implicit type conversions
  440. =========================
  441. Suppose that instances of two types ``A`` and ``B`` are used in a project, and
  442. that an ``A`` can easily be converted into an instance of type ``B`` (examples of this
  443. could be a fixed and an arbitrary precision number type).
  444. .. code-block:: cpp
  445. py::class_<A>(m, "A")
  446. /// ... members ...
  447. py::class_<B>(m, "B")
  448. .def(py::init<A>())
  449. /// ... members ...
  450. m.def("func",
  451. [](const B &) { /* .... */ }
  452. );
  453. To invoke the function ``func`` using a variable ``a`` containing an ``A``
  454. instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++
  455. will automatically apply an implicit type conversion, which makes it possible
  456. to directly write ``func(a)``.
  457. In this situation (i.e. where ``B`` has a constructor that converts from
  458. ``A``), the following statement enables similar implicit conversions on the
  459. Python side:
  460. .. code-block:: cpp
  461. py::implicitly_convertible<A, B>();
  462. .. note::
  463. Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom
  464. data type that is exposed to Python via pybind11.
  465. .. _static_properties:
  466. Static properties
  467. =================
  468. The section on :ref:`properties` discussed the creation of instance properties
  469. that are implemented in terms of C++ getters and setters.
  470. Static properties can also be created in a similar way to expose getters and
  471. setters of static class attributes. It is important to note that the implicit
  472. ``self`` argument also exists in this case and is used to pass the Python
  473. ``type`` subclass instance. This parameter will often not be needed by the C++
  474. side, and the following example illustrates how to instantiate a lambda getter
  475. function that ignores it:
  476. .. code-block:: cpp
  477. py::class_<Foo>(m, "Foo")
  478. .def_property_readonly_static("foo", [](py::object /* self */) { return Foo(); });
  479. Unique pointers
  480. ===============
  481. Given a class ``Example`` with Python bindings, it's possible to return
  482. instances wrapped in C++11 unique pointers, like so
  483. .. code-block:: cpp
  484. std::unique_ptr<Example> create_example() { return std::unique_ptr<Example>(new Example()); }
  485. .. code-block:: cpp
  486. m.def("create_example", &create_example);
  487. In other words, there is nothing special that needs to be done. While returning
  488. unique pointers in this way is allowed, it is *illegal* to use them as function
  489. arguments. For instance, the following function signature cannot be processed
  490. by pybind11.
  491. .. code-block:: cpp
  492. void do_something_with_example(std::unique_ptr<Example> ex) { ... }
  493. The above signature would imply that Python needs to give up ownership of an
  494. object that is passed to this function, which is generally not possible (for
  495. instance, the object might be referenced elsewhere).
  496. .. _smart_pointers:
  497. Smart pointers
  498. ==============
  499. This section explains how to pass values that are wrapped in "smart" pointer
  500. types with internal reference counting. For the simpler C++11 unique pointers,
  501. refer to the previous section.
  502. The binding generator for classes, :class:`class_`, takes an optional second
  503. template type, which denotes a special *holder* type that is used to manage
  504. references to the object. When wrapping a type named ``Type``, the default
  505. value of this template parameter is ``std::unique_ptr<Type>``, which means that
  506. the object is deallocated when Python's reference count goes to zero.
  507. It is possible to switch to other types of reference counting wrappers or smart
  508. pointers, which is useful in codebases that rely on them. For instance, the
  509. following snippet causes ``std::shared_ptr`` to be used instead.
  510. .. code-block:: cpp
  511. py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
  512. Note that any particular class can only be associated with a single holder type.
  513. To enable transparent conversions for functions that take shared pointers as an
  514. argument or that return them, a macro invocation similar to the following must
  515. be declared at the top level before any binding code:
  516. .. code-block:: cpp
  517. PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
  518. .. note::
  519. The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
  520. placeholder name that is used as a template parameter of the second
  521. argument. Thus, feel free to use any identifier, but use it consistently on
  522. both sides; also, don't use the name of a type that already exists in your
  523. codebase.
  524. One potential stumbling block when using holder types is that they need to be
  525. applied consistently. Can you guess what's broken about the following binding
  526. code?
  527. .. code-block:: cpp
  528. class Child { };
  529. class Parent {
  530. public:
  531. Parent() : child(std::make_shared<Child>()) { }
  532. Child *get_child() { return child.get(); } /* Hint: ** DON'T DO THIS ** */
  533. private:
  534. std::shared_ptr<Child> child;
  535. };
  536. PYBIND11_PLUGIN(example) {
  537. py::module m("example");
  538. py::class_<Child, std::shared_ptr<Child>>(m, "Child");
  539. py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
  540. .def(py::init<>())
  541. .def("get_child", &Parent::get_child);
  542. return m.ptr();
  543. }
  544. The following Python code will cause undefined behavior (and likely a
  545. segmentation fault).
  546. .. code-block:: python
  547. from example import Parent
  548. print(Parent().get_child())
  549. The problem is that ``Parent::get_child()`` returns a pointer to an instance of
  550. ``Child``, but the fact that this instance is already managed by
  551. ``std::shared_ptr<...>`` is lost when passing raw pointers. In this case,
  552. pybind11 will create a second independent ``std::shared_ptr<...>`` that also
  553. claims ownership of the pointer. In the end, the object will be freed **twice**
  554. since these shared pointers have no way of knowing about each other.
  555. There are two ways to resolve this issue:
  556. 1. For types that are managed by a smart pointer class, never use raw pointers
  557. in function arguments or return values. In other words: always consistently
  558. wrap pointers into their designated holder types (such as
  559. ``std::shared_ptr<...>``). In this case, the signature of ``get_child()``
  560. should be modified as follows:
  561. .. code-block:: cpp
  562. std::shared_ptr<Child> get_child() { return child; }
  563. 2. Adjust the definition of ``Child`` by specifying
  564. ``std::enable_shared_from_this<T>`` (see cppreference_ for details) as a
  565. base class. This adds a small bit of information to ``Child`` that allows
  566. pybind11 to realize that there is already an existing
  567. ``std::shared_ptr<...>`` and communicate with it. In this case, the
  568. declaration of ``Child`` should look as follows:
  569. .. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
  570. .. code-block:: cpp
  571. class Child : public std::enable_shared_from_this<Child> { };
  572. Please take a look at the :ref:`macro_notes` before using this feature.
  573. .. seealso::
  574. The file :file:`example/example8.cpp` contains a complete example that
  575. demonstrates how to work with custom reference-counting holder types in
  576. more detail.
  577. .. _custom_constructors:
  578. Custom constructors
  579. ===================
  580. The syntax for binding constructors was previously introduced, but it only
  581. works when a constructor with the given parameters actually exists on the C++
  582. side. To extend this to more general cases, let's take a look at what actually
  583. happens under the hood: the following statement
  584. .. code-block:: cpp
  585. py::class_<Example>(m, "Example")
  586. .def(py::init<int>());
  587. is short hand notation for
  588. .. code-block:: cpp
  589. py::class_<Example>(m, "Example")
  590. .def("__init__",
  591. [](Example &instance, int arg) {
  592. new (&instance) Example(arg);
  593. }
  594. );
  595. In other words, :func:`init` creates an anonymous function that invokes an
  596. in-place constructor. Memory allocation etc. is already take care of beforehand
  597. within pybind11.
  598. Catching and throwing exceptions
  599. ================================
  600. When C++ code invoked from Python throws an ``std::exception``, it is
  601. automatically converted into a Python ``Exception``. pybind11 defines multiple
  602. special exception classes that will map to different types of Python
  603. exceptions:
  604. .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
  605. +--------------------------------------+------------------------------+
  606. | C++ exception type | Python exception type |
  607. +======================================+==============================+
  608. | :class:`std::exception` | ``RuntimeError`` |
  609. +--------------------------------------+------------------------------+
  610. | :class:`std::bad_alloc` | ``MemoryError`` |
  611. +--------------------------------------+------------------------------+
  612. | :class:`std::domain_error` | ``ValueError`` |
  613. +--------------------------------------+------------------------------+
  614. | :class:`std::invalid_argument` | ``ValueError`` |
  615. +--------------------------------------+------------------------------+
  616. | :class:`std::length_error` | ``ValueError`` |
  617. +--------------------------------------+------------------------------+
  618. | :class:`std::out_of_range` | ``ValueError`` |
  619. +--------------------------------------+------------------------------+
  620. | :class:`std::range_error` | ``ValueError`` |
  621. +--------------------------------------+------------------------------+
  622. | :class:`pybind11::stop_iteration` | ``StopIteration`` (used to |
  623. | | implement custom iterators) |
  624. +--------------------------------------+------------------------------+
  625. | :class:`pybind11::index_error` | ``IndexError`` (used to |
  626. | | indicate out of bounds |
  627. | | accesses in ``__getitem__``, |
  628. | | ``__setitem__``, etc.) |
  629. +--------------------------------------+------------------------------+
  630. | :class:`pybind11::value_error` | ``ValueError`` (used to |
  631. | | indicate wrong value passed |
  632. | | in ``container.remove(...)`` |
  633. +--------------------------------------+------------------------------+
  634. | :class:`pybind11::error_already_set` | Indicates that the Python |
  635. | | exception flag has already |
  636. | | been initialized |
  637. +--------------------------------------+------------------------------+
  638. When a Python function invoked from C++ throws an exception, it is converted
  639. into a C++ exception of type :class:`error_already_set` whose string payload
  640. contains a textual summary.
  641. There is also a special exception :class:`cast_error` that is thrown by
  642. :func:`handle::call` when the input arguments cannot be converted to Python
  643. objects.
  644. .. _opaque:
  645. Treating STL data structures as opaque objects
  646. ==============================================
  647. pybind11 heavily relies on a template matching mechanism to convert parameters
  648. and return values that are constructed from STL data types such as vectors,
  649. linked lists, hash tables, etc. This even works in a recursive manner, for
  650. instance to deal with lists of hash maps of pairs of elementary and custom
  651. types, etc.
  652. However, a fundamental limitation of this approach is that internal conversions
  653. between Python and C++ types involve a copy operation that prevents
  654. pass-by-reference semantics. What does this mean?
  655. Suppose we bind the following function
  656. .. code-block:: cpp
  657. void append_1(std::vector<int> &v) {
  658. v.push_back(1);
  659. }
  660. and call it from Python, the following happens:
  661. .. code-block:: pycon
  662. >>> v = [5, 6]
  663. >>> append_1(v)
  664. >>> print(v)
  665. [5, 6]
  666. As you can see, when passing STL data structures by reference, modifications
  667. are not propagated back the Python side. A similar situation arises when
  668. exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
  669. functions:
  670. .. code-block:: cpp
  671. /* ... definition ... */
  672. class MyClass {
  673. std::vector<int> contents;
  674. };
  675. /* ... binding code ... */
  676. py::class_<MyClass>(m, "MyClass")
  677. .def(py::init<>)
  678. .def_readwrite("contents", &MyClass::contents);
  679. In this case, properties can be read and written in their entirety. However, an
  680. ``append`` operaton involving such a list type has no effect:
  681. .. code-block:: pycon
  682. >>> m = MyClass()
  683. >>> m.contents = [5, 6]
  684. >>> print(m.contents)
  685. [5, 6]
  686. >>> m.contents.append(7)
  687. >>> print(m.contents)
  688. [5, 6]
  689. To deal with both of the above situations, pybind11 provides a macro named
  690. ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based conversion
  691. machinery of types, thus rendering them *opaque*. The contents of opaque
  692. objects are never inspected or extracted, hence they can be passed by
  693. reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
  694. the declaration
  695. .. code-block:: cpp
  696. PYBIND11_MAKE_OPAQUE(std::vector<int>);
  697. before any binding code (e.g. invocations to ``class_::def()``, etc.). This
  698. macro must be specified at the top level, since instantiates a partial template
  699. overload. If your binding code consists of multiple compilation units, it must
  700. be present in every file preceding any usage of ``std::vector<int>``. Opaque
  701. types must also have a corresponding ``class_`` declaration to associate them
  702. with a name in Python, and to define a set of available operations:
  703. .. code-block:: cpp
  704. py::class_<std::vector<int>>(m, "IntVector")
  705. .def(py::init<>())
  706. .def("clear", &std::vector<int>::clear)
  707. .def("pop_back", &std::vector<int>::pop_back)
  708. .def("__len__", [](const std::vector<int> &v) { return v.size(); })
  709. .def("__iter__", [](std::vector<int> &v) {
  710. return py::make_iterator(v.begin(), v.end());
  711. }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
  712. // ....
  713. Please take a look at the :ref:`macro_notes` before using this feature.
  714. .. seealso::
  715. The file :file:`example/example14.cpp` contains a complete example that
  716. demonstrates how to create and expose opaque types using pybind11 in more
  717. detail.
  718. .. _eigen:
  719. Transparent conversion of dense and sparse Eigen data types
  720. ===========================================================
  721. Eigen [#f1]_ is C++ header-based library for dense and sparse linear algebra. Due to
  722. its popularity and widespread adoption, pybind11 provides transparent
  723. conversion support between Eigen and Scientific Python linear algebra data types.
  724. Specifically, when including the optional header file :file:`pybind11/eigen.h`,
  725. pybind11 will automatically and transparently convert
  726. 1. Static and dynamic Eigen dense vectors and matrices to instances of
  727. ``numpy.ndarray`` (and vice versa).
  728. 1. Eigen sparse vectors and matrices to instances of
  729. ``scipy.sparse.csr_matrix``/``scipy.sparse.csc_matrix`` (and vice versa).
  730. This makes it possible to bind most kinds of functions that rely on these types.
  731. One major caveat are functions that take Eigen matrices *by reference* and modify
  732. them somehow, in which case the information won't be propagated to the caller.
  733. .. code-block:: cpp
  734. /* The Python bindings of this function won't replicate
  735. the intended effect of modifying the function argument */
  736. void scale_by_2(Eigen::Vector3f &v) {
  737. v *= 2;
  738. }
  739. To see why this is, refer to the section on :ref:`opaque` (although that
  740. section specifically covers STL data types, the underlying issue is the same).
  741. The next two sections discuss an efficient alternative for exposing the
  742. underlying native Eigen types as opaque objects in a way that still integrates
  743. with NumPy and SciPy.
  744. .. [#f1] http://eigen.tuxfamily.org
  745. .. seealso::
  746. The file :file:`example/eigen.cpp` contains a complete example that
  747. shows how to pass Eigen sparse and dense data types in more detail.
  748. Buffer protocol
  749. ===============
  750. Python supports an extremely general and convenient approach for exchanging
  751. data between plugin libraries. Types can expose a buffer view [#f2]_, which
  752. provides fast direct access to the raw internal data representation. Suppose we
  753. want to bind the following simplistic Matrix class:
  754. .. code-block:: cpp
  755. class Matrix {
  756. public:
  757. Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) {
  758. m_data = new float[rows*cols];
  759. }
  760. float *data() { return m_data; }
  761. size_t rows() const { return m_rows; }
  762. size_t cols() const { return m_cols; }
  763. private:
  764. size_t m_rows, m_cols;
  765. float *m_data;
  766. };
  767. The following binding code exposes the ``Matrix`` contents as a buffer object,
  768. making it possible to cast Matrices into NumPy arrays. It is even possible to
  769. completely avoid copy operations with Python expressions like
  770. ``np.array(matrix_instance, copy = False)``.
  771. .. code-block:: cpp
  772. py::class_<Matrix>(m, "Matrix")
  773. .def_buffer([](Matrix &m) -> py::buffer_info {
  774. return py::buffer_info(
  775. m.data(), /* Pointer to buffer */
  776. sizeof(float), /* Size of one scalar */
  777. py::format_descriptor<float>::value, /* Python struct-style format descriptor */
  778. 2, /* Number of dimensions */
  779. { m.rows(), m.cols() }, /* Buffer dimensions */
  780. { sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
  781. sizeof(float) }
  782. );
  783. });
  784. The snippet above binds a lambda function, which can create ``py::buffer_info``
  785. description records on demand describing a given matrix. The contents of
  786. ``py::buffer_info`` mirror the Python buffer protocol specification.
  787. .. code-block:: cpp
  788. struct buffer_info {
  789. void *ptr;
  790. size_t itemsize;
  791. std::string format;
  792. int ndim;
  793. std::vector<size_t> shape;
  794. std::vector<size_t> strides;
  795. };
  796. To create a C++ function that can take a Python buffer object as an argument,
  797. simply use the type ``py::buffer`` as one of its arguments. Buffers can exist
  798. in a great variety of configurations, hence some safety checks are usually
  799. necessary in the function body. Below, you can see an basic example on how to
  800. define a custom constructor for the Eigen double precision matrix
  801. (``Eigen::MatrixXd``) type, which supports initialization from compatible
  802. buffer objects (e.g. a NumPy matrix).
  803. .. code-block:: cpp
  804. /* Bind MatrixXd (or some other Eigen type) to Python */
  805. typedef Eigen::MatrixXd Matrix;
  806. typedef Matrix::Scalar Scalar;
  807. constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
  808. py::class_<Matrix>(m, "Matrix")
  809. .def("__init__", [](Matrix &m, py::buffer b) {
  810. typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
  811. /* Request a buffer descriptor from Python */
  812. py::buffer_info info = b.request();
  813. /* Some sanity checks ... */
  814. if (info.format != py::format_descriptor<Scalar>::value)
  815. throw std::runtime_error("Incompatible format: expected a double array!");
  816. if (info.ndim != 2)
  817. throw std::runtime_error("Incompatible buffer dimension!");
  818. auto strides = Strides(
  819. info.strides[rowMajor ? 0 : 1] / sizeof(Scalar),
  820. info.strides[rowMajor ? 1 : 0] / sizeof(Scalar));
  821. auto map = Eigen::Map<Matrix, 0, Strides>(
  822. static_cat<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides);
  823. new (&m) Matrix(map);
  824. });
  825. For reference, the ``def_buffer()`` call for this Eigen data type should look
  826. as follows:
  827. .. code-block:: cpp
  828. .def_buffer([](Matrix &m) -> py::buffer_info {
  829. return py::buffer_info(
  830. m.data(), /* Pointer to buffer */
  831. sizeof(Scalar), /* Size of one scalar */
  832. /* Python struct-style format descriptor */
  833. py::format_descriptor<Scalar>::value,
  834. /* Number of dimensions */
  835. 2,
  836. /* Buffer dimensions */
  837. { (size_t) m.rows(),
  838. (size_t) m.cols() },
  839. /* Strides (in bytes) for each index */
  840. { sizeof(Scalar) * (rowMajor ? m.cols() : 1),
  841. sizeof(Scalar) * (rowMajor ? 1 : m.rows()) }
  842. );
  843. })
  844. For a much easier approach of binding Eigen types (although with some
  845. limitations), refer to the section on :ref:`eigen`.
  846. .. seealso::
  847. The file :file:`example/example7.cpp` contains a complete example that
  848. demonstrates using the buffer protocol with pybind11 in more detail.
  849. .. [#f2] http://docs.python.org/3/c-api/buffer.html
  850. NumPy support
  851. =============
  852. By exchanging ``py::buffer`` with ``py::array`` in the above snippet, we can
  853. restrict the function so that it only accepts NumPy arrays (rather than any
  854. type of Python object satisfying the buffer protocol).
  855. In many situations, we want to define a function which only accepts a NumPy
  856. array of a certain data type. This is possible via the ``py::array_t<T>``
  857. template. For instance, the following function requires the argument to be a
  858. NumPy array containing double precision values.
  859. .. code-block:: cpp
  860. void f(py::array_t<double> array);
  861. When it is invoked with a different type (e.g. an integer or a list of
  862. integers), the binding code will attempt to cast the input into a NumPy array
  863. of the requested type. Note that this feature requires the
  864. :file:``pybind11/numpy.h`` header to be included.
  865. Data in NumPy arrays is not guaranteed to packed in a dense manner;
  866. furthermore, entries can be separated by arbitrary column and row strides.
  867. Sometimes, it can be useful to require a function to only accept dense arrays
  868. using either the C (row-major) or Fortran (column-major) ordering. This can be
  869. accomplished via a second template argument with values ``py::array::c_style``
  870. or ``py::array::f_style``.
  871. .. code-block:: cpp
  872. void f(py::array_t<double, py::array::c_style | py::array::forcecast> array);
  873. The ``py::array::forcecast`` argument is the default value of the second
  874. template paramenter, and it ensures that non-conforming arguments are converted
  875. into an array satisfying the specified requirements instead of trying the next
  876. function overload.
  877. Vectorizing functions
  878. =====================
  879. Suppose we want to bind a function with the following signature to Python so
  880. that it can process arbitrary NumPy array arguments (vectors, matrices, general
  881. N-D arrays) in addition to its normal arguments:
  882. .. code-block:: cpp
  883. double my_func(int x, float y, double z);
  884. After including the ``pybind11/numpy.h`` header, this is extremely simple:
  885. .. code-block:: cpp
  886. m.def("vectorized_func", py::vectorize(my_func));
  887. Invoking the function like below causes 4 calls to be made to ``my_func`` with
  888. each of the array elements. The significant advantage of this compared to
  889. solutions like ``numpy.vectorize()`` is that the loop over the elements runs
  890. entirely on the C++ side and can be crunched down into a tight, optimized loop
  891. by the compiler. The result is returned as a NumPy array of type
  892. ``numpy.dtype.float64``.
  893. .. code-block:: pycon
  894. >>> x = np.array([[1, 3],[5, 7]])
  895. >>> y = np.array([[2, 4],[6, 8]])
  896. >>> z = 3
  897. >>> result = vectorized_func(x, y, z)
  898. The scalar argument ``z`` is transparently replicated 4 times. The input
  899. arrays ``x`` and ``y`` are automatically converted into the right types (they
  900. are of type ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and
  901. ``numpy.dtype.float32``, respectively)
  902. Sometimes we might want to explicitly exclude an argument from the vectorization
  903. because it makes little sense to wrap it in a NumPy array. For instance,
  904. suppose the function signature was
  905. .. code-block:: cpp
  906. double my_func(int x, float y, my_custom_type *z);
  907. This can be done with a stateful Lambda closure:
  908. .. code-block:: cpp
  909. // Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization)
  910. m.def("vectorized_func",
  911. [](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) {
  912. auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); };
  913. return py::vectorize(stateful_closure)(x, y);
  914. }
  915. );
  916. In cases where the computation is too complicated to be reduced to
  917. ``vectorize``, it will be necessary to create and access the buffer contents
  918. manually. The following snippet contains a complete example that shows how this
  919. works (the code is somewhat contrived, since it could have been done more
  920. simply using ``vectorize``).
  921. .. code-block:: cpp
  922. #include <pybind11/pybind11.h>
  923. #include <pybind11/numpy.h>
  924. namespace py = pybind11;
  925. py::array_t<double> add_arrays(py::array_t<double> input1, py::array_t<double> input2) {
  926. auto buf1 = input1.request(), buf2 = input2.request();
  927. if (buf1.ndim != 1 || buf2.ndim != 1)
  928. throw std::runtime_error("Number of dimensions must be one");
  929. if (buf1.shape[0] != buf2.shape[0])
  930. throw std::runtime_error("Input shapes must match");
  931. auto result = py::array(py::buffer_info(
  932. nullptr, /* Pointer to data (nullptr -> ask NumPy to allocate!) */
  933. sizeof(double), /* Size of one item */
  934. py::format_descriptor<double>::value(), /* Buffer format */
  935. buf1.ndim, /* How many dimensions? */
  936. { buf1.shape[0] }, /* Number of elements for each dimension */
  937. { sizeof(double) } /* Strides for each dimension */
  938. ));
  939. auto buf3 = result.request();
  940. double *ptr1 = (double *) buf1.ptr,
  941. *ptr2 = (double *) buf2.ptr,
  942. *ptr3 = (double *) buf3.ptr;
  943. for (size_t idx = 0; idx < buf1.shape[0]; idx++)
  944. ptr3[idx] = ptr1[idx] + ptr2[idx];
  945. return result;
  946. }
  947. PYBIND11_PLUGIN(test) {
  948. py::module m("test");
  949. m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
  950. return m.ptr();
  951. }
  952. .. seealso::
  953. The file :file:`example/example10.cpp` contains a complete example that
  954. demonstrates using :func:`vectorize` in more detail.
  955. Functions taking Python objects as arguments
  956. ============================================
  957. pybind11 exposes all major Python types using thin C++ wrapper classes. These
  958. wrapper classes can also be used as parameters of functions in bindings, which
  959. makes it possible to directly work with native Python types on the C++ side.
  960. For instance, the following statement iterates over a Python ``dict``:
  961. .. code-block:: cpp
  962. void print_dict(py::dict dict) {
  963. /* Easily interact with Python types */
  964. for (auto item : dict)
  965. std::cout << "key=" << item.first << ", "
  966. << "value=" << item.second << std::endl;
  967. }
  968. Available types include :class:`handle`, :class:`object`, :class:`bool_`,
  969. :class:`int_`, :class:`float_`, :class:`str`, :class:`bytes`, :class:`tuple`,
  970. :class:`list`, :class:`dict`, :class:`slice`, :class:`none`, :class:`capsule`,
  971. :class:`iterable`, :class:`iterator`, :class:`function`, :class:`buffer`,
  972. :class:`array`, and :class:`array_t`.
  973. In this kind of mixed code, it is often necessary to convert arbitrary C++
  974. types to Python, which can be done using :func:`cast`:
  975. .. code-block:: cpp
  976. MyClass *cls = ..;
  977. py::object obj = py::cast(cls);
  978. The reverse direction uses the following syntax:
  979. .. code-block:: cpp
  980. py::object obj = ...;
  981. MyClass *cls = obj.cast<MyClass *>();
  982. When conversion fails, both directions throw the exception :class:`cast_error`.
  983. It is also possible to call python functions via ``operator()``.
  984. .. code-block:: cpp
  985. py::function f = <...>;
  986. py::object result_py = f(1234, "hello", some_instance);
  987. MyClass &result = result_py.cast<MyClass>();
  988. The special ``f(*args)`` and ``f(*args, **kwargs)`` syntax is also supported to
  989. supply arbitrary argument and keyword lists, although these cannot be mixed
  990. with other parameters.
  991. .. code-block:: cpp
  992. py::function f = <...>;
  993. py::tuple args = py::make_tuple(1234);
  994. py::dict kwargs;
  995. kwargs["y"] = py::cast(5678);
  996. py::object result = f(*args, **kwargs);
  997. .. seealso::
  998. The file :file:`example/example2.cpp` contains a complete example that
  999. demonstrates passing native Python types in more detail. The file
  1000. :file:`example/example11.cpp` discusses usage of ``args`` and ``kwargs``.
  1001. Default arguments revisited
  1002. ===========================
  1003. The section on :ref:`default_args` previously discussed basic usage of default
  1004. arguments using pybind11. One noteworthy aspect of their implementation is that
  1005. default arguments are converted to Python objects right at declaration time.
  1006. Consider the following example:
  1007. .. code-block:: cpp
  1008. py::class_<MyClass>("MyClass")
  1009. .def("myFunction", py::arg("arg") = SomeType(123));
  1010. In this case, pybind11 must already be set up to deal with values of the type
  1011. ``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an
  1012. exception will be thrown.
  1013. Another aspect worth highlighting is that the "preview" of the default argument
  1014. in the function signature is generated using the object's ``__repr__`` method.
  1015. If not available, the signature may not be very helpful, e.g.:
  1016. .. code-block:: pycon
  1017. FUNCTIONS
  1018. ...
  1019. | myFunction(...)
  1020. | Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> NoneType
  1021. ...
  1022. The first way of addressing this is by defining ``SomeType.__repr__``.
  1023. Alternatively, it is possible to specify the human-readable preview of the
  1024. default argument manually using the ``arg_t`` notation:
  1025. .. code-block:: cpp
  1026. py::class_<MyClass>("MyClass")
  1027. .def("myFunction", py::arg_t<SomeType>("arg", SomeType(123), "SomeType(123)"));
  1028. Sometimes it may be necessary to pass a null pointer value as a default
  1029. argument. In this case, remember to cast it to the underlying type in question,
  1030. like so:
  1031. .. code-block:: cpp
  1032. py::class_<MyClass>("MyClass")
  1033. .def("myFunction", py::arg("arg") = (SomeType *) nullptr);
  1034. Binding functions that accept arbitrary numbers of arguments and keywords arguments
  1035. ===================================================================================
  1036. Python provides a useful mechanism to define functions that accept arbitrary
  1037. numbers of arguments and keyword arguments:
  1038. .. code-block:: cpp
  1039. def generic(*args, **kwargs):
  1040. # .. do something with args and kwargs
  1041. Such functions can also be created using pybind11:
  1042. .. code-block:: cpp
  1043. void generic(py::args args, py::kwargs kwargs) {
  1044. /// .. do something with args
  1045. if (kwargs)
  1046. /// .. do something with kwargs
  1047. }
  1048. /// Binding code
  1049. m.def("generic", &generic);
  1050. (See ``example/example11.cpp``). The class ``py::args`` derives from
  1051. ``py::list`` and ``py::kwargs`` derives from ``py::dict`` Note that the
  1052. ``kwargs`` argument is invalid if no keyword arguments were actually provided.
  1053. Please refer to the other examples for details on how to iterate over these,
  1054. and on how to cast their entries into C++ objects.
  1055. Partitioning code over multiple extension modules
  1056. =================================================
  1057. It's straightforward to split binding code over multiple extension modules,
  1058. while referencing types that are declared elsewhere. Everything "just" works
  1059. without any special precautions. One exception to this rule occurs when
  1060. extending a type declared in another extension module. Recall the basic example
  1061. from Section :ref:`inheritance`.
  1062. .. code-block:: cpp
  1063. py::class_<Pet> pet(m, "Pet");
  1064. pet.def(py::init<const std::string &>())
  1065. .def_readwrite("name", &Pet::name);
  1066. py::class_<Dog>(m, "Dog", pet /* <- specify parent */)
  1067. .def(py::init<const std::string &>())
  1068. .def("bark", &Dog::bark);
  1069. Suppose now that ``Pet`` bindings are defined in a module named ``basic``,
  1070. whereas the ``Dog`` bindings are defined somewhere else. The challenge is of
  1071. course that the variable ``pet`` is not available anymore though it is needed
  1072. to indicate the inheritance relationship to the constructor of ``class_<Dog>``.
  1073. However, it can be acquired as follows:
  1074. .. code-block:: cpp
  1075. py::object pet = (py::object) py::module::import("basic").attr("Pet");
  1076. py::class_<Dog>(m, "Dog", pet)
  1077. .def(py::init<const std::string &>())
  1078. .def("bark", &Dog::bark);
  1079. Alternatively, we can rely on the ``base`` tag, which performs an automated
  1080. lookup of the corresponding Python type. However, this also requires invoking
  1081. the ``import`` function once to ensure that the pybind11 binding code of the
  1082. module ``basic`` has been executed.
  1083. .. code-block:: cpp
  1084. py::module::import("basic");
  1085. py::class_<Dog>(m, "Dog", py::base<Pet>())
  1086. .def(py::init<const std::string &>())
  1087. .def("bark", &Dog::bark);
  1088. Naturally, both methods will fail when there are cyclic dependencies.
  1089. Note that compiling code which has its default symbol visibility set to
  1090. *hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the
  1091. ability to access types defined in another extension module. Workarounds
  1092. include changing the global symbol visibility (not recommended, because it will
  1093. lead unnecessarily large binaries) or manually exporting types that are
  1094. accessed by multiple extension modules:
  1095. .. code-block:: cpp
  1096. #ifdef _WIN32
  1097. # define EXPORT_TYPE __declspec(dllexport)
  1098. #else
  1099. # define EXPORT_TYPE __attribute__ ((visibility("default")))
  1100. #endif
  1101. class EXPORT_TYPE Dog : public Animal {
  1102. ...
  1103. };
  1104. Pickling support
  1105. ================
  1106. Python's ``pickle`` module provides a powerful facility to serialize and
  1107. de-serialize a Python object graph into a binary data stream. To pickle and
  1108. unpickle C++ classes using pybind11, two additional functions must be provided.
  1109. Suppose the class in question has the following signature:
  1110. .. code-block:: cpp
  1111. class Pickleable {
  1112. public:
  1113. Pickleable(const std::string &value) : m_value(value) { }
  1114. const std::string &value() const { return m_value; }
  1115. void setExtra(int extra) { m_extra = extra; }
  1116. int extra() const { return m_extra; }
  1117. private:
  1118. std::string m_value;
  1119. int m_extra = 0;
  1120. };
  1121. The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_
  1122. looks as follows:
  1123. .. code-block:: cpp
  1124. py::class_<Pickleable>(m, "Pickleable")
  1125. .def(py::init<std::string>())
  1126. .def("value", &Pickleable::value)
  1127. .def("extra", &Pickleable::extra)
  1128. .def("setExtra", &Pickleable::setExtra)
  1129. .def("__getstate__", [](const Pickleable &p) {
  1130. /* Return a tuple that fully encodes the state of the object */
  1131. return py::make_tuple(p.value(), p.extra());
  1132. })
  1133. .def("__setstate__", [](Pickleable &p, py::tuple t) {
  1134. if (t.size() != 2)
  1135. throw std::runtime_error("Invalid state!");
  1136. /* Invoke the in-place constructor. Note that this is needed even
  1137. when the object just has a trivial default constructor */
  1138. new (&p) Pickleable(t[0].cast<std::string>());
  1139. /* Assign any additional state */
  1140. p.setExtra(t[1].cast<int>());
  1141. });
  1142. An instance can now be pickled as follows:
  1143. .. code-block:: python
  1144. try:
  1145. import cPickle as pickle # Use cPickle on Python 2.7
  1146. except ImportError:
  1147. import pickle
  1148. p = Pickleable("test_value")
  1149. p.setExtra(15)
  1150. data = pickle.dumps(p, 2)
  1151. Note that only the cPickle module is supported on Python 2.7. The second
  1152. argument to ``dumps`` is also crucial: it selects the pickle protocol version
  1153. 2, since the older version 1 is not supported. Newer versions are also fine—for
  1154. instance, specify ``-1`` to always use the latest available version. Beware:
  1155. failure to follow these instructions will cause important pybind11 memory
  1156. allocation routines to be skipped during unpickling, which will likely lead to
  1157. memory corruption and/or segmentation faults.
  1158. .. seealso::
  1159. The file :file:`example/example15.cpp` contains a complete example that
  1160. demonstrates how to pickle and unpickle types using pybind11 in more detail.
  1161. .. [#f3] http://docs.python.org/3/library/pickle.html#pickling-class-instances
  1162. Generating documentation using Sphinx
  1163. =====================================
  1164. Sphinx [#f4]_ has the ability to inspect the signatures and documentation
  1165. strings in pybind11-based extension modules to automatically generate beautiful
  1166. documentation in a variety formats. The python_example repository [#f5]_ contains a
  1167. simple example repository which uses this approach.
  1168. There are two potential gotchas when using this approach: first, make sure that
  1169. the resulting strings do not contain any :kbd:`TAB` characters, which break the
  1170. docstring parsing routines. You may want to use C++11 raw string literals,
  1171. which are convenient for multi-line comments. Conveniently, any excess
  1172. indentation will be automatically be removed by Sphinx. However, for this to
  1173. work, it is important that all lines are indented consistently, i.e.:
  1174. .. code-block:: cpp
  1175. // ok
  1176. m.def("foo", &foo, R"mydelimiter(
  1177. The foo function
  1178. Parameters
  1179. ----------
  1180. )mydelimiter");
  1181. // *not ok*
  1182. m.def("foo", &foo, R"mydelimiter(The foo function
  1183. Parameters
  1184. ----------
  1185. )mydelimiter");
  1186. .. [#f4] http://www.sphinx-doc.org
  1187. .. [#f5] http://github.com/pybind/python_example