在加速的Python包装阵列 [英] Wrapping arrays in Boost Python
问题描述
我有一系列C ++的结构我想使用boost蟒蛇来包装。我碰到困难时，这些结构包含数组。我想以最小的开销要做到这一点，可惜我不能给自己结构的任何修改。因此，例如说我有
I have a series of C++ structures I am trying to wrap using boost python. I've run into difficulties when these structures contain arrays. I am trying to do this with minimal overhead and unfortunately I can't make any modifications to the structs themselves. So for instance say I have
struct Foo
{
int vals[3];
};
我希望能够访问这个在python如下：
I would like to be able to access this in python as follows:
f = Foo()
f.vals[0] = 10
print f.vals[0]
现在我使用的是一系列的get / set功能，它的作品，但很不雅观，不符合访问其他非阵列成员。这是我目前的解决办法：
Right now I am using a series of get/set functions which works but is very inelegant and inconsistent with accessing the other non-array members. Here is my current solution:
int getVals (Foo f, int index) { return f.vals[index]; }
void setVals (Foo& f, int index, int value) { f.vals[index] = value; }
boost::python::class_< Foo > ( "Foo", init<>() )
.def ( "getVals", &getVals )
.def ( "setVals", &setVals );
我很好的具有get / set函数（因为有某些情况下，我需要实现一个自定义的获取或设置操作），但我不知道如何把[]操作符来访问数组的元素。在其他类，它本身与[]操作符访问我已经能够使用 _ 的GetItem _ 和 _ setitem _ 已完美地工作，但我不知道我会怎么做这与类成员如果这甚至有可能。
I am fine with having the get/set functions (as there are certain cases where I need to implement a custom get or set operation) but I am not sure how to incorporate the [] operator to access the elements of the array. In other classes which themselves are accessible with the [] operator I have been able to use _getitem_ and _setitem_ which have worked perfectly, but I'm not sure how I would do this with class members if that would even be possible.
推荐答案
对于这样一个相对简单的问题，答案就相当复杂。提供解决方案之前，让我们先检查问题的深度：
For such a relatively simple question, the answer becomes rather involved. Before providing the solution, lets first examine the depth of the problem:
f = Foo()
f.vals[0] = 10
f.vals 返回一个中间对象，它提供 __的GetItem __ 和 __ setitem __ 方法。为Boost.Python的支持此，辅助类型将需要暴露为每个类型的数组，并且这些类型将提供索引支持
f.vals returns an intermediate object that provides __getitem__ and __setitem__ methods. For Boost.Python to support this, auxiliary types will need to be exposed for each type of array, and these types will provide indexing support.
语言之间的一个细微差别是对象的生命周期。考虑以下几点：
One subtle difference between the languages is object lifetime. Consider the following:
f = Foo()
v = f.vals
f = None
v[0] = 10
与正在通过引用计数管理Python对象的生命周期，˚F不拥有由引用的对象丘壑。因此，即使当˚F设置为按，由˚F引用的对象被销毁无 v 引用的对象仍然活着。这是对比的是C ++ 富键入需要被暴露出来，如富持有到<$ c中的记忆$ C>丘壑指。随着Boost.Python的，由 f.vals 返回的辅助对象需要扩展的对象由˚F引用的生命。
With Python object's lifetimes being managed through reference counting, f does not own the object referenced by vals. Hence, even though the object referenced to by f is destroyed when f is set to None, the object referenced by v remains alive. This is on contrast to the C++ Foo type needing to be exposed, as Foo owns the memory to which vals refers. With Boost.Python, the auxiliary object returned by f.vals needs to extend the life of object referenced by f.
通过审查的问题，让我们开始的方案。以下是基本阵列需要被曝光：
With the problem examined, lets start on the solution. Here are the basic arrays needing to be exposed:
struct Foo
{
int vals[3];
boost::array<std::string, 5> strs;
Foo() { std::cout << "Foo()" << std::endl; }
~Foo() { std::cout << "~Foo()" << std::endl; }
};
int more_vals[2];
辅助类型为富::丘壑和富::可疑交易报告需要提供最少的开销，同时支持索引。这是在完成 array_proxy ：
The auxiliary type for Foo::vals and Foo::strs needs to provide minimal overhead, while supporting indexing. This is accomplished in array_proxy:
/// @brief Type that proxies to an array.
template <typename T>
class array_proxy
{
public:
// Types
typedef T value_type;
typedef T* iterator;
typedef T& reference;
typedef std::size_t size_type;
/// @brief Empty constructor.
array_proxy()
: ptr_(0),
length_(0)
{}
/// @brief Construct with iterators.
template <typename Iterator>
array_proxy(Iterator begin, Iterator end)
: ptr_(&*begin),
length_(std::distance(begin, end))
{}
/// @brief Construct with with start and size.
array_proxy(reference begin, std::size_t length)
: ptr_(&begin),
length_(length)
{}
// Iterator support.
iterator begin() { return ptr_; }
iterator end() { return ptr_ + length_; }
// Element access.
reference operator[](size_t i) { return ptr_[i]; }
// Capacity.
size_type size() { return length_; }
private:
T* ptr_;
std::size_t length_;
};
通过做辅助类型，剩下的一块是添加揭露索引功能在Python辅助类型的能力。 Boost.Python的的<一个href=\"http://www.boost.org/doc/libs/1_54_0/libs/python/doc/v2/indexing.html#indexing_suite\"><$c$c>indexing_suite提供了钩子通过一个基于策略的方法索引支持添加到暴露的类型。下面的 ref_index_suite 类是一个策略类履行<一个href=\"http://www.boost.org/doc/libs/1_54_0/libs/python/doc/v2/indexing.html#DerivedPolicies\"><$c$c>DerivedPolicies类型要求：
With the auxiliary type done, the remaining piece is to add the ability to expose indexing capabilities to the auxiliary type in Python. Boost.Python's indexing_suite provides hooks to add indexing support to exposed types through a policy-based approach. The ref_index_suite class below is a policy class fulfilling the DerivedPolicies type requirements:
/// @brief Policy type for referenced indexing, meeting the DerivedPolicies
/// requirement of boost::python::index_suite.
///
/// @note Requires Container to support:
/// - value_type and size_type types,
/// - value_type is default constructable and copyable,
/// - element access via operator[],
/// - Default constructable, iterator constructable,
/// - begin(), end(), and size() member functions
template <typename Container>
class ref_index_suite
: public boost::python::indexing_suite<Container,
ref_index_suite<Container> >
{
public:
typedef typename Container::value_type data_type;
typedef typename Container::size_type index_type;
typedef typename Container::size_type size_type;
// Element access and manipulation.
/// @brief Get element from container.
static data_type&
get_item(Container& container, index_type index)
{
return container[index];
}
/// @brief Set element from container.
static void
set_item(Container& container, index_type index, const data_type& value)
{
container[index] = value;
}
/// @brief Reset index to default value.
static void
delete_item(Container& container, index_type index)
{
set_item(container, index, data_type());
};
// Slice support.
/// @brief Get slice from container.
///
/// @return Python object containing
static boost::python::object
get_slice(Container& container, index_type from, index_type to)
{
using boost::python::list;
if (from > to) return list();
// Return copy, as container only references its elements.
list list;
while (from != to) list.append(container[from++]);
return list;
};
/// @brief Set a slice in container with a given value.
static void
set_slice(
Container& container, index_type from,
index_type to, const data_type& value
)
{
// If range is invalid, return early.
if (from > to) return;
// Populate range with value.
while (from < to) container[from++] = value;
}
/// @brief Set a slice in container with another range.
template <class Iterator>
static void
set_slice(
Container& container, index_type from,
index_type to, Iterator first, Iterator last
)
{
// If range is invalid, return early.
if (from > to) return;
// Populate range with other range.
while (from < to) container[from++] = *first++;
}
/// @brief Reset slice to default values.
static void
delete_slice(Container& container, index_type from, index_type to)
{
set_slice(container, from, to, data_type());
}
// Capacity.
/// @brief Get size of container.
static std::size_t
size(Container& container) { return container.size(); }
/// @brief Check if a value is within the container.
template <class T>
static bool
contains(Container& container, const T& value)
{
return std::find(container.begin(), container.end(), value)
!= container.end();
}
/// @brief Minimum index supported for container.
static index_type
get_min_index(Container& /*container*/)
{
return 0;
}
/// @brief Maximum index supported for container.
static index_type
get_max_index(Container& container)
{
return size(container);
}
// Misc.
/// @brief Convert python index (could be negative) to a valid container
/// index with proper boundary checks.
static index_type
convert_index(Container& container, PyObject* object)
{
namespace python = boost::python;
python::extract<long> py_index(object);
// If py_index cannot extract a long, then type the type is wrong so
// set error and return early.
if (!py_index.check())
{
PyErr_SetString(PyExc_TypeError, "Invalid index type");
python::throw_error_already_set();
return index_type();
}
// Extract index.
long index = py_index();
// Adjust negative index.
if (index < 0)
index += container.size();
// Boundary check.
if (index >= long(container.size()) || index < 0)
{
PyErr_SetString(PyExc_IndexError, "Index out of range");
python::throw_error_already_set();
}
return index;
}
};
每个辅助型，需要通过与Boost.Python的的boost ::蟒蛇:: class_＆LT暴露; ...＆GT; 。这可能是一个有点乏味，所以单一的辅助功能将有条件注册类型。
Each auxiliary type needs to be exposed through Boost.Python with boost::python::class_<...>. This can be a bit tedious, so a single auxiliary function will conditionally register types.
/// @brief Conditionally register a type with Boost.Python.
template <typename T>
void register_array_proxy()
{
typedef array_proxy<T> proxy_type;
// If type is already registered, then return early.
namespace python = boost::python;
bool is_registered = (0 != python::converter::registry::query(
python::type_id<proxy_type>())->to_python_target_type());
if (is_registered) return;
// Otherwise, register the type as an internal type.
std::string type_name = std::string("_") + typeid(T).name();
python::class_<proxy_type>(type_name.c_str(), python::no_init)
.def(ref_index_suite<proxy_type>());
}
此外，模板参数推导将用来提供了一种简单的API向用户
Additionally, template argument deduction will be used to provide an easy API to the user:
/// @brief Create a callable Boost.Python object from an array.
template <typename Array>
boost::python::object make_array(Array array)
{
// Deduce the array_proxy type by removing all the extents from the
// array.
...
// Register an array proxy.
register_array_proxy<...>();
}
在Python的被访问，富::丘壑需要从 INT [3] 来转化 array_proxy＆LT; INT＆GT; 。模板类可以作为将创建适当类型的 array_proxy 仿函数。在 array_proxy_getter 下面提供了这个功能。
When being accessed from Python, Foo::vals need to be transformed from int[3] to array_proxy<int>. A template class can serve as a functor that will create array_proxy of the appropriate type. The array_proxy_getter below provides this capability.
/// @brief Functor used used convert an array to an array_proxy for
/// non-member objects.
template <typename NativeType,
typename ProxyType>
struct array_proxy_getter
{
public:
typedef NativeType native_type;
typedef ProxyType proxy_type;
/// @brief Constructor.
array_proxy_getter(native_type array): array_(array) {}
/// @brief Return an array_proxy for a member array object.
template <typename C>
proxy_type operator()(C& c) { return make_array_proxy(c.*array_); }
/// @brief Return an array_proxy for non-member array object.
proxy_type operator()() { return make_array_proxy(*array_); }
private:
native_type array_;
};
该函子的实例将被包装在一个可调用的boost ::蟒蛇::对象。的 make_array 的单一入口点扩展：
Instances of this functor will be wrapped in a callable boost::python::object. The single entry point of make_array is expanded:
/// @brief Create a callable Boost.Python object from an array.
template <typename Array>
boost::python::object make_array(Array array)
{
// Deduce the array_proxy type by removing all the extents from the
// array.
...
// Register an array proxy.
register_array_proxy<...>();
// Create function.
return boost::python::make_function(
array_proxy_getter<Array>(array),
...);
}
最后，对象生存期需要进行管理。 Boost.Python的提供挂钩指定如何对象的寿命应该通过它的<一个管理href=\"http://www.boost.org/doc/libs/1_54_0/libs/python/doc/v2/CallPolicies.html#CallPolicies-concept\"><$c$c>CallPolices概念。在这种情况下，<一href=\"http://www.boost.org/doc/libs/1_54_0/libs/python/doc/v2/with_custodian_and_ward.html#with_custodian_and_ward_postcall-spec\"><$c$c>with_custodian_and_ward_postcall可用于实施该 array_proxy＆LT; INT＆GT; 从 foo_vals（）延伸的实例的生命周期返回富从创建它。
Finally, object lifetime needs to be managed. Boost.Python provides hooks to designate how object lifetimes should be managed through its CallPolices concept. In this case, with_custodian_and_ward_postcall can be used to enforce that the array_proxy<int> returned from foo_vals() extends the lifetime of instance of foo from which it was created.
// CallPolicy type used to keep the owner alive when returning an object
// that references the parents member variable.
typedef boost::python::with_custodian_and_ward_postcall<
0, // return object (custodian)
1 // self or this (ward)
> return_keeps_owner_alive;
下面是完整的例子支持非成员和成员本地和 Boost.Array 一维数组：
#include <string>
#include <typeinfo>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/indexing_suite.hpp>
namespace detail {
template <typename> struct array_trait;
/// @brief Type that proxies to an array.
template <typename T>
class array_proxy
{
public:
// Types
typedef T value_type;
typedef T* iterator;
typedef T& reference;
typedef std::size_t size_type;
/// @brief Empty constructor.
array_proxy()
: ptr_(0),
length_(0)
{}
/// @brief Construct with iterators.
template <typename Iterator>
array_proxy(Iterator begin, Iterator end)
: ptr_(&*begin),
length_(std::distance(begin, end))
{}
/// @brief Construct with with start and size.
array_proxy(reference begin, std::size_t length)
: ptr_(&begin),
length_(length)
{}
// Iterator support.
iterator begin() { return ptr_; }
iterator end() { return ptr_ + length_; }
// Element access.
reference operator[](size_t i) { return ptr_[i]; }
// Capacity.
size_type size() { return length_; }
private:
T* ptr_;
std::size_t length_;
};
/// @brief Make an array_proxy.
template <typename T>
array_proxy<typename array_trait<T>::element_type>
make_array_proxy(T& array)
{
return array_proxy<typename array_trait<T>::element_type>(
array[0],
array_trait<T>::static_size);
}
/// @brief Policy type for referenced indexing, meeting the DerivedPolicies
/// requirement of boost::python::index_suite.
///
/// @note Requires Container to support:
/// - value_type and size_type types,
/// - value_type is default constructable and copyable,
/// - element access via operator[],
/// - Default constructable, iterator constructable,
/// - begin(), end(), and size() member functions
template <typename Container>
class ref_index_suite
: public boost::python::indexing_suite<Container,
ref_index_suite<Container> >
{
public:
typedef typename Container::value_type data_type;
typedef typename Container::size_type index_type;
typedef typename Container::size_type size_type;
// Element access and manipulation.
/// @brief Get element from container.
static data_type&
get_item(Container& container, index_type index)
{
return container[index];
}
/// @brief Set element from container.
static void
set_item(Container& container, index_type index, const data_type& value)
{
container[index] = value;
}
/// @brief Reset index to default value.
static void
delete_item(Container& container, index_type index)
{
set_item(container, index, data_type());
};
// Slice support.
/// @brief Get slice from container.
///
/// @return Python object containing
static boost::python::object
get_slice(Container& container, index_type from, index_type to)
{
using boost::python::list;
if (from > to) return list();
// Return copy, as container only references its elements.
list list;
while (from != to) list.append(container[from++]);
return list;
};
/// @brief Set a slice in container with a given value.
static void
set_slice(
Container& container, index_type from,
index_type to, const data_type& value
)
{
// If range is invalid, return early.
if (from > to) return;
// Populate range with value.
while (from < to) container[from++] = value;
}
/// @brief Set a slice in container with another range.
template <class Iterator>
static void
set_slice(
Container& container, index_type from,
index_type to, Iterator first, Iterator last
)
{
// If range is invalid, return early.
if (from > to) return;
// Populate range with other range.
while (from < to) container[from++] = *first++;
}
/// @brief Reset slice to default values.
static void
delete_slice(Container& container, index_type from, index_type to)
{
set_slice(container, from, to, data_type());
}
// Capacity.
/// @brief Get size of container.
static std::size_t
size(Container& container) { return container.size(); }
/// @brief Check if a value is within the container.
template <class T>
static bool
contains(Container& container, const T& value)
{
return std::find(container.begin(), container.end(), value)
!= container.end();
}
/// @brief Minimum index supported for container.
static index_type
get_min_index(Container& /*container*/)
{
return 0;
}
/// @brief Maximum index supported for container.
static index_type
get_max_index(Container& container)
{
return size(container);
}
// Misc.
/// @brief Convert python index (could be negative) to a valid container
/// index with proper boundary checks.
static index_type
convert_index(Container& container, PyObject* object)
{
namespace python = boost::python;
python::extract<long> py_index(object);
// If py_index cannot extract a long, then type the type is wrong so
// set error and return early.
if (!py_index.check())
{
PyErr_SetString(PyExc_TypeError, "Invalid index type");
python::throw_error_already_set();
return index_type();
}
// Extract index.
long index = py_index();
// Adjust negative index.
if (index < 0)
index += container.size();
// Boundary check.
if (index >= long(container.size()) || index < 0)
{
PyErr_SetString(PyExc_IndexError, "Index out of range");
python::throw_error_already_set();
}
return index;
}
};
/// @brief Trait for arrays.
template <typename T>
struct array_trait_impl;
// Specialize for native array.
template <typename T, std::size_t N>
struct array_trait_impl<T[N]>
{
typedef T element_type;
enum { static_size = N };
typedef array_proxy<element_type> proxy_type;
typedef boost::python::default_call_policies policy;
typedef boost::mpl::vector<array_proxy<element_type> > signature;
};
// Specialize boost::array to use the native array trait.
template <typename T, std::size_t N>
struct array_trait_impl<boost::array<T, N> >
: public array_trait_impl<T[N]>
{};
// @brief Specialize for member objects to use and modify non member traits.
template <typename T, typename C>
struct array_trait_impl<T (C::*)>
: public array_trait_impl<T>
{
typedef boost::python::with_custodian_and_ward_postcall<
0, // return object (custodian)
1 // self or this (ward)
> policy;
// Append the class to the signature.
typedef typename boost::mpl::push_back<
typename array_trait_impl<T>::signature, C&>::type signature;
};
/// @brief Trait class used to deduce array information, policies, and
/// signatures
template <typename T>
struct array_trait:
public array_trait_impl<typename boost::remove_pointer<T>::type>
{
typedef T native_type;
};
/// @brief Functor used used convert an array to an array_proxy for
/// non-member objects.
template <typename Trait>
struct array_proxy_getter
{
public:
typedef typename Trait::native_type native_type;
typedef typename Trait::proxy_type proxy_type;
/// @brief Constructor.
array_proxy_getter(native_type array): array_(array) {}
/// @brief Return an array_proxy for a member array object.
template <typename C>
proxy_type operator()(C& c) { return make_array_proxy(c.*array_); }
/// @brief Return an array_proxy for a non-member array object.
proxy_type operator()() { return make_array_proxy(*array_); }
private:
native_type array_;
};
/// @brief Conditionally register a type with Boost.Python.
template <typename Trait>
void register_array_proxy()
{
typedef typename Trait::element_type element_type;
typedef typename Trait::proxy_type proxy_type;
// If type is already registered, then return early.
namespace python = boost::python;
bool is_registered = (0 != python::converter::registry::query(
python::type_id<proxy_type>())->to_python_target_type());
if (is_registered) return;
// Otherwise, register the type as an internal type.
std::string type_name = std::string("_") + typeid(element_type).name();
python::class_<proxy_type>(type_name.c_str(), python::no_init)
.def(ref_index_suite<proxy_type>());
}
/// @brief Create a callable Boost.Python object that will return an
/// array_proxy type when called.
///
/// @note This function will conditionally register array_proxy types
/// for conversion within Boost.Python. The array_proxy will
/// extend the life of the object from which it was called.
/// For example, if `foo` is an object, and `vars` is an array,
/// then the object returned from `foo.vars` will extend the life
/// of `foo`.
template <typename Array>
boost::python::object make_array_aux(Array array)
{
typedef array_trait<Array> trait_type;
// Register an array proxy.
register_array_proxy<trait_type>();
// Create function.
return boost::python::make_function(
array_proxy_getter<trait_type>(array),
typename trait_type::policy(),
typename trait_type::signature());
}
} // namespace detail
/// @brief Create a callable Boost.Python object from an array.
template <typename T>
boost::python::object make_array(T array)
{
return detail::make_array_aux(array);
}
struct Foo
{
int vals[3];
boost::array<std::string, 5> strs;
Foo() { std::cout << "Foo()" << std::endl; }
~Foo() { std::cout << "~Foo()" << std::endl; }
};
int more_vals[2];
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<Foo>("Foo")
.add_property("vals", make_array(&Foo::vals))
.add_property("strs", make_array(&Foo::strs))
;
python::def("more_vals", make_array(&more_vals));
}
和使用，测试访问，切片，类型检查和生命周期管理：
And usage, testing access, slicing, type checking, and lifetime management:
>>> from example import Foo, more_vals
>>> def print_list(l): print ', '.join(str(v) for v in l)
...
>>> f = Foo()
Foo()
>>> f.vals[0] = 10
>>> print f.vals[0]
10
>>> f.vals[0] = '10'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Invalid assignment
>>> f.vals[100] = 10
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: Index out of range
>>> f.vals[:] = xrange(100,103)
>>> print_list(f.vals)
100, 101, 102
>>> f.strs[:] = ("a", "b", "c", "d", "e")
>>> print_list(f.strs)
a, b, c, d, e
>>> f.vals[-1] = 30
>>> print_list(f.vals)
100, 101, 30
>>> v = f.vals
>>> del v[:-1]
>>> print_list(f.vals)
0, 0, 30
>>> print_list(v)
0, 0, 30
>>> x = v[-1:]
>>> f = None
>>> v = None
~Foo()
>>> print_list(x)
30
>>> more_vals()[:] = xrange(50, 100)
>>> print_list(more_vals())
50, 51
这篇关于在加速的Python包装阵列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
