JNA如何在要传递给本机库的结构中填充指向结构的指针字段? [英] JNA How to populate a pointer-to-structure field within a structure to be PASSED to the native library?

问题描述

我需要将JNA结构传递到包含指向结构的指针字段(可能包含零个或多个结构)的本机层.

这是父"结构:

public class VkRenderPassCreateInfo extends Structure {
    public int attachmentCount;
    public VkAttachmentDescription.ByReference pAttachments;
}

(其他字段，为简洁起见，省略了@FieldOrder和ByReference/Value类)

这是子"结构:

public class VkAttachmentDescription extends Structure {
    public int flags;
    // ... lots and lots of other simple fields
}

根据JNA文档(在此处)数组指针字段应为Structure.ByReference字段.

在其他帖子中，填充此字段的标准方法是:

1. 将字段初始化为按引用结构

2. 使用Structure::toArray

从字段中分配结构数组

3. 填充数组

所以:

// Init structure fields
renderPass.pAttachments = new VkRenderPassCreateInfo.ByReference();
renderPass.attachmentCount = size;

// Allocate memory
VkAttachmentDescription[] attachments = (VkAttachmentDescription[]) renderPass.pAttachments.toArray(size);

// Populate array
for(int n = 0; n < size; ++n) {
    attachments[n].flags = ...
    // and so on for other fields
}

1-这是在结构内初始化和分配指向结构的指针字段的正确方法吗?似乎有很多混帐?

2-上面的方法对于摆弄大小的结构很好，但是我要处理的一些结构具有大量的字段，子结构等.我假设我可以在该结构上构建一系列JNA结构. Java方面，并将它们直接设置到父结构中，但是toArray方法意味着我必须然后将所有内容复制到生成的数组中?是否有更好/更简单的方法，这意味着我不必创建和复制本来就已经在Java端已经拥有的数据?

3-JNA提供了一个StringArray帮助器类，该类处理结构中字符串数组的类似情况:

// Array of strings maintained on the Java side
List<String> strings = ...

// Nice and easy means of populating the JNA structure
structure.pStrings = new StringArray(strings.toArray(String[]::new));
...

// Send the data
library.fireandForget(structure);

这是我试图通过上面的结构代码实现的目标，但显然仅用于字符串大小写-我是否错过了其他类似的帮助程序?

请注意，以上内容是将传递到本机层的结构，我没有尝试检索任何内容.

只是为了解决这个问题，尽管上面的方法可以解决所有琐碎的情况，但对于大多数琐碎的情况来说，却是一大堆样板代码.我正在努力找到最简单/最好的方法来构建要传递给本机方法的复杂结构图.似乎缺少示例或教程，或者也许我不是在问正确的问题(?)任何指向示例，教程或传递包含其他结构的指针的结构的示例代码的指针都是非常赞赏.

因此，我尝试了多种方法，当调用本机库时，所有这些方法都会导致Illegal memory access错误.

我要发送的数据是由应用程序构建的-它可以是构建器模式，用户的选择等.无论如何，结果是VkAttachmentDescription的列表，然后我需要将其作为文件发送. 父级" VkRenderPassCreateInfo中的结构指针"字段.

在Java端使用JNA VkAttachmentStructure的原因是其中一些结构包含大量字段.即调用Structure::toArray然后逐字段填充结果数组是站不住脚的:代码量巨大，容易出错且更改起来很脆弱(例如忘记复制新字段).我可以创建另一个类来抽象JNA类，但这只会解决问题.

这是代码的作用:

// Application builds the attachments
final List<VkAttachmentDescription> attachments = ...

...

// At some point we then send the render pass including the attachments

// Populate the render pass descriptor
final VkRenderPassCreateInfo info = new VkRenderPassCreateInfo();
info.pAttachments = ??? <--- what?
// ... other fields

// Send the descriptor
library.sendRenderPass(info);

尝试1:天真地将指向结构的指针设置为数组:

final VkRenderPassCreateInfo info = new VkRenderPassCreateInfo();
final var array = attachments.toArray(VkAttachmentDescription.ByReference[]::new);
info.pAttachments = array[0];
library.sendRenderPass(info);

结果是内存访问错误，我没想到这会起作用！

尝试2:使用Structure :: toArray(int)并将字段设置为第一个元素

final VkAttachmentDescription.ByReference[] array = (VkAttachmentDescription.ByReference[]) new VkAttachmentDescription.ByReference().toArray(attachments.size());

for(int n = 0; n < size; ++n) {
    array[n] = attachments.get(n);
}

info.pAttachments = array[0];

library.sendRenderPass(info);

结果相同.

尝试3:使用Structure :: toArray(array)

Structure中还有一个替代的toArray方法，该方法采用数组，但似乎与调用整数版本没有什么不同?

尝试4:逐字段复制

final VkAttachmentDescription.ByReference[] array = (VkAttachmentDescription.ByReference[]) new VkAttachmentDescription.ByReference().toArray(attachments.size());

for(int n = 0; n < size; ++n) {
    array[n].field = attachments.get(n).field;
    // ...lots of other fields
}

info.pAttachments = array[0];

library.sendRenderPass(info);

这有效，但令人讨厌.

我显然完全不了解JNA.我的主要坚持点是Structure::toArray创建了一个空结构数组，这些结构必须逐场填充，但是我已经已经拥有包含所有内容的结构数组填充-如何将指向结构的指针字段设置为该数组(即与StringArray助手等效的对象)?在我的脑海中似乎做一件如此简单的事情，但我根本找不到任何如何做自己想做的事的例子(除了琐碎的逐场复制的例子).

困扰我的另一件事是，父结构字段必须为ByReference，这意味着代码中的所有其他结构都必须被引用?再次感觉就像我做错了.

解决方案

您需要解决的问题(以及Illegal memory access错误的来源)是，接受您的数组的C端代码期望Pointer到连续块的内存中.在C语言中，只需要第一个元素的内存地址，再加上大小偏移量即可；要访问array [1]，您会发现array [0]的内存，并因结构的大小而偏移.

在您的情况下，您已为此块中的每个结构分配了非连续内存:

// Application builds the attachments
final List<VkAttachmentDescription> attachments = ...

每个VkAttachmentDescription都映射到其自己的内存，并且尝试在第一个结构的末尾读取内存会导致错误.如果在实例化这些VkAttachmentDescription对象时无法控制使用哪个内存，最终将导致重复您的内存需求，并且必须将本机内存从非连续块复制到连续块.

编辑后添加:如您其他答案中所指出的那样，如果您仅使用Java方面的VkAttachmentDescription结构，而不将其传递给C函数，则本机内存可能还没有写过.以下基于Pointer.get*()方法的解决方案直接从C内存读取，因此它们需要在某个时候进行write()调用.

假设除了以List<VkAttachmentDescription>开头外别无选择，您要做的第一件事就是分配C所需的连续内存.让我们获取所需的字节大小:

int size = attachments.size();
int bytes = attachments.get(0).size();

我们需要分配size * bytes内存.

您在这里有两个选择:使用Memory对象(Pointer的子类)直接分配内存，或使用Structure.toArray.对于直接分配:

Memory mem = new Memory(size * bytes);

如果我们这样定义引用，我们可以直接将mem用作Pointer:

public class VkRenderPassCreateInfo extends Structure {
    public int attachmentCount;
    public Pointer pAttachments;
}

那么这很简单:

info.pAttachments = mem;

现在剩下的就是将字节从非连续内存复制到分配的内存中.我们可以逐字节进行操作(更容易了解C端字节级别的情况):

for (int n = 0; n < size; ++n) {
    Pointer p = attachments.get(n).getPointer();
    for (int b = 0; b < bytes; ++b) {
        mem.setByte(n * bytes + b, p.getByte(b));
    }
}

或者我们可以逐个结构地进行操作:

for (int n = 0; n < size; ++n) {
    byte[] attachment = attachments.get(n).getPointer().getByteArray(0, bytes);
    mem.write(n * bytes, attachment, 0, bytes);
}

(性能折衷:数组实例化开销与Java <-> C调用的关系.)

现在，缓冲区已写入，您可以将其发送到C，它在其中需要结构数组，并且它不知道区别……字节就是字节！

编辑后添加:我认为可以使用useMemory()更改本机内存支持，然后直接写入新的(连续的)位置.此代码未经测试，但我怀疑可能确实有效:

for (int n = 0; n < size; ++n) {
    attachments.get(n).useMemory(mem, n * bytes);
    attachments.get(n).write();
}

就个人而言，由于我们只是在复制已存在的内容，因此我更喜欢这种基于Memory的映射.但是...有些编码员是受虐狂.

如果您想更加类型安全"，可以在结构内部使用ByReference类声明，并使用toArray()创建Structure数组. 您已在代码中列出了一种使用ByReference类型创建数组的方法.可行，或者您也可以使用(默认ByValue)类型创建它，然后将Pointer提取到第一个元素，以在将其分配给结构字段时创建ByReference类型:

VkAttachmentDescription[] array = 
    (VkAttachmentDescription[]) new VkAttachmentDescription().toArray(attachments.size());

然后您可以这样设置:

info.pAttachments = new VkAttachmentDescription.ByReference(array[0].getPointer());

在这种情况下，将值(从列表(由单独分配的内存块支持的结构)复制)复制到(连续内存的)数组会比较复杂，因为内存映射的类型更窄，但是遵循与Memory映射相同的模式.您发现的一种方法是手动复制结构的每个元素！ (糟糕)另一种可能使您免受某些复制/粘贴错误的影响的方法是使用Reflection(JNA在后台执行的操作).这也是很多工作，并且重复了JNA的工作，所以这很丑陋且容易出错.但是，仍然有可能将原始本机字节从不连续的存储块复制到连续的存储块. (在这种情况下，为什么不直接进入Memory却显示了我的偏见.)您可以像Memory示例中那样遍历字节，如下所示:

for (int n = 0; n < size; ++n) {
    Pointer p = attachments.get(n).getPointer();
    Pointer q = array[n].getPointer();
    for (int b = 0; b < bytes; ++b) {
        q.setByte(b, p.getByte(b));
    }
}

或者您可以像这样读取块中的字节:

for (int n = 0; n < size; ++n) {
    byte[] attachment = attachments.get(n).getPointer().getByteArray(0, bytes);
    array[n].getPointer().write(0, attachment, 0, bytes);
}

请注意，我尚未测试此代码；它写到本机端而不是Java结构，所以我认为它可以照常工作，但是如果有内置Java，则可能需要在上述循环的结尾处调用array[n].read()从C读取到Java到我不知道的C复制.

为响应您的父结构字段必须为ByReference":如上所示，Pointer映射有效，并且以类型安全"和(或可能)可读性"为代价提供了更多的灵活性. .您不需要在其他地方使用ByReference，正如我在toArray()所示的那样，您只需要在Structure字段中使用它即可(您可以将其定义为Pointer并完全消除对ByReference的需要...但是如果您要这样做，为什么不直接复制到Memory缓冲区呢?我在这里打败一匹死马！).

最后，如果您知道最终将拥有多少个元素(或该数字的上限)，则理想的解决方案是在一开始使用连续内存实例化数组.然后，无需创建VkAttachmentDescription的新实例，您只需从数组中获取一个预先存在的实例即可.只要您从一开始就连续使用它们，就可以过度分配并且不使用它们，这没关系.您要传递给C的全部是结构数和第一个结构的地址，它并不关心您是否有多余的字节.

I need to pass a JNA structure to the native layer that contains a pointer-to-structure field (may contain zero-or-more structures).

Here is the 'parent' structure:

public class VkRenderPassCreateInfo extends Structure {
    public int attachmentCount;
    public VkAttachmentDescription.ByReference pAttachments;
}

(Other fields, @FieldOrder and ByReference/Value classes omitted for brevity)

And here is the 'child' structure:

public class VkAttachmentDescription extends Structure {
    public int flags;
    // ... lots and lots of other simple fields
}

As per the JNA documentation (here) the pointer-to-array field should be a Structure.ByReference field.

From other posts the standard approach to populating this field is:

1. init the field to a structure-by-reference

2. allocate an array of structures from the field using Structure::toArray

3. populate the array

So:

// Init structure fields
renderPass.pAttachments = new VkRenderPassCreateInfo.ByReference();
renderPass.attachmentCount = size;

// Allocate memory
VkAttachmentDescription[] attachments = (VkAttachmentDescription[]) renderPass.pAttachments.toArray(size);

// Populate array
for(int n = 0; n < size; ++n) {
    attachments[n].flags = ...
    // and so on for other fields
}

1 - Is this the correct approach for initialising and allocating a pointer-to-structure field within a structure? Seems a lot of mucking around?

2 - The above works fine for fiddling sized structures but some of the ones I'm dealing with have a huge number of fields, sub-structures, etc. I had assumed I could just build an array of JNA structures on the Java side and set those directly into the parent structure, but the toArray approach means I have to then copy everything to the generated array? Is there a better/simpler approach that means I don't have to create and copy data that I essentially already have on the Java side?

3 - JNA provides a StringArray helper class that handles a similar case for an array of strings within a structure:

// Array of strings maintained on the Java side
List<String> strings = ...

// Nice and easy means of populating the JNA structure
structure.pStrings = new StringArray(strings.toArray(String[]::new));
...

// Send the data
library.fireandForget(structure);

This is sort-of what I'm trying to achieve with the structure code above but obviously is only for the string case - are there other similar helpers that I've missed?

Please note that the above is passing the structures to the native layer, I am not trying to retrieve anything.

EDIT 1: Just to qualify the point of this question - although the above works it results in a buge amount of boiler-plate code for all but the most trivial cases. I'm struggling to work out the simplest/best way of building a complex graph of structures to be passed to a native method. There seems a shortage of examples or tutorials, or maybe I'm just not asking the right question (?) Any pointers to examples, tutorials or example code of passing structures containing pointers to other structures would be very appreciated.

EDIT 2: So I've tried numerous approaches all of which result in Illegal memory access errors when I invoke the native library.

The data that I want to send is constructed by the application - it could be a builder pattern, selections by the user, etc. In any case the result is a list of VkAttachmentDescription that I then need to send as a pointer-to-structure field in the 'parent' VkRenderPassCreateInfo.

The reason for using the JNA VkAttachmentStructure on the Java side is that some these structures contain a large number of fields. i.e. calling Structure::toArray and then populating the resultant array field-by-field is just not tenable: the amount of code would be huge, error-prone and brittle to change (e.g. forget to copy a new field). I could create another class to abstract the JNA class but that would just move the problem.

Here is what the code is doing:

// Application builds the attachments
final List<VkAttachmentDescription> attachments = ...

...

// At some point we then send the render pass including the attachments

// Populate the render pass descriptor
final VkRenderPassCreateInfo info = new VkRenderPassCreateInfo();
info.pAttachments = ??? <--- what?
// ... other fields

// Send the descriptor
library.sendRenderPass(info);

Attempt 1: Naively set the pointer-to-structure to an array:

final VkRenderPassCreateInfo info = new VkRenderPassCreateInfo();
final var array = attachments.toArray(VkAttachmentDescription.ByReference[]::new);
info.pAttachments = array[0];
library.sendRenderPass(info);

Result is a memory access error, I didn't expect this to work!

Attempt 2: Use Structure::toArray(int) and set the field to the first element

final VkAttachmentDescription.ByReference[] array = (VkAttachmentDescription.ByReference[]) new VkAttachmentDescription.ByReference().toArray(attachments.size());

for(int n = 0; n < size; ++n) {
    array[n] = attachments.get(n);
}

info.pAttachments = array[0];

library.sendRenderPass(info);

Same result.

Attempt 3: Use Structure::toArray(array)

There is an alternative toArray method in Structure that takes an array but it doesn't seem to do anything different to calling the integer version?

Attempt 4: Copy field-by-field

final VkAttachmentDescription.ByReference[] array = (VkAttachmentDescription.ByReference[]) new VkAttachmentDescription.ByReference().toArray(attachments.size());

for(int n = 0; n < size; ++n) {
    array[n].field = attachments.get(n).field;
    // ...lots of other fields
}

info.pAttachments = array[0];

library.sendRenderPass(info);

This works but is nasty.

I'm obviously completely missing something about JNA. My main sticking point is that Structure::toArray creates an array of empty structures that have to be populated field-by-field, but I already have the array of structures with everything populated - how can I set the pointer-to-structure field to that array (i.e. an equivalent of the StringArray helper)? It seems such a simple thing to do in my head but I simply cannot find any examples of how to do what I want (other than trivial ones that copy field-by-field).

The other thing that is bothering me is that fact that the parent structure field has to be ByReference which means every other structure in the code has to be by-reference? Again it just feels like I'm doing this all wrong.

解决方案

The issue you need to solve (and the source of the Illegal memory access errors) is that the C-side code accepting your array is expecting a Pointer to a contiguous block of memory. In C, you only need the memory address of the first element, plus the size offset; to access array[1] you find the memory of array[0] and offset by the size of the structure.

In your case, you have allocated non-contiguous memory for each structure in this block:

// Application builds the attachments
final List<VkAttachmentDescription> attachments = ...

Each VkAttachmentDescription is mapped to its own memory, and attempting to read the memory at the end of the first structure causes the error. If you do not have control over which memory is used when these VkAttachmentDescription objects are instantiated, you will end up duplicating your memory requirements, and having to copy over the native memory from the non-contiguous block to the contiguous block.

Edited to add: As pointed out in your other answer, if you've only worked with the VkAttachmentDescription Structure on the Java side and not passed it to a C function, the native memory may not have been written. The below solutions based on Pointer.get*() methods read directly from the C memory, so they would require a write() call to be made at some point.

Assuming you have no choice other than to start with a List<VkAttachmentDescription>, the first thing you need to do is allocate the contiguous memory that C needs. Let's get the byte sizes we need:

int size = attachments.size();
int bytes = attachments.get(0).size();

We need to allocate size * bytes of memory.

You have two options here: directly allocate the memory using a Memory object (a subclass of Pointer) or use Structure.toArray. For the direct allocation:

Memory mem = new Memory(size * bytes);

We can directly use mem as a Pointer if we define the reference like this:

public class VkRenderPassCreateInfo extends Structure {
    public int attachmentCount;
    public Pointer pAttachments;
}

Then it's a simple:

info.pAttachments = mem;

Now all that is left is to copy the bytes from the non-contiguous memory into your allocated memory. We can do it byte by byte (easier to see what's happening at the byte level on the C side):

for (int n = 0; n < size; ++n) {
    Pointer p = attachments.get(n).getPointer();
    for (int b = 0; b < bytes; ++b) {
        mem.setByte(n * bytes + b, p.getByte(b));
    }
}

Or we can do it structure by structure:

for (int n = 0; n < size; ++n) {
    byte[] attachment = attachments.get(n).getPointer().getByteArray(0, bytes);
    mem.write(n * bytes, attachment, 0, bytes);
}

(Performance tradeoff: array instantiation overhead vs. Java<-->C calls.)

Now that the buffer is written, you can then send it to C, where it expects the array of structures, and it won't know the difference... bytes are bytes!

Edited to add: I think it's possible to change the native memory backing using useMemory() and then directly write to the new (contiguous) location. This code is untested but I suspect might actually work:

for (int n = 0; n < size; ++n) {
    attachments.get(n).useMemory(mem, n * bytes);
    attachments.get(n).write();
}

Personally, since we're just making a copy of something that already exists, I would prefer this Memory-based mapping. However... some coders are masochists.

If you want to be a bit more "type safe" you can use the ByReference class declaration inside the structure and create the Structure array using toArray(). You've listed in your code one way of creating the array using the ByReference type. That works, or you can also create it with the (default ByValue) type and then extract the Pointer to the first element later to create the ByReference type when assigning it to the structure field:

VkAttachmentDescription[] array = 
    (VkAttachmentDescription[]) new VkAttachmentDescription().toArray(attachments.size());

Then you can set it like this:

info.pAttachments = new VkAttachmentDescription.ByReference(array[0].getPointer());

In this case, it's a bit more complex copying the values over from the List (of structures backed by individually allocated memory blocks) to the array (of contiguous memory) because the memory mapping is more narrowly typed, but it follows the same pattern as for the Memory mapping. One way, which you've discovered, is to manually copy over every element of the Structure! (Ugh.) Another way which might save you from some copy/paste errors is to use Reflection (what JNA does under the hood). That's a lot of work, too, and duplicates what JNA does, so that's ugly and error-prone. However, it is still possible copy over the raw native bytes from the non-contiguous memory block to the contiguous one. (In which case... why not just go straight to Memory but my bias is showing.) You can iterate over bytes like in the Memory example, like this:

for (int n = 0; n < size; ++n) {
    Pointer p = attachments.get(n).getPointer();
    Pointer q = array[n].getPointer();
    for (int b = 0; b < bytes; ++b) {
        q.setByte(b, p.getByte(b));
    }
}

Or you can read the bytes in chunks like this:

for (int n = 0; n < size; ++n) {
    byte[] attachment = attachments.get(n).getPointer().getByteArray(0, bytes);
    array[n].getPointer().write(0, attachment, 0, bytes);
}

Note that I haven't tested this code; it writes to the native side but not the Java structure so I think it will work as is, but you may need an array[n].read() call at the end of the above loop to read from C to Java in case there's a built-in Java to C copy that I'm not aware of.

In response to your "parent structure field has to be ByReference": As shown above, a Pointer mapping works and allows a bit more flexibility at the cost of "type safety" and maybe (or not) "readability". You don't need to use ByReference elsewhere as I've shown with the toArray() where you only need it for the Structure field (which you could just define as a Pointer and completely eliminate the need for ByReference ... but if you're doing that why not just copy to the Memory buffer? I'm beating a dead horse here!).

Finally, the ideal solution if you know how many elements you will eventually have (or an upper bound on that number), would be to instantiate the array using contiguous memory at the very beginning. Then instead of creating a new instance of VkAttachmentDescription you could just fetch a pre-existing one from the array. It's okay if you over-allocate and don't use them all as long as you use them contiguously from the beginning. All you're passing to C is the # of structures and the address of the first one, it doesn't care if you've got extra bytes.

这篇关于JNA如何在要传递给本机库的结构中填充指向结构的指针字段?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！