如何在Python中循环遍历HDF5组，从而根据掩码删除行? [英] How can I loop over HDF5 groups in Python removing rows according to a mask?

问题描述

我有一个HDF5文件，其中包含许多不同的组，所有这些组都具有相同的行数.我还有一个布尔掩码，用于保留或删除行.我想遍历HDF5文件中的所有组，并根据掩码删除行.

I have an HDF5 file containing a number of different groups all of which have the same number of rows. I also have a Boolean mask for rows to keep or remove. I would like to iterate over all groups in the HDF5 file removing rows according to the mask.

推荐的方法以递归方式访问所有组是visit(callable)，但是我不知道如何将蒙版传递给可调用对象.

The recommended method to recursively visit all groups is visit(callable), but I can't work out how to pass my mask to the callable.

这里有一些代码希望演示我想做的事，但是那行不通:

Here is some code hopefully demonstrating what I would like to do but which doesn't work:

def apply_mask(name, *args):
    h5obj[name] = h5obj[name][mask]

with h5py.File(os.path.join(directory, filename), 'r+') as h5obj:
    h5obj.visit(apply_mask, mask)

哪个会导致错误

TypeError: visit() takes 2 positional arguments but 3 were given

如何使我的面罩阵列进入此功能?

How can I get my mask array into this function?

推荐答案

我最终通过一系列hacky解决方法实现了这一目标.如果有更好的解决方案，我将对此感兴趣！

I eventually achieved this with a series of hacky workarounds. If there is a better solution I'd be interested to know about it!

with h5py.File(os.path.join(directory, filename), 'r+') as h5obj:
    # Use the visit callable to append to a list of key names
    h5_keys = []
    h5obj.visit(h5_keys.append)
    # Then loop over those keys and, if they're datasets rather than
    # groups, remove the invalid rows
    for h5_key in h5_keys:
        if isinstance(h5obj[h5_key], h5py.Dataset):
            tmp = np.array(h5obj[h5_key])[mask]
            # There is no way to simply change the dataset because its
            # shape is fixed, causing a broadcast error, so it is
            # necessary to delete and then recreate it.
            del h5obj[h5_key]
            h5obj.create_dataset(h5_key, data=tmp)

这篇关于如何在Python中循环遍历HDF5组，从而根据掩码删除行?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！