来自DataFrame的Dask数组 [英] Dask Array from DataFrame

问题描述

有没有一种方法可以轻松地将数值的DataFrame转换为Array？类似于带有熊猫DataFrame的 values 。我似乎找不到使用提供的API进行此操作的任何方法，但我认为这是一个常见操作。

Is there a way to easily convert a DataFrame of numeric values into an Array? Similar to values with a pandas DataFrame. I can't seem to find any way to do this with the provided API, but I'd assume it's a common operation.

推荐答案

< h3>编辑：是的，现在这很简单

您可以使用 .values 属性

x = df.values

较旧的，现在不正确的答案

目前，没有简单的方法可以做到这一点。这是因为dask.array需要知道其所有块的长度，而dask.dataframe不知道该长度。

Older, now incorrect answer

At the moment there is no trivial way to do this. This is because dask.array needs to know the length of all of its chunks and dask.dataframe doesn't know this length. This can not be a completely lazy operation.

话虽如此，您可以使用 dask.delayed 如下：

That being said, you can accomplish it using dask.delayed as follows:

import dask.array as da
from dask import compute

def to_dask_array(df):
    partitions = df.to_delayed()
    shapes = [part.values.shape for part in partitions]
    dtype = partitions[0].dtype

    results = compute(dtype, *shapes)  # trigger computation to find shape
    dtype, shapes = results[0], results[1:]

    chunks = [da.from_delayed(part.values, shape, dtype) 
              for part, shape in zip(partitions, shapes)]
    return da.concatenate(chunks, axis=0)

这篇关于来自DataFrame的Dask数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！