"ValueError:块大小不能超过尺寸大小"尝试将xarray写入netcdf时 [英] "ValueError: chunksize cannot exceed dimension size" when trying to write xarray to netcdf

问题描述

尝试将xarray对象写入netcdf文件时出现以下错误:

I got the following error when trying to write a xarray object to netcdf file:

"ValueError: chunksize cannot exceed dimension size"  

数据对于我的内存来说太大了，需要分块.
该例程基本上如下:

The data is too big for my memory and needs to be chunked.
The routine is basically as follows:

import xarray as xr  
ds=xr.open_dataset("somefile.nc",chunks={'lat':72,'lon':144}  
myds=ds.copy()
#ds is 335 (time) on 720 on 1440 and has variable var  
def some_function(x):
  return x*2
myds['newvar']=xr.DataArray(np.apply_along_axis(some_function,0,ds['var']))  
myds.drop('var')  
myds.to_netcdf("somenewfile.nc")

所以基本上，我只是操纵内容并重写.但是，大块似乎是不好的.与重分配到一个数组相同.我既不能重写ds. 知道如何追踪错误或解决这个问题吗?

So basically, I just manipulate the content and rewrite. Nevertheless, chunks seem to be bad. Same with rechunking to one array. I neither can rewrite ds. Any idea how to track the error down or solve this?

netCDF4版本是1.2.4
xarray(以前的X射线)版本为0.8.2
快捷版本为0.10.1

netCDF4 version is 1.2.4
xarray (former xray) version is 0.8.2
dask version is 0.10.1

推荐答案

这是编写命令中引擎的问题. 如果您正在使用块，则需要将引擎从netcdf4(默认)更改为scipy！

It was an issue of the engine in the writing command. You need to change the engine from netcdf4 (default) to scipy if you are using chunks!

myds.to_netcdf("somenewfile.nc",engine='scipy')

netcdf4软件包无法写入此类文件.

The netcdf4 package is NOT capable of writing such files.

这篇关于"ValueError:块大小不能超过尺寸大小"尝试将xarray写入netcdf时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！