如何使用Python读取NetCDF文件并写入CSV [英] How to read NetCDF file and write to CSV using Python
问题描述
我的目的是从netcdf文件访问数据并以以下格式写入CSV文件.
My aim is to access data from a netcdf file and write to a CSV file in the following format.
Latitude Longitude Date1 Date2 Date3
100 200 <-- MIN_SFC values -->
到目前为止，我已经访问了变量，将标头写入文件并填充了纬度/经度.
So far I have accessed the variables, written the header to the file and populated the lat/lons.
如何访问指定的lon，lat坐标和日期的MIN_SFC值，然后将其写入CSV文件.
How can I access the MIN_SFC values for specified lon,lat coordinates and dates and then write to a CSV file.
我是python新手，如果有更好的方法可以解决这个问题，请告诉我.
I'm a python newbie if there is a better way to go about this please let me know.
NetCDF文件信息:
NetCDF file info:
Dimensions:
time = 7
latitude = 292
longitude =341
Variables:
float MIN_SFC (time=7, latitude = 292, longitude = 341)
这是我尝试过的:
from netCDF4 import Dataset, num2date
filename = "C:/filename.nc"
nc = Dataset(filename, 'r', Format='NETCDF4')
print nc.variables
print 'Variable List'
for var in nc.variables:
print var, var.units, var.shape
# get coordinates variables
lats = nc.variables['latitude'][:]
lons = nc.variables['longitude'][:]
sfc= nc.variables['Min_SFC'][:]
times = nc.variables['time'][:]
# convert date, how to store date only strip away time?
print "Converting Dates"
units = nc.variables['time'].units
dates = num2date (times[:], units=units, calendar='365_day')
#print [dates.strftime('%Y%m%d%H') for date in dates]
header = ['Latitude', 'Longitude']
# append dates to header string
for d in dates:
print d
header.append(d)
# write to file
import csv
with open('Output.csv', 'wb') as csvFile:
outputwriter = csv.writer(csvFile, delimiter=',')
outputwriter.writerow(header)
for lat, lon in zip(lats, lons):
outputwriter.writerow( [lat, lon] )
# close the output file
csvFile.close()
# close netcdf
nc.close()
更新:
我已经更新了写入CSV文件的代码，这是一个属性错误，因为纬度/经度是双精度的.
I've updated the code that writes the CSV file, there's an attribute error, because the lat/lon are doubles.
AttributeError:'numpy.float32'对象没有属性'append'
AttributeError: 'numpy.float32' object has no attribute 'append'
在python中强制转换为字符串的任何方法吗?你认为它会起作用吗?
Any way to cast to a string in python? Do you think it'll work?
当我在控制台上打印值时，我注意到许多返回为-"的值.我想知道这是否代表定义为-32767.0的fillValue或missingValue.
I've noticed a number of values returned as "--" when I printed values to the console. I'm wondering if this represents the fillValue or missingValue defined as -32767.0.
我还想知道3d数据集的变量是否应通过lats = nc.variables ['latitude'] [:] [:]或lats = nc.variables ['latitude'] [:] [ :，:]?
I'm also wondering whether the variables of the 3d dataset should be accessed by lats = nc.variables['latitude'][:][:] or lats = nc.variables['latitude'][:][:,:] ?
# the csv file is closed when you leave the block
with open('output.csv', 'wb') as csvFile:
outputwriter = csv.writer(csvFile, delimiter=',')
for time_index, time in enumerate(times): # pull the dates out for the header
t = num2date(time, units = units, calendar='365_day')
header.append(t)
outputwriter.writerow(header)
for lat_index, lat in enumerate(lats):
content = lat
print lat_index
for lon_index, lon in enumerate(lons):
content.append(lon)
print lon_index
for time_index, time in enumerate(times): # for a date
# pull out the data
data = sfc[time_index,lat_index,lon_index]
content.append(data)
outputwriter.writerow(content)
推荐答案
我会将数据加载到Pandas中，这有助于对时间序列数据进行分析和绘图以及写入CSV.
I would load the data into Pandas, which facilitates the analysis and plotting of time series data, as well as writing to CSV.
因此，这是一个真实的工作示例，该示例从指定的lon，lat位置中提取了波高的时间序列，并将其从全局预测模型数据集中提取.
So here's a real working example which pulls a time series of wave heights from a specified lon,lat location out of a global forecast model dataset.
注意:这里我们访问OPeNDAP数据集，因此我们可以从远程服务器提取所需的数据，而无需下载文件.但是netCDF4对于删除的OPeNDAP数据集或本地NetCDF文件的工作原理完全相同，这是非常有用的功能！
Note: here we access an OPeNDAP dataset so we can just extract the data we need from a remote server without downloading files. But netCDF4 works exactly the same for a remove OPeNDAP dataset or a local NetCDF file, which is a very useful feature!
import netCDF4
import pandas as pd
import matplotlib.pyplot as plt
# NetCDF4-Python can read a remote OPeNDAP dataset or a local NetCDF file:
url='http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/WW3/Global/Best'
nc = netCDF4.Dataset(url)
nc.variables.keys()
lat = nc.variables['lat'][:]
lon = nc.variables['lon'][:]
time_var = nc.variables['time']
dtime = netCDF4.num2date(time_var[:],time_var.units)
# determine what longitude convention is being used [-180,180], [0,360]
print lon.min(),lon.max()
# specify some location to extract time series
lati = 41.4; loni = -67.8 +360.0 # Georges Bank
# find closest index to specified value
def near(array,value):
idx=(abs(array-value)).argmin()
return idx
# Find nearest point to desired location (could also interpolate, but more work)
ix = near(lon, loni)
iy = near(lat, lati)
# Extract desired times.
# 1. Select -+some days around the current time:
start = dt.datetime.utcnow()- dt.timedelta(days=3)
stop = dt.datetime.utcnow()+ dt.timedelta(days=3)
# OR
# 2. Specify the exact time period you want:
#start = dt.datetime(2013,6,2,0,0,0)
#stop = dt.datetime(2013,6,3,0,0,0)
istart = netCDF4.date2index(start,time_var,select='nearest')
istop = netCDF4.date2index(stop,time_var,select='nearest')
print istart,istop
# Get all time records of variable [vname] at indices [iy,ix]
vname = 'Significant_height_of_wind_waves_surface'
#vname = 'surf_el'
var = nc.variables[vname]
hs = var[istart:istop,iy,ix]
tim = dtime[istart:istop]
# Create Pandas time series object
ts = pd.Series(hs,index=tim,name=vname)
# Use Pandas time series plot method
ts.plot(figsize(12,4),
title='Location: Lon=%.2f, Lat=%.2f' % ( lon[ix], lat[iy]),legend=True)
plt.ylabel(var.units);
#write to a CSV file
ts.to_csv('time_series_from_netcdf.csv')
都会创建此图以验证您是否拥有所需的数据:
which both creates this plot to verify that you've got the data you wanted:
，还将所需的CSV文件time_series_from_netcdf.csv写入磁盘.
and also writes the desired CSV file time_series_from_netcdf.csv to disk.
您还可以在Wakari上查看，下载和/或运行此示例 .
这篇关于如何使用Python读取NetCDF文件并写入CSV的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
