pandas df.to_csv("file.csv"编码="utf-8")仍会给垃圾号减号 [英] Pandas df.to_csv("file.csv" encode="utf-8") still gives trash characters for minus sign

问题描述

关于Pandas的to_csv(...等...)，我已经阅读了有关Python 2限制的内容.我打了吗?我正在使用Python 2.7.3

I've read something about a Python 2 limitation with respect to Pandas' to_csv( ... etc ...). Have I hit it? I'm on Python 2.7.3

当字符串中出现≥和-时，这将产生垃圾字符.除此之外，出口是完美的.

This turns out trash characters for ≥ and - when they appear in strings. Aside from that the export is perfect.

df.to_csv("file.csv", encoding="utf-8") 

有什么解决方法吗?

df.head()是这样的:

df.head() is this:

demography  Adults ≥49 yrs  Adults 18−49 yrs at high risk||  \
state                                                           
Alabama                 32.7                             38.6   
Alaska                  31.2                             33.2   
Arizona                 22.9                             38.8   
Arkansas                31.2                             34.0   
California              29.8                             38.8  

csv输出是这个

state,  Adults ≥49 yrs,   Adults 18−49 yrs at high risk||
0,  Alabama,    32.7,   38.6
1,  Alaska, 31.2,   33.2
2,  Arizona,    22.9,   38.8
3,  Arkansas,31.2,  34
4,  California,29.8, 38.8

整个代码是这样的:

import pandas
import xlrd
import csv
import json

df = pandas.DataFrame()
dy = pandas.DataFrame()
# first merge all this xls together


workbook = xlrd.open_workbook('csv_merger/vaccoverage.xls')
worksheets = workbook.sheet_names()


for i in range(3,len(worksheets)):
    dy = pandas.io.excel.read_excel(workbook, i, engine='xlrd', index=None)
    i = i+1
    df = df.append(dy)

df.index.name = "index"

df.columns = ['demography', 'area','state', 'month', 'rate', 'moe']

#Then just grab month = 'May'

may_mask = df['month'] == "May"
may_df = (df[may_mask])

#then delete some columns we dont need

may_df = may_df.drop('area', 1)
may_df = may_df.drop('month', 1)
may_df = may_df.drop('moe', 1)


print may_df.dtypes #uh oh, it sees 'rate' as type 'object', not 'float'.  Better change that.

may_df = may_df.convert_objects('rate', convert_numeric=True)

print may_df.dtypes #that's better

res = may_df.pivot_table('rate', 'state', 'demography')
print res.head()


#and this is going to spit out an array of Objects, each Object a state containing its demographics
res.reset_index().to_json("thejson.json", orient='records')
#and a .csv for good measure
res.reset_index().to_csv("thecsv.csv", orient='records', encoding="utf-8")

推荐答案

您的错误"输出是显示为CP1252的UTF-8.

Your "bad" output is UTF-8 displayed as CP1252.

在Windows上，如果文件开头没有字节顺序标记(BOM)字符，则许多编辑器都将使用默认的ANSI编码(在US Windows上为CP1252)而不是UTF-8.尽管BOM对UTF-8编码没有意义，但其UTF-8编码的状态可作为某些程序的签名.例如，即使在非Windows操作系统上，Microsoft Office的Excel也需要它.试试:

On Windows, many editors assume the default ANSI encoding (CP1252 on US Windows) instead of UTF-8 if there is no byte order mark (BOM) character at the start of the file. While a BOM is meaningless to the UTF-8 encoding, its UTF-8-encoded presence serves as a signature for some programs. For example, Microsoft Office's Excel requires it even on non-Windows OSes. Try:

df.to_csv('file.csv',encoding='utf-8-sig')

该编码器将添加BOM.

That encoder will add the BOM.

这篇关于 pandas df.to_csv("file.csv"编码="utf-8")仍会给垃圾号减号的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！