在终端中编码，解码阿拉伯语时出现问题 [英] Issues while encoding, decoding arabic language in terminal

问题描述

在我的脚本中，余弦相似度首先需要在执行余弦相似度之前将阿拉伯字符串转换为向量 在Linux下的终端上->问题，同时将阿拉伯字符串转换为矢量，产生阿拉伯语为:

In my script Cosine similarity need first, to convert an Arabic string into a vector before perform Cosine similarity on terminal under Linux --> problem while convert Arabic string to vector producing Arabic as:

[u'\u0627\u0644\u0634\u0645\u0633 \u0645\u0634\u0631\u0642\u0647 \u0646\u0647\u0627\u0631\u0627', u'\u0627\u0644\u0633\u0645\u0627\u0621 \u0632\u0631\u0642\u0627\u0621']

我的脚本:

train_set = ["السماء زرقاء", "الشمس مشرقه نهارا"] #Documents
test_set = ["الشمس التى فى السماء مشرقه","السماء زرقاء"] #Query
stopWords = set(stopwords.words('english'))

vectorizer = CountVectorizer(stop_words = stopWords)
transformer = TfidfTransformer()
trainVectorizerArray = vectorizer.fit_transform(train_set).toarray()
testVectorizerArray = vectorizer.transform(test_set).toarray()
print 'Fit Vectorizer to train set', trainVectorizerArray
print 'Transform Vectorizer to test set', testVectorizerArray
cx = lambda a, b : round(np.inner(a, b)/(LA.norm(a)*LA.norm(b)), 3)

for vector in trainVectorizerArray:
    print vector
    for testV in testVectorizerArray:
        print testV
        cosine = cx(vector, testV)
        print cosine

推荐答案

您的结果是一个字符串列表，只需将字符串加入即可，您会得到一个清晰的句子:

Your result is a list of strings, just join the string and you will get a clear sentence:

>>> print "\n".join(a)
الشمس مشرقه نهارا
السماء زرقاء

这篇关于在终端中编码，解码阿拉伯语时出现问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！