令牌函数将随机字符附加在字符串中 [英] Token function appends random character with string‏

问题描述

我正在使用strtok函数将字符串分成所需的部分.

我正在使用以下代码:

i am using strtok function to break the string into desirable parts.

i am using the following code :

const char seps[6]   = "\n\t";
char *token;
token = strtok( a, seps );

char *arr[5]={NULL};
arr[i]=token; //Inside the loop i assign each token of a line into arr[i]
string buf= arr[3]; // Description token is in the arr[3]. buf string is initialized by arr[3]'s value and is later used for printing.

但是，每当我破坏令牌时，对于描述字段的某些令牌，随机行都会附加一个额外的字符.
例如
亲爱的父母显示为亲爱的父母uè

在4行中，这些多余的字符会附加以表示最后2行左右的描述标记.其他令牌都不会发生这种情况.

你能建议我一些我可能会忽略或遗忘的东西吗?

However whenever i break my tokens, for some tokens for the description field, an extra character is appended to random rows.
For e.g.
Dear parents is shown as Dear parentsuè

Out of 4 lines these extra character appends to say the description tokens of last 2 lines or so. This thing does not happens with nay of other tokens.

Can you suggest me something which i may be overlooking or missing?

while( getline( myTfile, s1 ) )
{
	char * a = new char[s1.size() + 1];
	std::copy(s1.begin(), s1.end(), a);
	const char toks[6]   = "\n\t";
	char *token;
	token=NULL;
	token = strtok(a,toks);
	char *arr[5]={NULL}; int i=0;
	while( token != NULL )
	{
		arr[i]=token;
		i++;
		token++;
		token = strtok(NULL,toks);/* Get next token: */
	}
	string buf= arr[3];         // Description
}

输入就像:

+ 123123123 243.20 textstring1
-123454355 123.10 textstring2


现在，对于textstring2或lie输入中的某个随机字符串，在随机行后附加一个额外的字符.
例如
亲爱的父母显示为亲爱的父母uè

Input is some thing like :

+ 123123123 243.20 textstring1
- 123454355 123.10 textstring2


Now for textstring2 or some random string in the lie input, an extra character is appended to random rows.
For e.g.
Dear parents is shown as Dear parentsuè

推荐答案

需要考虑的几件事:

A few things to consider:

1. std::copy()不会终止您的字符数组，因此，在数组末尾立即有可能出现额外的字符.您应该用strcpy(a, s1.c_str());
2. 形式的一行替换它.您的令牌字符不包含空格，但是，这可能是故意的.
3. 您应该真正使用char* toks = "\n\t"
4. 表达式token=NULL;是多余的，因为它紧随其后是strtok()调用.
5. 您在while循环中有表达式token++，这毫无用处.

1. std::copy() will not null terminate your character array, so you immediately have the potential of extra characters at the end of your array. You should replace this with a line of the form strcpy(a, s1.c_str());
2. Your token characters do not include a space, however, this may be deliberate.
3. You should really use char* toks = "\n\t" for your splitter characters to ensure it is properly const (i.e. read only).
4. The expression token=NULL; is redundant as it is immediately followed by a strtok() call.
5. Similarly you have the expression token++ in your while loop, which serves no purpose.

您看到的多余字符是上面第1点的结果.根据您尝试处理数据的方式，可能还有其他需要考虑的事情.

The extra characters you are seeing are the result of point 1 above. There are probably other things to consider depending on how you are trying to process your data.

您还可能考虑使用STL类型，例如可以简化字符串的处理.使用C char指针/数组时，您面临的典型错误是什么.

为了您的灵感.获取字符串标记并将其存储在向量中的函数可能如下所示:

You may also think of using STL types which for example may ease the handling of strings. What you are facing a typical errors when working with C char pointers/arrays.

For your inspiration. A function to get the tokens of a string and store them in a vector may look as follows:

//! Tokenize the given string str with given delimiter. If no delimiter is given whitespace is used.
void Tokenize(const std::string& str, std::vector<std::string> & tokens, const std::string& delimiters = " ")
{
        tokens.clear();
	// Skip delimiters at beginning.
	std::string::size_type lastPos = str.find_first_not_of(delimiters, 0);
	// Find first "non-delimiter".
	std::string::size_type pos = str.find_first_of(delimiters, lastPos);

	while (std::string::npos != pos || std::string::npos != lastPos)
	{
		// Found a token, add it to the vector.
		tokens.push_back(str.substr(lastPos, pos - lastPos));
		// Skip delimiters.  Note the "not_of"
		lastPos = str.find_first_not_of(delimiters, pos);
		// Find next "non-delimiter"
		pos = str.find_first_of(delimiters, lastPos);
        }
}

这篇关于令牌函数将随机字符附加在字符串中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！