strtok / strtok_r困境 [英] strtok/strtok_r woes

问题描述

我用strtok_r编写了一些代码来解析带有两个

级别的字符串。分隔符。代码工作就像一个魅力，直到它有一天突然爆发
。你看，我在输入

字符串本身上应用strtok，只要变量字符串被传递给

函数，一切都是hunky dory。然后有一天有人决定将
传递给我的代码一个恒定的字符串----一切都崩溃了

就像多米诺骨牌一样。


我相信这是许多程序员可能陷入的陷阱。

我现在已经改变了我的函数的签名来处理输​​入

字符串作为常量字符串，我现在制作

字符串的本地副本并对其进行操作。当然，我现在必须确保我将
释放动态分配的内存。

Ciao

KB

[我已将我的源列表发布到comp.sources.d]

解决方案

我程序的修改源列表使用strtok_r，我在其中输入字符串的

本地副本并在其上运行strtok

//解析表单字符串的程序：

// Name，SS＃，Emp＃，Other＃:: Name，SS＃，Emp＃，Other＃:: Name，SS＃，Emp＃，

其他＃

//并提取与

员工相关的名称和其他属性

#include< stdio.h>

＃包括< iostream.h>

#include< fstream.h>


#include< list.h>

externC char * strtok_r（char * s1，const char * s2，char **持续）;


#define MAX_DATA_LEN 128

#define EMPLOYEE_DELIMITER_STR"：" ;

#define ATTR_DELIMITER_STR"，"

#define ATTR_DELIMITER_CHAR''，''

struct EmployeeInfo

{

char _name [MAX_DATA_LEN];

size_t _attributeCount;

size_t * _attribute;


EmployeeInfo（const char * str）：_ attributeCount（0），_ attribute（NULL）

{

if（！strstr（str，ATTR_DELIMITER_STR））

strcpy（_name，str）;

else

{

char * temp1 = new char [strlen（str）+ 1] ;

strcpy（temp1，str）;

char * temp2 = temp1;


char * holdingBuf [1];


strcpy（_name，strtok_r（temp1，（const char *）ATTR_DELIMITER_STR，

holdingBuf））;

temp1 + =（ strlen（temp1）+ 1）;

_attributeCount = numChars（temp1，ATTR_DELIMITER_CHAR）+ 1;

_a ttribute = new size_t [_attributeCount];

memset（_attribute，''\'''，_ attributeCount * sizeof（* _ attribute））;


char * stringWithTokens = temp1;

char * token;

int i;

for（i = 0;（token = strtok_r（stringWithTokens，（ const char *）

ATTR_DELIMITER_STR，holdingBuf））！= NULL;

i ++）

{

stringWithTokens = NULL;

_attribute [i] = atoi（令牌）;

}


_attributeCount = i;


删除[] temp2;

}

}


virtual~EmployeeInfo（）

{

if（_attributeCount）delete [] _attribute;

}


void toString（）

{

cout<< " \ nEmployee Name：" << _name<< endl;

for（size_t i = 0; i< _attributeCount; i ++）

cout<< 属性 << i<< ： << _attribute [i]<<结束;

}


受保护：

static size_t numChars（const char * str，const char delimiter）

{

size_t retVal = 0;

char ch;

for（size_t i = 0; ch = str [i] ; i ++）

{

if（ch == delimiter）

retVal ++;

}

返回retVal;

}

};

void

parseNameString（const char * nameString）

{

char * temp1 = new char [strlen（nameString）+ 1];

strcpy（temp1，nameString）;

char * temp2 = temp1;

char * holdingBuf [1];


list< EmployeeInfo *> empInfoList;


char * stringWithTokens = temp1;

char * token;

while（（token = strtok_r（stringWithTokens，（ const char *）

EMPLOYEE_DELIMITER_STR，holdingBuf））！= NULL）

{

stringWithTokens = NULL;

EmployeeInfo * newInfo = new EmployeeInfo（token）;

empInfoList.push_back（newInfo）;

}


delete [] temp2 ;


list< EmployeeInfo *> :: iterator itor，endOfList = empInfoList.end（）;


for（itor = empInfoList。 begin（）; itor！= endOfList; itor ++）

{

EmployeeInfo * thisInfo = * itor;

thisInfo-> toString（） ;

}


for（itor = empInfoList.begin（）; itor！= endOfList; itor ++）

{


EmployeeInfo * thisInfo = * itor;

删除thisInfo;

}

}


int

main（int argc，char * argv []）

{

if（argc< 2）

{

cerr<< 用法： << argv [0]<< " < inputfile中取代。退出.... <<

endl;

退出（-1）;

}


ifstream inFile（argv [1]）;

if（！inFile）

{

cerr<< 读取文件时出错 << argv [1]<< "退出.... <<

endl;

退出（-2）;

}


char inputStr [MAX_DATA_LEN + 1];


/ *

while（inFile>> inputStr）

{

cout<< **解析字符串 << inputStr<< " ** \ n \ n" ;;

parseNameString（inputStr）;

cout<< " \ n \ n \ n";

}

* /


while（inFile.getline（ inputStr，MAX_DATA_LEN））

{

cout<< **解析字符串 << inputStr<< " ** \ n \ n" ;;

parseNameString（inputStr）;

cout<< \ n \ n \ nn;

}


退出（0）;

}

将整个源列表发布到comp.lang.c新闻组

是一个无意中的错误，深感遗憾。

kb***@kaxy.com 写道：< blockquote class =post_quotes>我用strtok_r编写了一些代码，用于解析具有两个
level的字符串。分隔符。代码工作就像一个魅力，直到有一天突然爆发。你看，我在输入
字符串本身上应用strtok，只要变量字符串被传递给
函数，一切都是hunky dory。然后有一天有人决定将一个恒定的字符串传递给我的代码----所有的东西都像一张多米诺骨牌一样崩溃了。

我相信这是一个陷阱许多程序员可能会失败。
我现在已经改变了我的函数的签名，将输入的字符串视为一个常量字符串，我现在制作一个
字符串的本地副本并进行操作那。当然，我现在必须确保我释放动态分配的内存。
Ciao
KB

[我已将我的源列表发布到comp.sources .d]

char * strtok（char *，const char *）;


strtok插入''\\ \\ 0''代替每个分隔符

连续调用。传递一个字符串文字肯定会导致麻烦。传递字符串缓冲区。 ;）


事实上，臭名昭着的臭名昭着小丑的书也做了你所做的事！

远离他的书，除非你想要练习

进行调试。 ;）


问候，

Jonathan。


-

"女性应该附带文件。 - 戴夫

I had written some code using strtok_r for parsing strings with two
"levels" of delimiters. The code was working like a charm, until it
suddenly broke one day. You see, I was applying strtok on the input
string itself, and as long as a variable string was passed to the
function, everything was hunky dory. Then one day somebody decided to
pass a constant string to my code ---- and everything came collapsing
down like a domino.

I believe that this is a pitfall into which many programmers may fall.
I have now changed the signature of my function to treat the input
string as a constant string, and I now making a local copy of the
string and operating on that. Of course, I now have to ensure that I
free up the dynamically allocated memory.
Ciao
KB

[I have posted my source listing to comp.sources.d]

解决方案

Modified source listing for my program using strtok_r in which I make a
local copy of the input string and operate strtok on that
// Program that parses strings of the form:
// Name, SS#, Emp#, Other#::Name, SS#, Emp#, Other#::Name, SS#, Emp#,
Other#
// and extracts the name and other attributes associated with an
employee
#include <stdio.h>
#include <iostream.h>
#include <fstream.h>

#include <list.h>

extern "C" char *strtok_r(char *s1, const char *s2, char **lasts);

#define MAX_DATA_LEN 128
#define EMPLOYEE_DELIMITER_STR ":"
#define ATTR_DELIMITER_STR ","
#define ATTR_DELIMITER_CHAR '',''
struct EmployeeInfo
{
char _name[MAX_DATA_LEN];
size_t _attributeCount;
size_t *_attribute;

EmployeeInfo(const char *str):_attributeCount(0), _attribute(NULL)
{
if(!strstr(str,ATTR_DELIMITER_STR))
strcpy(_name, str);
else
{
char *temp1 = new char[strlen(str) + 1];
strcpy(temp1, str);
char *temp2 = temp1;

char* holdingBuf[1];

strcpy(_name, strtok_r(temp1, (const char *) ATTR_DELIMITER_STR,
holdingBuf));
temp1 += (strlen(temp1) + 1);
_attributeCount = numChars(temp1, ATTR_DELIMITER_CHAR) + 1;
_attribute = new size_t[_attributeCount];
memset(_attribute, ''\0'', _attributeCount*sizeof(*_attribute));

char* stringWithTokens = temp1;
char* token;
int i;
for(i =0;(token = strtok_r(stringWithTokens, (const char *)
ATTR_DELIMITER_STR, holdingBuf)) != NULL;
i++)
{
stringWithTokens = NULL;
_attribute[i] = atoi(token);
}

_attributeCount = i;

delete [] temp2;
}
}

virtual ~EmployeeInfo()
{
if(_attributeCount) delete [] _attribute;
}

void toString()
{
cout << "\nEmployee Name: " << _name << endl;
for(size_t i = 0; i < _attributeCount; i++)
cout << "Attribute " << i << ": " << _attribute[i] << endl;
}

protected:
static size_t numChars(const char* str, const char delimiter)
{
size_t retVal = 0;
char ch;
for(size_t i = 0; ch = str[i]; i++)
{
if(ch == delimiter)
retVal++;
}
return retVal;
}
};
void
parseNameString(const char* nameString)
{
char *temp1 = new char[strlen(nameString) + 1];
strcpy(temp1, nameString);
char *temp2 = temp1;
char* holdingBuf[1];

list<EmployeeInfo*> empInfoList;

char* stringWithTokens = temp1;
char* token;
while((token = strtok_r(stringWithTokens, (const char *)
EMPLOYEE_DELIMITER_STR, holdingBuf)) != NULL)
{
stringWithTokens = NULL;
EmployeeInfo *newInfo = new EmployeeInfo(token);
empInfoList.push_back(newInfo);
}

delete [] temp2;

list<EmployeeInfo*>::iterator itor, endOfList=empInfoList.end();

for(itor = empInfoList.begin(); itor != endOfList; itor++)
{
EmployeeInfo* thisInfo = *itor;
thisInfo->toString();
}

for(itor = empInfoList.begin(); itor != endOfList; itor++)
{

EmployeeInfo* thisInfo = *itor;
delete thisInfo;
}
}

int
main(int argc, char* argv[])
{
if(argc < 2)
{
cerr << "usage: " << argv[0] << " <inputfile>. Exiting ...." <<
endl;
exit(-1);
}

ifstream inFile(argv[1]);
if(!inFile)
{
cerr << "Error reading file " << argv[1] << " Exiting ...." <<
endl;
exit(-2);
}

char inputStr[MAX_DATA_LEN + 1];

/*
while(inFile >> inputStr)
{
cout << "** Parsing string " << inputStr << " **\n\n";
parseNameString(inputStr);
cout << "\n\n\n";
}
*/

while(inFile.getline(inputStr, MAX_DATA_LEN))
{
cout << "** Parsing string " << inputStr << " **\n\n";
parseNameString(inputStr);
cout << "\n\n\n";
}

exit(0);
}

The posting of the entire source listing to the comp.lang.c newsgroup
was an inadvertent mistake which is deeply regretted.

kb***@kaxy.com wrote:

I had written some code using strtok_r for parsing strings with two
"levels" of delimiters. The code was working like a charm, until it
suddenly broke one day. You see, I was applying strtok on the input
string itself, and as long as a variable string was passed to the
function, everything was hunky dory. Then one day somebody decided to
pass a constant string to my code ---- and everything came collapsing
down like a domino.

I believe that this is a pitfall into which many programmers may fall.
I have now changed the signature of my function to treat the input
string as a constant string, and I now making a local copy of the
string and operating on that. Of course, I now have to ensure that I
free up the dynamically allocated memory.
Ciao
KB

[I have posted my source listing to comp.sources.d]

char* strtok (char*, const char*);

strtok works by inserting ''\0'' in place of the delimiter each
successive call. Passing a string literal will definitely
cause trouble. Pass a string buffer. ;)

In fact, Herb-the-infamous-clown''s book does what you did too!
Stay away from his books, unless you want an exercise
in debugging. ;)

Regards,
Jonathan.

--
"Women should come with documentation." - Dave

这篇关于strtok / strtok_r困境的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！