Re: strtok question

Stu Cazzo <SCazzo@xxxxxxxxx> wrote:
Hi all,
I have a question on why strtok is doing what it's doing for my
splitString( string2 ); call.

Below is the output for the entire program:

token was: word1
token was: word2
token was: word3
token was: word1
token was: word3
empty field found - token <(null)>


The splitString( string1 ); works as expected, 3 tokens are found.
The splitString( string2 ); does not work as I expected.
I was expecting this:

token was: word1
empty field found - token <(null)>
token was: word3

Why does it not see the empty field for lineToken2?

Because that's not how strtok() works. The man page on my machine
for strtok() actually makes it rather clear:

A sequence of two or more contiguous delimiter characters in the
parsed string is considered to be a single delimiter. Delimiter
characters at the start or end of the string are ignored. Put
another way: the tokens returned by strtok() are always non-empty
strings.

So if you have more than one ',' in a row all of them are treated
the same as a single ','.

Is there a better way to strip out the tokens for the case I have
where there is no space between my delimiter?

I guess you will have to write your own function for that, probably
repeatedly using strchr() or strstr().

Regards, Jens
--
\ Jens Thoms Toerring ___ jt@xxxxxxxxxxx
\__________________________ http://toerring.de
.

Relevant Pages

  • Re: use of strtok( )
    ... tokens that are themselves strings, ... The original delimiter is gone, ... calls to strtok() will not find it. ... The second argument is a *set* of delimiter characters, ...
    (comp.lang.c)
  • strtok question
    ... The splitString(string1); works as expected, 3 tokens are found. ... The splitString(string2); does not work as I expected. ... Why does it not see the empty field for lineToken2? ...
    (comp.lang.c)
  • Re: BOOLEAN VBA FUNCTION !!!
    ... String2 As String) As Boolean ... Dim arr1 As Variant ... If ALL the tokens in String2 are also tokens of String1, ...
    (microsoft.public.excel.programming)
  • Re: Simple text parsing error
    ... As for the delimiter I'm using spaces, ... Okay, that's the problem, the tokens vary. ... It is rarely justified to declare an arbitrary array size. ... Then I can split the response code and the article numbers. ...
    (comp.lang.java.help)
  • Re: Lex Question
    ... use that delimiter to read the rest of the message? ... read using the same lex rule. ... Your best bet is a pre-processor that passes tokens to ... character class of the delimiter character). ...
    (comp.unix.programmer)