Re: tokens

On Aug 10, 5:11 pm, Harald van Dijk <true...@xxxxxxxxx> wrote:
On Sun, 10 Aug 2008 17:01:18 -0700, mdh wrote:
Is that why my example of ( ) is not a single token as the first "("
could be the start of a lot of different things?

No, your example of ( ) is not a single token because the two can be
separated.

Sorry...then I did not make myself clear. I am agreeing with you. ()
are 2 tokens, as each ( has a a meaning to the compiler.

The reason this is somewhat confusing is that on p125 of K&R2, they
define "tokens" ( and the quotation marks are theirs, not mine) as a
pair of parentheses, a pair of brackets perhaps including a number. I
assume that their definition is for the sake of the example, then.
.

Relevant Pages

  • Re: The Groovy Programming Language
    ... the benefit of whitespace to separate logically distinct symbols ... The use of parentheses or some other tokens ... Making those parentheses optional creates non-obvious changes in the ... I can design a language superior to Groovy. ...
    (comp.lang.lisp)
  • Re: A
    ... the characters 'fn_name' become the ... They will not form two separate tokens of '', ... parentheses are not special characters (that is, ...
    (comp.unix.shell)
  • Re: Is This Expression Parsing Feasible?
    ... to right) noting the tokenwith the highest precedence and making ... sub-trees out of those tokens and the tokens ... What about parentheses in the expression? ... tree nodes are iteratively evaluated (with the same recursive/stack ...
    (comp.compilers)
  • Re: Google-like query tokenizer
    ... Meaning that tokens are separated by spaces unless grouped with ... parentheses. ... Can the StringTokenizer do this? ...
    (comp.lang.java.programmer)
  • Re: problem with splitting on "words"
    ... Charlotte Hee wrote: ... why not capture the tokens you're interested in. ... You don't really need parentheses, ...
    (perl.beginners)