[RFC] C token counter

From: Arthur J. O'Dwyer (ajo_at_nospam.andrew.cmu.edu)
Date: 04/30/04 

Date: Fri, 30 Apr 2004 01:37:06 -0400 (EDT)

  Request for comments on the following program:
http://www.contrib.andrew.cmu.edu/~ajo/workshop/tokens.c

  It's a token counter for the C programming language, following
the outline kind-of-described here:
http://www.kochandreas.com/home/language/tests/TOKENS.HTM
Basically, it's supposed to give a reasonable approximation of
the number of "atomic tokens" present in a C source file.

  To those reading in c.l.c: Are there any glaring mistakes or
poor coding styles in the program itself? And did I miss any
corner cases --- does the program produce a wrong number of
tokens for any valid C99 program?

  To those reading in c.l.m: Andreas, this is for you. (:
I'm just posting it generally in case anyone has any comments
along the lines of "gee, C is nifty!" or "gee, C can't do
anything!"

  Please set follow-ups appropriately in your reply: comp.lang.c
probably doesn't care about Practical Language Comparison ;) and
comp.lang.misc definitely doesn't want long pedantic debates about
whether #define is one token or two.[1] ;)

-Arthur

[1] - I'm counting it as one token in my program, even though
it looks like technically it's not a "token" in any sense of the
word, Standard-ly speaking.

Quantcast