Re: Parsing

KDawg44 wrote:
I am writing a top-down parser in Perl for a simple grammar. I would
like to split on a change in a character set. Is there a way to split
the tokens based on a change of a character set (like changing from a /
\w+/ to /+/ or something like that.

Basically I want to be able to split var = var1 + var 2; into tokens
(var, =, var1, +, var2, ;). I could do this with split / / but I
would also like var=var1+var2; to split into the same tokens.

How can I parse these two and get the same thing?

my @tokens = split /\s*([-+=;])\s*/, $string;

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
.

Relevant Pages

  • Re: Parsing
    ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ... Hard-coding the lexing and parsing steps as others have shown you is ... probably not what you want for anything but the simplest grammar. ...
    (comp.lang.perl.misc)
  • Re: invalid single quote
    ... The input has already been broken into tokens before any ... macro processing is done and the result of a macro expansion is itself a ... But the single quote ', being a punctuation, belongs ... to the basic C character set. ...
    (comp.lang.c)
  • Re: Parsing
    ... Gunnar Hjalmarsson wrote: ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ... Tad McClellan ...
    (comp.lang.perl.misc)
  • Compiler in Perl
    ... I am writing a top-down parser in Perl for a simple grammar. ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ...
    (perl.beginners)
  • Parsing
    ... I am writing a top-down parser in Perl for a simple grammar. ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ...
    (comp.lang.perl.misc)