Parsing



I am writing a top-down parser in Perl for a simple grammar. I would
like to split on a change in a character set. Is there a way to split
the tokens based on a change of a character set (like changing from a /
\w+/ to /+/ or something like that.

Basically I want to be able to split var = var1 + var 2; into tokens
(var, =, var1, +, var2, ;). I could do this with split / / but I
would also like var=var1+var2; to split into the same tokens.

How can I parse these two and get the same thing?

Thanks.

Kevin
.



Relevant Pages

  • Compiler in Perl
    ... I am writing a top-down parser in Perl for a simple grammar. ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ...
    (perl.beginners)
  • Re: Is perl better? :(((
    ... >Recently I tried port one perl script. ... >thousands SQLs retrieve and store/summarize rezults. ... like the problem would be in character set conversions. ... results when using JDBC OCI driver; ...
    (comp.lang.java.databases)
  • Re: Is perl better? :(((
    ... >Recently I tried port one perl script. ... >thousands SQLs retrieve and store/summarize rezults. ... like the problem would be in character set conversions. ... results when using JDBC OCI driver; ...
    (comp.lang.java.programmer)
  • RE: Oracle: Embedded Perl & 7/8 bit character sets
    ... after several days of investigations we found out that using the non-standard character set US7ASCII also affected the C++ programs. ... I did not use any specific DBI connect routine such as telling DBD to share the connection. ... It seems to be that the Oracle client libraries provides the result in a way that the Perl driver cannot decode. ...
    (perl.dbi.users)
  • Re: Parsing
    ... like to split on a change in a character set. ... the tokens based on a change of a character set (like changing from a / ... Hard-coding the lexing and parsing steps as others have shown you is ... probably not what you want for anything but the simplest grammar. ...
    (comp.lang.perl.misc)