Jim's
Tutorials

Fall 2016
course
navigation

Lexing

Though I spend last week remembering Racket, this week I found the parts I actually forgot (which entailed more time spent on relearning).
I went over Matt Might's slides on lexing to prepare for the Python lexer.
I also spend some time getting a sample calculator parser working (no actual arithmetic is done, just lexing).
#lang racket #| Calculator lexer | By Logan Davis | | A simple example of the Racket lexer. |# (require parser-tools/lex) (define end-of-file #f) ; A lexer to recognize common calculator symbols (define calc-lexer (lexer [(union #\+ #\- #\* #\\) (printf "found operator ~a \n" lexeme)] [(repetition 0 +inf.0 (char-range #\0 #\9)) (printf "found value ~a \n" lexeme)] [(repetition 0 +inf.0 (union (char-range #\a #\z) (char-range #\A #\Z))) (printf "found string ~a \n" lexeme)] [#\( (printf "found left paren \n")] [#\) (printf "found right paren \n")] [(union #\space #\newline "") (void)] [(eof) (set! end-of-file #t)])) (define test (open-input-string "+d-( *\\ c a 5) 6 7 345 ABC")) ;;test string for parsing ; takes a input stream and parses until eof (define reader (lambda (file) (if end-of-file '() (and (calc-lexer file) (reader file))))) (reader test)
Only after that did I find Matt Might already has an example like this on his site. :(
I spend a lot more time in racket's docs about regular expressions and lexers this week as well. In prep for parsing python I poked around their docs on the subject too.
I have also started looking into BNF.
http://cs.marlboro.edu/ courses/ fall2016/jims_tutorials/ ldavis/ Sep_14
last modified Wednesday September 14 2016 11:27 am EDT