Nick Says

I spent this week preparing for Matt Might's first project, the python lexer (ooh!). I read his article about lexical analysis in Racket which I think I tried to dig into before without having any background on the subject...I have also been poring over the Racket documentation for lexers while attempting to implement the python lexer. Matt Might provides a skeleton to fill out as far as the lexer goes, so I've been reading the lexical specification of python searching for the specific expressions to identify using the lexer. I have started the lexer code but have not finished it, so there's an incomplete version in my attachments.

Jim says

I encourage you to look at examples smaller than the entire python language.

It also may make sense to look at tools for this stuff in languages that are more familiar ... and then head towards racket ... as we get better at racket itself.

Possibilities :

• http://www.dabeaz.com/ply/
• https://github.com/dabeaz/sly <== more recent
• https://medium.com/@gajus/parsing-absolutely-anything-in-javascript-using-earley-algorithm-886edcc31e5e ... https://nearley.js.org/

attachments

		last modified	size
	pylex.rkt	Sat Apr 27 2024 01:59 pm	5.4K