Next
Up
Prev
Home
This
[tokenizers.ipk]
6.7. Tokenisers
Tokenise various languages.
Start
python
section to
interscript/tokenisers/__init__.py[1 /1 ]
1:
#line 4 "tokenizers.ipk"
2:
# tokenisers package
3:
6.7.1. Python Tokeniser
6.7.2. Parsers
Next
Up
Prev
Home
This
[tokenizers.ipk]