6.7. Tokenisers

Tokenise various languages.
Start python section to interscript/tokenisers/__init__.py[1 /1 ]
     1: #line 4 "tokenizers.ipk"
     2: # tokenisers package
     3: 


6.7.1. Python Tokeniser
6.7.2. Parsers