complexion
A fast tokenizer/lexer core that is well tested.
Found 14 results for tokeniser
A fast tokenizer/lexer core that is well tested.
Tokenize JavaScript
An HTML5 lexer
A regular expression (RexExp) stream tokenizer.
HTML and CSS lexer aimed at code with fatal errors, accepts mixed coding languages
Is a character on a given index a closing of an HTML attribute?
Is given character suitable to be in an HTML attribute's name?
A simple, Twitter-aware tokenizer.
Time a JavaScript tokenizer
An HTML5 lexer
RFC1459 and IRCv3 protocol tokeniser
Simple, lightweight Markdown-like language with extensible grammar.
Tokenizes a text from left to right and returns the list of tokens with their types.
Piecinise your strings!