Function tokenize

tokenize(
source: string,
definitions: Iterable<TokenDefinition, any, any>,
): Token[]
Parameters
- source: string
- definitions: Iterable<TokenDefinition, any, any>
Returns Token[]
- Defined in tokens.ts:14