Function tokenize

  • tokenize(source: string, unit?: string): Token[]

  • Shortcut for tokenize(source, bnf.literals);

    Parameters

    • source: string
    • Optionalunit: string

    Returns Token[]

Settings

Member Visibility